The invention discloses a GPU Box hardware decoupling
system based on a Retimer card and a PCIe Switch
chip, and belongs to the technical field of
computer hardware architecture and high-speed
interconnection. According to the
system, a Retimer card and a PCIe Switch
chip are integrated in an independent GPU Box, and a decoupling link of a CPU
server and a GPU acceleration card is constructed; the Retimer card realizes 30-meter long-distance PCIe
signal transmission and breaks through physical distance limitation; the PCIe Switch
chip pools GPU resources through a dynamic routing and MRIOV technology, supports flexible allocation of computing power by multiple servers, and realizes Peer-to-Peer direct connection communication between GPUs. Aiming at a
large model reasoning scene, the
system optimizes KV cache
bandwidth allocation and
video memory and memory cooperative scheduling, so that the 100B parameter model reasoning
throughput is greatly improved; and meanwhile, the
usability of the system is greatly improved through fault isolation and hot plug design. According to the method, the problems of physical binding of the CPU and the GPU, limited transmission distance, rigid
resource allocation and the like in a traditional architecture are solved, and the method is suitable for large-scale AI calculation and distributed
GPU cluster deployment.