Server, server node and node interconnection system

Abstract

The application discloses a kind of server, server node and node mutual contact system, it is related to server technical field, including: data processor, at least one exchanger, storage device and at least one graphics processor, data processor includes network data transceiver and remote direct memory access engine, remote direct memory access engine is used to transmit the network data received by network transceiver to storage device through exchanger, graphics processor and storage device can be carried out data transmission by exchanger, server does not need to carry and copy network data for multiple times in the process of processing network data, can save bandwidth, improve the efficiency of data transmission and processing.

Description

Technical Field

[0001] This application relates to the field of server technology, and in particular to a server, server node, node interconnection system, data transmission method, device and storage medium. Background Technology

[0002] Currently, when a server processes received network data, it first needs to store the network data in the CPU's memory, then transfer it from the CPU's memory to the GPU's video memory, where the GPU processes the network data before transferring it back to the CPU's memory, and finally to the storage device. In this process, the network data needs to be moved and copied multiple times, which affects data transmission efficiency. Summary of the Invention

[0003] This application provides a server, server node, and node interconnection system to at least solve the problem of low data transmission efficiency in related technologies.

[0004] This application provides a server, including: a data processor, at least one switch, a storage device, and at least one graphics processor; the switch includes an upstream port and at least two downstream ports; wherein, the upstream port is connected to the data processor, at least one downstream port is connected to the storage device, and at least one downstream port is connected to at least one graphics processor. The data processor includes a network transceiver, which is equipped with a remote direct memory access engine. The network transceiver is used to receive network data and transmit the network data to a storage device or at least one graphics processor via a switch through the remote direct memory access engine. At least one graphics processor and storage device transmit data via a switch.

[0005] This application also provides a server node, on which at least one server as described above is provided.

[0006] This application also provides a node interconnection system, including a network switch and at least one server node as described above, wherein the at least one server node is communicatively connected through the network switch.

[0007] The server, server node, and node interconnection system of this application include a server comprising a data processor, at least one switch, a storage device, and at least one graphics processor. The data processor includes a network transceiver and a remote direct memory access engine. The remote direct memory access engine is used to transmit network data received by the network transceiver to the storage device through the switch. The graphics processor and the storage device can transmit data through the switch. The server does not need to repeatedly move and copy network data during the processing of network data, which can save bandwidth and improve the efficiency of data transmission and processing. Attached Figure Description

[0008] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of an existing type of server; Figure 2 A schematic diagram of a server provided for one embodiment of this application; Figure 3 A schematic diagram of a server provided for another embodiment of this application; Figure 4 A schematic diagram of a server node provided in one embodiment of this application; Figure 5 A schematic diagram of a node interconnection system provided in one embodiment of this application;

[0010] Figure 6 A flowchart illustrating a data transmission method provided in one embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0012] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0013] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] like Figure 1 The diagram shown illustrates a conventional server. This server includes a network interface card (NIC), a central processing unit (CPU), a switch, multiple graphics processors (GPUs), and storage devices. The upstream port of the switch is connected to the CPU, the CPU is connected to both the NIC and the storage devices, and the multiple downstream ports of the switch are connected to the GPUs.

[0015] The server receives network data through the network card and stores it in the CPU's memory. When the network data needs to be processed, it is transferred from the CPU's memory to the GPU's video memory. The GPU processes the network data and then transfers it back to the CPU's memory before finally storing it in the storage device. In this process, the network data needs to be moved and copied multiple times, which affects the data transmission efficiency.

[0016] Therefore, in response to the above problems, such as Figure 2 As shown, this application provides a server, including: a data processor 110, at least one switch 120, a storage device 130, and at least one graphics processor 140; Switch 120 includes an upstream port and at least two downstream ports; wherein the upstream port is connected to data processor 110, at least one downstream port is connected to storage device 130, and at least one downstream port is connected to at least one graphics processor 140. Data processor 110 includes network transceiver 111, which is equipped with a remote direct memory access engine. The network transceiver 111 is used to receive or send network data, and the remote direct memory access engine is used to transfer network data to storage device 130 through switch 120. At least one graphics processor 140 and storage device 130 transmit data via switch 120.

[0017] The network transceiver 111 can be a separate functional module within the data processor 110. Alternatively, in this embodiment, the network transceiver 111 can be integrated into the data processor 110.

[0018] In this embodiment, the data processor 110 can be a data processing unit (DPU), which can be used to improve the performance of complex application scenarios such as network acceleration, storage acceleration, virtualization computing, and high-performance computing.

[0019] The network transceiver 111 can be an Ethernet card, an unlimited bandwidth (IB) network card, or other device capable of network data transmission and reception. Optionally, the network transceiver 111 can be a high-speed network card of 200GbE or 400GbE.

[0020] The Remote Direct Memory Access Engine is used to perform Remote Direct Memory Access (RDMA) operations. The Remote Direct Memory Access Engine is used to bypass the remote central processing unit and operating system kernel, so that the data processor 110 can directly read and write the memory of the storage device 130.

[0021] Switch 120 is used to provide a signal channel for data processor 110, storage device 130 and graphics processor 140 so that data processor 110, storage device 130 and graphics processor 140 can transmit signals through the signal channel.

[0022] In this embodiment, the switch 120 can be a PCIe switch, which is a multi-port switching / expansion chip operating at the PCIe protocol layer, used to expand PCIe topology, fan-out ports, cascade devices, and forward data packets within the PCIe domain. The storage device 130 is used to store data. The storage device 130 can be volatile semiconductor memory (DDR memory), non-volatile memory (NVMe SSD), or other commonly used storage devices.

[0023] The graphics processing unit (GPU) 140 is used to perform computations on network data. For example, when the server is used for training an artificial intelligence model, the network data can be the training data for the artificial intelligence model, and the GPU 140 can be used to perform operations such as forward propagation and inference on the training data.

[0024] The following uses the application of the server in this application to an artificial intelligence model training scenario as an example to illustrate the data transmission process of the server in this application: Specifically, the network transceiver 111 receives training data over the network, and the remote direct memory access engine transmits the training data to the storage device 130 via the switch 120 for storage.

[0025] When it is necessary to train an artificial intelligence model, the graphics processor 140 obtains training data pre-stored in the storage device 130 through the switch 120, and then uses the training data to train the artificial intelligence model.

[0026] After the artificial intelligence model performs inference operations, the graphics processor 140 can use the exchange 120 to write the inference data obtained by the graphics processor 140 through the inference operations to the storage device 130.

[0027] The above data transfer process does not require copying data to the local memory of the central processing unit or data processor, which can improve the data flow speed and transmission efficiency.

[0028] In this embodiment, the server's upstream port is connected to a data processor, and at least one downstream port of the switch is connected to a storage device. The at least one downstream port is also connected to at least one graphics processor. The data processor includes a network transceiver and a remote direct memory access engine. The network transceiver is used to receive network data, and the remote direct memory access engine is used to transmit the network data received by the transceiver to the storage device through the switch. At least one graphics processor and the storage device transmit data through the switch, thereby improving data loading and data read / write efficiency.

[0029] In one embodiment, switch 120 may be a switch with a large number of lanes, such as a PCIe switch with 320 lanes, to meet the communication needs of multiple connected devices.

[0030] In another embodiment, when the number of downstream ports or channels of a single switch 120 cannot meet the application requirements, the server may also include at least two switches 120 connected in parallel. By connecting multiple switches 120 in parallel, more downstream ports and signal channels can be provided to meet the communication needs of the server.

[0031] like Figure 3 As shown, in one embodiment, the data processor 110 may further include a central processing unit 112 and a root port controller 113. The central processing unit 112 is used to obtain device information, which is the information of the devices connected to the upstream and downstream interfaces of the switch 120. Based on the device information, it determines channel resource configuration information and sends it to the root port controller 113, so that the root port controller 113 configures the channel resources of the switch 120 according to the channel resource configuration information.

[0032] The central processing unit 112 can enumerate the connected devices, obtain device information, and independently discover, configure, and manage the devices connected to its downstream ports.

[0033] The central processing unit 112 can enumerate access devices through operations such as reading and writing configuration space, allocating BAR (Base Address Register) resources, and mapping configuration message interrupts.

[0034] The CPU 112 can scan the bus, read the configuration space, and identify information such as the type, manufacturer, signal, version, device capabilities, and supported behaviors of the accessed devices. The CPU 112 can also read the BAR (Base Address Register) register of the accessed devices and allocate a segment of available system physical address space to the BAR register, enabling the CPU 112 to access the device's internal resources through memory mapping. The message interrupt mapping can be configured as an MSI / MSI-X (Message Signaled Interrupt) message interrupt mapping. The MSI / MSI-X message interrupt mapping configuration determines the address, memory, corresponding processor core, and driver function for each interrupt event of the device. Upon receiving interrupt information from the device, the CPU determines the corresponding processor core to execute the interrupt event based on the MSI / MSI-X message interrupt mapping configuration.

[0035] The central processing unit 112 can dynamically determine the channel resource configuration information based on a preset allocation strategy, according to information such as the type and number of devices connected to the upstream and downstream interfaces. The channel resource configuration information may include the number of root ports and the link width. The root port controller 113 is used to allocate channel resources of the switch 120 according to channel resource configuration information, such as configuring the switch as a root port with a corresponding number and link width.

[0036] The central processing unit 112 can be a processor integrated into the data processor 110. In this embodiment, the central processing unit 112 can be an ARM architecture processor.

[0037] The link width can be the number of channels allocated. For example, when the switch 120 has 64 channels, the root port controller 113 can allocate it as 8 root ports with a link width of 8 channels; or, allocate it as 16 root ports with a link width of 4 channels.

[0038] In this embodiment, the central processing unit obtains device information, determines channel resource configuration information based on the device information, and sends it to the root port controller. The root port controller then configures the channel resources of the switch according to the channel resource configuration information, dynamically adjusting the number of ports and link width allocated to the switch to meet the application needs of different compute-to-memory ratio scenarios.

[0039] Optionally, the central processing unit can also load corresponding drivers for devices connected to downstream ports of the switch, such as graphics processors and storage devices. For example, the central processing unit can load corresponding CUDA, ROCm, CANN, etc. drivers for the connected graphics processors according to the model of the graphics processor, so as to realize driver loading and management of the connected devices.

[0040] In one embodiment, the central processing unit is further configured to activate the root port controller in response to a power-on operation, causing the root port controller to sequentially control at least two downstream ports to be turned on according to a preset power-on sequence, thereby enabling power-on of at least one switch, at least one graphics processor, and a storage device.

[0041] The power-on sequence of each device at the downstream port can be configured according to requirements.

[0042] In this embodiment, after the data processor is powered on first, its central processing unit starts the operating system, which in this embodiment can be a Linux operating system. Subsequently, the root port controller is activated to sequentially power on devices connected to downstream ports.

[0043] Optionally, the power-on sequence is the switch, graphics processor, and storage device.

[0044] In this embodiment, the process can proceed as follows: First, the data processor is powered on, causing it to load the boot program or firmware. For example, in this embodiment, after the data processor is powered on, the boot ROM inside the data processor loads the DOCA firmware to initialize the data processor. Next, the switch is powered on, enabling the switch to perform link training with the data processor and establish a communication connection between them. Then, the graphics processor is powered on. When the link training is complete, the data processor enumerates the number of connected graphics processors, obtains device information, and loads the corresponding drivers for the graphics processors. For example, it loads drivers for CUDA, ROCm, CANN, etc., based on the model of the graphics processor. After the storage device is powered on, the data processor mounts the operating system's file system, allowing the operating system to access data on the storage device. Simultaneously, the data processor establishes a direct memory access mapping table (GPUDirect Storage mapping table) between the graphics processor and the storage device, associating the address of the graphics processor's video memory with the address of the storage area of ​​the storage device.

[0045] In this embodiment, at least one switch, storage device, and at least one graphics processor are powered on in a preset power-on sequence to ensure that each device starts up after stable power supply and avoids failure of data processor enumeration access to devices due to incorrect power supply sequence.

[0046] In one embodiment, the central processing unit is also configured to initialize the network transceiver after the storage device is powered on, establish an address mapping relationship between the network transceiver, the storage device, and at least one graphics processor, and enable the remote direct memory access engine to transmit network data to the storage device or at least one graphics processor through a switch.

[0047] Establishing an address mapping relationship between a network transceiver, a storage device, and at least one graphics processor can be done by registering in the memory region (MR) of the storage device and the graphics processor, so as to allow the remote direct memory access engine to directly access the memory of the storage device or the video memory of the graphics processor.

[0048] In one embodiment, the network transceiver is further configured to receive task requests, and the central processing unit is configured to respond to the task requests by allocating corresponding virtual resources for the tasks in the task requests, so that the server can use the virtual resources to execute the tasks.

[0049] A task request is used to request the server to execute the task corresponding to the task request. In this embodiment, the task can be a training task for an artificial intelligence model. In other embodiments, the task can be set according to actual application requirements.

[0050] Virtual resources can include the server's computing and / or storage resources. For example, a separate memory address space, port, and channel resources can be allocated to the task to avoid interference between the task and other tasks that the server is executing or about to execute.

[0051] Optionally, the data processor 110 can use port virtualization technology (IOMMU) to allocate independent virtual functions (VFs) to different tasks. Each virtual function can have an independent PCIe configuration space, base address register (BAR), and interrupt resources. Each virtual function can be managed and used by the server as a virtual independent PCIe device. The server allocates a virtual function for each task, thereby achieving isolated processing of each task.

[0052] In this embodiment, after receiving a task request, the central processing unit allocates corresponding virtual resources to the corresponding task to avoid interference between the task and other tasks being executed or about to be executed by the server.

[0053] In one embodiment, the switch 120 is equipped with a direct memory access engine for enabling direct data transfer between the video memory of at least one graphics processor and the memory of a storage device.

[0054] In this embodiment, the data processor 110, at least one switch 120, storage device 130, and at least one graphics processor 140 can interact with each other based on the Peripheral Component Interconnect Express (PCIe) bus.

[0055] A direct memory access engine can be a peer-to-peer direct memory access engine. Peer-to-peer direct memory access (P2P DMA) technology allows direct data transfer between two devices connected on the same PCIe bus (such as through a PCIe switch) without the data needing to pass through a central processing unit (CPU) or system memory (RAM).

[0056] In other embodiments, the direct memory access engine may also employ other interconnect technologies that enable direct data transfer between the graphics processor's video memory and the storage device's memory.

[0057] In this embodiment, a point-to-point direct memory access engine is used to realize direct data transfer between the graphics processor's video memory and the storage device's memory. The entire process does not require passing through the local memory of the data processor, enabling end-to-end direct transmission from the network to the storage or central processing unit, achieving zero-copy data transfer, and improving data transfer efficiency.

[0058] In one embodiment, the graphics processor is also used to generate a check code, and send the check code and the data to be transmitted to the storage device through a switch, so that the storage device performs cyclic redundancy check based on the check code. Alternatively, the storage device can also be used to generate a checksum, which is then sent to the graphics processor via a switch, allowing the graphics processor to perform a cyclic redundancy check based on the checksum.

[0059] Cyclic Redundancy Check (CRC) refers to a process where the sender calculates a fixed-length checksum based on the data and a pre-defined algorithm, and sends the checksum along with the data. The receiver recalculates the checksum using the same algorithm upon receiving the data. If the calculated checksum does not match the received checksum, it indicates that an error occurred during data transmission.

[0060] When a graphics processor and storage device are connected to a switch via the PCIe bus, a checksum can be encapsulated along with the data to be transmitted into a Transaction Layer Packet (TLP). A TLP is a core data packet in the PCIe protocol used to transmit data between endpoints. A TLP consists of a header, the data to be transmitted, and a checksum. The header may include sender information, destination address, TLP type (such as memory read / write, configuration read / write, etc.), and data length.

[0061] In this embodiment, when the graphics processor and storage device transmit data through a switch, a checksum and the data to be transmitted are sent together. This allows the receiver to verify the integrity of the data based on the checksum, and enables timely retransmission in case of data transmission errors, thereby improving the reliability of the received data.

[0062] In one embodiment, the server can be used to train an artificial intelligence model, and the graphics processor can be used to write intermediate values ​​from the calculations performed by the artificial intelligence model into the cache of the storage device via a switch; the graphics processor can also be used to write the result data calculated by the artificial intelligence model into the storage area of ​​the storage device via a switch.

[0063] Intermediate values ​​can be intermediate data during model execution, such as input and / or output values, activation values, etc., of each layer of an artificial intelligence model. In this embodiment, intermediate values ​​can also be key-value cache data.

[0064] The artificial intelligence model can be a large language model such as GPT-4, Llama 3, or DeepSeek-V3, or other artificial intelligence models.

[0065] When the input to the next layer of an AI model depends on the output of a certain layer, intermediate activation values ​​can be temporarily stored in the cache of a storage device. After the AI ​​model calculates the final result, the result data is then written to the storage area of ​​the storage device for persistent storage. This allows for the reuse of intermediate activation values, reduces the computational load of the AI ​​model, and improves the training efficiency of the AI ​​model.

[0066] In one embodiment, the data processor further includes a key-value processing engine for serializing or deserializing key-value cache (KV Cache) data.

[0067] Serialization is the process of converting objects and data structures (such as dictionaries, lists, and instances of classes) in program memory into a data format that can be stored or transmitted, such as JSON, XML, binary byte streams, and Protocol Buffers. In this embodiment, key-value cache data is serialized to facilitate cross-process and cross-network transmission of key-value cache data and to preserve the complex data structures of key-value data.

[0068] Deserialization refers to restoring serialized data to its original object or data structure.

[0069] In traditional server architectures, the host CPU performs serialization / deserialization and protocol processing on key-value cache data, consuming the host's computing resources. Furthermore, the key-value cache data needs to be relayed through the CPU, resulting in significant data transmission latency.

[0070] In this embodiment, by setting a key-value processing engine in the data processor, the key-value processing engine is used to serialize or deserialize key-value cached data, thereby freeing up host resources, eliminating the need for relaying data in the host central processing unit, and reducing data transmission latency.

[0071] In one embodiment, the remote direct memory access engine is also used to write serialized or deserialized key-value cache data to a graphics processor or storage device.

[0072] Specifically, the remote direct memory access engine can write the serialized or deserialized key-value cache data output by the key-value processing engine to the graphics processor's video memory, or write the serialized or deserialized key-value cache data output by the key-value processing engine to the memory of the storage device.

[0073] After serializing or deserializing key-value cache data using a key-value processing engine, the serialized or deserialized key-value cache data is written to a graphics processor or storage device via a remote direct memory access engine, thereby reducing the host's computational overhead.

[0074] In one embodiment, the switch further includes a communication module, through which the server communicates with the communication modules of other switches of other servers, enabling the graphics processor to interact with the graphics processors connected to other switches.

[0075] The communication module is used to enable data interaction between the server's graphics processor and other graphics processors connected to the switch.

[0076] When the switch is a PCIe switch, the communication module can be a non-transparent bridge (NTB) module for the PCIe switch. A non-transparent bridge is a special PCIe bridging function that allows two PCIe domains with independent address spaces and independent hosts to communicate while maintaining logical isolation between them. In other embodiments, the communication module can also employ existing communication modules with the same functionality.

[0077] In this embodiment, the server and other servers can be servers on the same node. Through the communication module, computing resources and data can be shared between servers on the same node, which facilitates collaborative work between servers on the same node.

[0078] In this embodiment, the graphics processors of different servers can communicate with each other through the communication module of the switch, thereby realizing point-to-point communication between the graphics processors of different data processor control domains, which facilitates the collaborative work of multiple servers.

[0079] In one embodiment, the server is located at a first node, the first node is connected to at least one second node via a network switch, the second node is equipped with at least one server, and the first node and at least one second node interact with each other via the network switch.

[0080] The structure of the server in the second node can be the same as the server structure in this application.

[0081] Network switches with higher transmission rates can be selected, such as 800Gb / s network switches, to improve the data transmission efficiency between nodes.

[0082] The first and second nodes can share data through the network data transceiver of the server located on that node, thereby enabling cross-node data scheduling.

[0083] In this embodiment, the first node and the second node interact with each other through a network switch, which can realize data sharing, data synchronization, data storage, etc. between nodes, and realize the sharing of node computing power and storage power.

[0084] In one embodiment, the first node and at least one second node are connected to the master node via a network switch. The master node is used to divide the inference task into at least two inference subtasks when it receives an inference task, and send the at least two inference subtasks to the corresponding nodes of the first node and at least one second node, so that the nodes execute the inference subtasks.

[0085] The master node can divide the inference task into multiple inference subtasks by means of equal distribution or according to the computing power of each node, and the master node determines each inference subtask.

[0086] After executing the inference subtask, the first and second nodes can store the inference results obtained from the inference subtask on the storage device of the server of that node, and share them with other nodes through the server network data transceiver.

[0087] The first node and the second node can determine the storage location of the data for the inference subtask based on the data address information received when the inference subtask execution request is received. The inference subtask execution request is used to request the node to execute the inference subtask. The data address information may include the storage file offset, storage object ID, block length, and other information. The storage file offset is used to record the storage address, the block length is used to determine the length of the data block, and the storage object ID is used to determine the specific storage device.

[0088] In this embodiment, the master node may have at least one server as described in the above embodiment. The master node is used to distribute inference tasks, divide the inference tasks into multiple inference subtasks and distribute them to the first node and the second node, and use multiple nodes to complete the inference tasks. In this way, by sharing the computing power of each node, the efficiency of completing the inference tasks is improved.

[0089] In one embodiment, the first node and at least one second node are connected to the master node via a network switch. The master node is used to divide the target data into at least two target sub-data when it receives the target data, and send the at least two target sub-data to the corresponding nodes among the first node and at least one second node, so that the nodes store the target sub-data.

[0090] The target data can be large data with a volume exceeding the preset value. After receiving a large amount of target data, the master node fragments it to make it easier to transmit.

[0091] The master node can use existing sharding methods to divide the target data into multiple target sub-data, such as dividing it equally or according to the storage space of each node.

[0092] The target sub-data stored by the first and second nodes can be stored in the storage devices of the servers located at those nodes, and shared with other nodes through the network transceivers of the servers located at those nodes. The storage location of the target sub-data in the storage device can be determined based on the data address information received when a data reception request is received. The data reception request is used to request the node to receive the target sub-data, and the data address information can include the storage file offset, storage object ID, block length, etc., of the target sub-data. The storage file offset records the storage address, the block length determines the length of the data block, and the storage object ID identifies the specific storage device. The master node can have at least one server. The master node can be used to divide the target data into multiple target sub-data and distribute them to the first and second nodes, achieving storage of large amounts of data across multiple nodes, thereby improving data storage efficiency by sharing the storage capacity of each node.

[0093] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0094] like Figure 4 As shown, embodiments of this application also provide a server node 200, on which at least one server 210 as described above is provided.

[0095] In one embodiment, when the server node includes at least two servers 210, the at least two servers 210 communicate with each other through a communication module, enabling the graphics processors of the at least two servers to interact with each other.

[0096] like Figure 5 As shown, embodiments of this application also provide a node interconnection system, including a network switch 300 and at least one server node 200 as described above, wherein the at least one server node 200 is communicatively connected through the network switch 300.

[0097] In one embodiment, the server node includes a master node, a first node, and a second node. The master node is used to divide the inference task into at least two inference subtasks when it receives an inference task, and send the at least two inference subtasks to the corresponding nodes of the first node and at least one of the second nodes, so that the nodes execute the inference subtasks.

[0098] In one embodiment, the server node includes a master node, a first node, and a second node. The master node is used to divide the target data into at least two target sub-data when it receives target data, and send the at least two target sub-data to the corresponding nodes of the first node and at least one second node, so that the nodes store the target sub-data.

[0099] like Figure 6 As shown, embodiments of this application also provide a data transmission method, applied to a server as described in any of the above claims, comprising: Step S110: Receive network data and transmit the network data to a storage device or at least one graphics processor via a switch; Step S120: Transfer the processed data of the graphics processor to the storage device via a switch; and / or transfer the data stored in the storage device to the graphics processor via a switch.

[0100] In one embodiment, the switch is equipped with a point-to-point direct memory access engine, which enables direct data transfer between the video memory of at least one graphics processor and the memory of a storage device.

[0101] In one embodiment, transferring the processed data from the graphics processor to a storage device via a switch further includes: Generate a checksum, and send the checksum and the data to be transmitted to the storage device through the switch, so that the storage device can perform cyclic redundancy check based on the checksum. The method of transferring data stored in the storage device to the graphics processor via a switch also includes: Generate a checksum, and send the checksum and data to the graphics processor via an exchange, so that the graphics processor can perform cyclic redundancy check based on the checksum.

[0102] In one embodiment, the data processor further includes a key-value processing engine, and the method further includes: using the key-value processing engine to perform serialization or deserialization processing of key-value cached data.

[0103] In one embodiment, the method further includes: using a remote direct memory access engine to write serialized or deserialized key-value cache data to a graphics processor or storage device.

[0104] In one embodiment, the method further includes: Obtain device information, which includes information about the devices connected to the upstream and downstream interfaces of the switch. Determine the channel resource configuration information based on the device information and send it to the root port controller, so that the root port controller can configure the channel resources of the switch according to the channel resource configuration information.

[0105] In one embodiment, the method further includes: In response to the power-on operation, the root port controller is activated, causing the root port controller to sequentially control the opening of at least two downstream ports according to the preset power-on sequence, thereby enabling the power-on of at least one switch, at least one graphics processor, and storage device.

[0106] In one embodiment, the method further includes: After the storage device is powered on, the network transceiver is initialized to put it into data receiving mode.

[0107] In one embodiment, the method further includes: Receive task requests, respond to task requests, and allocate corresponding virtual resources for the requested tasks so that the server can use the virtual resources to execute the tasks.

[0108] In one embodiment, the method further includes: The graphics processor (GPU) communicates with other switches on other servers via the communication module, enabling data exchange between GPUs connected to other switches.

[0109] In one embodiment, the method further includes: Train the artificial intelligence model; The intermediate activation values ​​during the computation of the artificial intelligence model are written to the cache of the storage device through the switch; The results calculated by the artificial intelligence model are written to the storage area of ​​the storage device through the switch.

[0110] In one embodiment, the server is located at the first node, and the method further includes: Data interaction is performed with at least one second node via a network switch.

[0111] In one embodiment, the method further includes: Upon receiving an inference task, the inference task is divided into at least two inference subtasks, and the at least two inference subtasks are sent to the corresponding nodes in the first node and at least one second node, so that the nodes execute the inference subtasks.

[0112] In one embodiment, the method further includes: Upon receiving target data, the target data is divided into at least two target sub-data, and the at least two target sub-data are sent to the corresponding nodes in the first node and at least one second node, so that the nodes store the target sub-data.

[0113] Embodiments of this application also provide a data transmission apparatus, including: A network data receiving module is used to receive network data and transmit the network data to a storage device or at least one graphics processor via a switch. A data transmission module is used to transmit data processed by the graphics processor to a storage device via a switch; and / or to transmit data stored in the storage device to the graphics processor via a switch.

[0114] In one embodiment, the data transmission module is used to generate a check code, and send the check code and the data to be transmitted to the storage device through a switch, so that the storage device performs cyclic redundancy check based on the check code; or, it generates a check code, and sends the check code and the data to the graphics processor through a switch, so that the graphics processor performs cyclic redundancy check based on the check code.

[0115] In one embodiment, the data processor further includes a key-value processing engine, and the apparatus further includes a key-value processing module for using the key-value processing engine to perform serialization or deserialization processing of key-value cached data.

[0116] In one embodiment, the apparatus further includes a key-value cache data writing module, configured to write serialized or deserialized key-value cache data to a graphics processor or storage device using a remote direct memory access engine.

[0117] In one embodiment, the apparatus further includes: The channel resource configuration information determination module is used to obtain device information, which includes information about the devices connected to the upstream and downstream interfaces of the switch. Based on the device information, the module determines the channel resource configuration information and sends it to the root port controller, enabling the root port controller to configure the channel resources of the switch according to the channel resource configuration information.

[0118] In one embodiment, the apparatus further includes: The power-on control module is used to activate the root port controller in response to the power-on operation, so that the root port controller controls at least two downstream ports to turn on in sequence according to the preset power-on sequence, thereby enabling the power-on of at least one switch, at least one graphics processor and storage device.

[0119] In one embodiment, the apparatus further includes: The initialization module is used to initialize the network transceiver after the storage device is powered on, so that the network transceiver is in the data receiving state.

[0120] In one embodiment, the apparatus further includes: The virtual resource allocation module is used to receive task requests and, in response to the task requests, allocate corresponding virtual resources for the requested tasks so that the server can use the virtual resources to execute the tasks.

[0121] In one embodiment, the apparatus further includes: The communication connection establishment module is used to establish communication connections with other switches on other servers through the communication module, enabling the graphics processor to interact with other graphics processors connected to the same switch.

[0122] In one embodiment, the apparatus further includes: The model training module is used to train artificial intelligence models; The activation value writing module is used to write intermediate values ​​calculated by the artificial intelligence model to the cache of the storage device via the exchange. The result writing module is used to write the result data calculated by the artificial intelligence model to the storage area of ​​the storage device through the exchange.

[0123] In one embodiment, the server is located at the first node, and the apparatus further includes: The data interaction module is used to interact with at least one second node via a network switch.

[0124] In one embodiment, the apparatus further includes: The inference task execution module is used to divide the inference task into at least two inference subtasks when it receives an inference task, and send the at least two inference subtasks to the corresponding nodes in the first node and at least one second node, so that the nodes execute the inference subtasks.

[0125] In one embodiment, the apparatus further includes: The target data storage module is used to divide the target data into at least two target sub-data when it receives target data, and send the at least two target sub-data to the corresponding nodes of the first node and at least one second node, so that the nodes store the target sub-data.

[0126] For a description of the features in the embodiment corresponding to the data transmission device, please refer to the relevant description in the embodiment corresponding to the data transmission method, which will not be repeated here.

[0127] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described data transmission method embodiments.

[0128] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data transmission method embodiments when it is run.

[0129] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0130] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data transmission method embodiments.

[0131] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data transmission method embodiments.

[0132] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be performed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0133] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0134] The foregoing has provided a detailed description of a server, server node, and node interconnection system provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A server, characterized in that, include: A data processor, at least one switch, a storage device, and at least one graphics processor; The switch includes an upstream port and at least two downstream ports; wherein the upstream port is connected to the data processor, at least one downstream port is connected to the storage device, and at least one downstream port is connected to the at least one graphics processor; The data processor includes a network data transceiver, which is equipped with a remote direct memory access engine. The network data transceiver is used to receive network data and transmit the network data to the storage device or the at least one graphics processor through the switch via the remote direct memory access engine. The at least one graphics processor and the storage device transmit data through the switch.

2. The server according to claim 1, characterized in that, The switch is equipped with a point-to-point direct memory access engine, which is used to enable direct data transfer between the video memory of the at least one graphics processor and the memory of the storage device.

3. The server according to claim 1, characterized in that, The graphics processor is also used to generate a check code, and send the check code and the data to be transmitted to the storage device through the switch, so that the storage device performs cyclic redundancy check based on the check code. Alternatively, the storage device is also used to generate a checksum, and send the checksum and data to the graphics processor through the switch, so that the graphics processor performs cyclic redundancy check based on the checksum.

4. The server according to claim 1, characterized in that, The data processor also includes a key-value processing engine, which is used to perform serialization or deserialization processing on key-value cached data.

5. The server according to claim 4, characterized in that, The remote direct memory access engine is also used to write the serialized or deserialized key-value cache data to the graphics processor or the storage device.

6. The server according to claim 1, characterized in that, The data processor further includes a central processing unit and a root port controller. The central processing unit is used to acquire device information, which is the information of the devices connected to the upstream and downstream interfaces of the switch. Based on the device information, the central processing unit determines channel resource configuration information and sends it to the root port controller, so that the root port controller configures the channel resources of the switch according to the channel resource configuration information.

7. The server according to claim 6, characterized in that, The central processing unit is also configured to activate the root port controller in response to a power-on operation, so that the root port controller sequentially controls the opening of at least two downstream ports according to a preset power-on sequence, thereby enabling the power-on of at least one switch, at least one graphics processor and the storage device.

8. The server according to claim 7, characterized in that, The central processing unit is also used to initialize the network transceiver after the storage device is powered on, so that the network transceiver is in a data receiving state.

9. The server according to claim 1, characterized in that, The data processor further includes a central processing unit, and the network data transceiver is also used to receive task requests. The central processing unit is used to respond to the task request and allocate corresponding virtual resources for the task in the task request, so that the server can use the virtual resources to execute the task.

10. The server according to claim 1, characterized in that, The switch also includes a communication module, through which the server communicates with the communication modules of other switches of other servers, enabling the graphics processor to interact with the graphics processors connected to the other switches.

11. The server according to claim 10, characterized in that, The server and the other servers are located on the same node.

12. The server according to claim 1, characterized in that, The server is used to train the artificial intelligence model, and the graphics processor is used to write intermediate values ​​calculated by the artificial intelligence model into the cache area of ​​the storage device through the switch; the graphics processor is also used to write the result data calculated by the artificial intelligence model into the storage area of ​​the storage device through the switch.

13. The server according to claim 1, characterized in that, The server is located at a first node, which is connected to at least one second node via a network switch. The second node has at least one server, and the first node and the at least one second node interact with each other via the network switch.

14. The server according to claim 13, characterized in that, The first node and the at least one second node are connected to the master node through a network switch. When the master node receives an inference task, it divides the inference task into at least two inference subtasks and sends the at least two inference subtasks to the corresponding nodes among the first node and the at least one second node, so that the nodes execute the inference subtasks.

15. The server according to claim 13, characterized in that, The first node and the at least one second node are connected to the master node through a network switch. The master node is used to divide the target data into at least two target sub-data when it receives target data, and send the at least two target sub-data to the corresponding node among the first node and the at least one second node, so that the node stores the target sub-data.

16. A server node, characterized in that, The server node is equipped with at least one server as described in any one of claims 1-15.

17. The server node according to claim 16, characterized in that, When the server node includes at least two servers, the at least two servers communicate with each other through a communication module, enabling the graphics processors of the at least two servers to interact with each other.

18. A node interconnection system, characterized in that, It includes a network switch and at least one server node as described in any one of claims 16-17, wherein the at least one server node is communicatively connected through the network switch.

19. The node interconnection system according to claim 18, characterized in that, The server node includes a master node, a first node, and a second node. The master node is used to divide the inference task into at least two inference subtasks when it receives an inference task, and send the at least two inference subtasks to the corresponding node of the first node or the at least one second node, so that the node executes the inference subtask.

20. The node interconnection system according to claim 18, characterized in that, The server node includes a master node, a first node, and a second node. The master node is used to divide the target data into at least two target sub-data when it receives the target data, and send the at least two target sub-data to the corresponding node among the first node and the at least one second node, so that the node stores the target sub-data.

Images