A GPU network interconnection method and device
By presetting forwarding tables and data caches in the SRAM of the GPU, the connection and data transmission between GPUs are directly realized in hardware, which solves the problem that traditional data center networks cannot meet the needs of large-scale GPU interconnection and realizes efficient GPU data transmission.
Patent Information
- Application Number
- CN202411903948.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-23
- Publication Date
- 2025-04-04
AI Technical Summary
Traditional data center networks cannot meet the needs of large-scale GPU interconnection in terms of bandwidth and latency, especially in deep learning and large-scale training scenarios, efficient communication and synchronization between GPUs become bottlenecks.
By presetting forwarding tables and data caches in the GPU's static random access memory SRAM, connection management tasks are generated, connection and data transmission between GPUs are realized directly in hardware, connection protocols and packet formats are simplified, and packets are used in Ethernet format.
It greatly reduces the delay of data transmission between GPUs, improves data transmission efficiency, and can meet the bandwidth and delay requirements of large-scale GPU interconnection.
Smart Images
Figure CN119759839A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a method and device for GPU network interconnection. Background Art
[0002] Traditional data center networks are mainly used for interconnection between servers in a data center and between servers and storage. With the widespread application of AI and large models, data centers need a large number of GPUs (Graphics Processing Units) or TPUs (Tensor Processing Units) for distributed training and deep learning of AI, and the demand for GPUs and TPUs is also increasing. For some intelligent computing centers, the number of GPUs even exceeds the number of CPUs (Central Processing Units). At the same time, for some servers used for deep learning, the number of GPUs on a server is far greater than the number of CPUs. In this case, the interconnection of GPUs has also become a crucial issue.
[0003] Traditional GPU interconnection usually uses PCI E for interconnection when the number of GPUs is small and the computing power is not strong. However, with the increase in the amount of data processed by a single GPU and the increase in the number of GPUs in a cluster, PCI E technology is far from meeting the interconnection needs of GPUs.
[0004] At present, most mainstream GPU manufacturers in the industry use their own technologies to achieve GPU interconnection. For example, Nvidia uses private technology NVLink to interconnect GPUs, and the GPU interconnection bandwidth can reach hundreds of Gb, while AMD also uses its own Infinity Fabric technology. Of course, there are also standard technologies in the industry, such as RoCEv2 based on Ethernet, but for scenarios such as tens of thousands of GPU / TPU interconnection, this technology is difficult to meet the requirements of large models or deep learning scenarios in terms of latency and bandwidth.
[0005] Compared with traditional CPU communication, communication between GPUs requires much more data volume because there are many samples and training parameters to be transmitted. For models with hundreds of millions of nodes, the scale of parameters that need to be transmitted instantly reaches billions, that is, hundreds of GB. In addition, this type of communication has the characteristics of collective communication and requires data transmission in the form of broadcast, diffusion, many-to-one or one-to-many. Therefore, it has high requirements for network bandwidth. At the same time, in order to improve the efficiency of large model training, all GPUs need to synchronize parameters and data, otherwise the efficiency of model learning will be greatly reduced. Therefore, GPU communication also has extremely stringent requirements for transmission latency, or the lower the better.
[0006] The traditional data center server GPU data transmission method has the following problems:
[0007] 1. Traditional server data transmission is based on a complex TCP / IP software protocol stack, in which packets are encapsulated, including TCP, IP addresses, Ethernet and other packet headers. Even when RDMA technology is used, the remote CPU is not involved, but TCP, IP and other packet headers still need to be encapsulated and decapsulated. This encapsulation and decapsulation as well as multiple layers of protocol headers increase the delay of sending and receiving, and reduce the efficiency of transmission. For the scenario of data transmission between GPUs, when the requirements for delay and efficiency are extremely stringent, the method based on the traditional protocol stack cannot meet the requirements in terms of delay and efficiency.
[0008] 2. Traditional data center senders and receivers mostly establish connections based on TCP connections for data transmission. TCP requires three handshakes to establish a connection, which takes a long time and is inefficient. In addition, TCP's sliding window technology is based on software implementation, which is inefficient and has a large latency. For GPU interconnection scenarios, the establishment and maintenance of TCP connections greatly affect the efficiency of GPU data transmission.
[0009] Therefore, the traditional data center network cannot meet the needs of GPU interconnection in terms of bandwidth and latency. At the same time, as the number of GPU cards mounted on a server becomes increasingly dense, the traditional data center network is increasingly unable to meet the needs of large-scale GPU interconnection in terms of networking architecture and transmission protocols. Summary of the invention
[0010] The embodiments of the present application provide a method and device for GPU network interconnection, which can improve data transmission efficiency and reduce transmission delay.
[0011] In a first aspect, a method for interconnecting a GPU network is provided, comprising:
[0012] Generate a connection management task according to the forwarding table of the source GPU and the address of the destination GPU, wherein the forwarding table includes the address of each destination GPU and the number of the corresponding network output interface, the connection management task includes the destination GPU information connected thereto, the destination GPU information includes the number of the network output interface, the IP address and the data cache, and the forwarding table and the data cache are preset in a static random access memory SRAM;
[0013] According to the connection management task, a connection request message is sent to the destination GPU, and a timer is started to record a timestamp when the connection request message is sent;
[0014] If a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time, the connection is established successfully;
[0015] Otherwise, the connection establishment fails and the connection request message is resent.
[0016] In some embodiments, the connection request message and the connection response message both include: the address of the destination GPU, the address of the source GPU, the type field, the task number, the data packet type, the data packet sequence number, the data packet content, and the check code;
[0017] The address of the destination GPU of the connection request message is the same as the address of the source GPU of the connection response message, and the address of the source GPU of the connection request message is the same as the address of the destination GPU of the connection response message.
[0018] In some embodiments, obtaining a verification result after verifying the address, type field and data packet type of the destination GPU in the connection request message via the destination GPU;
[0019] According to the obtained verification result, a connection response message having a data packet type corresponding to the verification result is sent.
[0020] In some embodiments, when the connection is successfully established, the method further comprises: using the timer to calculate a round trip delay RTT between sending the connection request message and receiving the connection response message;
[0021] Furthermore, the method further includes: obtaining the size of a sending window for sending a service data packet based on the round-trip delay RTT and the sending rate of the network output interface of the destination GPU information.
[0022] In some embodiments, the method further comprises: sending a service data packet;
[0023] Sending a service data packet specifically includes:
[0024] Based on the size of the sending window, the service data of the HBM memory of the source GPU is copied in batches to the data cache corresponding to the network output interface of the destination GPU information through the DMA engine, and encapsulated into a number of service data packets;
[0025] According to the connection management task, a service data packet is sent to the destination GPU, and a data response message sent by the destination GPU based on the service data packet is received.
[0026] In some embodiments, sending a data response message specifically includes: sending a data response message every 1 / 2 of the round trip time RTT, and the data packet sequence number of the data response message is the largest data packet sequence number among the currently verified service data packets.
[0027] In some embodiments, if the data response message is not received within 1 / 2 of the round-trip time RTT, the sending rate is reduced.
[0028] In some embodiments, the service data packet and the data response message both include: the address of the destination GPU, the address of the source GPU, the type field, the task number, the data packet type, the data packet sequence number, the data packet content, and the checksum;
[0029] The address of the destination GPU of the service data packet is the same as the address of the source GPU of the data response message, and the address of the source GPU of the service data packet is the same as the address of the destination GPU of the data response message.
[0030] In some embodiments, obtaining a verification result after the destination GPU verifies the address, task number and check code in the service data packet via the destination GPU;
[0031] Determine whether to send a data response message based on the obtained verification result.
[0032] In a second aspect, a device for interconnecting a GPU network is provided, which is applied to interconnecting at least two GPUs, and at least one of the GPUs is used as a source GPU, and at least another GPU is used as a destination GPU, and includes:
[0033] A GPU network management module, which is used to: generate a connection management task according to a forwarding table of a source GPU and an address of a destination GPU; according to the connection management task, send a connection request message to the destination GPU, start a timer, and record a timestamp when the connection request message is sent; if a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time, the connection is successfully established; otherwise, the connection establishment fails, and the connection request message is resent; when the connection is successfully established, calculate a round-trip delay RTT from sending the connection request message to receiving the connection response message, and based on the round-trip delay RTT and the sending rate of the network output interface of the destination GPU information, obtain the size of a sending window for sending a service data packet; wherein the forwarding table includes the address of each destination GPU and the number of the corresponding network output interface, the connection management task includes the destination GPU information connected thereto, the destination GPU information includes the number of the network output interface, the IP and the data cache, and the forwarding table and the data cache are preset in a static random access memory SRAM;
[0034] A DMA engine is used to: based on the size of the sending window, copy the service data of the HBM memory of the source GPU in batches to the data cache corresponding to the network output interface of the destination GPU information, and encapsulate them into a number of service data packets;
[0035] Furthermore, the GPU network management module is further used to: send a service data packet to the destination GPU according to the connection management task, and receive a data response message sent by the destination GPU based on the service data packet.
[0036] The beneficial effects of the technical solution provided by this application include:
[0037] The present application pre-sets a forwarding table and a data cache in a static random access memory SRAM, generates a connection management task through the forwarding table of the source GPU and the address of the destination GPU, and uses the connection management task to send a connection request message to establish a connection and subsequent data transmission. The present application can be directly implemented in the SRAM hardware of the GPU without the participation of the protocol stack and the CPU, which greatly reduces the latency of data transmission between GPUs and improves data transmission efficiency.
[0038] This application simplifies the connection protocol and data packet format, avoids the use of complex TCP connections and TCP / IP packet headers between GPUs, and uses Ethernet format for messages or data packets, which is compatible with existing Ethernet networks. At the same time, a lot of simplifications are made, making the processing at both ends of the GPU simple and efficient, and can be used for large-scale GPU interconnection. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0040] Figure 1 A flowchart of a method for interconnecting GPU networks provided in an embodiment of the present application;
[0041] Figure 2 A block diagram of a device for interconnecting GPU networks provided in an embodiment of the present application;
[0042] Figure 3 The data packet format provided in the embodiment of the present application. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.
[0044] The embodiment of the present application provides a method for interconnecting GPU networks, which is applied to the interconnection of at least two GPUs, wherein at least one of the GPUs is used as a source GPU, and at least another GPU is used as a destination GPU, wherein the GPUs each include a forwarding table and a network output interface, and in order to simultaneously transmit data to multiple GPUs, each GPU has multiple network output interfaces to interconnect with surrounding GPUs, and in order to distinguish each network output interface, each network output interface has its own unique number and IP address, and each network output interface is configured with a data cache of a certain capacity, such as Figure 2 As shown in the figure, each GPU is configured with 6 network output interfaces, and this number can be adjusted according to specific circumstances.
[0045] Each network output interface is configured with an independent data cache, so that the GPU can send data from all network output interfaces at the same time. The GPU uses a multi-tasking method to transmit data with other GPUs. Each data transmission task is responsible for the entire life cycle of a connection session. The connection occupies a network output interface and its data cache. The data transmission task starts when the connection is established and ends when the connection ends. Each task maintains a connection separately, sends and receives data separately, and has no relationship with other connection tasks. This increases security and reduces the additional system complexity caused by network sharing and congestion control.
[0046] The GPU forwarding table is a two-dimensional table structure stored in the GPU's static random access memory SRAM, which is composed of the address of the destination GPU and the number of the corresponding network output interface. The address of the GPU can be the chip number. In this application, in order to be compatible with Ethernet, each GPU is configured with a unique MAC address. The setting of the forwarding table is set in advance by software programming. During the execution of tasks by the GPU cluster, the topological structure of the GPU will not change, so the forwarding table remains basically unchanged.
[0047] Specifically, see Figure 1 As shown, the method comprises the following steps:
[0048] 101: Generate a connection management task according to a forwarding table of a source GPU and an address of a destination GPU, wherein the forwarding table includes the address of each destination GPU and the number of the corresponding network output interface, the connection management task includes information of a destination GPU connected thereto, the destination GPU information includes the number of the network output interface, the IP address and the data cache, and the forwarding table and the data cache are preset in a static random access memory SRAM.
[0049] For two devices in the network to transmit data, they must first establish a connection. Similarly, for two GPUs to send data, they must first establish a connection. The GPU network management module is responsible for establishing the connection, which first generates a connection management task, which is responsible for establishing and maintaining the connection.
[0050] Before establishing a connection, the GPU network management module will first generate a connection management task to maintain and manage the connection. This application uses multiple connection management tasks to manage connections and network output interfaces. In this application, a connection management task is responsible for the establishment and management of a connection, and a connection has a network output interface to transmit data. Therefore, if there are multiple connections and multiple network output interfaces to transmit data, the GPU network management module will generate multiple connection management tasks.
[0051] The GPU network management module will use the corresponding network output interface found in the forwarding table of the source GPU as the interface for data transmission based on the address of the destination GPU provided by the source GPU, and clear the data cache corresponding to the network output interface. The generated connection management task will maintain the binding relationship between this connection (the connection has a task number), the network output interface, and the data cache until the connection is terminated. The connection management task is also responsible for generating the sequence number of the sent data packet and recording the received response sequence number to determine whether there is packet loss or other phenomena. The connection management task will also maintain a fixed-size data cache window through hardware (i.e., SRAM) to determine whether congestion occurs during the data transmission process, and maintain a timer that records the transmission delay of the data packet through hardware.
[0052] Each GPU maintains a global task number; this number is generated when a connection management task is generated and is recycled when the task ends.
[0053] When the task is ready, it starts to establish a transmission connection with the destination GPU.
[0054] 102: According to the connection management task, send a connection request message to the destination GPU, and start a timer to record a timestamp when the connection request message is sent.
[0055] First, the connection management task generates a connection request message. It should be noted that the message and data packet formats used in this application are as follows: Figure 3 As shown, in order to be compatible with Ethernet, this application adopts the same packet header format as Ethernet, but the difference from the data packet sent by the traditional CPU is that this application does not have the traditional IP layer and TCP layer, and the packet format of IP and TCP is not adopted in the data packet part, but is simplified. The header part is compatible with the Ethernet format in order to allow the data packet to be forwarded on the existing Ethernet switching device. The message or data packet format in this application is specifically composed of the following parts:
[0056] Destination address: 6 bytes, the address of the destination GPU; compatible with the MAC address, giving each GPU a unique MAC address.
[0057] Source address: 6 bytes, address of the source GPU; compatible with the MAC address.
[0058] Type field: 2 bytes. This application selects 0x0877 as the type field. This type is used to distinguish it from existing protocols in Ethernet.
[0059] Task number: 2 bytes, representing the connection number.
[0060] Data packet type: 1 byte, indicating the type of data message carried in the data packet or message, mainly including connection establishment message, connection response message, business data message, connection end message, end response message and other messages.
[0061] 0x1, representing a connection establishment message;
[0062] 0x2: represents the connection response message;
[0063] 0x4: represents business data message;
[0064] 0x8: represents the connection end message;
[0065] 0x10: End response message indicating the end of the connection;
[0066] Only when the data packet type is 0x4 (representing a business data message), the data packet sequence number and data packet content have data, otherwise there is no data.
[0067] Data packet sequence number: 4 bytes, indicating the sequence number of the service data packet in the connection.
[0068] Check code FCS: 4 bytes, is the check of the data packet part, using the CRC-32 check algorithm.
[0069] It is understandable that the address of the destination GPU of the connection request message is the same as the address of the source GPU of the connection response message, and the address of the source GPU of the connection request message is the same as the address of the destination GPU of the connection response message.
[0070] The destination address of the connection request message is the address of the opposite GPU of the data transmission, the source address is the address of the GPU, and the type field is 0x0877. The task number is generated by the connection management task, and the task number is globally unique in the GPU, so the GPU task number can be used as the number of the connection. The data packet type of the connection request message is 0x1, indicating that the connection is established. At the same time, when the connection management task sends the connection request message, it starts a timer to record the timestamp of sending the connection request message.
[0071] While sending the connection request message, the source GPU starts a timer to record the timestamp when the connection request message is sent.
[0072] 103: Determine whether a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time.
[0073] 104: If a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time, it means that the connection is established successfully.
[0074] 105: If the connection response message sent by the destination GPU based on the connection request message is not received within the predetermined time, a timeout occurs, indicating that the connection establishment fails, and the process returns to step 102 to resend the connection request message.
[0075] The present application pre-sets a forwarding table and a data cache in a static random access memory SRAM, generates a connection management task through the forwarding table of the source GPU and the address of the destination GPU, and uses the connection management task to send a connection request message to establish a connection and subsequent data transmission. The present application can be directly implemented in the SRAM hardware of the GPU without the participation of the protocol stack and the CPU, which greatly reduces the latency of data transmission between GPUs and improves data transmission efficiency.
[0076] This application simplifies the connection protocol and data packet format, avoids the use of complex TCP connections and TCP / IP packet headers between GPUs, and uses Ethernet format for messages or data packets, which is compatible with existing Ethernet networks. At the same time, a lot of simplifications are made, making the processing at both ends of the GPU simple and efficient, and can be used for large-scale GPU interconnection.
[0077] In the above step 102, the destination GPU sends a connection response message based on the connection request message, which specifically includes the following steps:
[0078] 201: Obtain a verification result after the destination GPU verifies the address, type field, and data packet type of the destination GPU in the connection request message.
[0079] 202: According to the obtained verification result, a connection response message having a data packet type corresponding to the verification result is sent.
[0080] It is understandable that after the destination GPU receives the connection request message:
[0081] The first step is to determine the destination address to determine whether the destination address of the connection request message is the address of its own GPU. If not, the connection request message is discarded, the destination GPU will not send a connection response message, and the source GPU will not be able to receive the connection response message within the scheduled time, so it will time out, and naturally know that the connection establishment failed; if so, proceed to the next step.
[0082] Step 2: If the destination address is the address of your own GPU, learn and verify the outbound interface IP.
[0083] The third step is to check whether the type field is 0x0877; if it is, proceed to the next step; if not, discard the connection request message, the destination GPU will not send a connection response message, and the source GPU will not be able to receive the connection response message within the scheduled time, so it will time out, and naturally know that the connection establishment failed.
[0084] The fourth step is to detect the data packet type. If the data packet type is not 0x1, the connection request message is discarded, the destination GPU will not send a connection response message, and the source GPU will not be able to receive the connection response message within the scheduled time, so it will time out and naturally know that the connection establishment has failed. If the data packet type is 0x1, it means that the connection request message is a connection establishment request, which means that the verification has passed. The verification result is that the connection is successfully established. Corresponding to the verification result, the destination GPU generates a connection response message with a data packet type of 0x2. While sending the connection response message, prepare to start receiving service data packets.
[0085] After the source GPU receives the connection response message, it means that the connection is established successfully and the service data can be sent normally. The connection during the service data packet sending is unchanged, and the task number is also unchanged.
[0086] At the same time, when the connection is successfully established, the timer is used to calculate the round trip delay RTT from sending the connection request message to receiving the connection response message. This time is used to estimate the delay of the entire link from the sender to the receiver.
[0087] Based on the round-trip time RTT and the sending rate of the network output interface of the destination GPU information, the size of the sending window for sending the service data packet can be calculated.
[0088] It is understandable that the above-mentioned predetermined time can be reasonably set artificially according to actual needs. For example, as an example, the above-mentioned predetermined time can be set to 2 times of the round-trip time RTT.
[0089] After the connection is successfully established in the above step 104, a service data packet may be sent; wherein sending the service data packet specifically includes the following steps:
[0090] 301: Based on the size of the sending window, the service data of the HBM memory of the source GPU is copied in batches to the data cache corresponding to the network output interface of the destination GPU information through the DMA engine, and encapsulated into a plurality of service data packets.
[0091] In step 301, the source GPU sends an instruction to the DMA engine to start sending service data. The specific process is as follows:
[0092] The DMA engine copies the service data in the HBM memory to the SRAM, encapsulates it into several service data packets, and then starts to send the service data packets. After the SRAM starts to send the service data packets, it will start a sending window to record the amount of data sent, and at the same time number the sent service data packets. The sent service data packets will contain the data packet number. The fixed window used in this method is easy to implement in hardware, and the window size is determined according to the sending rate and round-trip delay RTT of the network output interface sending the data.
[0093] Send window size = round-trip time RTT × interface rate.
[0094] When sending a service data packet, the data packet sequence number field will be filled with the data packet sequence number of each service data packet sent. The sequence number starts from 0, the sequence number occupies 4 bytes, and the maximum sequence number can be 2 32 -1.
[0095] 302: According to the connection management task, send a service data packet to the destination GPU, and receive a data response message sent by the destination GPU based on the service data packet; the address of the destination GPU of the service data packet is the same as the address of the source GPU of the data response message, and the address of the source GPU of the service data packet is the same as the address of the destination GPU of the data response message.
[0096] The DMA engine will continuously copy data to the send window in the SRAM until the window is full or until the send is completed. At the same time, the window will continuously move forward according to the sequence number of the corresponding message received, and the data with the received response sequence number will be deleted.
[0097] Since the size of the sending window is equal to the round-trip delay RTT × interface rate, that is, in the absence of congestion, the data in the sending window will be sent within a round-trip delay RTT cycle. Therefore, under normal circumstances, the window will keep moving.
[0098] After receiving the service data packet, the destination GPU checks and verifies whether the address, task number and check code of the destination GPU are all correct to obtain the verification result. If the verification result is that they are all correct, a data response message is sent to the source GPU, otherwise no data response message is sent. In order to avoid sending data response messages too frequently, the present application adopts the method of sending a data response message every 1 / 2 of the round-trip delay RTT. The data packet sequence number of the data response message is the largest data packet sequence number among the currently verified service data packets, and the service data packets before this sequence number have been received.
[0099] When the data response message of this sequence number is sent to the source GPU, the source GPU will delete the data before this sequence number and then move the sending window forward.
[0100] If the data response message is not received within 1 / 2 of the round-trip delay RTT, the source GPU considers that congestion has occurred and starts to reduce the sending rate. At this time, the movement of the sending window will also slow down. If the data response message is still not received, there may be severe congestion or packet loss. At this time, the sending rate continues to be reduced, and some messages are resent, and the messages are resent from the beginning of the sending window.
[0101] See also Figure 2 As shown, an embodiment of the present application also provides a GPU network interconnection device, which is applied to the interconnection of at least two GPUs, and at least one of the GPUs is used as a source GPU, and at least another GPU is used as a destination GPU, which includes a GPU network management module and a DMA engine.
[0102] The GPU network management module is used to: generate a connection management task according to the forwarding table of the source GPU and the address of the destination GPU; according to the connection management task, send a connection request message to the destination GPU, start a timer, and record the timestamp when the connection request message is sent; if a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time, the connection is successfully established; otherwise, the connection establishment fails, and the connection request message is resent; when the connection is successfully established, calculate the round-trip delay RTT from sending the connection request message to receiving the connection response message, and obtain the size of the sending window for sending the service data packet based on the round-trip delay RTT and the sending rate of the network output interface of the destination GPU information; wherein the forwarding table includes the address of each destination GPU and the number of the corresponding network output interface, the connection management task includes the destination GPU information connected thereto, and the destination GPU information includes the number of the network output interface, IP, and data cache.
[0103] The DMA engine is used to: based on the size of the sending window, copy the service data of the HBM memory of the source GPU in batches to the data cache corresponding to the network output interface of the destination GPU information, and encapsulate them into several service data packets.
[0104] The GPU network management module is further used to: send a service data packet to the destination GPU according to the connection management task, and receive a data response message sent by the destination GPU based on the service data packet.
[0105] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.
Claims
1. A method for interconnecting GPU networks, characterized in that: It includes: Generate a connection management task according to the forwarding table of the source GPU and the address of the destination GPU, wherein the forwarding table includes the address of each destination GPU and the number of the corresponding network output interface, the connection management task includes the destination GPU information connected thereto, the destination GPU information includes the number of the network output interface, the IP address and the data cache, and the forwarding table and the data cache are preset in a static random access memory SRAM; According to the connection management task, a connection request message is sent to the destination GPU, and a timer is started to record a timestamp when the connection request message is sent; If a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time, the connection is established successfully; Otherwise, the connection establishment fails and the connection request message is resent.
2. The method for interconnecting GPU networks according to claim 1, wherein: The connection request message and the connection response message both include: the address of the destination GPU, the address of the source GPU, the type field, the task number, the data packet type, the data packet sequence number, the data packet content, and the check code; The address of the destination GPU of the connection request message is the same as the address of the source GPU of the connection response message, and the address of the source GPU of the connection request message is the same as the address of the destination GPU of the connection response message.
3. The method for interconnecting GPU networks as claimed in claim 2, characterized in that: Obtaining a verification result after the destination GPU verifies the address, type field, and data packet type of the destination GPU in the connection request message via the destination GPU; According to the obtained verification result, a connection response message having a data packet type corresponding to the verification result is sent.
4. The method for interconnecting GPU networks according to claim 1, wherein: When the connection is successfully established, the method further comprises: using the timer to calculate a round trip delay RTT from sending the connection request message to receiving the connection response message; Furthermore, the method further includes: obtaining the size of a sending window for sending a service data packet based on the round-trip delay RTT and the sending rate of the network output interface of the destination GPU information.
5. The method for interconnecting GPU networks as claimed in claim 4, characterized in that: The method further comprises: sending a service data packet; Sending a service data packet specifically includes: Based on the size of the sending window, the service data of the HBM memory of the source GPU is copied in batches to the data cache corresponding to the network output interface of the destination GPU information through the DMA engine, and encapsulated into a number of service data packets; According to the connection management task, a service data packet is sent to the destination GPU, and a data response message sent by the destination GPU based on the service data packet is received.
6. The method for interconnecting GPU networks as claimed in claim 5, characterized in that: Sending a data response message specifically includes: sending a data response message every 1 / 2 of the round trip time RTT, and the data packet sequence number of the data response message is the largest data packet sequence number among the currently verified service data packets.
7. The method for interconnecting GPU networks as claimed in claim 6, characterized in that: If the data response message is not received within 1 / 2 of the round-trip time RTT, the sending rate is reduced.
8. The method for interconnecting GPU networks as claimed in claim 5, characterized in that: The service data packet and the data response message both include: the address of the destination GPU, the address of the source GPU, the type field, the task number, the data packet type, the data packet sequence number, the data packet content, and the check code; The address of the destination GPU of the service data packet is the same as the address of the source GPU of the data response message, and the address of the source GPU of the service data packet is the same as the address of the destination GPU of the data response message.
9. The method for interconnecting GPU networks as claimed in claim 8, characterized in that: Obtaining a verification result after the destination GPU verifies the address, task number and check code in the service data packet through the destination GPU; Determine whether to send a data response message based on the obtained verification result.
10. A GPU network interconnection device, which is applied to the interconnection of at least two GPUs, wherein at least one of the GPUs is used as a source GPU and at least another GPU is used as a destination GPU, characterized in that: It includes: A GPU network management module, which is used to: generate a connection management task according to the forwarding table of the source GPU and the address of the destination GPU, send a connection request message to the destination GPU according to the connection management task, and start a timer to record a timestamp when the connection request message is sent; If a connection response message sent by the destination GPU based on the connection request message is received within a predetermined time, the connection is successfully established; otherwise, the connection fails to be established and the connection request message is resent; When the connection is successfully established, the round-trip delay RTT from sending the connection request message to receiving the connection response message is calculated, and based on the round-trip delay RTT and the sending rate of the network output interface of the destination GPU information, the size of the sending window for sending the service data packet is obtained; wherein the forwarding table includes the address of each destination GPU and the number of the corresponding network output interface, the connection management task includes the destination GPU information connected thereto, the destination GPU information includes the number of the network output interface, the IP and the data cache, and the forwarding table and the data cache are preset in a static random access memory SRAM; A DMA engine is used to: based on the size of the sending window, copy the service data of the HBM memory of the source GPU in batches to the data cache corresponding to the network output interface of the destination GPU information, and encapsulate them into a number of service data packets; Furthermore, the GPU network management module is further used to: send a service data packet to the destination GPU according to the connection management task, and receive a data response message sent by the destination GPU based on the service data packet.