Data transmission method and apparatus, and computer, storage medium and program product

Abstract

Disclosed in the present application are a data transmission method and apparatus, and a computer, a storage medium and a program product. The method comprises: in response to a data sending request, acquiring, from the data sending request, first service data, a first graphics processing unit that sends the first service data, and a second graphics processing unit that receives the first service data, wherein the first graphics processing unit is a graphics processing unit in a first service device; determining, from among S queue pairs associated with the first graphics processing unit, a first queue pair for sending the first service data, wherein S is a positive integer; and acquiring a first hardware port corresponding to the first queue pair, and by means of the first hardware port and the first queue pair, sending the first service data to the second graphics processing unit in a second service device. By means of the present application, the efficiency of data transmission can be improved, and the hardware utilization rate and communication performance of a service device can be enhanced.

Description

Data transmission methods, devices, computers, storage media and software products

[0001] This application claims priority to Chinese Patent Application No. 202510470479.3, filed on April 14, 2025, entitled “Data Transmission Method, Apparatus, Computer, Storage Medium and Program Product”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to a data transmission method, apparatus, computer, storage medium, and program product. Background Technology

[0003] The High-Performance Communication Library (HPL) is a communication library designed for hybrid expert systems. It optimizes GPU data processing and supports low-precision arithmetic to achieve large-scale processing tasks. The HPL's inter-machine (i.e., between service devices) communication method is implemented based on the GPU Shared Memory (SHMEM) communication library and utilizes the InfiniBand Reliable Connection (IBRC) method provided by SHMEM. However, this approach suffers from drawbacks. Since IBRC only supports single queue pairs (QPs), in dual-port network architectures, both ports cannot be fully utilized, leading to reduced data transmission efficiency and lower hardware utilization and communication performance of service devices. Summary of the Invention

[0004] This application provides a data transmission method, apparatus, computer, storage medium, and program product, which can improve data transmission efficiency and enhance the hardware utilization and communication performance of service equipment.

[0005] One embodiment of this application provides a data transmission method, which is executed by a first service device, and the method includes:

[0006] In response to a data transmission request, a first graphics processing unit obtains first service data from the data transmission request, transmits the first service data, and a second graphics processing unit receives the first service data; the first graphics processing unit is a graphics processing unit in the first service device.

[0007] From the S queue pairs associated with the first graphics processing unit, determine the first queue pair used to send the first service data; S is a positive integer;

[0008] Obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

[0009] One embodiment of this application provides a data transmission apparatus, which is adapted to a first service device, and the apparatus includes:

[0010] The data acquisition module is used to respond to a data transmission request, acquire first service data from the data transmission request, send the first service data to a first graphics processing unit, and receive the first service data to a second graphics processing unit; the first graphics processing unit is a graphics processing unit in the first service device.

[0011] The queue pair determination module is used to determine the first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit; S is a positive integer.

[0012] The data transmission module is used to obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

[0013] The device also includes:

[0014] The channel construction module is used to construct D first data channels through the first graphics processing unit; D is an integer multiple of S;

[0015] The data association module is used to associate the D first data channels with S queue pairs based on the channel numbers corresponding to the D first data channels and S respectively; each first data channel is associated with a queue pair, and each queue pair is associated with one or more first data channels;

[0016] This queue, for a specific module, can be used for:

[0017] The first graphics processing unit selects a first target data channel from the D first data channels based on the channel operation status of the D first data channels, and determines the queue pair associated with the first target data channel as the first queue pair for sending the first service data.

[0018] The queue pair determination module can be used for:

[0019] Based on the data volume of the first business data and S, determine the number of first channels and construct H second data channels; H is an integer multiple of S, and H is the number of first channels;

[0020] The first business data is split into H business sub-data, and a second data channel is allocated to each business sub-data.

[0021] Based on the channel number of the second data channel corresponding to each service sub-data, a first queue pair for sending that service sub-data is determined.

[0022] Specifically, when acquiring the first hardware port corresponding to the first queue pair, the data sending module can be used for:

[0023] Obtain the network topology corresponding to the first service device, and the first queue identifier of the first queue pair;

[0024] The network topology and the first queue identifier are hashed to obtain the target queue port number of the first queue pair.

[0025] Obtain the P hardware ports associated with the first graphics processing unit. Based on the queue port parameters corresponding to the P hardware ports, determine the hardware port corresponding to the queue port parameter that matches the target queue port number as the first hardware port corresponding to the first queue pair; P is a positive integer.

[0026] The device also includes:

[0027] The parameter acquisition module is used to obtain the queue port numbers corresponding to S queue pairs and to obtain A candidate classification parameters; A is a positive integer.

[0028] The port clustering module is used to cluster the S queue port numbers based on A candidate classification parameters to obtain P port number clusters corresponding to the A candidate classification parameters.

[0029] The difference detection module is used to determine the degree of classification difference corresponding to each candidate classification parameter based on the number of queue port numbers included in the P port number clusters corresponding to each candidate classification parameter; the degree of classification difference is used to represent the difference between the number of P port numbers of the corresponding candidate classification parameter.

[0030] The parameter determination module is used to determine the candidate classification parameter with the smallest degree of classification difference as the target classification parameter, and to assign queue port parameters to P hardware ports based on the target classification parameter.

[0031] The first service device includes M hardware devices, where M is a positive integer;

[0032] When sending the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair, the data sending module can be used to:

[0033] Create a work queue element for the first business data and add the work queue element to the first queue pair;

[0034] Send a data transmission instruction to the first device hardware associated with the first graphics processing unit, and obtain a work queue element from the first queue pair through the first device hardware and the data transmission instruction; the M device hardwares include the first device hardware;

[0035] The first business data corresponding to the work queue element is sent to the second graphics processing unit in the second service device through the first device hardware.

[0036] Specifically, when adding work queue elements to the first queue pair, this data sending module can be used to:

[0037] Obtain the enqueue position and dequeue position of the first queue pair in the first service device, and determine the number of data to be processed corresponding to the first queue pair in the first service device based on the enqueue position and dequeue position.

[0038] If the amount of data to be processed is less than the queue capacity of the first queue pair, then add the work queue element to the first queue pair;

[0039] If the amount of data to be processed is greater than or equal to the queue capacity of the first queue pair, the work queue elements are cached until the amount of data to be processed is less than the queue capacity of the first queue pair, at which point the work queue elements are added to the first queue pair.

[0040] The device also includes:

[0041] The signaling generation module is used to generate first signaling data for the first service data when the first service data has been sent.

[0042] The signaling sending module is used to determine a second queue pair for sending first signaling data from S queue pairs, and send the first signaling data to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair, so that the second graphics processing unit can obtain the first service data based on the first signaling data and detect the first service data.

[0043] Specifically, when sending the first signaling data to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair, the signaling sending module can be used for:

[0044] Generate a signaling queue element for the first signaling data and add the signaling queue element to the second queue pair;

[0045] When a signaling queue element is obtained from the second queue pair through the second hardware port corresponding to the second queue pair, the queue status of the second queue pair is updated to the locked state, and the first signaling data is sent to the second graphics processing unit; the locked state is used to indicate that the second queue pair cannot be accessed;

[0046] When the first signaling data is successfully sent, the queue status of the second queue pair is updated to running status; the running status indicates that the second queue pair is allowed to access.

[0047] The device also includes:

[0048] The data receiving module is used to receive the second service data sent by the third service device through the third queue and write the second service data into the third queue.

[0049] The information acquisition module is used to, when receiving second signaling data for the second service data, obtain the description parameters for the second service data from the second signaling data, and obtain the actual received parameters corresponding to the description parameters from the second service data;

[0050] The data processing module is used to obtain the second service data from the third queue pair if the described parameters are the same as the actual received parameters.

[0051] The data processing module is also used to send a data repair request to the third service device based on the parameter difference between the description parameters and the actual received parameters if the description parameters are different from the actual received parameters.

[0052] The first service device includes M graphics processing units, each of which includes the first graphics processing unit, where M is a positive integer; the device also includes:

[0053] The unit group generation module is used to obtain the unit number data corresponding to M graphics processing units, and add the M graphics processing units to M unit groups based on the unit number data.

[0054] The queue pair building module is used to build S queue pairs between the graphics processing unit of the first service device in each unit group and the other graphics processing units in the unit group.

[0055] The port mapping module is used to map the S queue pairs corresponding to each unit group to the hardware port of that unit group.

[0056] One embodiment of this application provides a computer device, including a processor, a memory, and an input / output interface;

[0057] The processor is connected to a memory and an input / output interface, respectively. The input / output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device containing the processor executes the data transmission method in one aspect of the embodiments of this application.

[0058] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the data transmission method of one aspect of this application.

[0059] One aspect of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional embodiments of this application. In other words, when the computer instructions are executed by the processor, they implement the methods provided in various optional embodiments of this application.

[0060] Implementing the embodiments of this application will have the following beneficial effects:

[0061] In this embodiment, in response to a data transmission request, first service data, a first graphics processing unit for transmitting the first service data, and a second graphics processing unit for receiving the first service data are obtained from the data transmission request. The first graphics processing unit is a graphics processing unit in a first service device. From S queue pairs associated with the first graphics processing unit, a first queue pair for transmitting the first service data is determined; S is a positive integer. The first hardware port corresponding to the first queue pair is obtained, and the first service data is transmitted to the second graphics processing unit in the second service device through the first hardware port and the first queue pair. Through the above process, data processing of multiple queue pairs is realized, and each queue pair can correspond to a hardware port. This allows access to different hardware ports to be realized even when there are multiple hardware ports in the service device, thereby making full use of the hardware ports in the service device, improving the hardware utilization and communication performance of the service device. Furthermore, by using multiple queue pairs, parallel data processing can be realized, so that each hardware port can perform data processing when there are multiple hardware ports, thereby improving data transmission efficiency. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 is a network interaction architecture diagram for data transmission provided in an embodiment of this application;

[0064] Figure 2 is an example diagram of a unit group provided in an embodiment of this application;

[0065] Figure 3 is a schematic diagram of a data transmission scenario provided in an embodiment of this application;

[0066] Figure 4 is a flowchart of a data transmission method provided in an embodiment of this application;

[0067] Figure 5 is a schematic diagram of an inter-unit communication structure provided in an embodiment of this application;

[0068] Figure 6 is a schematic diagram of a communication library provided in an embodiment of this application;

[0069] Figure 7 is a flowchart of a specific data transmission method provided in an embodiment of this application;

[0070] Figure 8 is a schematic diagram of a data transmission device provided in an embodiment of this application;

[0071] Figure 9 is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0073] Where this application requires the collection of object data (such as user data), a prompt interface or pop-up window will be displayed before and during the collection process. This prompt interface or pop-up window is used to inform the user that certain data (such as business data in this application) is being collected. The data acquisition steps will only begin after the user confirms the prompt interface or pop-up window; otherwise, the process will end. Furthermore, the acquired user data will be used in reasonable and legal scenarios or for legitimate purposes. Optionally, in scenarios where user data needs to be used but user authorization has not been obtained, authorization can be requested from the user, and the user data will be used only after authorization is granted. In other words, the use of user data in this application complies with relevant laws and regulations.

[0074] The following explanations may be provided for some of the terms that may be used in this application:

[0075] 1. SHMEM is a parallel programming interface designed for GPU clusters to enable efficient and scalable communication. It creates a global address space for data that spans multiple GPU memories, supporting data sharing and parallel computing between GPUs.

[0076] 2. InfiniBand is a computer network communication standard for high-performance computing. It features extremely high throughput and extremely low latency, and is used for data interconnection between computers. InfiniBand is also used for direct or switched interconnection between servers and storage systems, as well as interconnection between storage systems.

[0077] 3. A communication library is a software library used to enable data transmission and communication between devices or systems.

[0078] In this embodiment of the application, please refer to Figure 1. Figure 1 is a network interaction architecture diagram for data transmission provided by this embodiment of the application. As shown in Figure 1, this application can be considered to implement a data transmission system 101. The data transmission system 101 may include N service devices, where N is a positive integer, such as service devices 101a, 101b, and 101c shown in Figure 1. Each service device may include M graphics processing units (GPUs), where M is a positive integer. As shown in Figure 1, service device 101b includes M graphics processing units 102b, such as graphics processing units 1021, ... and graphics processing unit 102M. The M graphics processing units corresponding to each service device can form M unit groups. Each unit group consists of one graphics processing unit included in each of the N service devices. In the same unit group, S queue pairs can be established between any two graphics processing units, where S is a positive integer. For example, assuming there are service devices 101a, 101b and 101c, a unit group may include a graphics processing unit in service device 101a, a graphics processing unit in service device 101b and a graphics processing unit in service device 101c.

[0079] Specifically, please refer to Figure 2, which is an example diagram of a unit group provided in an embodiment of this application. As shown in Figure 2, assuming N is 3 and M is 3, that is, the data transmission system includes service device 201, service device 202 and service device 203, wherein service device 201 includes graphics processing unit 2011, graphics processing unit 2012 and graphics processing unit 2013, service device 202 includes graphics processing unit 2021, graphics processing unit 2022 and graphics processing unit 2023, and service device 203 includes graphics processing unit 2031, graphics processing unit 2032 and graphics processing unit 2033. At this point, the graphics processing units included in the N service devices can be divided into M unit groups, as shown in Figure 2: unit group 2041, unit group 2042, and unit group 2043. Unit group 2041 includes graphics processing units 2011, 2021, and 2031; unit group 2042 includes graphics processing units 2012, 2022, and 2032; and unit group 2043 includes graphics processing units 2013, 2023, and 2033. Data interaction is possible between the graphics processing units within each unit group. Furthermore, any two graphics processing units within a unit group can communicate via S queue pairs, where S is a positive integer. In this system, one graphics processing unit (GPU) corresponds to one piece of hardware. That is, a service device comprising M GPUs includes M pieces of hardware. Hardware refers to the computer hardware, such as a network interface card (NIC), used to enable communication between the service device and the computer network. This system architecture enables data transmission based on multiple queue pairs (QPs), achieving parallel data processing and improving data processing efficiency. Furthermore, when GPUs transmit data using multiple QPs, regardless of the number of hardware ports on the corresponding hardware device, multiple hardware ports can be used simultaneously, thereby improving the hardware utilization and communication performance of the service device. A QP is a resource that combines a send queue and a receive queue for data transmission. A hardware port is the interface between the service device and the outside world; a piece of hardware can include one or more hardware ports.

[0080] It is understood that the service device mentioned in the embodiments of this application can be a computer device, which includes, but is not limited to, terminal devices or servers. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. The service device includes a graphics processing unit and integrates device hardware. The terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, smart speakers, digital cameras, webcams, and other mobile internet devices (MIDs) with network access capabilities, or terminal devices in scenarios such as trains, ships, and flights. The servers mentioned above can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-road cooperation, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0081] Specifically, please refer to Figure 3, which is a schematic diagram of a data transmission scenario provided in an embodiment of this application. As shown in Figure 3, the first service device 301 can respond to a data transmission request and obtain first service data, a first graphics processing unit 3011 for transmitting the first service data, and a second graphics processing unit 3021 for receiving the first service data from the data transmission request. The first service device 301 can determine a first queue pair for transmitting the first service data from the S queue pairs (such as queue pair 1, ... and queue pair S) associated with the first graphics processing unit 3011. The number of the first queue pairs is one or more. The first graphics processing unit 3011 corresponds to the first device hardware 3012, which includes P hardware ports, where P is a positive integer, such as hardware ports 301a and 301b shown in Figure 3. The first service device 301 can obtain the first hardware port corresponding to the first queue pair and send the first service data to the second graphics processing unit 3021 in the second service device 302 through the first hardware port and the first queue pair. For example, assuming the first queue pair is queue pair S, which corresponds to hardware port 301a, the first service device 301 can send the first service data to the second graphics processing unit 3021 in the second service device 302 through queue pair S and hardware port 301a. Both the first service device 301 and the second service device 302 can be any service device in the data transmission system. Since this application uses multiple QPs for data transmission, when data transmission is needed, as long as there is a QP that can receive new data (such as an idle QP or a QP with an incomplete queue space), the data to be transmitted can be processed for transmission, achieving parallel data transmission and thus improving data transmission efficiency.

[0082] Optionally, the data involved in the embodiments of this application may be stored in a computer device, or may be stored based on cloud storage technology or a blockchain network, without limitation.

[0083] Further, please refer to Figure 4, which is a flowchart of a data transmission method provided in an embodiment of this application. As shown in Figure 4, this data transmission process can be implemented by any service device, which can be referred to as the first service device. The data transmission process includes the following steps:

[0084] Step S401: Respond to the data transmission request, obtain the first service data, the first graphics processing unit that transmits the first service data, and the second graphics processing unit that receives the first service data from the data transmission request.

[0085] In this embodiment, the first service device can respond to a data transmission request and obtain first service data, a first graphics processing unit that transmits the first service data, and a second graphics processing unit that receives the first service data from the data transmission request. The first graphics processing unit is a graphics processing unit within the first service device. The first service data can be a single piece of data or data composed of multiple pieces of data; that is, the first service data can be a single piece of data or a group of data. The data transmission request is used to instruct the first service device to transmit data to the second service device, specifying the type of data to be transmitted, the graphics processing unit from which the data is transmitted, and the graphics processing unit to which the data is sent.

[0086] Optionally, the first service device can respond to a data transmission request and obtain first service data, a first graphics processing unit that transmits the first service data, and a receiving graphics processing unit that receives the first service data from the data transmission request. If the first graphics processing unit and the receiving graphics processing unit belong to the same unit group, then the receiving graphics processing unit is identified as the second graphics processing unit for receiving the first service data; if the first graphics processing unit and the receiving graphics processing unit do not belong to the same unit group, then the second service device to which the receiving graphics processing unit belongs is obtained, and the graphics processing unit in the second service device that belongs to the unit group to which the first graphics processing unit belongs is identified as the second graphics processing unit for receiving the first service data. As shown in Figure 2, assuming that the first graphics processing unit is graphics processing unit 2011 and the receiving graphics processing unit is graphics processing unit 2022, where graphics processing unit 2011 belongs to unit group 2041 and graphics processing unit 2022 belongs to unit group 2042, that is, graphics processing unit 2011 and graphics processing unit 2022 do not belong to the same unit group, then the first service device can obtain the second service device (here, service device 202) to which graphics processing unit 2022 belongs, and can determine the graphics processing unit (here, graphics processing unit 2021) belonging to unit group 2041 in the second service device as the second graphics processing unit used to receive the first service data.

[0087] Step S402: Determine the first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit.

[0088] In this embodiment, the first service device can determine a first queue pair for sending first service data from S queue pairs associated with the first graphics processing unit, where S is a positive integer. Here, the S queue pairs associated with the first graphics processing unit refer to the S queue pairs established between the first graphics processing unit and the second graphics processing unit. Specifically, the first service device can obtain the target data channel associated with the first service data and determine the queue pair corresponding to the target data channel as the first queue pair for sending the first service data. The queue pair is a virtual interface between hardware and software used for information transmission. Each queue pair includes two queues, which can be denoted as a first queue and a second queue. When one queue in a queue pair is used as a send queue (SQ), the other queue is used as a receive queue (RQ).

[0089] The first service device can determine the target data channel associated with the first business data from existing data channels. Specifically, the first service device can construct D first data channels through the first graphics processing unit; D is an integer multiple of S. Optionally, to improve the convenience of parallel data transmission within the service device, D can be the same as S. The D first data channels refer to the data channels corresponding to the first graphics processing unit and the second graphics processing unit. By setting D to an integer multiple of S, each queue pair can correspond to the same number of data channels, thus making the data processed by each queue pair more balanced. Based on the channel numbers corresponding to the D first data channels and S, the D first data channels are associated with S queue pairs; each first data channel is associated with one queue pair, and each queue pair is associated with one or more first data channels. For example, the remainder between the channel number corresponding to the i-th first data channel and S can be used to determine the queue identifier associated with the i-th first data channel, and the queue pair associated with the queue identifier of the i-th first data channel is determined as the queue pair associated with the i-th first data channel, where i is a natural number less than D. For example, if the channel number of the i-th first data channel (Channel i) is i, then the queue identifier of the queue pair associated with the i-th first data channel is i%S. Here, "%" refers to the modulo operation. Of course, in this method, the queue identifiers corresponding to the S queue pairs are sequentially from 0 to S-1. When the queue identifiers corresponding to the S queue pairs are sequentially from 1 to S, the remainder between the channel number corresponding to the i-th first data channel and S can be added by one to obtain the queue identifier associated with the i-th first data channel, i.e., i%S+1. In simple terms, the first service device can cyclically map D first data channels to S queue pairs, thus associating the D first data channels with the S queue pairs. Cyclic mapping means that one piece of data is read from each of the two sets of data (D first data channels and S queue pairs) to form an association relationship. When a set of data is finished being read, the next read starts from the beginning of that set of data, until both sets of data (D first data channels and S queue pairs) have been read. This allows us to determine the queue pair associated with each first data channel and the first data channel associated with each queue pair.

[0090] For example, referring to Figure 5, which is a schematic diagram of an inter-unit communication structure provided in an embodiment of this application. As shown in Figure 5, there are S queue pairs between graphics processing unit 1 and graphics processing unit 2. Graphics processing unit 1 includes D data channels. It can be assumed that data channel 1 is associated with queue pair 1, data channel 2 is associated with queue pair 2, ..., and data channel D is associated with queue pair S. For example, assuming S is 2 and D is 4, then data channel 1 is associated with queue pair 1, data channel 2 is associated with queue pair 2, data channel 3 is associated with queue pair 1, and data channel 4 is associated with queue pair 2.

[0091] At this time, when the first service device determines the first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit, the first service device can, through the first graphics processing unit, select a first target data channel from the D first data channels based on the channel operating status of the D first data channels, and determine the queue pair associated with the first target data channel as the first queue pair for sending the first service data. The channel operating status can include a channel working state and a channel idle state. The channel working state indicates that the corresponding first data channel is processing data, and the channel idle state indicates that the corresponding first data channel is not processing data. Specifically, the first service device can determine the first data channel whose channel operating state is idle as the first target data channel. Optionally, the first service device can split the first service data into H service sub-data, select the first target data channels corresponding to the H service sub-data from the D first data channels based on the channel operating status of the D first data channels, and determine the queue pair associated with the first target data channel corresponding to each service sub-data as the first queue pair for sending that service sub-data, thereby obtaining the first queue pairs corresponding to the H service sub-data.

[0092] Alternatively, the first service device can create data channels for the data to be sent. Specifically, the first service device can determine the number of first channels and construct H second data channels based on the data volume of the first business data and S; H is an integer multiple of S, and H is the number of first channels. Specifically, the first service device can obtain the unit data volume, obtain the data volume multiple between the data volume of the first business data and the unit data volume, and adjust the data volume multiple based on S to obtain the number of first channels. This data volume multiple refers to the quotient between the data volume of the first business data and the unit data volume. The unit data volume refers to the theoretical data volume that a queue can process in a single run; that is, when the number of data processed by a queue in a single run is the unit data volume, the queue has higher data processing efficiency. When adjusting the data volume multiple based on S, the value closest to the data volume multiple and an integer multiple of S can be determined as the number of first channels. Furthermore, the first service device can split the first service data into H service sub-data, and allocate a second data channel for each service sub-data; based on the channel number of the second data channel corresponding to each service sub-data, determine the first queue pair used to send the service sub-data. This process can be referred to the above process of associating D first data channels with S queue pairs.

[0093] This process enables data transmission based on multiple QPs, allowing multiple QPs to transmit data in parallel, thereby improving data transmission efficiency. Furthermore, generating data channels based on integer multiples of QPs ensures balanced processing and full utilization of QPs when transmitting the first business data, thus improving the resource utilization of the data transmission system.

[0094] Step S403: Obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

[0095] In this embodiment, the first service device can obtain the first hardware port corresponding to the first queue pair. Through the first hardware port, the first service data is sent to the second graphics processing unit in the second service device via the first queue pair. The second service device can then obtain the first service data from the first queue pair through the second graphics processing unit. Optionally, when there are H service sub-data corresponding to different first queue pairs, the first service device can obtain the first hardware ports corresponding to each of the H first queue pairs. The j-th service sub-data is sent to the second graphics processing unit through the first hardware port corresponding to the j-th service sub-data and the first queue pair corresponding to the j-th service sub-data, where j is a positive integer less than or equal to H. Similarly, H service sub-data can be sent to the second graphics processing unit separately. In short, H service sub-data can be sent to the second graphics processing unit separately through the first hardware ports and first queue pairs corresponding to the H service sub-data. This method allows all first hardware ports to be used in the data transmission process when there are one or more first hardware ports, achieving full operation of the hardware ports, thereby improving the hardware utilization of the service device and enhancing communication performance.

[0096] The correspondence between queue pairs and hardware ports can be determined through the queue port number (UDP src port) of the queue pair. Specifically, the first service device can obtain the network topology corresponding to the first service device and the first queue identifier of the first queue pair. The network topology refers to the physical or logical layout formed by the connection of the first service device and its related devices (such as the device hardware included in the first service device, the hardware ports in the device hardware, etc.). The network topology and the first queue identifier are hashed to obtain the target queue port number of the first queue pair. P hardware ports associated with the first graphics processing unit are obtained. Based on the queue port parameters corresponding to the P hardware ports, the hardware port corresponding to the queue port parameter that matches the target queue port number is determined as the first hardware port corresponding to the first queue pair. The queue port parameter corresponding to a hardware port is used to represent the conditions that the queue port number corresponding to the queue pair that can access the hardware port must meet. Simply put, the queue port parameter can be considered to include the queue port number that matches the queue port parameter, or the queue port number has the characteristics of the queue port parameter that matches the queue port number; P is a positive integer. For example, if P is 2, the queue port parameter corresponding to hardware port 1 is "odd", and the queue port parameter corresponding to hardware port 2 is "even", then the parity of the target queue port number can be obtained. If the target queue port number is odd, then hardware port 1 is determined as the first hardware port corresponding to the first queue pair; if the target queue port number is even, then hardware port 2 is determined as the first hardware port corresponding to the first queue pair.

[0097] Furthermore, the first service device includes M hardware devices, where M is a positive integer, with one graphics processing unit corresponding to one hardware device. The first service device can send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair. Specifically, the first service device can create a work queue element (WQE) for the first service data and add the work queue element to the first queue pair, specifically adding the work queue element to the first queue pair corresponding to the first queue in the first service device. Here, WQE is a queue element that can be used for Remote Direct Memory Access (RDMA). Furthermore, the first service device can send a data transmission command to the first hardware device associated with the first graphics processing unit, and obtain the work queue element from the first queue pair through the first hardware device and the data transmission command; the M hardware devices include the first hardware device. This data transmission command is used to indicate that the first hardware device has data to be sent; this data transmission command can be considered a doorbell notification, a lightweight asynchronous event notification mechanism. The first service device can send the first business data corresponding to the work queue element to the second graphics processing unit in the second service device through the first device hardware.

[0098] Optionally, when adding a work queue element to the first queue pair, the first service device can obtain the inbound queue position (producer index, PI) and debound queue position (consumer index, CI) of the first queue pair in the first service device. Based on the inbound and debound queue positions, the number of pending data corresponding to the first queue pair in the first service device is determined. This number of pending data can be denoted as PI-CI, representing the number of WQEs that have not yet been sent in the first queue pair. PI represents the position of the most recently enqueued WQE in the first queue pair, and CI represents the position of the dequeued WQEs in the first queue pair. Each time a new WQE is added to the first queue pair, PI can be updated, and the first service device can periodically obtain CI through its hardware.

[0099] Furthermore, if the amount of data to be processed is less than the queue capacity of the first queue pair, then the work queue element is added to the first queue pair. Optionally, the enqueue position of the first queue pair in the first service device can be updated based on the position of the work queue element in the first queue pair. The queue capacity can represent the total number of WQEs that can be stored in the first queue pair, or it can represent the depth of the first queue pair. If the amount of data to be processed is greater than or equal to the queue capacity of the first queue pair, it means that there are no free slots in the first queue pair. The first service device can cache the work queue element until the amount of data to be processed is less than the queue capacity of the first queue pair, at which point the work queue element is added to the first queue pair.

[0100] Optionally, the first service device can send the first service data to the second graphics processing unit in the second service device via the first hardware port and the first queue pair using InfiniBand GPUDirect Async (IBGDA) technology. Through this process, data transmission can be implemented via the GPU, eliminating the need for a central processing unit (CPU) in the data transmission process, thereby minimizing latency and improving data transmission efficiency. IBGDA is a technology that directly transmits data between the GPU memory and the memory of the remote system, bypassing the CPU to achieve asynchronous communication.

[0101] Further, please refer to Figure 6, which is a schematic diagram of a communication library provided in an embodiment of this application. As shown in Figure 6, this application can be considered to implement a service-agnostic multi-to-multi collective communication (Alltoall) performance acceleration library, also known as a remote memory transport (Remote Memory Transport) high-performance communication library. This multi-to-multi collective communication performance acceleration library can include a shared communication library and a high-performance communication library. The shared communication library is used to generate pre-defined port numbers for queue pairs, and the high-performance communication library is used for end-to-end (i.e., data transmission between graphics processing units) data transfer. This multi-to-multi collective communication performance acceleration library can be divided into three parts: 1. Establishing a communication connection; 2. Multi-channel data transmission; 3. Sending signaling data. The above processes can be implemented based on IBGDA to improve data transmission efficiency. Specifically, these three parts can be as follows:

[0102] Step 1: Establish a communication connection.

[0103] In this embodiment, the data transmission system can establish unit groups among multiple GPUs. These unit groups can be considered SHMEM communication groups. Within each unit group, multiple communication connections are established between any two GPUs; that is, multiple queue pairs are established between any two GPUs within each unit group. Specifically, assuming the data transmission system includes N service devices, each service device includes M graphics processing units (GPUs), and the GPU number can be considered a tuple (x, y), where x refers to the service device number and y refers to the GPU's unit number within the service device, then all GPUs can be divided into M unit groups. The unit number y of the GPUs included in each unit group is the same. Within each unit group, all GPUs establish communication connections with other GPUs, and each communication connection corresponds to multiple queue pairs (QPs). As shown in Figure 2, in unit group 2041, among graphics processing units 2011, 2021, and 2031, a communication connection is established between any two graphics processing units, creating S queue pairs between them. For example, there are S queue pairs between graphics processing unit 2011 and graphics processing unit 2021. This example only illustrates a portion of the communication connections in the service device shown in Figure 2. In other words, within any unit group, S queue pairs can be established for any two graphics processing units within that unit group.

[0104] Each QP has two identifiers: a queue identifier (QP num) and a queue port number (UDP src port). UDP stands for User Datagram Protocol. The queue identifier is generated by the hardware of the first service device. The hardware can record the queue information of the S queue pairs corresponding to the graphics processing unit. The queue information of a queue pair can include the queue identifier and other information, including but not limited to the device number of the service device to which the queue pair belongs and the hardware port number of the corresponding device hardware. The queue port number is a pre-defined port number generated by the shared communication library (SHMEM). Queue port numbers are assigned to queue pairs based on these pre-defined port numbers. Specifically, taking the first service device as an example, the first service device can obtain the network topology and generate multiple pre-defined port numbers based on the network topology and the device hash algorithm of the first service device. Any two pre-defined port numbers are different. The first service device can determine the queue port numbers corresponding to the S queue pairs from these pre-defined port numbers. Alternatively, the first service device can hash the network topology and the queue identifier of the r-th queue pair to obtain the queue port number of the r-th queue pair, where r is a positive integer less than or equal to S. Based on this process, the queue port numbers of all queue pairs included in the first service device can be obtained. The selection of hardware ports and switch ports by the QP is accomplished through a 5-tuple hash (including the queue port number of the QP). The method described above for determining the queue port number of the QP utilizes the characteristic of linear hash offset, ensuring that the QP is evenly distributed across the hardware ports (e.g., dual-port NICs) at the sending end, the hardware ports (e.g., dual-port NICs) at the receiving end, and the network switch. This improves the load balancing of the service device's hardware, reduces the probability of hash collisions on the network switch, and thus enhances network security and communication performance for data transmission.

[0105] Step 2, multi-channel data transmission.

[0106] In this embodiment, the step may include the following process: associating the data channel with the queue pair -> issuing work queue elements -> issuing a data transmission reminder. In simple terms, the first service device can write the first service data into the target data channel, map the target data channel to the S queue pairs established between the first graphics processing unit and the second graphics processing unit, determine the first queue pair corresponding to the target data channel; then generate work queue elements for the first service data, add the work queue elements to the first queue pair; and send a data transmission instruction (i.e., issue a data transmission reminder) to the device hardware corresponding to the first graphics processing unit, wherein the device hardware here can be a Remote Direct Memory Access (RDMA) network card. The first service device can then send the first service data to the second graphics processing unit in the second service device via the device hardware.

[0107] In this process, when the first service device distributes and combines data, each GPU communicates with multiple GPUs within the unit group established in step 1. Assume the current GPU's number within the unit group is GPU1, and the other GPUs in the unit group are numbered GPU2, GPU3, GPU4, ..., GPUN. Taking the communication between GPU1 and GPU2 as an example, GPU1 establishes multiple channels (i.e., data channels), each channel handling different data, and multiple channels transmit data in parallel. Each channel is mapped to a QP created in step 1: assuming that in step 1, GPU1 establishes S QPs with each of the other GPUs in the group, then the queue port number of the QP mapped to Channel i is i%S (the number of channels is an integer multiple of the number of QPs; for example, the number of channels can be equal to the number of QPs). Within each channel, once the data to be sent to a target GPU is ready, a corresponding RDMA Work Queue Element (WQE) is created and populated into the corresponding QP. Then, a Doorbell is used to notify the RDMA device hardware that data to be sent is available. See steps S402 and S403 in Figure 4 for details. This process evenly distributes data channels across queue pairs, and the queue pairs are also evenly distributed across the hardware ports of the device. This achieves load balancing for the service device, improves application balance across queue pairs and hardware ports, thereby increasing hardware utilization and communication performance.

[0108] Step 3: Send signaling data.

[0109] In this embodiment, after the data transmission operation in step 2 is completed, signaling data needs to be sent. This signaling data is a control signal and other data associated with the corresponding service data. Similar to step 2, channel and QP mapping is first performed to determine which QP to use for transmitting the signaling data. Further, the first service device can directly assemble the signaling data using WQE within the GPU, and then use the atomic transmission mode of IBGDA to send the signaling data to the target GPU, i.e., the aforementioned second graphics processing unit. During data transmission, it is essential to ensure that the same QP is used within the same channel, transmitting data first and then signaling data, thereby ensuring the integrity and security of the data transmission by transmitting the service data and its related descriptive information (i.e., signaling data).

[0110] Specifically, please refer to Figure 7, which is a flowchart of a data transmission method provided in an embodiment of this application. As shown in Figure 7, the data transmission process includes the following steps:

[0111] Step S701: Establish a communication connection.

[0112] In this embodiment, the first service device includes M graphics processing units, where M is a positive integer. Each graphics processing unit corresponds to one piece of hardware, and each piece of hardware includes P hardware ports, where P is a positive integer. The data transmission system includes N service devices, where N is a positive integer, and each service device includes M graphics processing units.

[0113] Specifically, the first service device can obtain the unit number data corresponding to each of the M graphics processing units (GPUs), and add the M GPUs to the M unit groups based on the unit number data. The unit number data indicates the number of the corresponding GPU in the corresponding service device. Each service device can add the M GPUs to the M unit groups based on the unit number data corresponding to its own M GPUs. It can be assumed that the unit number data of the GPUs in a unit group are the same, as shown in the example in Figure 2. Further, in each unit group, the GPUs of the first service device in that unit group are linked to other GPUs in that unit group to form S queue pairs. That is, S queue pairs are formed between any two GPUs in the same unit group. The S queue pairs corresponding to each unit group are mapped to the hardware port of that unit group. In other words, the queue pairs associated with a GPU are mapped to the hardware port of the device hardware corresponding to that GPU, as described in step S403 of Figure 4. The queue pairs associated with a graphics processing unit include S queue pairs between that graphics processing unit and any other graphics processing unit in the unit group to which it belongs. For example, the composition of a possible service device can be seen in Table 1:

[0114] Table 1

[0115] As shown in Table 1, service device 1 includes M GPUs. Each GPU belongs to a unit group. One GPU corresponds to one device hardware. One device hardware includes P hardware ports. For example, "hardware port 1.k" represents the kth hardware port in the device hardware corresponding to GPU1, and "hardware port Mk" represents the kth hardware port in the device hardware corresponding to GPUM, etc. k is a positive integer less than or equal to P.

[0116] One possible composition of the GPU queue pairs can be found in Table 2:

[0117] Table 2

[0118] As shown in Table 2, assuming a unit group includes GPU1 to GPUN, then S queue pairs are established between GPU1 and GPU2, ..., and S queue pairs are established between GPU1 and GPUN. For example, "Queue pair 12." represents a queue pair between GPU1 and GPU2, and "Queue pair 1N." represents a queue pair between GPU1 and GPUN, etc.

[0119] When mapping queue pairs to hardware ports, P queue port parameters corresponding to each hardware port can be obtained. The hardware port that matches the queue port number is identified as the queue pair corresponding to that queue port number. Optionally, the queue port parameters corresponding to each of the P hardware ports can be provided manually. Alternatively, the first service device can obtain the queue port numbers corresponding to each of the S queue pairs and obtain A candidate classification parameters; A is a positive integer, where the candidate classification parameters refer to the data characteristics used when mapping queue pairs to hardware ports, such as parity and remainder with respect to P. The first service device can perform clustering processing on the S queue port numbers based on the A candidate classification parameters to obtain P port number clusters corresponding to each of the A candidate classification parameters. Based on the number of queue port numbers included in each of the P port number clusters corresponding to each candidate classification parameter, the classification difference degree corresponding to that candidate classification parameter is determined; the classification difference degree is used to represent the difference between the number of P port numbers of the corresponding candidate classification parameter. For example, the first service device can determine the classification difference degree of a candidate classification parameter by taking the average of the differences between the number of any two port numbers among the P port numbers of the candidate classification parameters. Further, the first service device can determine the candidate classification parameter with the smallest classification difference degree as the target classification parameter, and allocate queue port parameters to the P hardware ports based on the target classification parameter. The smallest classification difference degree indicates that when the queue pairs are mapped to hardware ports using the candidate classification parameters corresponding to this degree of classification difference, the queue pairs are evenly distributed across the hardware ports, which can improve the load balancing of the hardware ports.

[0120] Step S702: Respond to the data transmission request, obtain the first service data, the first graphics processing unit that transmits the first service data, and the second graphics processing unit that receives the first service data from the data transmission request.

[0121] In the embodiments of this application, please refer to the relevant description in step S401 of FIG4.

[0122] Step S703: Divide the first business data into H business sub-data and determine the target data channels corresponding to each of the H business sub-data.

[0123] In this embodiment of the application, please refer to the relevant description in step S402 of FIG4. The target data channels corresponding to the H service sub-data can be the first target data channel shown in step S402 of FIG4, or the second data channel.

[0124] Step S704: From the S queue pairs associated with the first graphics processing unit, determine the first queue pairs corresponding to the H target data channels respectively.

[0125] In this embodiment of the application, the process can be referred to the relevant description in step S402 of FIG4. In the service device, one data channel corresponds to one queue pair, and one queue pair can correspond to one or more data channels. The first service device can determine the first queue pairs corresponding to H target data channels from the S queue pairs between the first graphics processing unit and the second graphics processing unit.

[0126] Step S705: Generate work queue elements corresponding to H business sub-data respectively, and add the work queue elements corresponding to H business sub-data respectively to the first queue pair corresponding to H business sub-data respectively.

[0127] In this embodiment, the first service device can generate a work queue element corresponding to the j-th service sub-data and add the work queue element corresponding to the j-th service sub-data to the first queue pair corresponding to the j-th service sub-data. Similarly, the work queue elements corresponding to H service sub-data can be added to the first queue pairs corresponding to H service sub-data respectively. Optionally, the processing of the H service sub-data can be simultaneous and asynchronous, which can improve the efficiency of data processing. For details, please refer to the relevant description in step S403 of Figure 4.

[0128] Step S706: Obtain a work queue element from the first queue pair through the first hardware port corresponding to each first queue pair, and send the business sub-data corresponding to the work queue element to the second graphics processing unit in the second service device.

[0129] In the embodiments of this application, the process can be referred to the relevant description of step S403 in Figure 4.

[0130] Step S707: When the transmission of the first service data is completed, first signaling data is generated for the first service data, and the first signaling data is sent to the second graphics processing unit.

[0131] In this embodiment, when the transmission of the first service data is completed, the first service device can generate first signaling data for the first service data; from S queue pairs, a second queue pair for transmitting the first signaling data is determined, and the first signaling data is transmitted to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair, so that the second graphics processing unit can obtain the first service data based on the first signaling data and detect the first service data. The transmission process of the first signaling data can be referred to as the transmission process of the first service data. The first signaling data is descriptive information of the first service data, including but not limited to the data quantity and data composition. The data quantity indicates the number of data transmitted to the second service device, that is, how many complete data sets are transmitted to the second service device; the data composition indicates the composition and slicing method of the data transmitted to the second service device.

[0132] Optionally, when sending the first signaling data to the second graphics processing unit via the second queue pair and its corresponding second hardware port, the first service device can generate a signaling queue element for the first signaling data and add it to the second queue pair. This signaling queue element is a queue element that can be used for remote direct memory access, similar to the working queue element for the first service data. The difference is that the queue element for signaling data is referred to as the signaling queue element, and the queue element for service data is referred to as the working queue element. When a signaling queue element is obtained from the second queue pair via its corresponding second hardware port, the queue status of the second queue pair is updated to a locked state, and the first signaling data is sent to the second graphics processing unit. The locked state indicates that the second queue pair cannot be accessed. When the first signaling data is successfully sent, the queue status of the second queue pair is updated to a running state. The running state indicates that the second queue pair is allowed to be accessed.

[0133] Optionally, the first service device receives the second service data sent by the third service device through the third queue pair and writes the second service data into the third queue pair; when receiving the second signaling data for the second service data, it obtains the description parameters for the second service data from the second signaling data and the actual received parameters corresponding to the description parameters from the second service data; if the description parameters are the same as the actual received parameters, it obtains the second service data from the third queue pair; if the description parameters are different from the actual received parameters, it sends a data repair request to the third service device based on the parameter difference between the description parameters and the actual received parameters. For example, the description parameters for the second service data include "data quantity: 5; data composition: H' sub-data", while the actual received parameters include "data quantity: 4; data composition: H" sub-data". Therefore, the third service device sent 5 complete data packets to the first service device, but the first service device only received 4 complete data packets. This indicates that the data received by the first service device is abnormal. Based on the difference between the description parameters and the actual received parameters, a data repair request can be sent to the third service device to repair the received second service data, thereby improving the security and accuracy of data transmission. Similarly, this can be used for any process that detects the service data corresponding to the signaling data based on the signaling data, such as the process described above where the second service device detects the first service data based on the first signaling data.

[0134] In this embodiment, in response to a data transmission request, first service data, a first graphics processing unit for transmitting the first service data, and a second graphics processing unit for receiving the first service data are obtained from the data transmission request. The first graphics processing unit is a graphics processing unit in a first service device. From S queue pairs associated with the first graphics processing unit, a first queue pair for transmitting the first service data is determined; S is a positive integer. The first hardware port corresponding to the first queue pair is obtained, and the first service data is transmitted to the second graphics processing unit in the second service device through the first hardware port and the first queue pair. Through the above process, data processing of multiple queue pairs is realized, and each queue pair can correspond to a hardware port. This allows access to different hardware ports to be realized even when there are multiple hardware ports in the service device, thereby making full use of the hardware ports in the service device, improving the hardware utilization and communication performance of the service device. Furthermore, by using multiple queue pairs, parallel data processing can be realized, so that each hardware port can perform data processing when there are multiple hardware ports, thereby improving data transmission efficiency.

[0135] Optionally, when this application is used for training and inference tasks of Mixture of Experts (MOE) models, the technical solution of this application can realize all-to-all communication of the MOE model. In this case, the first business data refers to the model parameters or gradient adjustments of the MOE model, that is, the data that needs to be shared between various service devices during the training or inference process of the MOE model, thereby improving the Dispatch and Combine bandwidth (the amount of data transmitted per unit time) in the MOE model and reducing the training / inference time of large models. This application can fully utilize the bandwidth of multiple hardware ports and improve the hardware utilization of service devices.

[0136] Furthermore, tests were conducted on this application under different server environments, such as server environment 1 and server environment 2. Server environment 1 refers to the equipment model of the service device; server environment 1 is an artificial intelligence accelerator, and server environment 2 is a high-performance computer server. The test results are shown in Table 3.

[0137] Table 3

[0138] As shown in Table 3, the number of nodes refers to the number of service devices used in the test, the communication library version indicates the version of the communication library being tested, the existing communication library image indicates the prior art, and the communication library image of this application indicates the technical solution of this application (i.e., the many-to-many collective communication performance acceleration library). Tests were conducted in server environment 1 and server environment 2 for 2-machine dispatch and combine, and 4-machine dispatch and combine, respectively. 2 machines refer to 2 service devices, and 4 machines refer to 4 service devices. Table 3 shows that in server environment 1, the RDMA performance of 2-machine combine and dispatch is improved by 35.2% and 37.69%, respectively, and the LINK (high-speed interconnect) performance is improved by 35.2% and 37.69%, respectively; the RDMA performance of 4-machine dispatch and combine is improved by 105.96% and 119.32%, respectively, and the LINK performance is improved by 105.96% and 119.32%, respectively. In server environment 2, the RDMA performance of 2-machine merging and distribution is improved, and the RDMA performance of 4-machine Dispatch and Combine is improved by 120.43% and 111.52% respectively, and the LINK performance is improved by 120.43% and 111.52% respectively. It can be seen that the communication library implemented in this application has a significant performance improvement compared with the existing communication libraries.

[0139] As can be seen from Table 3, when the number of deployed service devices is 4, the technical solution of this application can achieve a performance improvement of about 100% compared with the existing technical solution. Therefore, in the training and inference deployment of the Mixture of Experts (MoE) model, the optimal number of deployed service devices is 4.

[0140] Further, please refer to Figure 8, which is a schematic diagram of a data transmission device provided in an embodiment of this application. This data transmission device can be a computer program (including program code, etc.) running on a computer device; for example, the data transmission device can be an application software. The device can be used to execute the corresponding steps in the method provided in the embodiment of this application. As shown in Figure 8, the data transmission device 800 can be used in the computer device in the embodiment corresponding to Figure 3. Specifically, the device may include: a data acquisition module 11, a queue pair determination module 12, and a data transmission module 13.

[0141] Data acquisition module 11 is used to respond to data transmission requests, acquire first service data from the data transmission request, a first graphics processing unit that transmits the first service data, and a second graphics processing unit that receives the first service data; the first graphics processing unit is a graphics processing unit in the first service device;

[0142] The queue pair determination module 12 is used to determine the first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit; S is a positive integer.

[0143] The data sending module 13 is used to obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

[0144] The device 800 also includes:

[0145] Channel construction module 14 is used to construct D first data channels through the first graphics processing unit; D is an integer multiple of S;

[0146] Data association module 15 is used to associate D first data channels with S queue pairs based on the channel numbers corresponding to the D first data channels and S respectively; each first data channel is associated with a queue pair, and each queue pair is associated with one or more first data channels;

[0147] This queue, for determining module 12, can be used for:

[0148] The first graphics processing unit selects a first target data channel from the D first data channels based on the channel operation status of the D first data channels, and determines the queue pair associated with the first target data channel as the first queue pair for sending the first service data.

[0149] The queue pair determination module 12 can be used for:

[0150] Based on the data volume of the first business data and S, determine the number of first channels and construct H second data channels; H is an integer multiple of S, and H is the number of first channels;

[0151] The first business data is split into H business sub-data, and a second data channel is allocated to each business sub-data.

[0152] Based on the channel number of the second data channel corresponding to each service sub-data, a first queue pair for sending that service sub-data is determined.

[0153] Specifically, when acquiring the first hardware port corresponding to the first queue pair, the data sending module 13 can be used for:

[0154] Obtain the network topology corresponding to the first service device, and the first queue identifier of the first queue pair;

[0155] The network topology and the first queue identifier are hashed to obtain the target queue port number of the first queue pair.

[0156] Obtain the P hardware ports associated with the first graphics processing unit. Based on the queue port parameters corresponding to the P hardware ports, determine the hardware port corresponding to the queue port parameter that matches the target queue port number as the first hardware port corresponding to the first queue pair; P is a positive integer.

[0157] The device 800 also includes:

[0158] Parameter acquisition module 16 is used to obtain the queue port numbers corresponding to S queue pairs and to obtain A candidate classification parameters; A is a positive integer.

[0159] The port clustering module 17 is used to perform clustering processing on S queue port numbers based on A candidate classification parameters to obtain P port number clusters corresponding to A candidate classification parameters.

[0160] The difference detection module 18 is used to determine the degree of classification difference corresponding to each candidate classification parameter based on the number of queue port numbers included in the P port number clusters corresponding to each candidate classification parameter; the degree of classification difference is used to represent the difference between the number of P port numbers of the corresponding candidate classification parameter.

[0161] The parameter determination module 19 is used to determine the candidate classification parameter with the smallest degree of classification difference as the target classification parameter, and to allocate queue port parameters to P hardware ports based on the target classification parameter.

[0162] The first service device includes M hardware devices, where M is a positive integer;

[0163] When sending the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair, the data sending module 13 can be used to:

[0164] Create a work queue element for the first business data and add the work queue element to the first queue pair;

[0165] Send a data transmission instruction to the first device hardware associated with the first graphics processing unit, and obtain a work queue element from the first queue pair through the first device hardware and the data transmission instruction; the M device hardwares include the first device hardware;

[0166] The first business data corresponding to the work queue element is sent to the second graphics processing unit in the second service device through the first device hardware.

[0167] Specifically, when adding a work queue element to the first queue pair, the data sending module 13 can be used to:

[0168] Obtain the enqueue position and dequeue position of the first queue pair in the first service device, and determine the number of data to be processed corresponding to the first queue pair in the first service device based on the enqueue position and dequeue position.

[0169] If the amount of data to be processed is less than the queue capacity of the first queue pair, then add the work queue element to the first queue pair;

[0170] If the amount of data to be processed is greater than or equal to the queue capacity of the first queue pair, the work queue elements are cached until the amount of data to be processed is less than the queue capacity of the first queue pair, at which point the work queue elements are added to the first queue pair.

[0171] The device 800 also includes:

[0172] Signaling generation module 20 is used to generate first signaling data for the first service data when the first service data transmission is completed;

[0173] The signaling sending module 21 is used to determine a second queue pair for sending first signaling data from S queue pairs, and send the first signaling data to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair, so that the second graphics processing unit can obtain the first service data based on the first signaling data and detect the first service data.

[0174] Specifically, when sending the first signaling data to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair, the signaling sending module 21 can be used to:

[0175] Generate a signaling queue element for the first signaling data and add the signaling queue element to the second queue pair;

[0176] When a signaling queue element is obtained from the second queue pair through the second hardware port corresponding to the second queue pair, the queue status of the second queue pair is updated to the locked state, and the first signaling data is sent to the second graphics processing unit; the locked state is used to indicate that the second queue pair cannot be accessed;

[0177] When the first signaling data is successfully sent, the queue status of the second queue pair is updated to running status; the running status indicates that the second queue pair is allowed to access.

[0178] The device 800 also includes:

[0179] Data receiving module 22 is used to receive the second service data sent by the third service device through the third queue pair and write the second service data into the third queue pair;

[0180] Information acquisition module 23 is used to, when receiving second signaling data for the second service data, obtain description parameters for the second service data from the second signaling data, and obtain the actual received parameters corresponding to the description parameters from the second service data;

[0181] The data processing module 24 is used to obtain the second service data from the third queue pair if the description parameters are the same as the actual received parameters.

[0182] The data processing module 24 is also used to send a data repair request to the third service device based on the parameter difference between the description parameters and the actual received parameters if the description parameters are different from the actual received parameters.

[0183] The first service device includes M graphics processing units, each of which includes the first graphics processing unit, where M is a positive integer; the device 800 also includes:

[0184] The unit group generation module 25 is used to obtain the unit number data corresponding to each of the M graphics processing units, and add the M graphics processing units to the M unit groups based on the unit number data.

[0185] The queue pair construction module 26 is used to construct S queue pairs between the graphics processing unit of the first service device in the unit group and other graphics processing units in the unit group in each unit group;

[0186] The port mapping module 27 is used to map the S queue pairs corresponding to each unit group to the hardware port of that unit group.

[0187] This application provides a data transmission apparatus that can respond to a data transmission request, obtain first service data, a first graphics processing unit for transmitting the first service data, and a second graphics processing unit for receiving the first service data from the data transmission request. The first graphics processing unit is a graphics processing unit in a first service device. From S queue pairs associated with the first graphics processing unit, a first queue pair for transmitting the first service data is determined; S is a positive integer. A first hardware port corresponding to the first queue pair is obtained, and the first service data is transmitted to the second graphics processing unit in a second service device through the first hardware port and the first queue pair. Through the above process, data processing of multiple queue pairs is realized, and each queue pair can correspond to a hardware port. This allows access to different hardware ports even when a service device has multiple hardware ports, thereby fully utilizing the hardware ports of the service device, improving the hardware utilization and communication performance of the service device. Furthermore, using multiple queue pairs enables parallel data processing, allowing each hardware port to perform data processing when multiple hardware ports exist, thereby improving data transmission efficiency.

[0188] Referring to Figure 9, which is a schematic diagram of the structure of a computer device provided in an embodiment of this application, the computer device in this embodiment may include one or more processors 901, a memory 902, and an input / output interface 903. The processor 901, memory 902, and input / output interface 903 are connected via a bus 904. The memory 902 stores a computer program, which includes program instructions. The input / output interface 903 receives and outputs data, such as for data interaction between the host machine and the computer device, or for data interaction between various virtual machines within the host machine. The processor 901 executes the program instructions stored in the memory 902.

[0189] The processor 901 can perform the following operations:

[0190] In response to a data transmission request, a first graphics processing unit obtains first service data from the data transmission request, transmits the first service data, and a second graphics processing unit receives the first service data; the first graphics processing unit is a graphics processing unit in the first service device.

[0191] From the S queue pairs associated with the first graphics processing unit, determine the first queue pair used to send the first service data; S is a positive integer;

[0192] Obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

[0193] In some feasible implementations, the processor 901 may be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0194] The memory 902 may include read-only memory and random access memory, and provides instructions and data to the processor 901 and the input / output interface 903. A portion of the memory 902 may also include non-volatile random access memory. For example, the memory 902 may also store device type information.

[0195] In practice, the computer device can execute the implementation methods provided in the steps of Figure 3 or Figure 6 through its built-in functional modules. For details, please refer to the implementation methods provided in the steps of Figure 3 or Figure 6, which will not be repeated here.

[0196] This application embodiment provides a computer device including a processor, an input / output interface, and a memory. The processor retrieves a computer program from the memory and executes the steps of the method shown in Figure 4 to perform data transmission operations. This application embodiment implements responding to a data transmission request, retrieving first service data from the data transmission request, a first graphics processing unit for transmitting the first service data, and a second graphics processing unit for receiving the first service data. The first graphics processing unit is a graphics processing unit in a first service device. From S queue pairs associated with the first graphics processing unit, a first queue pair for transmitting the first service data is determined; S is a positive integer. The first hardware port corresponding to the first queue pair is obtained, and the first service data is transmitted to the second graphics processing unit in a second service device through the first hardware port and the first queue pair. Through the above process, data processing with multiple queue pairs is achieved. Each queue pair can correspond to a hardware port, enabling access to different hardware ports even when the service device has multiple hardware ports. This fully utilizes the hardware ports of the service device, thereby improving the hardware utilization and communication performance of the service device. Furthermore, using multiple queue pairs enables parallel data processing, allowing each hardware port to process data when multiple hardware ports exist, thus improving data transmission efficiency.

[0197] This application also provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed using the data transmission method provided in the steps of FIG4. Specific implementations of the steps in FIG4 are described below. Furthermore, the beneficial effects of using the same method are also not described in detail here. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application. As an example, the computer program can be deployed to execute on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.

[0198] The computer-readable storage medium can be a data transmission device provided in any of the foregoing embodiments or an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0199] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional modes shown in FIG4. This achieves data processing with multiple queue pairs, where each queue pair can correspond to a hardware port. This allows access to different hardware ports even when the service device has multiple hardware ports, thereby fully utilizing the hardware ports of the service device and improving its hardware utilization and communication performance. Furthermore, the use of multiple queue pairs enables parallel data processing, allowing each hardware port to perform data processing when multiple hardware ports are present, thus improving data transmission efficiency.

[0200] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0201] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0202] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0203] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowcharts and / or structural diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data transmission device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data transmission device, create means for implementing the functions specified in one or more blocks of the flowcharts and / or one or more blocks of the structural diagrams. These computer program instructions can also be stored in a computer-readable storage medium capable of directing a computer or other programmable data transmission device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more blocks of the flowcharts and / or one or more blocks of the structural diagrams. These computer program instructions may also be loaded onto a computer or other programmable data transmission device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable device, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks in the structural diagram.

[0204] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0205] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0206] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data transmission method, characterized by, The method is performed by a first service device, and the method includes: In response to a data transmission request, a first graphics processing unit obtains first service data from the data transmission request, sends the first service data, and a second graphics processing unit receives the first service data; the first graphics processing unit is a graphics processing unit in the first service device. From the S queue pairs associated with the first graphics processing unit, determine the first queue pair used to send the first service data; S is a positive integer; Obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

2. The method of claim 1, wherein, The method further includes: D first data channels are constructed through the first graphics processing unit; D is an integer multiple of S; Based on the channel numbers corresponding to the D first data channels and the S, the D first data channels are associated with the S queue pairs; each first data channel is associated with a queue pair, and each queue pair is associated with one or more first data channels; The step of determining the first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit includes: Based on the channel operation status of the D first data channels, the first graphics processing unit selects a first target data channel from the D first data channels and determines the queue pair associated with the first target data channel as the first queue pair for sending the first service data.

3. The method of claim 1, wherein, The step of determining the first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit includes: Based on the data volume of the first service data and S, determine the number of first channels and construct H second data channels; H is an integer multiple of S, and H is the number of first channels; The first business data is divided into H business sub-data, and a second data channel is allocated to each business sub-data. Based on the channel number of the second data channel corresponding to each service sub-data, a first queue pair for sending that service sub-data is determined.

4. The method of claim 1, wherein, The step of obtaining the first hardware port corresponding to the first queue pair includes: Obtain the network topology corresponding to the first service device, and the first queue identifier of the first queue pair; The network topology and the first queue identifier are hashed to obtain the target queue port number of the first queue pair; Obtain the P hardware ports associated with the first graphics processing unit. Based on the queue port parameters corresponding to the P hardware ports, determine the hardware port corresponding to the queue port parameter that matches the target queue port number as the first hardware port corresponding to the first queue pair; P is a positive integer.

5. The method of claim 4, wherein, The method further includes: Obtain the queue port numbers corresponding to the S queue pairs, and obtain A candidate classification parameters; A is a positive integer; Based on the A candidate classification parameters, the S queue port numbers are clustered to obtain P port number clusters corresponding to the A candidate classification parameters respectively; Based on the number of queue port numbers included in the P port number clusters corresponding to each candidate classification parameter, the classification difference degree corresponding to that candidate classification parameter is determined; the classification difference degree is used to represent the difference between the number of P port numbers of the corresponding candidate classification parameter. The candidate classification parameter with the smallest classification difference is determined as the target classification parameter, and queue port parameters are assigned to the P hardware ports based on the target classification parameter.

6. The method of claim 1, wherein, The first service device includes M hardware devices, where M is a positive integer; The step of sending the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair includes: Create a work queue element for the first business data, and add the work queue element to the first queue pair; A data transmission instruction is sent to the first device hardware associated with the first graphics processing unit, and the work queue element is obtained from the first queue pair through the first device hardware and the data transmission instruction; the M device hardwares include the first device hardware; The first device hardware sends the first business data corresponding to the work queue element to the second graphics processing unit in the second service device.

7. The method of claim 6, wherein, Adding the work queue element to the first queue pair includes: Obtain the enqueue position and dequeue position of the first queue pair in the first service device, and determine the number of data to be processed corresponding to the first queue pair in the first service device based on the enqueue position and the dequeue position. If the amount of data to be processed is less than the queue capacity of the first queue pair, then the work queue element is added to the first queue pair; If the amount of data to be processed is greater than or equal to the queue capacity of the first queue pair, the work queue element is cached until the amount of data to be processed is less than the queue capacity of the first queue pair, at which point the work queue element is added to the first queue pair.

8. The method of claim 1, wherein, The method further includes: When the transmission of the first service data is completed, first signaling data is generated for the first service data; From the S queue pairs, a second queue pair for sending the first signaling data is determined. The first signaling data is then sent to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair, so that the second graphics processing unit can obtain the first service data based on the first signaling data and detect the first service data.

9. The method of claim 8, wherein, The step of sending the first signaling data to the second graphics processing unit through the second queue pair and the second hardware port corresponding to the second queue pair includes: Generate a signaling queue element for the first signaling data, and add the signaling queue element to the second queue pair; When the signaling queue element is obtained from the second queue pair through the second hardware port corresponding to the second queue pair, the queue status of the second queue pair is updated to a locked state, and the first signaling data is sent to the second graphics processing unit; the locked state is used to indicate that the second queue pair cannot be accessed; When the first signaling data is successfully sent, the queue status of the second queue pair is updated to running status; the running status indicates that the second queue pair is allowed to be accessed.

10. The method of claim 1, wherein, The method further includes: Receive the second service data sent by the third service device through the third queue pair, and write the second service data into the third queue pair; When receiving second signaling data for the second service data, obtain the description parameters for the second service data from the second signaling data, and obtain the actual received parameters corresponding to the description parameters from the second service data; If the description parameter is the same as the actual received parameter, then the second service data is obtained from the third queue pair; If the description parameter is different from the actual received parameter, a data repair request is sent to the third service device based on the parameter difference between the description parameter and the actual received parameter.

11. The method of claim 1, wherein, The first service device includes M graphics processing units, wherein the M graphics processing units include the first graphics processing unit, and M is a positive integer; the method further includes: Obtain the unit number data corresponding to the M graphics processing units respectively, and add the M graphics processing units to the M unit groups based on the unit number data; In each unit group, the graphics processing unit of the first service device in that unit group is used to establish S queue pairs with other graphics processing units in that unit group; Map the S queue pairs corresponding to each unit group to the hardware port of that unit group.

12. A data transmission apparatus, characterized by comprising: The apparatus is adapted for a first service device, the apparatus comprising: The data acquisition module is used to respond to a data transmission request, acquire first service data from the data transmission request, send the first service data to a first graphics processing unit, and receive the first service data to a second graphics processing unit; the first graphics processing unit is a graphics processing unit in the first service device. The queue pair determination module is used to determine a first queue pair for sending the first service data from the S queue pairs associated with the first graphics processing unit; S is a positive integer; The data sending module is used to obtain the first hardware port corresponding to the first queue pair, and send the first service data to the second graphics processing unit in the second service device through the first hardware port and the first queue pair.

13. A computer device, comprising: Includes processor, memory, and input / output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program which is adapted to be loaded and executed by the processor to cause the computer device having the processor to perform the method of any one of claims 1-11.

15. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the method of any one of claims 1-11.

Images