Data transmission method and device, computer equipment, medium and product

Abstract

The invention discloses a data transmission method and apparatus, a computer device, a medium and a product. The method comprises the steps of obtaining a data processing task carrying service representation data; the service characterization data is sent to an expert model in the second graphic processing unit; based on N first queue pairs associated with the expert model, performing block division processing on the service representation data to obtain data division blocks of the service representation data; if a target first queue pair meeting the data sending condition exists in the N first queue pairs, sending the target first queue pair to a first network transmission unit, sending a target division block matched with the target first queue pair to a second graphic processing unit through a target first network port in the N first network ports by the first network transmission unit; the target first network port is determined by the first network transmission unit based on the communication link parameter of the target first queue pair. Through the method, flow load balancing of multiple network ports can be realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data transmission method, a data transmission device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Currently, in the low-latency mode provided by the high-performance communication library, each Graphics Processing Unit (GPU) can directly use a high-performance communication technology (for example, InfiniBand GPUDirect Async, abbreviated as IBGDA) to achieve data exchange.

[0003] However, the inventors have found in practice that in the process of sending data in a graphics processing unit (for example, GPU1) to an expert model deployed in another graphics processing unit (for example, GPU2), a single queue pair can be constructed for each expert model so that the network card corresponding to GPU1 can indiscriminately use the unified network port provided by itself to perform data transmission each time it needs to transmit data read from GPU1. This means that when the number of network ports provided by the network card corresponding to GPU1 (for example, a multi-port network card) is multiple, the bandwidth of other network ports among the multiple network ports except the same network port will be wasted. Therefore, how to avoid the waste of network bandwidth of the multi-port network card during data transmission and improve the load balancing of each network port in the multi-port network card is a technical problem that needs to be solved urgently. Summary of the Invention

[0004] The embodiments of the present application provide a data transmission method, apparatus, computer equipment, medium, and product. When sending service representation data from a data transmitting end to a data receiving end, multiple queue pairs can be configured for each expert model, thereby transmitting data through multiple queue pairs. At the same time, when using a network transmission unit to transmit data, the communication link parameters of the multiple queue pairs can be used to flexibly select network ports for data transmission, thereby avoiding waste of network bandwidth of a multi-port network card and evenly selecting network ports for data transmission, thereby achieving load balancing for each network port.

[0005] In one aspect, an embodiment of the present application discloses a data transmission method. The method is applied to a data transmission system. The data transmission system includes a data sending end and a data receiving end. The data sending end includes a first graphics processing unit and a first network transmission unit. The data receiving end includes a second graphics processing unit. The method is performed by the first graphics processing unit. The method includes:

[0006] Acquire a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in a second graphics processing unit;

[0007] Based on N first queue pairs associated with the expert model, block partitioning processing is performed on the service characterization data to obtain data partition blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1;

[0008] If a target first queue pair that meets the data transmission condition exists among the N first queue pairs, the target first queue pair is sent to the first network transmission unit, so that the first network transmission unit obtains a target partition block that matches the target first queue pair, and sends the target partition block to the second graphics processing unit through the target first network port among the N first network ports;

[0009] The second graphics processing unit is used to send the target partition block to the expert model, the target first network port is determined by the first network transmission unit from N first network ports based on the communication link parameters of the target first queue pair; the target partition block is a partition block in the data partition block.

[0010] In one aspect, an embodiment of the present application discloses a data transmission method. A data transmission system includes a data transmitting end and a data receiving end. The data transmitting end includes a first graphics processing unit and a first network transmission unit. The data receiving end includes a second graphics processing unit. The method is executed by the second graphics processing unit. The method includes:

[0011] Obtain a target partition block that matches a target first queue pair, wherein the target partition block is sent by the first network transmission unit through a target first network port among a plurality of first network ports provided by the first network transmission unit; the target partition block is a partition block that matches the target first queue pair obtained from the data partition block by the first network transmission unit after receiving the target first queue pair in N first queues sent by the first graphics processing unit; the target first queue pair is the first queue pair among the N first queue pairs that meets the data sending conditions; the data partition block is obtained by the first graphics processing unit performing block partitioning processing on the business characterization data according to the N first queue pairs associated with the expert model after obtaining a data processing task carrying business characterization data, each first queue pair being a queue pair in the N groups of queue pairs associated with the expert model, N being an integer greater than 1, and the data processing task including a task of sending the business characterization data to the expert model in the second graphics processing unit; the target first network port is determined by the first network transmission unit from the N first network ports based on the communication link parameters of the target first queue pair;

[0012] Determine a target second queue pair mapped to the target first queue pair from N second queue pairs associated with the expert model; each second queue pair is a queue pair mapped to the first queue pair in the N groups of queue pairs;

[0013] When sending end indication information associated with the target second queue is detected, it is determined that each of the divided data blocks sent as the target divided blocks has been obtained, and the business characterization data corresponding to the divided data blocks is sent to the expert model. The sending end indication information is obtained by the first graphics processing unit performing quantitative statistics on the sent business characterization data.

[0014] In one aspect, an embodiment of the present application discloses a data transmission device, which includes:

[0015] A task acquisition unit, configured to acquire a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in the second graphics processing unit;

[0016] a data partitioning unit, configured to perform block partitioning processing on the service characterization data based on N first queue pairs associated with the expert model to obtain data partitioned blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1;

[0017] a queue pair sending unit, configured to, if a target first queue pair that meets the data sending condition exists among the N first queue pairs, send the target first queue pair to the first network transmission unit, so that the first network transmission unit obtains a target partition block that matches the target first queue pair, and sends the target partition block to the second graphics processing unit through the target first network port among the N first network ports;

[0018] The second graphics processing unit is used to send the target partition block to the expert model, the target first network port is determined by the first network transmission unit for the communication link parameters of the target first queue pair from N first network ports; the target partition block is a partition block in the data partition block.

[0019] In one embodiment, when the data partitioning unit is used to perform the step of partitioning the service characterization data into blocks based on the N first queue pairs associated with the expert model to obtain data partition blocks of the service characterization data, it is specifically used to:

[0020] Obtaining a queue pair relationship mapping table maintained by the first graphics processing unit; the queue pair relationship mapping table includes at least N groups of queue pairs associated with the expert model; the N groups of queue pairs include N first queue pairs and second queue pairs mapped respectively to the N first queue pairs;

[0021] Obtain N first queue pairs from N groups of queue pairs;

[0022] Determine the number of queues of the N first queue pairs as the first number of queues, determine N initial partition blocks corresponding to the service characterization data based on the first number of queues, and determine the amount of partitioned data for each initial partition block; the amount of partitioned data for each initial partition block is determined based on the first number of queues and the amount of data of the service characterization data;

[0023] Based on the maximum transmission unit associated with the first network transmission unit and the amount of divided data of each initial divided block, block division processing is performed on each initial divided block to obtain sub-divided blocks of each initial divided block, and the sub-divided blocks of each initial divided block are determined as data divided blocks of the service characterization data.

[0024] In one embodiment, the first graphics processing unit and the second graphics processing unit are graphics processing units in a communication group of a data transmission system;

[0025] a second queue pair acquisition unit, configured to determine the number of ports of the N first network ports as the first port number, and acquire N first queue pairs pre-configured by the first graphics processing unit and matching the first port number from the communication group; and acquire N second queue pairs pre-configured by the second graphics processing unit for the expert model from the communication group;

[0026] A queue pair relationship construction unit is used to establish a queue pair mapping relationship between N first queue pairs and N second queue pairs, and based on the established queue pair mapping relationship, determine the second queue pairs mapped to the N first queue pairs respectively; based on each first queue pair and each second queue pair mapped to the first queue pair respectively, construct N groups of queue pairs associated with the expert model; based on the N groups of queue pairs, determine a queue pair relationship mapping table.

[0027] In one embodiment, the first network transmission unit is used to perform data encapsulation processing on the target partition block according to the transport layer protocol indicated by the data processing task after obtaining the target partition block, obtain a business data packet carrying the target partition block, and determine to transmit the business data packet through the target first network port based on the communication link parameters of the target first queue pair.

[0028] In one embodiment, the queue pair second obtaining unit is further configured to obtain a first queue pair i and a first queue pair j from the N first queue pairs; i and j are integers less than or equal to N, and i is not equal to j;

[0029] A configuration unit is used to configure a first control signal storage space for the first queue pair i; the first control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair i; a second control signal storage space is configured for the first queue pair j; the second control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair j; the space size of the second control signal storage space is consistent with the space size of the first control signal storage space.

[0030] In one embodiment, the N initial partition blocks include a first initial partition block and a second initial partition block; and the data partitioning unit, when performing the step of determining the amount of partitioned data for each initial partition block, is specifically configured to:

[0031] Calculate the ratio of the data volume of the service representation data to N;

[0032] If the ratio is an integer, it is determined that the data volume of the service characterization data is divisible by N, and the ratio is used as the data volume of the first initial partition block and the second initial partition block respectively;

[0033] If the ratio is not an integer, it is determined that the data volume of the business characterization data cannot be divided by N. The comparison value is rounded to obtain an integer value of the ratio. When the integer value is used as the data volume of the first initial partition block, the difference between the data volume of the business characterization data and the integer value is used as the data volume of the second initial partition block.

[0034] In one embodiment, the storage address of the first graphics processing unit is a first storage address, and the storage address of the second graphics processing unit is a second storage address; the first graphics processing unit stores a unit data volume of a maximum transmission unit associated with the first network transmission unit;

[0035] A first storage unit is configured to store the data partition blocks of the service representation data in a first storage space corresponding to a first storage address when the data partition blocks are obtained;

[0036] A queue element determination unit is configured to determine the queue element of each first queue pair based on the first storage address, the second storage address, and the block data volume of the data partition block; the queue element of each first queue pair is configured to instruct the first network transmission unit to obtain a target partition block that matches the queue element of the target first queue pair from the first storage space; the block data volume of the data partition block is determined based on the number of queues of the N first queue pairs, the data volume of the service characterization data, and the unit data volume of the maximum transmission unit.

[0037] In one embodiment, the data partition block includes at least a first partition block and a second partition block;

[0038] A thread calling unit is used to call a first business processing thread configured for a first partition block, determine a first storage address and a second storage address, and perform queue element encapsulation processing on the first storage address, the second storage address, and the block data volume of the first partition block through the first business processing thread to obtain a first queue element corresponding to the first partition block; call a second business processing thread configured for a second partition block, determine the first storage address and the second storage address, and perform queue element encapsulation processing on the first storage address, the second storage address, and the block data volume of the second partition block through the second business processing thread to obtain a second queue element corresponding to the second partition block; configure the first queue element to a target first queue pair through the first business processing thread, and configure the second queue element to the target first queue pair through the second business processing thread to obtain a queue element of the target first queue pair.

[0039] In one embodiment, the target first queue pair includes at least a first reserved position and a second reserved position for storing queue elements;

[0040] The first storage unit is used to write the first queue element to the first reserved position in the target first queue pair when the first queue element is configured to the target first queue pair through the first business processing thread; and to write the second queue element to the second reserved position in the target first queue pair when the second queue element is configured to the target first queue pair through the second business processing thread.

[0041] In one embodiment, when the queue pair sending unit is used to execute the step of sending the target first queue pair to the first network transmission unit, it is specifically used to:

[0042] Call the data transmission thread configured for the target first queue pair, encapsulate the queue elements in the target first queue pair as a data transmission request, and send the data transmission request to the first network transmission unit, so that the first network transmission unit executes the step of obtaining the target partition block matching the queue elements of the target first queue pair from the first storage space according to the data transmission request.

[0043] In one embodiment, the task acquisition unit is further configured to acquire a task type of the data processing task;

[0044] The notification unit is configured to notify the data partitioning unit to perform block partitioning processing on the business characterization data based on N first queue pairs associated with the expert model to obtain data partition blocks of the business characterization data if the task type is a target task type that can be processed with low latency.

[0045] In one embodiment, the data processing task further includes a model reasoning task for performing model reasoning using an expert model deployed in the hybrid business model; the model reasoning task refers to a task of reasoning on business sample data using the expert model;

[0046] The task acquisition unit is further used to obtain business sample data corresponding to the model reasoning task; and obtain business text information associated with the business sample data;

[0047] The text processing unit is used to perform text segmentation processing on the business text information to obtain the text segmentation of the business text information; perform vector representation processing on the text segmentation to obtain the text representation vector for the text segmentation, and use the text representation vector as the business representation data in the data processing task.

[0048] In one embodiment, the queue pairs in the N groups of queue pairs include N second queue pairs configured for the expert model; the indication information generating unit is configured to, after sending each of the data partition blocks associated with the service characterization data to the second graphics processing unit, call the streaming multiprocessor data processing task to count the number of service characterization data to be sent to the expert model; and generate sending completion indication information based on the counted number of service characterization data;

[0049] The indication information sending unit is used to send the sending end indication information to the second graphics processing unit, so that the second graphics processing unit stores the sending end indication information in the control signal storage space configured for the N second queue pairs.

[0050] In one aspect, an embodiment of the present application discloses a data transmission device, which includes:

[0051] a partition block acquisition unit configured to acquire a target partition block matching a target first queue pair, wherein the target partition block is sent by the first network transmission unit via a target first network port among a plurality of first network ports provided by the first network transmission unit; the target partition block is a partition block matching the target first queue pair acquired from the data partition block by the first network transmission unit after receiving the target first queue pair from the N first queues sent by the first graphics processing unit; the target first queue pair is the first queue pair among the N first queue pairs that meets the data sending conditions; the data partition block is obtained by the first graphics processing unit performing block partitioning processing on the service characterization data according to the N first queue pairs associated with the expert model after acquiring a data processing task carrying service characterization data, wherein each first queue pair is a queue pair among the N groups of queue pairs associated with the expert model, where N is an integer greater than 1, and the data processing task includes a task of sending the service characterization data to the expert model in the second graphics processing unit; the target first network port is determined by the first network transmission unit from the N first network ports based on the communication link parameters of the target first queue pair;

[0052] a first queue pair obtaining unit, configured to determine a target second queue pair mapped to the target first queue pair from N second queue pairs associated with the expert model; each second queue pair is a queue pair mapped to the first queue pair in the N groups of queue pairs;

[0053] The data sending unit is configured to, upon detecting sending end indication information associated with the target second queue, determine that each of the divided data blocks sent as the target divided blocks has been acquired, and send the service representation data corresponding to the divided data block to the expert model, wherein the sending end indication information is obtained by the first graphics processing unit performing quantitative statistics on the sent service representation data.

[0054] In one embodiment, the data receiving end further includes a second network transmission unit; and the partition block acquisition unit, when used to execute the step of acquiring a target partition block matching the target first queue pair, is specifically configured to:

[0055] receiving a service data packet carrying a target partitioned block sent by the second network transmission unit; the service data packet is obtained by encapsulating the target partitioned block by the first network transmission unit;

[0056] Decapsulate the service data packet to obtain the target partition block.

[0057] In one embodiment, when the switch device deployed between the second network transmission unit and the first network transmission unit is a switch, the second network transmission unit is used to receive a service data packet sent by the first network transmission unit through a target second network port determined by the switch, wherein the target second network port is determined by the switch from N second network ports provided by the second network transmission unit according to the communication link parameters corresponding to the service data packet when the switch receives the service data packet sent by the first network transmission unit through the target first network port.

[0058] In one embodiment, the target partition block is a partition block carried in a service data packet, and the service data packet further carries a queue element of a target first queue pair; the queue element of the target first queue pair is used to indicate a second storage address of the target partition block in the second graphics processing unit;

[0059] The second storage unit is configured to store the target partition block to a second storage address based on the queue element of the target first queue pair carried in the service data packet when the target partition block is obtained.

[0060] In one embodiment, when the data sending unit is configured to execute the step of, upon detecting the sending end indication information associated with the target second queue, determining that each of the divided data blocks sent as the target divided blocks has been acquired, and sending the service representation data corresponding to the divided data block to the expert model, the data sending unit is specifically configured to:

[0061] detecting, in a control signal storage space configured for the target second queue pair, transmission end indication information associated with the target second queue pair;

[0062] Upon detecting that there is sending end indication information in the control signal storage space, determining that each of the divided data blocks sent as the target divided blocks has been acquired, transmitting the sending end indication information to the expert model, and generating a data read request for reading the service representation data through the expert model;

[0063] The expert model is controlled to read the service representation data from the second storage space corresponding to the second storage address according to the data read request, where the second storage address is the storage address of the target partition block stored in the second graphics processing unit.

[0064] In one embodiment, the data processing task also includes a model reasoning task of performing model reasoning using an expert model deployed in the hybrid business model; the model reasoning task refers to a task of reasoning on business sample data using an expert model;

[0065] The sample data acquisition unit is used to acquire the business sample data corresponding to the business representation data, perform model inference processing on the business sample data through the expert model, and obtain the model inference results corresponding to the business sample data.

[0066] On the one hand, an embodiment of the present application discloses a computer device, which includes a processor suitable for implementing one or more computer programs; and a computer-readable storage medium, which stores one or more computer programs, and the one or more computer programs are suitable for being loaded and executed by the processor in accordance with the above-mentioned data transmission method.

[0067] One aspect of the present application discloses a computer-readable storage medium storing one or more computer programs, wherein the one or more computer programs are suitable for being loaded by a processor and executing the above-mentioned data transmission method.

[0068] In one aspect, an embodiment of the present application discloses a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the above-mentioned data transmission method.

[0069] In an embodiment of the present application, when the data transmission method is applied to a first graphics processing unit included in a data transmission end of a data transmission system, the first graphics processing unit obtains a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in a second graphics processing unit; based on N first queue pairs associated with the expert model, the service characterization data is partitioned into blocks to obtain data partition blocks of the service characterization data; each first queue pair is a queue pair in N groups of queue pairs associated with the expert model, where N is an integer greater than 1; if a target first queue pair exists among the N first queue pairs that meets data transmission conditions, the target first queue pair is sent to a first network transmission unit (the data transmission end includes the first network transmission unit). The first network transmission unit obtains a target partition block that matches the target first queue pair, determines a target first network port from N first network ports based on communication link parameters configured for the target first queue pair, and sends the target partition block to the second graphics processing unit (i.e., the data receiving end) via the target first network port. Using this method, when sending service characterization data from the data transmission end to the data receiving end, multiple queue pairs can be configured for each expert model, and data transmission is performed through these multiple queue pairs. When using a network transmission unit to send data, the communication link parameters of multiple queue pairs can be used to flexibly select network ports for data transmission, avoiding the waste of network bandwidth of multi-port network cards and evenly selecting network ports for data transmission, thereby achieving load balancing for each network port. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0071] Figure 1 is a schematic diagram of the network structure provided in an embodiment of the present application;

[0072] Figure 2 This is a specific data transmission system architecture diagram provided in an embodiment of the present application;

[0073] Figure 3 This is a flow chart of a data transmission method provided in an embodiment of the present application;

[0074] Figure 4 This is a schematic diagram of a queue element generation process provided by an embodiment of the present application;

[0075] Figure 5This is a schematic diagram of a queue pair establishment result provided in an embodiment of the present application;

[0076] Figure 6 This is a flow chart of another data transmission method provided in an embodiment of the present application;

[0077] Figure 7 This is a schematic diagram of a control signal storage space corresponding to sending end indication information provided by an embodiment of the present application;

[0078] Figure 8 This is a schematic diagram of another control signal storage space corresponding to sending end indication information provided by an embodiment of the present application;

[0079] Figure 9 This is a schematic diagram of network traffic test results provided by an embodiment of the present application;

[0080] Figure 10 This is a structural diagram of a data transmission device provided in an embodiment of the present application;

[0081] Figure 11 is a structural diagram of another data transmission device provided in an embodiment of the present application;

[0082] Figure 12 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0084] Before explaining the embodiments of the present application, the following terms related to the embodiments of the present application are explained:

[0085] 1. GPU (Graphics Processing Unit): A dedicated processor originally designed to efficiently process computer graphics rendering tasks (such as light and shadow calculations, texture mapping) to accelerate the generation of images, videos, and game screens. With the evolution of technology, its application has expanded to the field of general parallel computing (such as scientific simulations and AI training), becoming the core hardware for high-performance computing. In the embodiment of the present application, the application scenario is to use the GPU for model reasoning, and the model involved is a hybrid expert model, for which the GPU can provide parallel processing capabilities.

[0086] 2. Communication libraries: These libraries primarily implement various communication protocols and functions within computer programs, provide programming interfaces, simplify communication operations, and support multiple communication methods. For example, DeepEP is a communication library designed for Mixture of Experts (MoE) and Expert Parallelism (EP). Another example is NCCL (NVIDIA Collective Communications Library), a multi-GPU / multi-node communication library designed to optimize collective communication operations for distributed deep learning training.

[0087] In addition to the communication library described above, in an embodiment of the present application, a GPU cluster communication library is also involved, which greatly optimizes data communication and processing in the GPU cluster by unifying a global address space and supporting multiple types of operations. In an embodiment of the present application, based on the IBGDA method that supports multiple QPs provided by the GPU cluster communication library, the low-latency mode of DeepEP is successfully adapted to the network architecture of a multi-port network card (i.e., a network card integrating multiple network ports), such as a dual-port network card, a three-port network card, etc. The first storage space of the first graphics processing unit, the second storage space of the second graphics processing unit, and the control signal storage space (including the first control signal storage space and the second control signal storage space in the application) involved in the embodiment of the present application are all part of the above-mentioned global address space.

[0088] 3. QP (Queue Pair): QP is the basic unit of communication of RDMA (Remote Direct Memory Access) technology. For each node, for example, the node can be a computer device or server in the network, and each node usually has one or more network interface cards (NIC) for data transmission. In an embodiment of the present application, the node is a server in the network. For example, the server here can include a server as a data sending end and a server as a data receiving end. Among them, for ease of understanding, the embodiment of the present application can collectively refer to the GPU deployed in the data sending end as a first graphics processing unit, and the network interface card (NIC) deployed in the data sending end that can directly interact with the first graphics processing unit through remote direct memory access technology is collectively referred to as a first network transmission unit. Similarly, the embodiment of the present application can collectively refer to the GPU deployed in the data receiving end as a second graphics processing unit, and the network interface card (NIC) deployed in the data receiving end that can directly interact with the second graphics processing unit through remote direct memory access technology is collectively referred to as a second network transmission unit.

[0089] Regardless of whether the node is a computer device or a server, each node can run multiple processes (corresponding to data transmission threads). These processes communicate via RDMA. Each process can use several QPs, and each local QP can be "associated" with a remote QP. In the embodiment of the present application, a set of queue pairs is used to represent local and remote QPs, where the set of queue pairs includes a first queue pair and a second queue pair. The first queue pair is the local (i.e., data sending end) QP, and the second queue pair is the remote (i.e., data receiving end) QP.

[0090] 4. RDMA: Remote Direct Memory Access, remote direct memory access, is a network communication technology that allows computers to directly access the memory of a remote server without CPU involvement. Its core goal is to achieve low-latency, high-throughput cross-node data transmission by eliminating data duplication and reducing kernel state switching. In the embodiments of the present application, the remote server may refer to the server corresponding to the data receiving end. When the data sending end and the data receiving end are different devices, the GPUs at both ends can directly transmit data via RDMA.

[0091] 5. IBGDA: InfiniBand GPUDirect Async is a communication acceleration technology for GPU clusters. It directly initiates InfiniBand network operations through the GPU, bypassing the CPU, reducing communication latency and energy consumption in AI training and large-model inference. Its core goal is to eliminate the dependence on the CPU in traditional GPU communication and improve the efficiency of heterogeneous computing clusters. In the embodiments of this application, IBGDA is applied to large-model training scenarios to optimize the all-to-all communication of the Mixed Experts Model (MoE) and improve communication efficiency.

[0092] 6. GDR: GPU Direct RDMA, GPU Direct Remote Direct Memory Access, is a technology that enables direct communication between GPU video memory and remote devices (such as network cards, other GPUs or storage devices) through the RDMA network protocol. The core goal is to bypass the transit between the CPU and host memory, reduce data transmission latency and improve throughput. This technology improves GPU communication efficiency to the microsecond level through hardware offload and protocol optimization, and is suitable for AI training, HPC (High performance computing) and real-time reasoning scenarios. In an embodiment of the present application, the first graphics processing unit writes the target partition block to the storage space of the second graphics processing unit, which is achieved through GDR technology; so that the expert model in the second graphics processing unit can realize real-time reasoning of the model by obtaining the business sample data corresponding to the target partition block from the storage space.

[0093] 7. UDP: User Datagram Protocol, User Datagram Protocol, is a connectionless protocol at the transport layer. It is designed for lightweight, low-latency data transmission and is suitable for scenarios with high real-time requirements but high tolerance for reliability.

[0094] 8. MoE (Mixture of Experts): This is an architectural design that achieves efficient computing and high performance by dynamically combining multiple sub-models (expert networks). Its core goal is to reduce computing costs while maintaining the scale of model parameters. It is particularly suitable for large-scale language models (LLM) and multimodal scenarios. In the training and inference process of large AI models, the MoE model plays an important role in efficiently processing and optimizing model performance. The MoE model decomposes complex problems into multiple sub-tasks, which are processed separately by multiple specialized "expert" models. Finally, the output of these experts is combined to achieve efficient and accurate predictions or decisions.

[0095] The embodiments of the present application are mainly used for the All-to-All communication optimization of the low-latency mode required in the inference decoding stage of the MoE model. The All-to-All communication is to ensure that the input data (such as the business sample data in the data processing task in this application) can be correctly distributed to each expert model, and the output results of the expert model (such as the model reasoning results in this application) are correctly aggregated. The two key steps of All-to-All communication are dispatch and combine. Among them, the purpose of dispatch is to distribute the input data to different expert models for processing, and the purpose of combine is to merge the output results of each expert model back into a complete output.

[0096] 9. Experts (i.e., expert models): Independent subnetworks in the MoE model that specialize in processing specific input patterns (such as small fully connected layers or Transformer modules) are dynamically selected and activated through the routing mechanism. In the embodiment of the present application, multiple experts (expert models) are distributed for each GPU in the GPU cluster (i.e., the GPU cluster corresponding to the communication group mentioned in the embodiment of the present application), and all expert models constitute the MoE model.

[0097] 10. src port: The source port number is an unsigned integer (range 0-65535) used to identify the sender in network communications. Together with the destination port number, it constitutes the endpoint identifier of the communication session. The source port number, combined with the source IP address, destination IP address, destination port number, and transport layer protocol, forms a quintuple that uniquely identifies a network communication link (i.e., the communication link parameters in the embodiments of this application). In the embodiments of this application, the network card can use this quintuple to determine the network port to which data is sent, achieving load balancing across multiple network ports.

[0098] 11. MTU: Maximum transmission unit. The maximum transmission unit refers to the maximum data block size that the data link layer can carry in network communications, usually in bytes. The value of MTU varies in different network types. For example, the default MTU of Ethernet is 1500 bytes. When the data to be transmitted exceeds the MTU, the data needs to be divided into multiple smaller data packets, a process called fragmentation. After receiving these fragments, the receiver reassembles them into the original data. The size of MTU affects the efficiency and performance of network transmission. If the MTU is set too large, it may cause excessive fragmentation, increase network overhead and transmission delay; if it is set too small, although the possibility of fragmentation is reduced, the number of data packets will increase, which will also affect transmission efficiency. In the embodiment of the present application, the MTU is associated with the network card (i.e., the network transmission unit in the embodiment of the present application), that is, each network card has a default value for the MTU, and the MTU of different network cards can be the same or different. Specifically, the MTU is configured for the network transmission unit through an external device. For example, the maximum transmission unit can be the default configuration parameter of the network transmission unit when it leaves the factory. If the maximum transmission unit needs to be adjusted, the network transmission unit can be configured and updated through an external device.

[0099] 12. Bond port: A technology that bonds multiple physical network interfaces into a single logical interface to improve bandwidth, redundancy, or load balancing.

[0100] 13. Token (Business Characterization Data): In this embodiment, a token refers to business characterization data. For example, a business sample data is "I like eating apples." 'I' like eating' and 'apples' are both the original form of a token (i.e., text segmentation). Each text segmentation is then vectorized to obtain the corresponding token, i.e., business characterization data.

[0101] In a feasible embodiment, when the data transmission method is implemented based on DeepEP, DeepEP can provide a low-latency mode and a standard mode. In low-latency mode, when data is transmitted between any two GPUs in a GPU communication group, the embodiment of the present application proposes that GDR (a GPU direct remote direct memory access) technology can be used between the two GPUs to send the token (i.e., business characterization data) stored in one GPU video memory (e.g., the storage space of the local GPU) directly to another GPU (e.g., the remote GPU) where the target expert model is located through the RDMA network protocol. For example, in an embodiment of the present application, the local GPU can determine, based on the currently acquired data processing task, that the token (i.e., business characterization data) currently stored in its own storage space needs to be divided into blocks, and different network ports mapped by different queues are used to send it to the remote GPU in accordance with the GDR technology, so as to bypass the transit of the CPU and host memory in low-latency mode to reduce the transmission delay during the data transmission process, and improve the throughput during data transmission through the different network ports of the multi-port network card mapped by the local GPU. In standard mode, when data is transmitted between any two GPUs in a GPU communication group, the sending GPU sends a token (i.e., service representation data) via the RDMA network protocol to the video memory of the proxy GPU (a proxy GPU is a GPU with the same logical number as the sending GPU on the device where the remote GPU is located). The proxy GPU then sends the token to the GPU where the target expert model is located (e.g., the remote GPU). Since low-latency mode involves direct GPU communication, eliminating the need for an intermediate proxy, data transmission is faster and less time-consuming in low-latency mode. However, standard mode requires an intermediate proxy, so it takes slightly longer. Therefore, if low latency (delay) is required, low-latency mode can be used; if not, standard mode can be used.

[0102] See Figure 1 , Figure 1 Schematic diagram of the network structure provided by the embodiment of the present application, which can be applied to the data transmission system involved in the embodiment of the present application. Figure 1As shown in FIG, the data transmission system is composed of a cluster of computer devices (such as device 11a, device 12a, device 13a and device 14a). Figure 1 It can be seen that the computer device can be a terminal device (such as device 11a, device 12a, device 13a) or a server (such as device 14a). It is understandable that Figure 1 The number of terminal devices and servers in the example is only for illustration. Any number of terminal devices and servers can be used according to the implementation requirements. In communication transmission, any computer device can be used as a network node, and any two nodes can be connected to each other for data transmission. Figure 1 As shown, any two computer devices in the data transmission system can communicate with each other through a network (i.e., a medium providing a communication link through a wired or wireless communication link or fiber optic cable, etc.), thereby performing data transmission. For example, device 11a can communicate with device 12a, and device 11a can also communicate with device 14a.

[0103] In one embodiment, any computer device in the data transmission system includes multiple GPUs, and the GPUs can communicate with each other via RDMA. The communication can be between GPUs in the same computer device or between GPUs across computer devices. For example, GPU1 in device 12a can communicate with GPU2 in device 13a.

[0104] It is understood that the terminal devices (such as device 11a, device 12a, and device 13a) may include, but are not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, smart home appliances, etc., and are not limited here. The server (such as device 14a) may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms, and is not limited here.

[0105] It should be understood that when the data transmission method involved in the embodiment of the present application is applied to the above-mentioned data transmission system, the corresponding data transmission system can also be as follows: Figure 2 As shown, Figure 2This is a specific data transmission system architecture diagram provided by an embodiment of the present application. The architecture diagram includes a data sending end (also known as a sending end) and a data receiving end (also known as a receiving end). The data sending end includes a first graphics processing unit (such as GPU1) and a first network transmission unit (such as network card 1), and the data receiving end includes a second graphics processing unit (such as GPU2) and a second network transmission unit (such as network card 2). Assuming that GPU1 and GPU2 are GPUs on different computer devices, and multiple expert models are deployed in GPU1 and GPU2 respectively, when the data transmission method provided by the embodiment of the present application is applied to Figure 2 In the data transmission system shown, the corresponding data transmission process may be as follows: the first graphics processing unit obtains a data processing task carrying service characterization data, where the data processing task includes a task of sending the service characterization data to an expert model in the second graphics processing unit (simply understood, the task is for GPU1 to send the service characterization data to the expert model in GPU2); based on N first queue pairs associated with the expert model, the service characterization data is partitioned to obtain data partition blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1; if a target first queue pair that meets the data transmission conditions exists among the N first queue pairs, the target first queue pair is sent to the first network transmission unit; the first network transmission unit obtains a target partition block that matches the target first queue pair from the partition blocks included in the data partition block, determines a target first network port from the N first network ports based on the communication link parameters configured for the target first queue pair, and sends the target partition block to the second graphics processing unit via the target first network port; after receiving the target partition block, the second graphics processing unit sends the target partition block to the expert model.

[0106] like Figure 2 As shown, taking a business characterization data token as an example, the following steps may be included: S1: the first graphics processing unit obtains the business characterization data token; S2: the first graphics processing unit performs block division processing on the token based on N first queues to obtain a data division block including k division blocks. Figure 2As shown, every two partition blocks correspond to a first queue pair, and there are N first queue pairs in total. S3: The first graphics processing unit determines whether there is a target first queue pair that meets the data sending conditions among the N first queue pairs. If so, execute S4: determine the target first queue pair; if not, repeat steps S1-S3, that is, continue to divide the second token (there is no first queue pair that meets the data sending conditions, which can be understood as the number of queue elements in the first queue pair is insufficient); or, optionally, there is no target first queue pair that meets the data sending conditions among the N first queue pairs, then it means that the data processing task corresponding to the token is wrong, and the data processing task can be directly ended. S5: The first graphics processing unit will target the first queue (specifically, the queue element in the target first queue pair, that is Figure 2 The target first queue pair includes a plurality of queue elements (i.e., WQEs) and sends them to the first network transmission unit. S6: The first network transmission unit obtains a target partition block from the first graphics processing unit using the queue elements in the target first queue pair. S7: The first network transmission unit determines a target network port based on the target first queue pair. S8: The first network transmission unit sends the target partition block to the second graphics processing unit via the target first network port.

[0107] Furthermore, the first network transmission unit transmits the target partition block to the second network transmission unit of the second graphics processing unit via the target first network port. The second network transmission unit then transmits the received target partition block to the second graphics processing unit. After the entire service representation data corresponding to the target partition block is completely transmitted to the second graphics processing unit, the expert model reads the service representation data including the target partition block from the storage space of the second graphics processing unit, so that the expert model performs model inference based on the service sample data corresponding to the service representation data.

[0108] It should be noted that the collection and processing of relevant data in this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0109] In conjunction with the above data transmission system, the data transmission method proposed in the embodiment of the present application will be described in detail below. Figure 3, is a flow chart of a data transmission method provided by an embodiment of the present application. The data transmission method is executed by a sending end, which includes a first graphics processing unit and a first network transmission unit. The first graphics processing unit and the first network transmission unit are integrated into a computer device. The computer device can be a terminal device or a server. The recipient of the data sent by the data sending end is a data receiving end. The data receiving end includes a first graphics processing unit and a first network transmission unit. The data receiving end and the data sending end can be the same computer device or different computer devices. Under this premise, the data transmission method includes but is not limited to the following steps:

[0110] S301: Acquire a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in a second graphics processing unit.

[0111] In some feasible embodiments, in addition to the task of sending the business characterization data to the expert model in the second graphics processing unit, the data processing task may also include a model reasoning task of using the expert model deployed in the hybrid business model to perform model reasoning. The model reasoning task refers to the task of reasoning on business sample data through the expert model, wherein the business sample data refers to the sample data used to instruct the expert model in the hybrid expert model to perform model reasoning in the model reasoning task. It should be understood that the data processing task may include multiple model reasoning tasks, and one model reasoning task needs to be sent to an expert model for model reasoning. It should be understood that model reasoning in the model reasoning task is another core process in the AI ​​model (including the large language model) in addition to model training, wherein the model training process optimizes the model performance by adjusting the model parameters, and the model reasoning process uses the trained model for prediction.

[0112] For any one of the model inference tasks, the process of the first graphics processing unit processing the model inference task includes: obtaining business sample data corresponding to the model inference task; the business sample data and the model inference task are matched. In the large language model, if the business sample data includes text-type media data (for example, text data), the text-type business sample data can be given to the large language model as business text information. Optionally, if the business sample data includes other types of media data (for example, audio data or video data), the other types of media data can be converted into text-type media data (for example, converted text data, where the converted text data can specifically include speech-converted text data corresponding to the audio data or video-associated text data corresponding to the video data). For example, if the business sample data is audio data, the audio data can be converted into speech-conversion text data through speech-to-text technology. For another example, if the business sample data is video data, the video-related text data associated with the video data (for example, the dialogue text, the plot description text, and the speech-conversion text corresponding to the audio data corresponding to the video data, etc.) can be obtained, and then the video-related text data associated with the video data can be used as business text information. Based on this, the embodiment of the present application can further perform text segmentation processing on the business text information when the business text information associated with the business sample data is obtained to obtain the text segmentation of the business text information. Further, the text segmentation is vectorized to obtain a text representation vector for the text segmentation, and the text representation vector is used as the business representation data in the data processing task.

[0113] For example, a business sample data may be "I like eating apples", where 'I' like' eating' and 'apples' are the original forms of tokens (i.e., text segmentation). Each text segmentation is then vectorized to obtain business representation data. Correspondingly, the business sample data includes 4 tokens (business representation data).

[0114] In one feasible embodiment, upon acquiring a data processing task, the first graphics processing unit needs to simultaneously obtain the task type of the data processing task. If the task type is a target task type that can be processed with low latency, step S302 is executed. Alternatively, if the task type is a target task type that does not require low latency processing, the standard mode is used (the execution process of the standard mode has been described above and will not be detailed here).

[0115] The expert model mentioned in the above process is the expert model in the second graphics processing unit. Optionally, in addition to the second graphics processing unit, other graphics processing units may also be included. These graphics processing units (including the first graphics processing unit and the second graphics processing unit) form a communication group, wherein any graphics processing unit can serve as a data sender or a data receiver. It can also be understood that the first graphics processing unit can transmit data to any other graphics processing unit in the communication group except the first graphics processing unit. The above-mentioned business sample data can be sent to multiple expert models at the same time. The multiple expert models can be expert models in the same graphics processing unit or in different graphics processing units. This application is not limited to this. In the embodiment of the present application, the data transmission method provided in the embodiment of the present application is explained by taking the first graphics processing unit sending business characterization data to an expert model in the second graphics processing unit as an example (in actual application scenarios, the business characterization data sent to an expert model are usually multiple).

[0116] S302: Based on N first queue pairs associated with the expert model, block division processing is performed on the service characterization data to obtain data division blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1.

[0117] In some feasible embodiments, after the first graphics processing unit determines that the task type of the data processing task is a target task type that can be processed with low latency, it obtains a queue pair relationship mapping table maintained by the first graphics processing unit; the queue pair relationship mapping table includes at least N groups of queue pairs associated with the expert model; the N groups of queue pairs include N first queue pairs and second queue pairs mapped to the N first queue pairs respectively; N first queue pairs are obtained from the N groups of queue pairs; because the queue pair relationship mapping table includes N groups of queue pairs of each expert model, the N first queue pairs of the model in the first graphics processing unit can be directly obtained based on the expert model in the data processing task. It should be clarified that the first queue pair and the second queue are a group of queue pairs with a logical association. In order to clearly distinguish between receiving and sending, they are described as the first queue pair and the second queue pair, wherein the first queue pair is the queue pair at the sending end and the second queue pair is the queue pair at the data receiving end. The number of queues in the N first queue pairs is determined as the number of first queues. Based on the number of first queues, N initial partition blocks corresponding to the service characterization data are determined, and the amount of partitioned data in each initial partition block is determined. The amount of partitioned data in each initial partition block is determined based on the number of first queues and the amount of service characterization data. For example, if an expert model is configured with two first queue pairs, two initial partition blocks for the service characterization data need to be determined.

[0118] In order to clearly explain how the amount of divided data for each initial partition is determined, the embodiment of the present application is explained using two initial partition blocks as an example. If the two initial partition blocks are the first initial partition block and the second initial partition block, the amount of divided data for each initial partition block is determined, including: calculating the ratio of the data amount of the business characterization data to N; if the ratio is an integer, determining that the data amount of the business characterization data is divisible by N, and using the ratio as the divided data amount of the first initial partition block and the second initial partition block respectively; if the ratio is a non-integer, determining that the data amount of the business characterization data is not divisible by N, rounding the ratio to obtain an integer value of the ratio; when the integer value is used as the divided data amount of the first initial partition block, the difference between the data amount of the business characterization data and the integer value is used as the divided data amount of the second initial partition block.

[0119] For example, if the amount of service characterization data is T bytes, corresponding to two first queue pairs, then two initial partition blocks are used. If T / 2 is an integer, then T / 2 is used as the data size of the two initial partition blocks. If T / 2 is not an integer, such as T is 413, then 413 / 2 is 206 with a remainder of 1. In this case, 206 is used as the data size of one initial partition block, and 207 (i.e., 413 - 206) is used as the data size of the other initial partition block.

[0120] For another example, if the amount of service characterization data is T bytes, corresponding to three first queue pairs, there are three corresponding initial partitions. If T / 3 is an integer, then T / 3 is used as the data size for the three initial partitions. If T / 3 is not an integer, for example, if T is 934, then 934 / 2 is 311 with a remainder of 1. In this case, 311 is used as the data size for any two of the three initial partitions, and 312 is used as the data size for the remaining initial partition. For another example, if T is 935, then 934 / 2 is 311 with a remainder of 2. In this case, 311 is used as the data size for any two of the three initial partitions, and 313 is used as the data size for the remaining initial partition. However, to ensure balanced data distribution, 311 can be used as the data size for any one of the three initial partitions, and the data size for the remaining two initial partitions can be determined as 312. This way, the data size for each initial partition is more balanced.

[0121] Furthermore, based on the unit data volume of the maximum transmission unit (MTU) associated with the first network transmission unit (the MTU is configured for the network transmission unit via an external device; for example, the MTU can be the default configuration parameter of the network transmission unit at the factory. If the MTU needs to be adjusted, the network transmission unit can be configured and updated via the external device), and the partitioned data volume of each initial partitioned block, each initial partitioned block is subjected to block partitioning to obtain sub-partitioned blocks of each initial partitioned block, and the sub-partitioned blocks of each initial partitioned block are determined as data partitioned blocks for service characterization data. A data partitioned block herein is a partitioned block set representing all partitioned blocks representing a service characterization data, including partitioned blocks obtained by partitioning each initial partitioned block. During data transmission, the greater the actual data volume carried by a data packet, the higher the communication efficiency. Therefore, in embodiments of the present application, the MTU is used for partitioning to improve communication efficiency and reduce overhead and delay during transmission. The unit data volume of the MTU can be stored in the first graphics processing unit or obtained by the first graphics processing unit in real time from the first network transmission unit.

[0122] In a feasible embodiment, taking two first queue pairs as an example, there are correspondingly two initial partition blocks. Assuming that the data volume of the service characterization data is T, and T / 2 is an integer, for each initial partition block, the number of partition blocks num_chunks_per_qp of the initial partition block can be determined based on the maximum transmission unit (MTU), that is, num_chunks_per_qp = (T / 2) / MTU. Correspondingly, the block data volume of each partition block in the initial partition block = (T / 2) / num_chunks_per_qp. Similarly, the block data volume of each partition block is also determined based on the result of (T / 2) / num_chunks_per_qp. If (T / 2) / num_chunks_per_qp is an integer, the block data volume of each partition block is the same. If (T / 2) / num_chunks_per_qp is not an integer, the data can be evenly divided based on the result.

[0123] In one feasible embodiment, when determining a data partition block for service characterization data, the data partition block is stored in a first storage space corresponding to a first storage address, so that during data transmission, data is retrieved from the first storage space for transmission. Simultaneously, a queue element for each first queue pair is determined based on the first storage address, the second storage address, and the block data volume of the data partition block. The storage address of the first graphics processing unit is the first storage address, and the storage address of the second graphics processing unit is the second storage address. The queue element of each first queue pair is used to instruct the first network transmission unit to retrieve a target partition block that matches the queue element of the target first queue pair from the first storage space. The block data volume of each partition block in the data partition block is determined based on the number of queues in the N first queue pairs, the data volume of the service characterization data, and the unit data volume of the maximum transmission unit. The target partition block is typically composed of multiple partition blocks. For example, if the target first queue pair includes three queue elements, the target partition block is composed of the partition blocks corresponding to these three queue elements.

[0124] Specifically, if a business representation data is divided into multiple partitions, queue elements are generated by multiple business processing threads. Taking the data partitions including the first partition and the second partition as an example, the process of generating the queue element may include: calling the first business processing thread configured for the first partition, determining the first storage address and the second storage address, and performing queue element encapsulation processing on the first storage address, the second storage address, and the block data volume of the first partition by the first business processing thread to obtain a first queue element corresponding to the first partition; at the same time, calling the second business processing thread configured for the second partition, determining the first storage address and the second storage address, and performing queue element encapsulation processing on the first storage address, the second storage address, and the block data volume of the second partition by the second business processing thread to obtain a second queue element corresponding to the second partition. The first storage address indicates the storage location of the first partition and / or the second partition at the sending end (i.e., the first graphics processing unit), and the second storage address indicates the storage location of the first partition and / or the second partition at the data receiving end (i.e., the second graphics processing unit). The first queue element is configured to the target first queue pair through the first business processing thread, and the second queue element is configured to the target first queue pair through the second business processing thread to obtain the queue element of the target first queue pair.

[0125] Specifically, when configuring the first queue element and the second queue element to the target first queue pair, it is necessary to first obtain the reserved positions of each queue element. When the number of partition blocks of each initial partition block is determined, positions are synchronously reserved for these partition blocks to write the queue elements. For example, for the first partition block and the second partition block, the first graphics processing unit will reserve a first reserved position for storing the first queue element and a second reserved position for storing the second queue element in the target first queue pair. When the first queue element is configured to the target first queue pair through the first business processing thread, the first queue element is written to the first reserved position in the target first queue pair; when the second queue element is configured to the target first queue pair through the second business processing thread, the second queue element is written to the second reserved position in the target first queue pair. Among them, the first reserved position and the second reserved position both refer to the positions in the sending queue of the first queue pair. It should also be understood that the order of writing to the reserved positions is related to the order of generating queue elements, that is, the first ones to be achieved are written first to ensure the positive order of the divided blocks and avoid data disorder. If disorder occurs, the data sent to the other end (that is, the data receiving end) will also be disordered, thereby affecting the reasoning effect of the model.

[0126] like Figure 4 As shown, it is a schematic diagram of the generation process of a queue element provided by an embodiment of the present application, from Figure 4 It can be seen that a business representation data is divided into four blocks (i.e., four chunks). Chunk0 and chunk1 are used to generate WQE0 and WQE1, and chunk2 and chunk3 are used to generate WQE2 and WQE3. During the generation process, multiple threads are called to execute WQE generation and writing. For example, call thread0 (business processing thread 0) to generate WQE0 based on the block data volume of chunk0 and the storage address of chunk0, and write WQE0 into the slot corresponding to QP0 (the first queue pair 0) (that is, the reserved position), call thread1 to generate WQE1 based on the block data volume of chunk1 and the storage address of chunk1, and write WQE1 into the slot corresponding to QP0; call thread2 to generate WQE2 based on the block data volume of chunk2 and the storage address of chunk2, and write WQE2 into the slot corresponding to QP1 (the first queue pair 1), call thread3 to generate WQE3 based on the block data volume of chunk3 and the storage address of chunk3, and write WQE3 into the slot corresponding to QP1. Figure 4 It can be seen that the use of multi-threaded processing can improve the generation efficiency of data (ie, WQE) to achieve efficient data transmission.

[0127] The queue pair relationship mapping table mentioned above is determined when the communication group is initialized after the communication group is determined. That is, each graphics processing unit in the communication group will maintain a queue pair relationship mapping table. Taking the first graphics processing unit as an example, the specific process of the corresponding queue pair relationship mapping table is as follows: obtain N first network ports of the first network transmission unit corresponding to the first graphics unit, determine the port number of the N first network ports as the first port number, obtain N first queue pairs pre-configured by the first graphics processing unit from the communication group that match the first port number; that is, the number of queue pairs corresponds to the number of network ports. At the same time, obtain N second queue pairs pre-configured by the second graphics processing unit for the expert model from the communication group, establish queue pair mapping relationships between the N first queue pairs and the N second queue pairs, and based on the established queue pair mapping relationships, determine the second queue pairs mapped to the N first queue pairs respectively; based on each first queue pair and each second queue pair mapped to the first queue pair, construct N groups of queue pairs associated with the expert model, and determine the queue pair relationship mapping table based on the N groups of queue pairs.

[0128] Simply put, for each expert model, the number of queue pairs created is associated with the network ports included in the network transmission unit corresponding to the graphics processing unit. A queue pair relationship mapping table corresponds to a graphics processing unit, and a queue pair relationship mapping table includes the queue pair mapping relationship between the graphics processing unit (that is, the network transmission unit corresponding to the graphics processing unit, and the network card or communication entity) and the expert models in other graphics processing units. In one embodiment, the number of first queue pairs in each GPU is determined based on the number of expert models in the communication group, the number of GPUs in the communication group, and the number of network ports associated with each GPU. Specifically, if there are x experts and n GPUs in the communication group, and the number of network ports of the network card corresponding to each GPU is 2, then the number of first queue pairs that each GPU needs to establish is: 2[x-(x / n)] QPs (each communication entity (GPU) will not establish a QP with itself).

[0129] For example, the number of expert models is x = 256, the number of GPUs is n = 16 GPUs, and the number of network ports of the network card corresponding to each GPU is 2. Then there will be 16 experts on each GPU, and 2*15*16=480 QPs (first queue pairs) will be established on each GPU (that is, the corresponding network card). At the same time, 480 second queue pairs are also included. The second queue pairs are queue pairs that have a queue pair mapping relationship with the first queue pairs, that is, queue pairs directly associated with each expert model at the opposite end (data receiving end, such as the second graphics processing unit).

[0130] For example, if there are two GPUs in the communication group, one GPU includes four expert models, and the number of network ports of the network card corresponding to one GPU is two, then eight QPs will be established on each GPU, that is, eight queue pairs, including eight first queue pairs and eight second queue pairs, as shown in Table 1:

[0131] Table 1

[0132]

[0133] As can be seen from Table 1, each expert model maintains two groups of queue pairs (the first group of queue pairs and the second group of queue pairs). For example, expert model 1 maintains two groups of queue pairs, where the first queue pair is QP11 and QP21, and the second queue pair on the opposite end is QP12 and QP22.

[0134] like Figure 5 As shown, taking the number of network ports of the network card as 2 as an example, when the number of expert models is x and the number of GPUs is n, the queue pairs configured on each GPU are as follows Figure 5 As shown in the figure, each expert model in each GPU is configured with two QPs. For example, Expert 1 in GPU0 is configured with QP0 and QP1. If GPU1 needs to send a token to the expert model (Expert 1) in GPU0, it can send it through the two queue pairs QP0 and QP1.

[0135] In a feasible embodiment, the process of initializing the queue pairs in the queue pair relationship mapping table also includes the configuration of a control signal storage space, which is used to store a control signal. The control signal is a transmission end indication information used to indicate that the data transmission of the transmitting end is completed. Taking the first queue pair i and the first queue pair j as an example, the specific creation process includes: configuring a first control signal storage space for the first queue pair i; the first control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair i; configuring a second control signal storage space for the first queue pair j; the second control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair j; the space size of the second control signal storage space is consistent with the space size of the first control signal storage space.

[0136] In this embodiment of the present application, the GPU cluster communication library can be directly used within the graphics processing unit to allocate memory with a symmetrical address space. In addition to allocating data storage space based on the GPU cluster communication library, two equally sized control signal storage spaces can also be allocated to store control signals for each queue. It should be noted that if a service representation data requires three QPs to send, three equally sized control signal storage spaces can be determined.

[0137] S303: If there is a target first queue pair that meets the data sending conditions among the N first queue pairs, the target first queue pair is sent to the first network transmission unit, so that the first network transmission unit obtains a target partition block that matches the target first queue pair, and sends the target partition block to the second graphics processing unit through the target first network port among the N first network ports.

[0138] The second graphics processing unit is used to send the target partition block to the expert model, the target first network port is determined by the first network transmission unit from N first network ports based on the communication link parameters of the target first queue pair; the target partition block is a partition block in the data partition block.

[0139] In some feasible embodiments, the first graphics processing unit can detect the number of queue elements included in each first queue pair in real time. When the number of queue elements included in the target first queue pair among the N first queue pairs reaches a set queue element threshold, indicating that the target first queue pair meets the data sending conditions, the first graphics processing unit can send the target first queue pair (specifically, the queue elements included in the target first queue pair) to the first network transmission unit. Furthermore, when the first graphics processing unit sends the queue elements included in the target first queue pair to the first network transmission unit, it does so by calling the data transmission thread configured for the target first queue pair, encapsulating the queue elements in the target first queue pair into a data transmission request, and sending the data transmission request to the first network transmission unit. Then, the first network transmission unit responds to the data transmission request and obtains the target partition block that matches the queue elements of the target first queue pair from the first storage space based on the queue elements in the target first queue pair. During this process, when the queue elements in the target first queue pair reach the queue element threshold of the device, the first graphics processing unit sends a trigger operation (corresponding to a notification mechanism, i.e., Ring Doorbell) to the first network transmission unit. After receiving the ring operation, the first network transmission unit notifies the first graphics processing unit that the queue elements in the target first queue pair can be sent over.

[0140] Among them, it should be noted that if there are multiple first queue pairs among the N first queue pairs that meet the data sending conditions, the first graphics processing unit can simultaneously call multiple data transmission threads to send data transmission requests to the first network transmission unit, so that the first network transmission unit can simultaneously receive multiple data transmission requests and call multiple data acquisition threads to obtain data from the first storage space in parallel, thereby improving data acquisition efficiency and thus improving data transmission efficiency.

[0141] In one feasible embodiment, the first network transmission unit is configured to, after obtaining the target partition block, perform data encapsulation processing on the target partition block according to the transport layer protocol indicated by the data processing task to obtain a service data packet of the target partition block; the first network transmission unit is further configured to, when determining the service data packet of the target partition block, determine the five-tuple carried in the packet header of the service data packet as the communication link parameters of the target first queue pair that matches the target partition block. In other words, after the first network transmission unit obtains the target partition block, the first network transmission unit can encapsulate the service data packet of the target partition block, and during the encapsulation process, can determine the communication link parameters of the target first queue pair. Therefore, after determining the communication link parameters of the target first queue pair, the first network transmission unit can perform hash processing on the communication link parameters of the target first queue pair to obtain a target link hash value of the communication link parameters of the target first queue pair, and perform port mapping processing on the target link hash value and the N first network ports, so that the target first network port can be determined from the N first network ports by hash mapping. The first network transmission unit sends the target partition block encapsulated in the business data packet to the second network transmission unit through the target first network port, and the second network transmission unit stores the target partition block in the second graphics processing unit, so that the second graphics processing unit sends the target partition block to the expert model. In low-latency mode, the first network transmission unit sends the target partition block (the target partition block is encapsulated in the business data packet) to the second network transmission unit through a switch device. In particular, during data transmission, GDR technology is used to bypass the transit of the CPU and host memory in low-latency mode to reduce the transmission delay during the data transmission process, and improve the throughput of the port network card mapped by the target first queue.

[0142] In addition, in an embodiment of the present application, when port mapping processing is performed on the target link hash value and N first network ports, two mapping methods, modulo mapping and table lookup mapping, which depend on the hash value of the communication link parameters (for example, a quintuple), can be used for mapping processing.

[0143] Among them, the first network transmission unit will encapsulate the target partition block into a business data packet, which may specifically include obtaining the transmission protocol (i.e., the transport layer protocol, such as the UPD protocol) between the first network transmission unit and the second network transmission unit, obtaining the various fields of the data packet corresponding to the transmission protocol (specifically, it may include the source port field, the destination port field, the data length field, etc.), and then obtaining the data corresponding to each field, and backfilling the obtained data corresponding to each field as the field value of the corresponding field to obtain the business data packet corresponding to the target partition block. Among them, it should be noted that the embodiment of the present application can determine the five-tuple of each first queue pair during the encapsulation process, and use the five-tuple as the communication link parameter of the first queue pair. It should be understood that the communication link parameter can provide a unique communication link for each first queue pair. Typically, a communication link parameter is a five-tuple, including a source UDP port number, a source IP address, a destination UDP port number, a destination IP address, and a transport layer protocol. When the source IP address, the destination IP address, the destination UDP port number, and the transport layer protocol are all exactly the same (i.e., data transmission with the same GPU), only the source UDP port number can be different.

[0144] Therefore, during the encapsulation process, when obtaining the communication link parameters, for two queue pairs configured for the same expert model, when the determined communication link parameters are the same, in order to achieve load balancing, the embodiment of the present application proposes that the source port number in the communication link parameters can be changed using two different port numbers to ensure that the target partition blocks corresponding to the two queue pairs can use different network ports for data transmission to achieve load balancing.

[0145] For example, in a network architecture based on RoCE (RDMA over Converged Ethernet) technology, the RDMA protocol transmits data through an Ethernet medium. After the first network transmission unit at the data sending end encapsulates the target partition block into a service data packet (such as a RoCE data packet, which is encapsulated using the UDP protocol), it can rely on the PFC (Priority Flow Control) mechanism to achieve lossless transmission when transmitting data through Ethernet. In this way, when the number of partition blocks received by the storage space of the data receiving end (i.e., the second storage space of the second graphics processing unit) reaches a threshold, it can also rely on the PFC to send pause frames in reverse hop by hop to prevent the upstream node from continuing to send the specified priority traffic, thereby avoiding packet loss.

[0146] The purpose of the embodiment of the present application is to achieve traffic load balancing of network ports. Therefore, there is usually more than one target first queue pair. When there are multiple target first queue pairs for the same expert model, the communication link parameters of the multiple target first queue pairs can be obtained first. Since different source port numbers are used for initialization when initializing the communication link parameters, the first network transmission unit can determine different network ports for data transmission based on different source ports for multiple target first queue pairs, thereby achieving traffic load balancing and avoiding congestion caused by always using the same network port for data transmission.

[0147] For example, when the number of multiple network ports (i.e., multiple first network ports) provided by the first network transmission unit is N, and N is 2, it means that the first network transmission unit can specifically include two first network ports (e.g., network port P11 and network port P12). Therefore, when the transmitting end involved in the embodiment of the present application sends service characterization data to the expert model of the data receiving end, it is necessary to obtain the N first queue pairs configured when sending the service characterization data (e.g., two first queue pairs, such as QP11 and QP12). If these two first queue pairs are queue pairs that meet the data sending conditions, then these two first queue pairs that meet the data sending conditions can be collectively referred to as target first queue pairs.

[0148] To facilitate distinction, in an embodiment of the present application, one of the two first queue pairs (e.g., QP11 and QP12) in the target first queue pair may be collectively referred to as a first first queue pair, and the other of the two first queue pairs (e.g., QP11 and QP12) in the target first queue pair may be collectively referred to as a second first queue pair. Similarly, in an embodiment of the present application, one of the two first network ports (e.g., network port P11 and network port P12) may be collectively referred to as a first first network port, and the other of the two first network ports (e.g., network port P11 and network port P12) may be collectively referred to as a second first network port.

[0149] Furthermore, in order to ensure that the two first network ports involved in the process of transmitting business characterization data can achieve load balancing, the transmitting end involved in the embodiment of the present application enables the first network transmission unit deployed in the transmitting end to transmit the target partition blocks (for example, the partition blocks matching QP11 and the partition blocks matching QP12) read from the first graphics processing unit and matching the queue elements in the target first queue pair through these two first network ports (for example, network port P11 and network port P12). Specifically, the network ports mapped respectively according to the hash values ​​of the five-tuples of the two first queue pairs (for example, QP11 and QP12) can be used to realize the transmission of the corresponding target partition blocks currently read.

[0150] For example, for the target partition block read that matches the queue element in the first first queue pair, the first network transmission unit can use the first first network port (for example, network port P11) of the two first network ports (for example, network port P11 and network port P12) for transmission, and for the target partition block read that matches the queue element in the second first queue pair, the first network transmission unit can use the second first network port (for example, network port P12) for transmission.

[0151] For another example, when the number N of the multiple network ports (i.e., multiple first network ports) provided by the first network transmission unit is 3, it means that the first network transmission unit can specifically include 3 first network ports (e.g., network port P11, network port P12, and network port P13). Therefore, when the transmitting end involved in the embodiment of the present application sends service characterization data to the expert model of the data receiving end, it is necessary to obtain the N first queue pairs configured when sending the service characterization data (e.g., 3 first queue pairs, such as QP11, QP12, and QP13). If these 3 first queue pairs are all queue pairs that meet the data sending conditions, then these 3 first queue pairs that meet the data sending conditions can be collectively referred to as target first queue pairs.

[0152] To facilitate distinction, in an embodiment of the present application, one first queue pair (for example, QP11) among the three first queue pairs (for example, QP11, QP12, and QP13) in the target first queue pairs may be collectively referred to as the first first queue pair, another first queue pair (for example, QP12) among the three first queue pairs in the target first queue pairs (for example, QP11, QP12, and QP13) may be collectively referred to as the second first queue pair, and yet another first queue pair (for example, QP13) among the three first queue pairs in the target first queue pairs (for example, QP11, QP12, and QP13) may be collectively referred to as the third first queue pair. Similarly, in the embodiment of the present application, one first network port (for example, network port P11) among the three first network ports (for example, network port P11, network port P12, and network port P13) may be collectively referred to as a first first network port, another first network port (for example, network port P12) among the three first network ports may be collectively referred to as a second first network port, and yet another first network port (for example, network port P13) among the three first network ports may be collectively referred to as a third first network port.

[0153] Furthermore, in order to ensure that the three first network ports involved in the process of transmitting business characterization data can achieve load balancing, the transmitting end involved in the embodiment of the present application enables the first network transmission unit deployed in the transmitting end to transmit the target partition blocks (for example, the partition blocks matching QP11, the partition blocks matching QP12, and the partition blocks matching QP13) read from the first graphics processing unit and matching the queue elements in the target first queue pair through these three first network ports (for example, network port P11, network port P12, and network port P13). Specifically, the network ports mapped respectively according to the hash values ​​of the five-tuples of the three two first queue pairs (for example, QP11, QP12, and QP13) can be used to realize the transmission of the corresponding target partition blocks currently read.

[0154] For example, for the target partition block read that matches the queue element in the first first queue pair, the first network transmission unit can use the first first network port (for example, network port P11) among the three first network ports (for example, network port P11, network port P12 and network port P13) for transmission; for the target partition block read that matches the queue element in the second first queue pair, the first network transmission unit can use the second first network port (for example, network port P12) for transmission; for the target partition block read that matches the queue element in the third first queue pair, the first network transmission unit can use the third first network port (for example, network port P13) for transmission.

[0155] It should be noted that if the block data volume of the target partition block indicated by the queue element in the target first queue pair is greater than the unit data volume of the maximum transmission unit, the first network transmission unit may further perform a secondary partitioning of the target partition block after obtaining the target partition block to ensure successful data transmission. This secondary partitioning does not change the target first network port of the target partition block; it merely divides the target partition block into multiple data items, which are still sent from that network port to the second network transmission unit.

[0156] In one embodiment, the first graphics processing unit can send data when detecting that the queue element in the target first queue pair meets the data sending condition. At the same time, the second graphics processing unit can also determine that the data sending can be ended (that is, determine that the data sending is completed) when detecting the sending end indication information. In an embodiment of the present application, the queue pairs in the N groups of queue pairs include N second queue pairs configured for the expert model, and the second queue pairs are queue pairs for receiving data at the data receiving end. After each of the data partition blocks associated with the business characterization data is sent to the second graphics processing unit, a stream multi-processor (SM, a core component on the GPU) is called to count the number of business characterization data to be sent to the expert model in the data processing task; that is, the number of business characterization data to be sent to the expert model in the data processing task, generate sending end indication information based on the counted number of business characterization data, and send the sending end indication information to the second graphics processing unit, so that the second graphics processing unit stores the sending end indication information in the control signal storage space configured for the N second queue pairs. It can also be understood that the first graphics processing unit uses the amount of service representation data as the sending end indication information. After sending it to the second graphics processing unit, the second graphics processing unit uses the sending end indication information as a control signal and stores it in the control signal storage space, so that the second graphics processing unit queries the control signal in real time to determine whether all the data has been sent.

[0157] In an embodiment of the present application, a data transmission process with a first graphics processing unit at a data sending end as the execution body is mainly described. When sending business characterization data to an expert model configured with multiple queue pairs, the first graphics processing unit can perform block division processing on the business characterization data according to the multiple queue pairs to obtain data division blocks of the business characterization data; when the first network transmission unit obtains a division block that meets the sending conditions, it selects a target network port from multiple network ports based on the communication links of the multiple queue pairs for data transmission. Due to the differences in the communication links of the multiple queue pairs in the present application, the first network transmission unit can evenly select network ports for data transmission, thereby achieving multi-port traffic load balancing at the sending end.

[0158] Figure 3 This paper mainly describes the data sending end as the execution subject. In order to reflect the integrity of the scheme, Figure 6 Taking the data receiving end as an example, the data transmission method provided in the embodiment of the present application is explained. The method is applied to the same data transmission system, which includes a data receiving end and a data sending end, wherein the data sending end includes a first graphics processing unit and a first network transmission unit, and the data receiving end includes a second graphics processing unit and a second network transmission unit. Figure 6 As shown, the data transmission method may include but is not limited to the following steps:

[0159] S601: Acquire a target partition block that matches a target first queue pair.

[0160] Among them, the target partition block is sent by the first network transmission unit through the target network end in the multiple network ports provided by the first network transmission unit; the target partition block is the partition block that matches the target first queue pair obtained from the data partition block by the first network transmission unit after receiving the target first queue pair in the N first queues sent by the first graphics processing unit; the target first queue pair is the first queue pair in the N first queue pairs that meets the data sending conditions; the data partition block is obtained by the first graphics processing unit after obtaining the data processing task carrying the business characterization data, and performing block partitioning processing on the business characterization data according to the N first queue pairs associated with the expert model. Each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, N is an integer greater than 1, and the data processing task includes the task of sending the business characterization data to the expert model in the second graphics processing unit. This process is also the generation process and sending process of the target partition block. For details, please refer to Figure 3 , I will not go into details here.

[0161] In the low-latency mode of an embodiment of the present application, the first network transmission unit sends data to the second network transmission unit through a switch device (the switch device specifically refers to a switch). The switch device is deployed between the first network transmission unit and the second network transmission unit, and its number is not limited. Since the second network transmission unit also includes multiple network ports, in order to achieve load balancing at the data receiving end, during the transmission of the service data packet, the switch device (specifically, it can be a switch) can flexibly determine the target second network port from the N second network ports provided by the second network transmission unit based on the communication link parameters corresponding to the service data packet, so that the second network transmission unit receives the service data packet through the target second network port. Specifically, the switch performs hash processing on the communication link parameters corresponding to the service data packet to obtain a link hash value of the communication link parameters of the service data packet, and performs port mapping processing on the link hash value and the N second network ports, so that the target second network port can be determined from the N second network ports by hash mapping. Furthermore, after receiving the service data packet, the second network transmission unit sends the service data packet to the second graphics processing unit through the data transmission thread configured for the second graphics processing unit.

[0162] In a feasible embodiment, after receiving the business data packet of the target partition block, the second graphics processing unit obtains the target partition block that matches the target first queue pair, that is, the process of obtaining the target partition block corresponding to the business data packet may include: the second graphics processing unit receives the business data packet carrying the target partition block sent by the second network transmission unit; the business data packet is obtained by the first network transmission unit encapsulating the target partition block; the second graphics processing unit decapsulates the business data packet to obtain the target partition block.

[0163] Furthermore, when the business data packet is decapsulated, the queue element of the target first queue pair corresponding to the target partition block can also be parsed out. The queue element of the target first queue pair is used to indicate the second storage address of the target partition block in the second graphics processing unit. Therefore, when the target partition block is obtained by decapsulation, the target partition block is stored in the second storage space corresponding to the second storage address based on the queue element of the target first queue pair carried in the business data packet.

[0164] S602: Determine a target second queue pair mapped to the target first queue pair from N second queue pairs associated with the expert model; each second queue pair is a queue pair mapped to the first queue pair in the N groups of queue pairs.

[0165] In this embodiment of the present application, queue pair mapping tables are maintained at both the data transmitting end and the data receiving end. Therefore, once the second graphics processing unit determines to which expert model the target partition block is to be sent, it can determine the target second queue pair to which the target first queue pair is mapped from the N second queue pairs associated with that expert model. The purpose of determining the target second queue pair is to facilitate data reception. For details, see step S603.

[0166] S603: When sending end indication information associated with the target second queue is detected, it is determined that each of the divided data blocks sent as the target divided blocks has been obtained, and the business characterization data corresponding to the divided data blocks is sent to the expert model. The sending end indication information is obtained by the first graphics processing unit performing quantitative statistics on the sent business characterization data.

[0167] In some feasible embodiments, the sending end indication information is obtained by the first graphics processing unit counting the number of business characterization data (i.e., token) sent. For example, if the total number of business characterization data sent by the sending end to the expert model A is 5, the sending end indication information is determined based on the number 5. Optionally, in addition to the number 5, the sending end indication information may also include the indication information of the expert model (used to uniquely identify the expert model). The sending end indication information is counted and sent to the second network transmission unit after the first network transmission unit completes sending all the business characterization data sent to the same expert model. After receiving the sending end indication information, the second network transmission unit stores it in the control signal storage space configured for the target second queue pair. In one possible implementation, the number of control signal storage spaces may be associated with the total number of queue pairs, that is, one queue pair is configured with one control signal storage space. In this case, the sending end only needs to send the number of tokens to the receiving end (the receiving end may directly store the tokens in the corresponding control signal storage space after receiving the tokens); or, the number of control signal storage spaces may be associated with the number of queue pairs configured for each expert model. For example, if an expert model is configured with two queue pairs, two control signal storage spaces are directly configured in the storage space of the graphics processing unit. In this case, the data that the sending end needs to transmit to the receiving end may specifically include the number of tokens and indication information of the corresponding expert model.

[0168] In a feasible embodiment, the second graphics processing unit detects the sending end indication information associated with the target second queue pair in the control signal storage space configured for the target second queue pair; upon detecting the presence of the sending end indication information in the control signal storage space, it is determined that each partition block in the partitioned data blocks sent as the target partition block has been obtained, the sending end indication information is transmitted to the expert model, and a data read request for reading the business characterization data is generated through the expert model; and the expert model is controlled to read the business characterization data from the second storage space corresponding to the second storage address according to the data read request.

[0169] like Figure 7 FIG. 1 is a schematic diagram of a control signal storage space corresponding to a sending end indication message. Figure 7 Take an expert model to configure two queue pairs as an example. Figure 7 As can be seen, in addition to the storage space for storing service representation data (such as the second storage space for storing the target partitioned blocks described above), this storage space may also include two control signal storage spaces, signal1 (a first control signal storage space) and signal2 (a second control signal storage space). The second graphics processing unit can sequentially check the two control signal storage spaces via a polling thread. If there is an indication that the expert model has been sent, it is determined that each partitioned block in the partitioned data blocks sent as target partitioned blocks has been obtained, indicating that all partitioned blocks for which the expert model needs to be sent have arrived.

[0170] Furthermore, in addition to sending the business representation data to the expert model, the data processing task also includes a model inference task that uses the expert model deployed in the hybrid business model to perform model inference. A model inference task refers to the task of inferring business sample data through the expert model. Therefore, after the expert model reads the task sample data (also called business sample data) corresponding to the business representation data, it performs model inference processing on the business sample data through the expert model to obtain a model inference result. For example, a business sample data is "If you want to take food from the refrigerator, what is the first step?" The expert model performs model inference on the business sample data and obtains a model inference result, such as "The first step is to open the refrigerator."

[0171] In the embodiments of the present application, the data transmission method is mainly described with respect to the data receiving end. The data receiving end mainly determines whether the data from the data sending end has arrived and how to process the data after arrival. At the data receiving end, it is possible to determine whether all the data from the data sending end has been sent by detecting the send end indication information in the control signal storage space. If the send end indication information is detected, it can be determined that all the data from the data sending end has been sent. In the embodiments of the present application, the send end indication information can be used to avoid out-of-order reception of data at the data receiving end, thereby ensuring efficient data transmission and reception.

[0172] In order to more clearly explain the data transmission method provided by the embodiment of the present application, the data transmission method provided by the embodiment of the present application is explained below in a manner in which a data sending end and a data receiving end interact with each other, wherein the data sending end includes a first graphics processing unit and a first network transmission unit, and the data receiving end includes a second graphics processing unit and a second network transmission unit. Figure 8 , is a flowchart of another data transmission method provided in an embodiment of the present application. The corresponding data transmission method includes but is not limited to the following steps:

[0173] S801: The first graphics processing unit obtains a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in the second graphics processing unit.

[0174] S802: The first graphics processing unit performs block partitioning processing on the service characterization data based on N first queue pairs associated with the expert model to obtain data partitioned blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1.

[0175] S803: The first graphics processing unit obtains a target first queue pair that meets the data sending condition among the N first queue pairs.

[0176] S804: The first graphics processing unit sends the target first queue pair to the first network transmission unit.

[0177] In this process, the first graphics processing unit transmits a queue element (WQE) in a target first queue pair to the first network transmission unit.

[0178] S805: The first network transmission unit obtains a target partition block that matches the target first queue pair, and determines a target first network port from the N first network ports based on the communication link parameters of the target first queue pair.

[0179] S806: The first network transmission unit sends the target divided block to the second network transmission unit through the target first network port.

[0180] In this process, the first network unit also needs to determine the target first network port and encapsulate the target partition block to obtain a service data packet. These steps have been described in detail in the above embodiment and will not be repeated here.

[0181] S807: The second network transmission unit sends the target partition block to the second graphics processing unit, so that the expert model in the second graphics processing unit can obtain the business sample data corresponding to the target partition block to perform model inference.

[0182] Based on the above explanation, in order to verify that the data transmission method can indeed achieve traffic load balancing, a large amount of data is used to illustrate the experimental configuration: the test is carried out in the environment of the H20 server (the server serves as the data sender and the H20 server is an AI computing server). The low-latency mode dispatch and combine conditions with 8, 16, and 32 GPU cards are tested respectively. The data types transmitted by dispatch are FP8 (float8, 8-bit floating point number) and BF16 (bfloat16, Brain Floating Point16). The number of experts is set to 256. The test results are shown in Table 2:

[0183] Table 2

[0184]

[0185]

[0186] As can be seen from Table 2, latency has steadily decreased in all experiments, with the latency reduction exceeding 25%. When testing FP8 transmission accuracy for dispatch between 16 cards, the latency reduction reached 47.43%.

[0187] As shown in Table 2 above, in an embodiment of the present application, for eight GPUs on the same server, eight network cards corresponding to the eight GPUs can be deployed simultaneously to ensure that each network card corresponds to one GPU. Based on this, in one achievable method, for data transmission using any two GPUs in the same server, one of the two GPUs that need to perform data transmission can be collectively referred to as the first graphics processing unit, and the other of the two GPUs that need to perform data transmission can be collectively referred to as the second graphics processing unit. When data transmission is performed between the first graphics processing unit and the second graphics processing unit, the data transmission method provided in the embodiment of the present application can be used to achieve load balancing of the network bandwidth during data transmission and reception.

[0188] In addition, if Figure 9 As shown, taking the communication library as DeepEP as an example, there are 8 network cards on a server, each network card has two physical ports (i.e., network ports), and the test is performed under the premise of binding the two physical ports to one bond port. In this embodiment of the application, the business logical port bound to the two physical ports of the first network card among the 8 network cards can be bond1 (i.e., the first business logical port, for example: Figure 9 The business logic port bound to the two physical ports of the second network card among the eight network cards can be bond2 (i.e., the second business logic port), for example: Figure 9 mlx5_bond_2 shown in the figure is referred to as No.2 bond port). Similarly, the business logical port bound to the two physical ports of the eighth network card among the eight network cards can be bond8 (i.e. the eighth business logical port). For example: Figure 9 The mlx5_bond_8 shown is referred to as No.8bond port).

[0189] like Figure 9 As shown in FIG, when different data transmission methods (the data transmission method under DeepEP before improvement and the data transmission method under DeepEP after improvement) are applied to the same server, the statistics of the data transmission in the same period on the server (for example, the first time period and the second time period, as shown in FIG) are shown. Figure 9 As shown, the first and second time periods can be the time periods corresponding to 4:00 PM on April 16, 2025, to illustrate network load conditions. The pre-improved data transmission method (e.g., the pre-improved DeepEP data transmission method) refers to a method in which data transmission between any two GPUs uses the same network port for data transmission without distinction. The improved data transmission method (e.g., the improved DeepEP data transmission method) refers to a method in which data transmission between any two GPUs uses different network ports for data transmission for different queue pairs through block partitioning.

[0190] for example, Figure 9 901 is the network traffic situation obtained by using the data transmission method under DeepEP before the improvement, and 902 is the network traffic situation obtained by using the data transmission method under DeepEP after the improvement. When mapping the two network ports to the same logical port bond7 (i.e. the seventh service logical port, for example: Figure 9As shown in mlx5_bond_7, referred to as No.7bond port), it can be seen from 901 that before the improvement, the network traffic of a network port under the logical port bond7 (i.e., network port 14, such as eth14 in 901) is 163.5 Gbps, and the network traffic of the other network port (i.e., network port 15, such as eth15 in 901) is 26.313 Gbps. It can be seen that the traffic data of the two ports are quite different and the traffic load is unbalanced; it can be seen from 902 that after the improvement, the network traffic of a network port under the logical port bond7 (i.e., network port 15, such as eth15 in 902) is 145.446 Gbps, and the network traffic of the other network port (i.e., network port 14, such as eth14 in 901) is 145.318 Gbps. It can be seen that the traffic data difference between the two ports is not much. Therefore, it can be concluded that the data transmission method provided in the embodiment of the present application can achieve traffic load balancing.

[0191] See Figure 10 , Figure 10 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of the present application. Figure 10 As shown, the data transmission device 1000 may be running in the first graphics processing unit, which may be running on a terminal device (for example, the above-mentioned device 11a, device 12a or 13a), or on a server (the above-mentioned device 14a). For example, the data transmission device 1000 may be an application software; it is understandable that the data transmission device 1000 may be used to execute the corresponding steps in the data transmission method provided in the embodiment of the present application. Figure 10 As shown, the data transmission device 1000 may include: a task acquisition unit 11, a data division unit 12, a queue pair sending unit 13, a queue pair second acquisition unit 14, a queue pair relationship construction unit 15, a configuration unit 16, a first storage unit 17, a queue element determination unit 18, a thread calling unit 19, a notification unit 20, a text processing unit 30, an indication information generation unit 40, and an indication information sending unit 50;

[0192] A task acquisition unit 11 is configured to acquire a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in the second graphics processing unit;

[0193] a data partitioning unit 12 configured to perform block partitioning processing on the service characterization data based on N first queue pairs associated with the expert model to obtain data partitioned blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1;

[0194] a queue pair sending unit 13 configured to, if a target first queue pair that meets the data sending condition exists among the N first queue pairs, send the target first queue pair to the first network transmission unit, so that the first network transmission unit obtains a target partition block that matches the target first queue pair, and sends the target partition block to the second graphics processing unit through the target first network port among the N first network ports;

[0195] The second graphics processing unit is used to send the target partition block to the expert model, the target first network port is determined by the first network transmission unit from N first network ports based on the communication link parameters of the target first queue pair; the target partition block is a partition block in the data partition block.

[0196] In one embodiment, when the data partitioning unit 12 is used to perform the step of partitioning the service characterization data into blocks based on the N first queue pairs associated with the expert model to obtain data partition blocks of the service characterization data, it is specifically used to:

[0197] Obtaining a queue pair relationship mapping table maintained by the first graphics processing unit; the queue pair relationship mapping table includes at least N groups of queue pairs associated with the expert model; the N groups of queue pairs include N first queue pairs and second queue pairs mapped respectively to the N first queue pairs;

[0198] Obtain N first queue pairs from N groups of queue pairs;

[0199] Determine the number of queues of the N first queue pairs as the first number of queues, determine N initial partition blocks corresponding to the service characterization data based on the first number of queues, and determine the amount of partitioned data for each initial partition block; the amount of partitioned data for each initial partition block is determined based on the first number of queues and the amount of data of the service characterization data;

[0200] Based on the maximum transmission unit associated with the first network transmission unit and the amount of divided data of each initial divided block, block division processing is performed on each initial divided block to obtain sub-divided blocks of each initial divided block, and the sub-divided blocks of each initial divided block are determined as data divided blocks of the service characterization data.

[0201] In one embodiment, the first graphics processing unit and the second graphics processing unit are graphics processing units in a communication group of a data transmission system;

[0202] The second queue pair obtaining unit 14 is configured to determine the number of ports of the N first network ports as the first port number, obtain N first queue pairs pre-configured by the first graphics processing unit and matching the first port number from the communication group; and obtain N second queue pairs pre-configured by the second graphics processing unit for the expert model from the communication group;

[0203] The queue pair relationship construction unit 15 is used to establish a queue pair mapping relationship between N first queue pairs and N second queue pairs, and based on the established queue pair mapping relationship, determine the second queue pairs mapped to the N first queue pairs respectively; based on each first queue pair and each second queue pair mapped to the first queue pair respectively, construct N groups of queue pairs associated with the expert model; based on the N groups of queue pairs, determine a queue pair relationship mapping table.

[0204] In one embodiment, the first network transmission unit is used to perform data encapsulation processing on the target partition block according to the transport layer protocol indicated by the data processing task after obtaining the target partition block, obtain a business data packet carrying the target partition block, and determine to transmit the business data packet through the target first network port based on the communication link parameters of the target first queue pair.

[0205] In one embodiment, the queue pair second obtaining unit 14 is further configured to obtain a first queue pair i and a first queue pair j from the N first queue pairs; i and j are integers less than or equal to N, and i is not equal to j;

[0206] Configuration unit 16 is used to configure a first control signal storage space for the first queue pair i; the first control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair i; configure a second control signal storage space for the first queue pair j; the second control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair j; the space size of the second control signal storage space is consistent with the space size of the first control signal storage space.

[0207] In one embodiment, the N initial partition blocks include a first initial partition block and a second initial partition block; when the data partitioning unit 12 is used to perform the step of determining the amount of partitioned data for each initial partition block, it is specifically used to:

[0208] Calculate the ratio of the data volume of the service representation data to N;

[0209] If the ratio is an integer, it is determined that the data volume of the service characterization data is divisible by N, and the ratio is used as the data volume of the first initial partition block and the second initial partition block respectively;

[0210] If the ratio is not an integer, it is determined that the data volume of the business characterization data cannot be divided by N. The comparison value is rounded to obtain an integer value of the ratio. When the integer value is used as the data volume of the first initial partition block, the difference between the data volume of the business characterization data and the integer value is used as the data volume of the second initial partition block.

[0211] In one embodiment, the storage address of the first graphics processing unit is a first storage address, and the storage address of the second graphics processing unit is a second storage address; the first graphics processing unit stores a unit data volume of a maximum transmission unit associated with the first network transmission unit;

[0212] The first storage unit 17 is configured to store the data partition blocks of the service characterization data in a first storage space corresponding to a first storage address when the data partition blocks are obtained;

[0213] The queue element determination unit 18 is used to determine the queue element of each first queue pair based on the first storage address, the second storage address and the block data volume of the data partition block; the queue element of each first queue pair is used to instruct the first network transmission unit to obtain a target partition block that matches the queue element of the target first queue pair from the first storage space; the block data volume of the data partition block is determined based on the number of queues of the N first queue pairs, the data volume of the service characterization data and the unit data volume of the maximum transmission unit.

[0214] In one embodiment, the data partition block includes at least a first partition block and a second partition block;

[0215] The thread calling unit 19 is used to call the first business processing thread configured for the first partition block, determine the first storage address and the second storage address, and perform queue element encapsulation processing on the first storage address, the second storage address and the block data volume of the first partition block through the first business processing thread to obtain the first queue element corresponding to the first partition block; call the second business processing thread configured for the second partition block, determine the first storage address and the second storage address, and perform queue element encapsulation processing on the first storage address, the second storage address and the block data volume of the second partition block through the second business processing thread to obtain the second queue element corresponding to the second partition block; configure the first queue element to the target first queue pair through the first business processing thread, and configure the second queue element to the target first queue pair through the second business processing thread to obtain the queue element of the target first queue pair.

[0216] In one embodiment, the target first queue pair includes at least a first reserved position and a second reserved position for storing queue elements;

[0217] The first storage unit 17 is further configured to:

[0218] When allocating the first queue element to the target first queue pair through the first business processing thread, writing the first queue element to a first reserved position in the target first queue pair;

[0219] When the second queue element is configured to the target first queue pair through the second service processing thread, the second queue element is written to the second reserved position in the target first queue pair.

[0220] In one embodiment, when the queue pair sending unit 13 is used to execute the step of sending the target first queue pair to the first network transmission unit, it is specifically used to:

[0221] Call the data transmission thread configured for the target first queue pair, encapsulate the queue elements in the target first queue pair as a data transmission request, and send the data transmission request to the first network transmission unit, so that the first network transmission unit executes the step of obtaining the target partition block matching the queue elements of the target first queue pair from the first storage space according to the data transmission request.

[0222] In one embodiment, the task acquisition unit 11 is further configured to acquire a task type of the data processing task;

[0223] The notification unit 20 is configured to notify the data partitioning unit to perform block partitioning processing on the business characterization data based on N first queue pairs associated with the expert model to obtain data partition blocks of the business characterization data if the task type is a target task type that can be processed with low latency.

[0224] In one embodiment, the data processing task further includes a model reasoning task for performing model reasoning using an expert model deployed in the hybrid business model; the model reasoning task refers to a task of reasoning on business sample data using the expert model;

[0225] The task acquisition unit 11 is further used to acquire business sample data corresponding to the model reasoning task; and acquire business text information associated with the business sample data;

[0226] The text processing unit 30 is used to perform text segmentation processing on the business text information to obtain text segmentation of the business text information; perform vector representation processing on the text segmentation to obtain text representation vectors for the text segmentation, and use the text representation vectors as business representation data in the data processing task.

[0227] In one embodiment, the queue pairs in the N groups of queue pairs include N second queue pairs configured for the expert model; the indication information generating unit 40 is configured to, after sending each of the data partition blocks associated with the service characterization data to the second graphics processing unit, call the streaming multiprocessor data processing task to count the number of service characterization data to be sent to the expert model; and generate sending completion indication information based on the counted number of service characterization data;

[0228] The indication information sending unit 50 is configured to send the sending end indication information to the second graphics processing unit, so that the second graphics processing unit stores the sending end indication information in the control signal storage space configured for the N second queue pairs.

[0229] It can be understood that the functions of each functional unit of the data transmission device 1000 provided in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment, and its specific implementation process can refer to the relevant description in the above method embodiment, which will not be repeated here.

[0230] In a feasible embodiment, the data transmission device 1000 provided in the embodiment of the present application can be implemented in software. The data transmission device 1000 can be stored in a memory. It can be software in the form of programs and plug-ins, and includes a series of units, including a task acquisition unit 11, a data division unit 12, and a queue pair sending unit 13; wherein the task acquisition unit 11, the data division unit 12, and the queue pair sending unit 13 are used to implement the data transmission method provided in the embodiment of the present application.

[0231] See Figure 11 , Figure 11 This is a schematic diagram of the structure of another data transmission device provided in an embodiment of the present application. Figure 11 As shown, the data transmission device 1100 may be running in the second graphics processing unit, and the second graphics processing unit may be running on a terminal device (for example, the above-mentioned device 11a, device 12a or 13a), or on a server (the above-mentioned device 14a). For example, the data transmission device 1100 may be an application software; it is understandable that the data transmission device 1100 may be used to execute the corresponding steps in the data transmission method provided in the embodiment of the present application. Figure 11 As shown, the data transmission device 1100 may include: a partition block acquisition unit 21, a queue pair first acquisition unit 22, a data sending unit 23, a second storage unit 24, and a sample data acquisition unit 25;

[0232] A partition block acquisition unit 21 is configured to acquire a target partition block matching a target first queue pair, wherein the target partition block is sent by the first network transmission unit via a target first network port among a plurality of first network ports provided by the first network transmission unit; the target partition block is a partition block matching the target first queue pair acquired from the data partition block by the first network transmission unit after receiving the target first queue pair from the N first queues sent by the first graphics processing unit; the target first queue pair is the first queue pair among the N first queue pairs that meets the data sending conditions; the data partition block is obtained by the first graphics processing unit performing block partitioning processing on the service characterization data according to the N first queue pairs associated with the expert model after acquiring a data processing task carrying service characterization data, wherein each first queue pair is a queue pair among the N groups of queue pairs associated with the expert model, where N is an integer greater than 1, and the data processing task includes a task of sending the service characterization data to the expert model in the second graphics processing unit; the target first network port is determined by the first network transmission unit from the N first network ports based on the communication link parameters of the target first queue pair;

[0233] a first queue pair obtaining unit 22, configured to determine a target second queue pair mapped to the target first queue pair from the N second queue pairs associated with the expert model; each second queue pair is a queue pair mapped to the first queue pair in the N groups of queue pairs;

[0234] The data sending unit 23 is configured to, upon detecting sending end indication information associated with the target second queue, determine that each of the divided data blocks sent as the target divided blocks has been acquired, and send the service representation data corresponding to the divided data block to the expert model. The sending end indication information is obtained by the first graphics processing unit performing quantitative statistics on the sent service representation data.

[0235] In one embodiment, the data receiving end further includes a second network transmission unit; when the partition block acquisition unit 21 is used to execute the step of acquiring the target partition block matching the target first queue pair, it is specifically used to:

[0236] receiving a service data packet carrying a target partitioned block sent by the second network transmission unit; the service data packet is obtained by encapsulating the target partitioned block by the first network transmission unit;

[0237] Decapsulate the service data packet to obtain the target partition block.

[0238] In one embodiment, when the switch device deployed between the second network transmission unit and the first network transmission unit is a switch, the second network transmission unit is used to receive a service data packet sent by the first network transmission unit through a target second network port determined by the switch, wherein the target second network port is determined by the switch from N second network ports provided by the second network transmission unit according to the communication link parameters corresponding to the service data packet when the switch receives the service data packet sent by the first network transmission unit through the target first network port.

[0239] In one embodiment, the target partition block is a partition block carried in a service data packet, and the service data packet further carries a queue element of a target first queue pair; the queue element of the target first queue pair is used to indicate a second storage address of the target partition block in the second graphics processing unit;

[0240] The second storage unit 24 is configured to store the target partition block to a second storage address based on the queue element of the target first queue pair carried in the service data packet when the target partition block is obtained.

[0241] In one embodiment, when the data sending unit 23 is configured to execute the step of determining that each of the divided data blocks sent as the target divided blocks has been acquired upon detecting the sending end indication information associated with the target second queue, and sending the service representation data corresponding to the divided data block to the expert model, the step is specifically configured to:

[0242] detecting, in a control signal storage space configured for the target second queue pair, transmission end indication information associated with the target second queue pair;

[0243] Upon detecting that there is sending end indication information in the control signal storage space, determining that each of the divided data blocks sent as the target divided blocks has been acquired, transmitting the sending end indication information to the expert model, and generating a data read request for reading the service representation data through the expert model;

[0244] The expert model is controlled to read the service representation data from the second storage space corresponding to the second storage address according to the data read request, where the second storage address is the storage address of the target partition block stored in the second graphics processing unit.

[0245] In one embodiment, the data processing task also includes a model reasoning task of performing model reasoning using an expert model deployed in the hybrid business model; the model reasoning task refers to a task of reasoning on business sample data using an expert model;

[0246] The sample data acquisition unit 25 is used to acquire the business sample data corresponding to the business representation data, and perform model inference processing on the business sample data through the expert model to obtain the model inference results corresponding to the business sample data.

[0247] It can be understood that the functions of each functional unit of the data transmission device 1100 provided in the embodiment of the present application can be specifically implemented according to the method in the above method embodiment. The specific implementation process can refer to the relevant description in the above method embodiment, and will not be repeated here.

[0248] In a feasible embodiment, the data transmission device 1100 provided in the embodiment of the present application can be implemented in software. The data transmission device 1100 can be stored in a memory. It can be software in the form of programs and plug-ins, and includes a series of units, including any partition block acquisition unit 21, a queue pair first acquisition unit 22, and a data sending unit 23; wherein the partition block acquisition unit 21, the queue pair first acquisition unit 22, and the data sending unit 23 are used to implement the data transmission method provided in the embodiment of the present application.

[0249] See Figure 12 , Figure 12 This is a structural diagram of a computer device provided in an embodiment of the present application. For example, the computer device may be the above-mentioned terminal device or server. For ease of understanding, the computer device is described here as a terminal device (a graphics processing unit and a network transmission unit are integrated in the terminal device). Figure 12 As shown, the computer device 1200 may include: a processor 1201, a network interface 1204 and a memory 1205. In addition, the above-mentioned computer device 1200 may also include: a user interface 1203, and at least one communication bus 1202. The communication bus 1202 is used to realize the connection and communication between these components. Optionally, the user interface 1203 may include a display screen (Display), a keyboard (Keyboard), and optionally, the user interface 1203 may also include a standard wired interface and a wireless interface. The network interface 1204 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1205 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1205 may optionally also be at least one storage device located away from the aforementioned processor 1201. As Figure 12 As shown, the memory 1205 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application.

[0250] In such Figure 12In the illustrated computer device 1200, the network interface 1204 provides network communication functionality; the user interface 1203 primarily provides an interface for user input; and the processor 1201 can be configured to invoke a device control application stored in the memory 1205 to execute the data transmission method described in any of the preceding embodiments, which will not be repeated here. Furthermore, the beneficial effects of employing the same method will not be repeated here.

[0251] In addition, it should be noted that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the aforementioned data transmission device (including data transmission device 1000 and data transmission device 1100), and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the data transmission method in the above embodiment, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0252] The computer-readable storage medium may be the data transmission device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0253] In addition, it should be noted that the embodiments of the present application also provide a computer program product, including a computer program, the computer program being stored in a computer-readable storage medium, and the computer program being suitable for being read and executed by a processor, so that a computer device having a processor performs the method provided in any of the corresponding embodiments described above. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiments involved in this application, please refer to the description of the method embodiments of this application.

[0254] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0255] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0256] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data transmission method, characterized in that: The method is applied to a data transmission system, the data transmission system including a data sending end and a data receiving end, the data sending end including a first graphics processing unit and a first network transmission unit, the data receiving end including a second graphics processing unit, the method being executed by the first graphics processing unit, and the method including: Acquire a data processing task carrying service characterization data; the data processing task includes a task of sending the service characterization data to an expert model in the second graphics processing unit; Based on N first queue pairs associated with the expert model, block partitioning processing is performed on the service characterization data to obtain data partition blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1; If a target first queue pair that meets the data transmission condition exists among the N first queue pairs, sending the target first queue pair to the first network transmission unit, so that the first network transmission unit obtains a target partition block that matches the target first queue pair, and sends the target partition block to the second graphics processing unit through the target first network port among the N first network ports; The second graphics processing unit is used to send the target partition block to the expert model, the target first network port is determined by the first network transmission unit from the N first network ports based on the communication link parameters of the target first queue pair; the target partition block is a partition block in the data partition block.

2. The method according to claim 1, characterized in that The block division processing of the service characterization data based on the N first queue pairs associated with the expert model to obtain data division blocks of the service characterization data includes: Obtaining a queue pair relationship mapping table maintained by the first graphics processing unit; the queue pair relationship mapping table includes at least the N groups of queue pairs associated with the expert model; the N groups of queue pairs include N first queue pairs and second queue pairs respectively mapped to the N first queue pairs; Obtain the N first queue pairs from the N groups of queue pairs; Determining the number of queues of the N first queue pairs as a first number of queues, determining N initial partition blocks corresponding to the service characterization data based on the first number of queues, and determining the amount of partitioned data of each initial partition block; the amount of partitioned data of each initial partition block is determined based on the first number of queues and the amount of data of the service characterization data; Based on a maximum transmission unit associated with the first network transmission unit and an amount of divided data of each initial divided block, block division processing is performed on each initial divided block to obtain sub-divided blocks of each initial divided block, and the sub-divided blocks of each initial divided block are determined as data divided blocks of the service characterization data.

3. The method according to claim 1, characterized in that The first graphics processing unit and the second graphics processing unit are graphics processing units in a communication group of the data transmission system; The method further comprises: Determine the port quantity of the N first network ports as a first port quantity, and obtain N first queue pairs pre-configured by the first graphics processing unit and matching the first port quantity from the communication group; Obtaining, from the communication group, N second queue pairs pre-configured by the second graphics processing unit for the expert model, establishing queue pair mapping relationships between the N first queue pairs and the N second queue pairs, and determining, based on the established queue pair mapping relationships, second queue pairs to which the N first queue pairs are respectively mapped; Constructing the N groups of queue pairs associated with the expert model based on each first queue pair and the second queue pair mapped to each first queue pair; Based on the N groups of queue pairs, the queue pair relationship mapping table is determined.

4. The method according to claim 1, wherein The method further comprises: Obtain a first queue pair i and a first queue pair j from the N first queue pairs; i and j are integers less than or equal to N, and i is not equal to j; A first control signal storage space is configured for the first queue pair i; the first control signal storage space is used to store a control signal sent by a second queue pair that has a queue pair mapping relationship with the first queue pair i; A second control signal storage space is configured for the first queue pair j; the second control signal storage space is used to store the control signal sent by the second queue pair that has a queue pair mapping relationship with the first queue pair j; the space size of the second control signal storage space is consistent with the space size of the first control signal storage space.

5. The method according to claim 2, characterized in that The N initial partition blocks include a first initial partition block and a second initial partition block; The step of determining the amount of divided data for each initial divided block includes: Calculating the ratio of the data volume of the service characterization data to N; If the ratio is an integer, determining that the data amount of the service characterization data is divisible by N, and using the ratio as the data amount of the first initial partition block and the second initial partition block respectively; If the ratio is a non-integer, it is determined that the data amount of the service characterization data is not divisible by N, the ratio is rounded to obtain an integer value of the ratio, and when the integer value is used as the divided data amount of the first initial partition block, the difference between the data amount of the service characterization data and the integer value is used as the divided data amount of the second initial partition block.

6. The method according to claim 1 or 2, characterized in that The storage address of the first graphics processing unit is a first storage address, and the storage address of the second graphics processing unit is a second storage address; the first graphics processing unit stores a unit data volume of a maximum transmission unit associated with the first network transmission unit; The method further comprises: When obtaining the data partition blocks of the service representation data, storing the data partition blocks in a first storage space corresponding to the first storage address; The queue element of each first queue pair is determined by the first storage address, the second storage address and the block data volume of the data partition block; the queue element of each first queue pair is used to instruct the first network transmission unit to obtain the target partition block that matches the queue element of the target first queue pair from the first storage space; the block data volume of the data partition block is determined based on the number of queues of the N first queue pairs, the data volume of the service characterization data and the unit data volume of the maximum transmission unit.

7. The method according to claim 6, characterized in that The data partition blocks include at least a first partition block and a second partition block; The method further comprises: Invoking a first business processing thread configured for the first partitioned block, determining the first storage address and the second storage address, and performing queue element encapsulation processing on the first storage address, the second storage address, and the block data volume of the first partitioned block by the first business processing thread to obtain a first queue element corresponding to the first partitioned block; calling a second business processing thread configured for the second partitioned block, determining the first storage address and the second storage address, and performing queue element encapsulation processing on the first storage address, the second storage address, and the block data volume of the second partitioned block by the second business processing thread to obtain a second queue element corresponding to the second partitioned block; The first queue element is configured to the target first queue pair through the first business processing thread, and the second queue element is configured to the target first queue pair through the second business processing thread to obtain the queue element of the target first queue pair.

8. The method according to claim 7, characterized in that The target first queue pair includes at least a first reserved position and a second reserved position for storing queue elements; The method further comprises: When allocating the first queue element to the target first queue pair through the first business processing thread, writing the first queue element to the first reserved position in the target first queue pair; When the second queue element is configured to the target first queue pair through the second business processing thread, the second queue element is written to the second reserved position in the target first queue pair.

9. The method according to claim 7, characterized in that The sending the target first queue pair to the first network transmission unit includes: Call the data transmission thread configured for the target first queue pair, encapsulate the queue elements in the target first queue pair as a data transmission request, and send the data transmission request to the first network transmission unit, so that the first network transmission unit executes the step of obtaining the target partition block matching the queue elements of the target first queue pair from the first storage space according to the data transmission request.

10. The method according to claim 1, characterized in that The method further comprises: Obtaining the task type of the data processing task; If the task type is a target task type that can be processed with low latency, the process jumps to executing a step of performing block division processing on the service characterization data based on the N first queue pairs associated with the expert model to obtain data division blocks of the service characterization data.

11. The method according to claim 1, wherein The data processing task also includes a model reasoning task of performing model reasoning using the expert model deployed in the hybrid business model; the model reasoning task refers to a task of reasoning on business sample data using the expert model; The method further comprises: Obtaining the business sample data corresponding to the model reasoning task; Acquire business text information associated with the business sample data, perform text segmentation processing on the business text information, and obtain text segmentation of the business text information; Vector representation processing is performed on the text segmentation to obtain a text representation vector for the text segmentation, and the text representation vector is used as business representation data in the data processing task.

12. The method according to claim 1, characterized in that The queue pairs in the N groups of queue pairs include N second queue pairs configured for the expert model; The method further comprises: After sending each of the data partition blocks associated with the service characterization data to the second graphics processing unit, calling a streaming multiprocessor to count the amount of the service characterization data in the data processing task for sending to the expert model; Generate sending end indication information according to the counted number of the service characterization data, and send the sending end indication information to the second graphics processing unit, so that the second graphics processing unit stores the sending end indication information in the control signal storage space configured for the N second queue pairs.

13. A data transmission method, characterized in that: The data transmission system includes a data sending end and a data receiving end, the data sending end includes a first graphics processing unit and a first network transmission unit, the data receiving end includes a second graphics processing unit, and the method is performed by the second graphics processing unit. The method includes: Obtaining a target partition block matching a target first queue pair, wherein the target partition block is sent by the first network transmission unit through a target first network port among a plurality of first network ports provided by the first network transmission unit; the target partition block is a partition block matching the target first queue pair obtained from a data partition block by the first network transmission unit after receiving the target first queue pair in N first queues sent by the first graphics processing unit; the target first queue pair is a first queue pair among the N first queue pairs that meets a data sending condition; the data partition block is obtained by the first graphics processing unit performing block partitioning processing on the service characterization data according to N first queue pairs associated with an expert model after obtaining a data processing task carrying service characterization data, each first queue pair being a queue pair in N groups of queue pairs associated with the expert model, N being an integer greater than 1, and the data processing task including a task of sending the service characterization data to the expert model in the second graphics processing unit; the target first network port is determined by the first network transmission unit from the N first network ports based on communication link parameters of the target first queue pair; Determining a target second queue pair mapped to the target first queue pair from N second queue pairs associated with the expert model; each second queue pair is a queue pair mapped to the first queue pair in the N groups of queue pairs; When sending end indication information associated with the target second queue is detected, it is determined that each of the divided data blocks sent as the target divided blocks has been obtained, and the service characterization data corresponding to the divided data blocks is sent to the expert model, where the sending end indication information is obtained by the first graphics processing unit performing quantity statistics on the sent service characterization data.

14. The method according to claim 13, characterized in that The target partition block is a partition block carried in a service data packet, and the service data packet also carries a queue element of the target first queue pair; The queue element of the target first queue pair is used to indicate a second storage address of the target partition block in the second graphics processing unit; The method further comprises: When the target partition block is obtained, the target partition block is stored in the second storage address based on the queue element of the target first queue pair carried in the service data packet.

15. The method according to claim 13, characterized in that The method of determining, upon detecting the sending end indication information associated with the target second queue, that each of the divided data blocks sent as the target divided blocks has been acquired, and sending the service representation data corresponding to the divided data blocks to the expert model includes: detecting, in a control signal storage space configured for the target second queue pair, sending end indication information associated with the target second queue pair; Upon detecting that the sending end indication information exists in the control signal storage space, determining that each of the divided data blocks sent as the target divided blocks has been acquired, transmitting the sending end indication information to the expert model, and generating, through the expert model, a data read request for reading the service representation data; The expert model is controlled to read the service representation data from a second storage space corresponding to a second storage address according to the data read request, where the second storage address is a storage address storing the target partition block in the second graphics processing unit.

16. A data transmission device, characterized in that: The device comprises: a task acquisition unit, configured to acquire a data processing task carrying service characterization data; the data processing task including a task of sending the service characterization data to an expert model in the second graphics processing unit; a data partitioning unit, configured to perform block partitioning processing on the service characterization data based on N first queue pairs associated with the expert model to obtain data partitioned blocks of the service characterization data; each first queue pair is a queue pair in the N groups of queue pairs associated with the expert model, where N is an integer greater than 1; a queue pair sending unit, configured to, if a target first queue pair that meets the data sending condition exists among the N first queue pairs, send the target first queue pair to the first network transmission unit, so that the first network transmission unit obtains a target partition block that matches the target first queue pair, and sends the target partition block to the second graphics processing unit through the target first network port among the N first network ports; The second graphics processing unit is used to send the target partition block to the expert model, the target first network port is determined by the first network transmission unit from the N first network ports based on the communication link parameters of the target first queue pair; the target partition block is a partition block in the data partition block.

17. A data transmission device, characterized in that: The device comprises: a partition block acquisition unit, configured to acquire a target partition block matching a target first queue pair, wherein the target partition block is sent by the first network transmission unit via a target first network port among a plurality of first network ports provided by the first network transmission unit; the target partition block is a partition block matching the target first queue pair acquired from a data partition block by the first network transmission unit after receiving the target first queue pair from the N first queues sent by the first graphics processing unit; the target first queue pair is a first queue pair among the N first queue pairs that meets a data sending condition; the data partition block is obtained by the first graphics processing unit performing block partitioning processing on the service characterization data according to the N first queue pairs associated with the expert model after acquiring a data processing task carrying service characterization data, each first queue pair being a queue pair among the N groups of queue pairs associated with the expert model, N being an integer greater than 1, and the data processing task comprising a task of sending the service characterization data to the expert model in the second graphics processing unit; the target first network port is determined by the first network transmission unit from the N first network ports based on communication link parameters of the target first queue pair; a first queue pair obtaining unit, configured to determine a target second queue pair mapped to the target first queue pair from N second queue pairs associated with the expert model; each second queue pair being a queue pair mapped to a first queue pair in the N groups of queue pairs; a data sending unit, configured to, upon detecting sending end indication information associated with the target second queue, determine that each of the divided data blocks sent as the target divided blocks has been acquired, and send the service representation data corresponding to the divided data block to the expert model, wherein the sending end indication information is obtained by the first graphics processing unit performing quantity statistics on the sent service representation data.

18. A computer device, characterized in that: The computer device comprises: a processor adapted to execute one or more computer programs; and A computer-readable storage medium storing one or more computer programs, wherein the one or more computer programs are suitable for being loaded by the processor and executing the data transmission method according to any one of claims 1 to 15.

19. A computer-readable storage medium, characterized in that The computer-readable storage medium stores one or more computer programs, and the one or more computer programs are suitable for being loaded by a processor and executing the data transmission method according to any one of claims 1 to 15.

20. A computer program product, characterized in that The computer program product includes a computer program, which is stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the data transmission method according to any one of claims 1 to 15.

Images