Data transmission method, apparatus, device, storage medium, and program product

Abstract

The application discloses a data transmission method and device, equipment, a storage medium and a program product, and relates to the technical field of communication. A first processing device acquires first data; acquires second data; transmits the first data through a first master processing unit of at least two master processing units corresponding to the first processing device; and transmits the second data through a second master processing unit and a slave processing unit of the at least two master processing units based on the data volume of the second data being greater than the data volume of the first data. The method accelerates the transmission of the second data through the slave processing unit, shortens the transmission completion time length of the second data, avoids a large difference between the transmission completion time length of the second data and the transmission completion time length of the first data, and further avoids a long tail phenomenon caused by the large data volume of the second data, thereby shortening the overall completion time length of data with different data volumes and improving the data transmission efficiency.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to data transmission methods, apparatus, devices, storage media, and program products. Background Technology

[0002] In the field of communication technology, there is a need for data transmission between computing nodes within a computing cluster. For example, a first computing node transmits data to multiple second computing nodes. If the first computing node processes different amounts of data transmitted from one second computing node to different second computing nodes using the same method, the transmission time for larger amounts of data may be longer than that for smaller amounts, resulting in a long-tail phenomenon in the transmission of large amounts of data. Summary of the Invention

[0003] This application provides a data transmission method, apparatus, device, storage medium, and program product for shortening the data transmission completion time for different data volumes.

[0004] In a first aspect, a data transmission method is provided. Taking the execution of the method by a first processing device as an example, the first processing device acquires first data; acquires second data; transmits the first data through a first main processing unit among at least two main processing units corresponding to the first processing device; and, based on the fact that the amount of the second data is greater than the amount of the first data, transmits the second data through a second main processing unit and a slave processing unit among at least two main processing units.

[0005] This method, when transmitting first data and second data with different data volumes, transmits the first data with a smaller data volume through the first main processing unit, and transmits the second data with a larger data volume through the second main processing unit and the slave processing unit. By accelerating the transmission of the second data through the slave processing unit, the transmission completion time of the second data is shortened, avoiding a large difference between the transmission completion time of the second data and the transmission completion time of the first data. This avoids the long-tail phenomenon caused by the large data volume of the second data, thereby shortening the overall completion time of transmitting data with different data volumes and improving data transmission efficiency.

[0006] In one possible implementation, the amount of second data transmitted by the second main processing unit and the second processing unit is determined based on their respective transmission speeds. In this approach, the amount of second data transmitted by the second main processing unit and the second processing unit is not statically allocated, but dynamically determined according to their respective transmission speeds. This allows the processing unit with the faster transmission speed to transmit more data, minimizing the transmission time of the second data and preventing the transmission completion time of the second data from being slowed down by the transmission bottleneck of a single processing unit.

[0007] In one possible implementation, the transmission links corresponding to the second main processing unit and the slave processing unit are different. In this manner, the second data is transmitted on different transmission links, thereby extending the transmission links compared to the transmission link corresponding to the second main processing unit, and thus expanding the bandwidth resources for transmitting the second data, enabling accelerated transmission of the second data through the additional extended bandwidth resources.

[0008] In one possible implementation, the receiving end of the second data is a second processing device, the transmission link corresponding to the second main processing unit is a direct link from the first processing device to the second processing device, and the transmission link corresponding to the secondary processing unit is a relay link from the first processing device to an intermediate device and then to the second processing device. In this method, the additional bandwidth resources are expanded through the relay link.

[0009] In one possible implementation, the first and second processing units are processing units in a computing cluster, and the intermediate unit is a host connected to the computing cluster; alternatively, the intermediate unit is a third processing unit in the computing cluster connected to the first and second processing units. In this approach, the host and the third processing unit can be flexibly utilized to implement the function of the relay device.

[0010] In one possible implementation, transmitting the second data from the processing unit may include: transmitting the second data from the processing unit via the relay link when the relay link is idle. Since the relay link requires the use of a link connected to the relay device, and this link may transmit data for other services, transmitting the second data via the relay link only when it is determined that the relay link is idle avoids transmission delays or packet loss caused by relay link congestion.

[0011] In one possible implementation, the second main processing unit and the slave processing unit share the same transmission link. In this approach, by having two processing units utilize the bandwidth resources of the same transmission link to transmit the second data, compared to having one processing unit utilize the bandwidth resources of the same transmission link, the bandwidth utilization of the transmission link can be improved. Furthermore, if there are equivalent multipaths on the transmission link, the bandwidth resources of the multiple paths can be fully utilized, thereby achieving accelerated transmission of the second data.

[0012] In one possible implementation, the method of transmitting second data through a second main processing unit and a slave processing unit in at least two main processing units may include: transmitting a first unit of data in the second data through the second main processing unit, and transmitting a second unit of data in the second data through the slave processing unit; when the second main processing unit has completed transmitting the first unit of data, transmitting the next first unit of data in the second data through the second main processing unit; when the slave processing unit has completed transmitting the second unit of data, transmitting the next second unit of data in the second data through the slave processing unit; and so on, until the transmission of the second data is completed. In this method, the second main processing unit transmits data sequentially from one position of the second data according to the granularity of the first unit of data, and the slave processing unit transmits data sequentially from another position of the second data according to the granularity of the second unit of data. If the transmission speed of the second main processing unit is faster, the second main processing unit will transmit more first unit of data; if the transmission speed of the slave processing unit is faster, the slave processing unit will transmit more second unit of data, so that the amount of data transmitted is determined by the transmission speed.

[0013] In another possible implementation, the method of transmitting the second data through a second main processing unit and a slave processing unit in at least two main processing units may include: dividing the second data into a first part and a second part by the second main processing unit; transmitting the first part of the data through the second main processing unit and transmitting the second part of the data through the slave processing unit. Since the transmission links corresponding to the second main processing unit and the slave processing unit are the same, resource allocation can be directly achieved by dividing the second data, simplifying the implementation process of transmitting the second data through the second main processing unit and the slave processing unit.

[0014] In one possible implementation, the method further includes: issuing a processing task to a slave processing unit via a second main processing unit, the processing task instructing the slave processing unit to transmit corresponding data from the second data. Thus, by issuing a processing task from the second main processing unit to the slave processing unit, the allocation of the second data is achieved, enabling the second main processing unit and the slave processing unit to collaboratively achieve parallel transmission of the second data based on the issuance and execution of the processing task.

[0015] In one possible implementation, when the second data is allocated by issuing processing tasks to the slave processing unit via the second main processing unit, the first processing device includes a memory region for task synchronization between the second main processing unit and the slave processing unit. Thus, by storing information related to the processing tasks in a memory region, the second main processing unit and the slave processing unit achieve task synchronization by reading or writing information in the memory region.

[0016] In one possible implementation, the memory region stores at least one of a task flag, an address flag, a granularity flag, a link flag, a completion flag, or an end flag. The task flag indicates whether the second main processing unit has issued a processing task; the address flag indicates the address offset of the data corresponding to the issued processing task; the granularity flag indicates the length of the unit data corresponding to the issued processing task; the link flag indicates whether the transmission link corresponding to the slave processing unit is idle; the completion flag indicates whether the slave processing unit has completed the issued processing task; and the end flag indicates whether the second data transmission is complete. Thus, by storing multiple different flags in the memory region, the second main processing unit and the slave processing unit can flexibly implement multiple task synchronization methods using various flags.

[0017] In one possible implementation, the first processing device is a processing device in a computing cluster, which includes n processing devices. These n processing devices transmit data to each other. The second data consists of data that meets acceleration conditions among the data transmitted between the n processing devices. Acceleration conditions include the data being ranked in descending order of size (the top n data items), and the sending and receiving ends of data of the same size level being different. The first data is all data other than the second data. This method can be applied to computing cluster scenarios to shorten the data transmission completion time between n processing devices in a computing cluster.

[0018] In one possible implementation, the second data also satisfies a benefit condition, which includes the proportion of the data volume difference to the total data volume of the second data being greater than a proportion threshold. The data volume difference is the difference between the data volume of the second data and the data volume of the first data. Therefore, when the difference between the data volumes of the second data and the first data is small, there is no need to enable acceleration from the processing unit, avoiding performance degradation caused by improper resource allocation during acceleration.

[0019] In one possible implementation, at least two main processing units are main threads, and slave processing units are slave threads. Therefore, this method can be applied to multi-threaded implementation scenarios to accelerate the transmission of second data through multi-threading.

[0020] In one possible implementation, the first processing device is a server, a neural processing unit (NPU), a graphics processing unit (GPU), a tensor processing unit (TPU), or a central processing unit (CPU). Therefore, this method can be flexibly applied to data transmission scenarios involving multiple servers, multiple NPUs, multiple GPUs, multiple TPUs, or multiple CPUs.

[0021] In one possible implementation, the first and second data are data from a variable all-to-all (all-to-allvariable, all-to-allv) set communication operation. This allows the method to improve the execution speed of the all-to-allv set communication operation.

[0022] Secondly, a data transmission device is provided, which includes a transceiver module and a processing module.

[0023] The transceiver module is used to perform receiving and / or transmitting related operations in the first aspect or any possible implementation of the first aspect; the processing module is used to perform other operations besides receiving and / or transmitting related operations in the first aspect or any possible implementation of the first aspect.

[0024] In one possible implementation, the transceiver module includes a receiving module and / or a sending module. The receiving module is used to perform receiving-related operations, and the sending module is used to perform sending-related operations.

[0025] In one possible implementation, the processing module is used to acquire first data; acquire second data; and the transceiver module is used to transmit the first data through a first main processing unit among at least two main processing units corresponding to the first processing device; and to transmit the second data through a second main processing unit and a slave processing unit among at least two main processing units, based on the fact that the amount of the second data is greater than the amount of the first data.

[0026] In one possible implementation, the amount of second data transmitted by the second main processing unit and the second processing unit is determined based on the transmission speed of the second main processing unit and the second processing unit, respectively.

[0027] In one possible implementation, the transmission links corresponding to the second master processing unit and the slave processing unit are different.

[0028] In one possible implementation, the receiving end of the second data is a second processing device, the transmission link corresponding to the second main processing unit is a direct link from the first processing device to the second processing device, and the transmission link corresponding to the processing unit is a relay link from the first processing device to the intermediate device and then to the second processing device.

[0029] In one possible implementation, the first processing device and the second processing device are processing devices in a computing cluster, and the intermediate device is a host connected to the computing cluster, or the intermediate device is a third processing device in the computing cluster connected to the first processing device and the second processing device.

[0030] In one possible implementation, the transceiver module is configured to transmit second data from the processing unit via the relay link when the relay link is idle.

[0031] In one possible implementation, the second master processing unit has the same transmission link as the slave processing unit.

[0032] In one possible implementation, the transceiver module is configured to transmit a first unit of data in the second data through a second main processing unit, and transmit a second unit of data in the second data through a processing unit; when the second main processing unit has completed transmitting the first unit of data, transmit the next first unit of data in the second data through the second main processing unit; when the processing unit has completed transmitting the second unit of data, transmit the next second unit of data in the second data through a processing unit; and so on, until the second data transmission is completed.

[0033] In one possible implementation, the processing module is used to divide the second data into a first part of data and a second part of data through the second main processing unit; the transceiver module is used to transmit the first part of data through the second main processing unit and transmit the second part of data from the processing unit.

[0034] In one possible implementation, the transceiver module is further configured to issue a processing task to the slave processing unit through the second main processing unit, the processing task being used to instruct the slave processing unit to transmit the corresponding data in the second data.

[0035] In one possible implementation, the first processing device includes a memory region for task synchronization between a second main processing unit and a slave processing unit.

[0036] In one possible implementation, the memory region stores at least one of a task flag, an address flag, a granularity flag, a link flag, a completion flag, or an end flag. The task flag is used to indicate whether the second main processing unit has issued a processing task, the address flag is used to indicate the address offset of the data corresponding to the issued processing task, the granularity flag is used to indicate the length of the unit data corresponding to the issued processing task, the link flag is used to indicate whether the transmission link corresponding to the slave processing unit is idle, the completion flag is used to indicate whether the slave processing unit has completed the issued processing task, and the end flag is used to indicate whether the second data has been transmitted.

[0037] In one possible implementation, the first processing device is a processing device in a computing cluster, which includes n processing devices. The n processing devices transmit data to each other. The second data is the data that meets the acceleration conditions among the data transmitted between the n processing devices. The acceleration conditions include the data volume being ranked in the top n from largest to smallest, and the sending and receiving ends of data of the same data volume level being different. The first data is the data other than the second data.

[0038] In one possible implementation, the second data also satisfies a benefit condition, which includes the proportion of the data volume difference to the data volume of the second data being greater than a proportion threshold, where the data volume difference is the difference between the data volume of the second data and the data volume of the first data.

[0039] In one possible implementation, at least two main processing units are main threads, and the slave processing units are slave threads.

[0040] In one possible implementation, the first processing device is a server, NPU, GPU, TPU, or CPU.

[0041] In one possible implementation, the first data and the second data are data from the alltoallv set communication operation.

[0042] Thirdly, a data transmission device is provided, comprising: a processor coupled to a memory storing at least one program instruction or code, the at least one program instruction or code being loaded and executed by the processor to enable the data transmission device to implement the method as described in the first aspect or any possible implementation thereof.

[0043] Optionally, the processor may be one or more, and the memory may be one or more.

[0044] Optionally, the memory may be integrated with the processor, or the memory may be separated from the processor.

[0045] In the specific implementation process, the memory can be a non-transitory memory, such as read-only memory (ROM), which can be integrated with the processor on the same chip or set on different chips. This application does not limit the type of memory or the way the memory and processor are set.

[0046] Fourthly, a computer-readable storage medium is provided, the storage medium storing at least one instruction, the instruction being loaded and executed by a processor to cause a computer to implement the method described in the first aspect or any possible implementation of the first aspect.

[0047] Fifthly, a computer program (product) is provided, the computer program (product) comprising: computer program code, which, when executed by a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0048] In a sixth aspect, a chip is provided, including a processor for retrieving and executing instructions stored in a memory, causing a device on which the chip is mounted to perform the methods described in the first aspect or any possible implementation thereof.

[0049] In a seventh aspect, another chip is provided, comprising: an input interface, an output interface, a processor, and a memory, wherein the input interface, the output interface, the processor, and the memory are connected via an internal connection path, and the processor is configured to execute code in the memory, wherein when the code is executed, the processor is configured to perform the method described in the first aspect or any possible implementation thereof.

[0050] It should be understood that the beneficial effects achieved by the technical solutions of the second to seventh aspects of this application and their corresponding possible implementations can be found in the technical effects of the first aspect and its corresponding possible implementations described above, and will not be repeated here. Furthermore, the data transmission device mentioned in the second aspect may be the chip mentioned in the sixth or seventh aspect, or the data transmission device may also be the device mentioned in the third aspect. Attached Figure Description

[0051] Figure 1 A schematic diagram illustrating a collection communication operation provided in an embodiment of this application;

[0052] Figure 2 A schematic diagram of a network topology provided for an embodiment of this application;

[0053] Figure 3A schematic diagram of a data transmission matrix and transmission completion time provided for an embodiment of this application;

[0054] Figure 4 A schematic diagram illustrating the implementation environment of a data transmission method provided in this application embodiment;

[0055] Figure 5 A flowchart illustrating a data transmission method provided in an embodiment of this application;

[0056] Figure 6 This application provides a schematic diagram for screening long tail currents.

[0057] Figure 7 A schematic diagram illustrating a data transfer process provided in an embodiment of this application;

[0058] Figure 8 A schematic diagram of a transmission link provided for an embodiment of this application;

[0059] Figure 9 A schematic diagram illustrating a data transfer process provided in an embodiment of this application;

[0060] Figure 10 This is a schematic diagram illustrating data transmission via a host relay, provided as an embodiment of this application.

[0061] Figure 11 This application provides a schematic diagram of the state transition of the main thread in an embodiment of the present application.

[0062] Figure 12 This application provides a schematic diagram of a state transition from a thread in an embodiment of the present application.

[0063] Figure 13 This application provides a schematic diagram of the state transition of a nested thread.

[0064] Figure 14 A schematic diagram illustrating inter-thread interaction provided in an embodiment of this application;

[0065] Figure 15 A schematic diagram illustrating another inter-thread interaction provided in an embodiment of this application;

[0066] Figure 16 A flowchart illustrating another data transmission method provided in this application embodiment;

[0067] Figure 17 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of this application;

[0068] Figure 18 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of this application;

[0069] Figure 19 This is a schematic diagram of another data transmission device provided in an embodiment of this application. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0071] In the field of communication technology, computing clusters are widely used in scenarios such as distributed systems or distributed models. Taking recommendation models as an example, distributed recommendation models are a technique that decomposes large-scale data and computational tasks into multiple smaller tasks and processes them in parallel on multiple computing nodes within a computing cluster. This aims to solve the problems of large data volumes and computational efficiency, thereby achieving more efficient recommendations.

[0072] For example, the embedding tables in the recommendation model are distributed across different computing nodes and require data exchange via collective communication (CC) before subsequent computation can begin. The embedding table maps discrete features (such as user identifiers, item identifiers, etc.) to low-dimensional dense vectors, enabling the model to process these vectors more effectively. Therefore, collective communication is a necessary operation for recommendation model applications, and its completion time significantly accounts for the end-to-end time consumption of the recommendation model. In this scenario, it is crucial to reduce the completion time of collective communication within the computing cluster to significantly improve the inference throughput of the recommendation model, thereby enhancing the user experience.

[0073] Collective communication is a global communication operation involving all computing nodes within a computing cluster. Optionally, collective communication includes, but is not limited to, operations such as broadcast, scatter, reduce-scatter, all-to-all, and all-to-allv. Specifically, a broadcast operation broadcasts data from one computing node to all other computing nodes; a scatter operation splits data and distributes it to different computing nodes; and a reduce-scatter operation performs a reduce operation on the data from all computing nodes and then distributes the result back to each computing node. Reduce operations include, but are not limited to: calculating the sum, calculating the product, finding the maximum value, finding the minimum value, logical AND, bitwise AND, logical OR, bitwise OR, logical XOR, and bitwise XOR.

[0074] The alltoall operation is a full exchange operation where each compute node sends data to all other compute nodes and also receives data from all other compute nodes. In other words, each compute node participating in the alltoall set communication exchanges data with other compute nodes. For example, see [link to example]. Figure 1 The diagram illustrates the interaction of the alltoall operation. Each compute node's receive and send buffers are arrays divided into several data blocks. The specific operation of alltoall is as follows: compute node i sends the j-th data block from its send buffer to compute node j, and compute node j places the received data block from compute node i into the i-th position of its receive buffer. After the alltoall set communication operation is completed, its effect is equivalent to a transposition of the data held by all compute nodes.

[0075] In all-to-all set communication, the data size sent and received by each computing node is the same, which is 1 / N of the input data, where N is the total number of nodes participating in the all-to-all set communication. The all-to-allv operation is similar to all-to-all, except that the data size sent and received by each computing node in the all-to-allv set communication operation can be freely specified. The all-to-allv set communication operation requires the amount of data transmitted between each operator as input, which means that the data sizes sent and received by each computing node may differ.

[0076] In related technologies, taking a network topology in which all computing nodes participating in the distributed recommendation model calculation within a computing cluster are interconnected to form a full mesh, as an example, each computing node reduces the completion time of one-to-many data transmission by enabling N-1 threads to transmit data to other N-1 computing nodes in parallel, thereby reducing the overall completion time of aggregate communication.

[0077] For example, such as Figure 2 As shown, the compute nodes are NPUs, and any two NPUs are connected via a high-speed direct link with identical bidirectional bandwidth. Figure 2 In the full mesh network topology shown, taking the example of 8 NPUs participating in all-to-all aggregate communication, each NPU will enable 7 threads to transmit data to the other 7 NPUs in parallel. Among them, one thread is used to transmit data to the corresponding NPU through a direct link, so as to make full use of all direct links. Figure 2 The example shown is of any NPU sending data to all other NPUs. Figure 2 The black arrow in the middle represents a direct connection link.

[0078] However, since the bandwidth of direct links is the same and there are no additional allocated bandwidth resources, the transmission speeds of each thread are similar. Therefore, if the amount of data transmitted from any NPU to different NPUs is uneven, such as in the all-to-allv aggregation communication scenario mentioned above, the transmission time for large data volumes will be longer than that for small data volumes, resulting in a longer overall completion time for aggregation communication and exhibiting a long-tail phenomenon. For example, in the practical application of distributed recommendation models, due to the different number of entries hit in the embedding table query, the maximum difference in the amount of data that needs to be transmitted between different NPUs can be as much as two times. Transmission of large data volumes will lead to a significant long-tail phenomenon, failing to effectively reduce the overall completion time of aggregation communication.

[0079] Taking a computing cluster consisting of 8 compute nodes as an example, see [link / reference]. Figure 3 The diagram shows the data transmission matrix and transmission completion time. The values ​​in the data transmission matrix represent data volume levels; the data volume level is positively correlated with the data size, with data volume level 2 representing twice the data volume of data volume level 1. For example... Figure 3 As shown, NPU1 sends data to NPU8 (i.e., ... Figure 3 The data volume level transmitted from NPU2 to NPU8 (i.e., 1 to 8) is 1. Figure 3 The data volume level transferred from NPU7 to NPU8 (i.e., 2 to 8) is 1.8. Figure 3 The data volume level of the 7 to 8 transfer is 1. It can be seen that the amount of data transferred from NPU2 to NPU8 is 1.8 times the amount of data transferred from other NPUs to NPU8. Figure 3 In the transmission completion times shown, the length of the horizontal bar represents the time required for the transmission to complete. The transmission completion time for 2 to 8 is significantly longer than that for 1 to 8.

[0080] In summary, for one-to-many data transmission scenarios in computing clusters, such as scenarios involving scatter operations or all-to-allv group communication, if any computing node sends different amounts of data to different computing nodes, but the data transmission method is the same, the transmission time of large data volumes may be longer than that of small data volumes. This results in a long-tail phenomenon in the transmission of large data volumes, leading to a longer overall completion time for one-to-many data transmission. The large data volumes can be referred to as long-tail data, and because the data is transmitted as a stream during transmission, long-tail data is also called a long-tail stream.

[0081] This application provides a data transmission method that can identify long-tail data in data of varying sizes. By accelerating the processing of long-tail data from the processing unit, the transmission latency of long-tail data is reduced, thereby reducing the overall completion time for transmitting data of varying sizes. The data transmission method provided in this application can be applied to any data transmission scenario, such as the ensemble communication scenario of the aforementioned distributed recommendation model, or the training scenario of other large artificial intelligence (AI) models. In one possible implementation, this method can be integrated as an ensemble communication operator into various ensemble communication libraries, providing a callable ensemble communication interface for upper-layer deep learning frameworks.

[0082] Optionally, the implementation environment of the data transmission method provided in this application embodiment includes multiple processing devices, which are directly or indirectly interconnected. These multiple processing devices can be multiple computing nodes in a computing cluster. In this application embodiment, the processing device can be a server or an xPU, where xPU represents different types of processing units, such as NPU, GPU, TPU, or CPU. For example, when the processing device is an xPU, the implementation environment of the data transmission method can be as follows: Figure 4 As shown in (1) or (2).

[0083] exist Figure 4 In (1), multiple servers are interconnected, different processing devices are deployed on different servers, and data transmission between different processing devices is achieved through the server's network interface card (NIC). The servers can be directly connected via Ethernet cables or fiber optic cables, or indirectly connected via network devices such as switches. This application does not limit the number of switches used in the indirect connection method; any number sufficient to enable interconnection between multiple servers is acceptable. Figure 4 (1) is illustrated using a connection through one switch as an example. In practical applications, the number of switches can be any number of more switches.

[0084] exist Figure 4 In (2), multiple processing devices are interconnected, and data transmission between different processing devices is achieved through a switching chip. All processing devices and the switching chip reside within the same server. The processing devices can be connected via links corresponding to high-speed interconnect expansion bus standards, such as peripheral component interconnect express (PCIe) links; or they can be indirectly connected via switching chips. This application does not limit the number of switching chips used in the indirect connection method, as long as it enables the interconnection of multiple processing devices. Figure 4(2) is illustrated using a connection through one switching chip as an example. In practical applications, the number of switching chips can be any number of more.

[0085] This application embodiment uses the execution of the method by a first processing device as an example for illustration. The first processing device can be... Figure 4 For any processing device in the implementation environment shown, the implementation method of other processing devices executing this method can be referred to the implementation method of the first processing device executing this method, and will not be described in detail again. See also Figure 5 The flowchart shown illustrates a data transmission method, which includes the following steps 501-504.

[0086] Step 501: Obtain the first data.

[0087] In this embodiment, the first data can be obtained by reading the first data locally stored in the first processing device, or by performing calculations or other processing on the data locally stored or received by the first processing device, and obtaining the first data based on the processed data. The first data can refer to one or more data blocks, or it can refer to data within a storage area, or it can refer to data corresponding to a specific service.

[0088] Step 502: Obtain the second data.

[0089] The method for obtaining the second data is similar to that for obtaining the first data, and will not be repeated here. The sources of the first and second data can be the same or different. Furthermore, the first and second data have different data volumes. The first and second data are data sent to different second processing devices, although they can also be data sent to the same second processing device. Optionally, the first and second data can be obtained simultaneously or separately; this application embodiment does not limit the order in which different data are obtained. In addition to obtaining the first and second data, third and fourth data can also be obtained, and this application embodiment does not limit the number of different data to be obtained. Taking the acquisition of first, second, third, and fourth data as an example, the data volumes of some data can be the same, but the data volumes of all data must not be completely identical; that is, at least one data volume is different from the data volumes of the others.

[0090] In one possible implementation, a first processing device acquires data transmitted to a plurality of second processing devices, wherein the amount of data sent to the plurality of second processing devices differs. For example, the data sent to second processing device 1 is first data, and the data sent to second processing device 2 is second data. Exemplarily, the first processing device is... Figure 1The computing node 1 shown can have the following data: the first data can be data A1 on computing node 1, the second data can be data B1 on computing node 1, the third data can be data C1 on computing node 1, and the fourth data can be data D1 on computing node 1; or, the first processing device is... Figure 1 The computing node 2 shown can have the following data: the first data can be data A2, the second data can be data B2, the third data can be data C2, and the fourth data can be data D2.

[0091] Step 503: Transmit the first data through the first main processing unit among the at least two main processing units corresponding to the first processing device.

[0092] In this embodiment, the first processing device corresponds to at least two main processing units, each of which is responsible for transmitting the corresponding data. The at least two main processing units may be main processing units included in the first processing device. In one possible scenario, the number of main processing units corresponding to the first processing device matches the amount of data acquired by the first processing device, and each main processing unit is used to transmit the data corresponding to that main processing unit. Taking the acquisition of first data, second data, third data, and fourth data by the first processing device as an example, the at least two main processing units corresponding to the first processing device include a first main processing unit, a second main processing unit, a third main processing unit, and a fourth main processing unit. The first main processing unit is responsible for transmitting the first data, the second main processing unit is responsible for transmitting the second data, the third main processing unit is responsible for transmitting the third data, and the fourth main processing unit is responsible for transmitting the fourth data. Optionally, a main processing unit may refer to a thread, a process, or a processor core, etc.

[0093] Step 504: Based on the fact that the amount of the second data is greater than the amount of the first data, the second data is transmitted through the second main processing unit and the slave processing unit in at least two main processing units.

[0094] In this embodiment, when the first processing device acquires first data and second data, since the amount of second data is greater than the amount of first data, in addition to transmitting the second data through the second main processing unit among at least two main processing units, the second data is also transmitted synchronously through a slave processing unit. That is, the second data is transmitted through both the second main processing unit and the slave processing unit among at least two main processing units. The slave processing unit is located on the first processing device or on other processing devices. Therefore, the transmission speed of the second data is accelerated by using the slave processing unit, avoiding the long-tail phenomenon caused by the large amount of second data, and shortening the overall completion time for transmitting the first and second data.

[0095] When the first processing unit acquires first data, second data, third data, and fourth data, and based on the fact that the amount of second data is greater than the amount of first data, the amount of second data is greater than the amount of third data, and the amount of second data is greater than the amount of fourth data, the second data is transmitted through a second main processing unit and a slave processing unit in at least two main processing units. That is, the amounts of different data acquired by the first processing unit vary, and the data with the largest amount of data among all the data acquired by the first processing unit can be called long-tail data; the second data is long-tail data. Optionally, the second data satisfies a benefit condition, which includes the proportion of the data size difference to the amount of second data being greater than a proportion threshold, where the data size difference is the difference between the amount of second data and the amount of first data. The type of slave processing unit corresponds to that of the main processing unit; a slave processing unit can also refer to a thread, process, or processor core, etc.

[0096] In one possible implementation, since the amount of the second data is greater than the amount of the first data, before transmitting the second data through the second main processing unit and the slave processing unit in at least two main processing units, the first main processing unit in at least two main processing units determines whether the first data it is responsible for transmitting needs acceleration. If it is determined that the first data does not need acceleration, the first data is transmitted through the first main processing unit. Then, the second main processing unit in at least two main processing units determines whether the second data it is responsible for transmitting needs acceleration. If it is determined that the second data needs acceleration, the second data is transmitted through the second main processing unit and the slave processing unit. The method for determining whether acceleration is needed includes: determining whether it is long-tail data; if it is long-tail data, determining whether it meets the benefit conditions; if the benefit conditions are met, acceleration is needed; otherwise, acceleration is not needed.

[0097] Taking the first or second data in this embodiment as a data block as an example, when the first processing device transmits seven data blocks through seven main processing units, one main processing unit is used to process one corresponding data block. Each main processing unit has the data volume of seven data blocks, and each main processing unit independently filters long-tail data from the seven data blocks. The filtered long-tail data corresponds to the second data. The rule for filtering long-tail data can be as follows: filter out the data block 1 with the largest data volume and the data block 2 with the second largest data volume from the seven data blocks, obtain the difference between the data volume of data block 1 and the data volume of data block 2. If the difference is greater than a proportional threshold, then data block 1 is determined to be long-tail data; if the difference is not greater than the proportional threshold, then no long-tail data is determined. The proportional threshold can be set based on experience or flexibly adjusted according to the application scenario. Therefore, the seven main processing units do not need to negotiate and can independently determine a unified long-tail data identification result, resulting in high efficiency in long-tail data identification.

[0098] After each main processing unit independently filters long-tail data from the seven data blocks, each main processing unit determines whether to accelerate the process through the slave processing units based on whether the corresponding data block is long-tail data. For example, taking main processing unit 1 corresponding to data block 1 as an example, if main processing unit 1 determines that data block 1 is long-tail data, it issues processing tasks to the slave processing units so that the slave processing units can process different parts of the data in data block 1 in parallel with the main processing unit 1 based on the processing tasks, thereby accelerating the processing of data block 1. If main processing unit 1 determines that there is no long-tail data, it does not enable the slave processing units for acceleration. In this case, the data size distribution of the seven data blocks is relatively uniform, so there is no need to enable the slave processing units for acceleration, avoiding performance degradation due to improper resource allocation during the acceleration process.

[0099] In this embodiment, since the main processing unit and the slave processing unit transmit the second data in parallel, the plane through which the second data is transmitted via the main processing unit and the plane through which the second data is transmitted via the main processing unit can be regarded as two transmission planes. Therefore, the acceleration of the transmission of the second data via the slave processing unit can be referred to as dual-plane acceleration.

[0100] In one possible implementation, the first processing device is a processing device in a computing cluster, which includes n processing devices, and the n processing devices transmit data to each other. The second data is the data among the data transmitted between the n processing devices that meets the acceleration conditions. The acceleration conditions include the data volume being ranked in the top n from largest to smallest, and the sending and receiving ends of data of the same data volume level being different. The first data is the data among the data transmitted between the n processing devices that does not meet the acceleration conditions.

[0101] In other words, in a computing cluster scenario, in addition to the fact that the amount of the second data is greater than the amount of the first data, the second data also needs to meet acceleration conditions before acceleration can be achieved by the processing unit. For example, the data transmitted between n processing devices includes n*(n-1) data blocks, meaning any processing device sends data blocks to all other processing devices and receives data blocks sent by all other processing devices. Since the data blocks are transmitted in the form of streams, one data block corresponds to one data stream, and different data streams can be distinguished by flow identification (FID).

[0102] For the scenario of aggregate communication between n processing devices, the n processing devices can determine the long-tail data in a distributed, non-negotiated manner. Taking an alltoallv aggregate communication operation between multiple processing devices as an example, the first data and the second data belong to the data in the alltoallv aggregate communication operation. Each processing device can independently determine the long-tail data among all the data in the alltoallv aggregate communication operation, and each main processing unit in each processing device can independently determine the long-tail data.

[0103] Taking the main processing unit as the main thread as an example, the filtering process for long tail streams in the alltoallv collection communication operation by any main thread can be as follows: any main thread possesses a data transmission matrix for all global processing devices to send data to each other, for example... Figure 3 The data transmission matrix shown uses one square in the matrix to represent one data stream. For example, any main thread sorts all data streams with FIDs in the data transmission matrix in descending order, obtaining a sorted traffic array. Starting from the first data stream in the traffic array, it sequentially checks whether it meets the acceleration conditions. Streams that meet the acceleration conditions are identified as long-tailed streams, continuing until a data stream that does not meet the acceleration conditions is identified, at which point the filtering process exits. For example... Figure 6 As shown, the data streams corresponding to the bolded elements in the data transmission matrix are the filtered long tail streams.

[0104] Taking the determination of whether data stream 1 transmitted from the first processing device to the second processing device meets the acceleration conditions as an example, the sending end (source end) and receiving end (destination end) of data stream 1 are determined based on its FID. It is then determined whether the sending end or receiving end of data stream 1 conflicts with the sending end or receiving end already identified as a long tail stream. If there is no conflict and the number of identified long tail streams is less than the number of multiple processing devices, then data stream 1 is determined to be a long tail stream. If there is a conflict, then data stream 1 does not meet the acceleration conditions, and data streams of the same data volume level as data stream 1 are also not long tail streams, thus exiting the filtering process. For example... Figure 6 As shown, if the data stream sent from NPU3 to NPU2 is determined to be a long tail stream, and the sender of the data stream sent from NPU3 to NPU4 conflicts with the sender of the data stream sent from NPU3 to NPU2, then the data stream sent from NPU3 to NPU4 is determined not to be a long tail stream. Since the data volume level of the data stream sent from NPU8 to NPU3 and the data stream sent from NPU3 to NPU4 is 1.2, the data stream sent from NPU8 to NPU3 is also determined not to be a long tail stream.

[0105] Optionally, after identifying the long tail streams through the above screening process, the long tail streams can be directly used as the second data in this embodiment of the application to accelerate the transmission of the long tail streams from the processing unit; alternatively, it can be further determined whether the long tail streams meet the benefit conditions, and if the benefit conditions are met, the long tail streams can then be used as the second data. In this case, in the alltoallv set communication scenario of this embodiment of the application, since the number of identified long tail streams is multiple, the benefit conditions can be expressed by the following formula (1).

[0106] (maxD1-maxD2) / maxD1 > Profit Threshold Formula (1)

[0107] Where maxD1 is the maximum flow that can be accelerated, representing the maximum amount of data in the filtered long-tailed flow; maxD2 is the maximum flow that cannot be accelerated, representing the maximum amount of data in the remaining non-long-tailed flow; and (maxD1-maxD2) / maxD1 represents the theoretical acceleration benefit generated by dual-plane acceleration. The long-tailed flow meets the benefit condition if the theoretical acceleration benefit is greater than the benefit threshold; otherwise, it does not meet the benefit condition. The benefit threshold can be set empirically or flexibly adjusted according to the application scenario; the benefit threshold here corresponds to the aforementioned proportional threshold.

[0108] In this embodiment of the application, considering that enabling dual-plane acceleration will not bring significant benefits when the data distribution is relatively uniform, and may cause tailing due to the inappropriate amount of data allocated to the slave processing unit, a benefit threshold judgment is added after the long tail stream is filtered out, so that dual-plane acceleration can be performed without enabling the slave processing unit when the data distribution is relatively uniform, thus avoiding the performance degradation caused by enabling dual-plane acceleration.

[0109] Therefore, the above process can accurately filter out the second data that needs to be accelerated, and then enable the slave processing unit to accelerate the transmission of the second data. In the embodiments of this application, the main processing unit and the slave processing unit are used to distinguish processing units with different functions. In fact, the main processing unit and the slave processing unit can be the same processing unit. For example, the main thread and the slave thread are both any threads in the thread pool, but the main thread and the slave thread perform different tasks, and the main thread and the slave thread can exchange synchronization information.

[0110] Taking processing units as threads and data transmission as data moving as an example, Figure 7This is a schematic diagram illustrating a data transfer process provided in an embodiment of this application. Data 1, data 2, and data 3 of processing device 1 are stored in high-bandwidth memory (HBM), with data 2 having a larger data volume than data 1 and data 3. The thread pool of processing device 1 includes threads 1 through 4. Processing device 1 sends data 1 to processing device 2 via thread 1, and sends data 2 to processing device 3 via threads 2 and 3, where one thread is the master thread and the other is the slave thread. Processing device 1 sends data 3 to processing device 4 via thread 4.

[0111] The following describes an implementation method for transmitting second data through the second main processing unit and the slave processing unit.

[0112] In this embodiment, the amount of second data transmitted by the second main processing unit and the slave processing unit is determined based on their respective transmission speeds. The second main processing unit is responsible for transmitting a portion of the second data, and the slave processing unit is responsible for transmitting the remaining portion. However, the amount of data transmitted by each unit is not statically allocated but dynamically determined based on the transmission speed. That is, during the transmission of the second data, the transmission speed of either the main or slave processing unit may change due to factors such as link or network conditions, and the amount of data allocated to each unit changes accordingly.

[0113] For example, the amount of second data transmitted is positively correlated with the transmission speed; that is, the faster the transmission speed, the more data is transmitted, and the slower the transmission speed, the less data is transmitted. Therefore, when transmitting second data through the second main processing unit and the slave processing unit, the processing unit with the faster transmission speed transmits more data. This allows the second data to be dynamically allocated between the second main processing unit and the slave processing unit, minimizing transmission time and preventing the overall completion time of the second data from being slowed down by the transmission bottleneck of one processing unit.

[0114] In one possible implementation, the receiving end of the second data is a second processing device, and the transmission link corresponding to the second main processing unit is a direct link from the first processing device to the second processing device. This direct link can be a direct connection between the first processing device and the second processing device, or it can be a link connected through multiple switching devices. Optionally, the transmission link corresponding to the processing unit includes, but is not limited to, the following two cases.

[0115] Case 1: The transmission link corresponding to the primary processing unit is different from the transmission link corresponding to the second primary processing unit.

[0116] In this scenario, the transmission link corresponding to the processing unit can be a direct link from the first processing device to the second processing device, or it can be a relay link from the first processing device to an intermediate device and then to the second processing device. Optionally, the first and second processing devices are processing devices in a computing cluster, and the intermediate device is a host connected to the computing cluster; or, the intermediate device is a third processing device in the computing cluster, which is connected to both the first and second processing devices. Thus, accelerated transmission of the second data is achieved by extending the additional link bandwidth.

[0117] The third processing unit can be any processing unit in the computing cluster other than the first and second processing units. For example, if the first processing unit is NPU1 and the second processing unit is NPU2, see [link to example]. Figure 8 The diagram shows a link where solid black arrows represent direct links from NPU1 to NPU2, and dashed black arrows represent relay links from NPU1 to NPU6 and then back to NPU2. In this case, the third processing device is NPU6. Since the network architecture between NPU1 and NPU8 is fully connected, the third processing device acting as a relay can be any NPU from NPU3 to NPU8. Figure 8 The example shown here is NPU6.

[0118] Scenario 2: The transmission link corresponding to the primary processing unit is the same as the transmission link corresponding to the second primary processing unit.

[0119] In this second scenario, although the transmission link corresponding to the slave processing unit is the same as that corresponding to the second master processing unit, the transmission efficiency of the direct link from the first processing device to the second processing device can be improved through the synchronous parallel transmission between the second master processing unit and the slave processing unit. For example, when the direct link is a direct connection between the first processing device and the second processing device, the bandwidth utilization of the direct link can be improved; when the direct link is a connection between the first processing device and the second processing device through a network including multiple switching devices, the multi-equivalent path between the first processing device and the switching devices can be fully utilized.

[0120] In both Case 1 and Case 2, the methods for transmitting the second data through the second main processing unit and the slave processing unit in at least two main processing units include, but are not limited to, the following two.

[0121] In method one, the first unit of data in the second data is transmitted through the second main processing unit, and the second unit of data in the second data is transmitted through the slave processing unit. When the second main processing unit has completed transmitting the first unit of data, the next first unit of data in the second data is transmitted through the second main processing unit; when the slave processing unit has completed transmitting the second unit of data, the next second unit of data in the second data is transmitted through the slave processing unit; and so on, until the transmission of the second data is complete. Thus, by transmitting the next unit of data after each unit of data has been transmitted, the second main processing unit and the slave processing unit achieve dynamic allocation of the second data between the second main processing unit and the slave processing unit.

[0122] For example, the second data is stored in the HBM of the first processing device, and transmitting the second data refers to moving the second data from the HBM of the first processing device to the HBM of the second processing device. Based on the above data transmission process, the second main processing unit and the slave processing unit can start from two different positions of the second data stored in the HBM of the first processing device, and sequentially move the second data according to the corresponding moving granularity. Optionally, the two different positions can be the beginning position and the end position. The moving granularity indicates the length of the unit data transmitted each time. The moving granularity of the second main processing unit and the slave processing unit can be determined based on the theoretical bandwidth ratio of the two planes.

[0123] In this embodiment, before transmitting the second data from the processing unit, it is first determined whether the relay link corresponding to the processing unit is idle. If the relay link is idle, the second data is then transmitted from the processing unit based on the relay link. This avoids data transmission delays caused by relay link congestion.

[0124] For example, let Figure 8Taking the illustrated application scenario as an example, NPU1 can obtain the global link status, which includes the link status from NPU1 to all other NPUs and the link status from all other NPUs to NPU2. The link status includes idle and occupied. Idle means no data is being transmitted on the link, while occupied means data is being transmitted on the link. Therefore, when both the link from NPU1 to NPU6 and the link from NPU6 to NPU2 are idle, the second unit of data in the second unit is transmitted from the processing unit via the relay link from NPU1 to NPU6 and then to NPU2. Similarly, when both the link from NPU1 to NPU5 and the link from NPU5 to NPU2 are idle, the next second unit of data in the second unit is transmitted from the processing unit via the relay link from NPU1 to NPU5 and then to NPU2. In other words, the idle relay links may differ depending on the second unit of data being transmitted, thus affecting the idle relay links used to transmit different second units of data from the processing unit.

[0125] Method 2 involves dividing the second data into a first part and a second part using a second main processing unit; the first part is transmitted via the second main processing unit, and the second part is transmitted via a slave processing unit. This method directly allocates the data volume to be transmitted by the second main processing unit and the slave processing unit based on task granularity, improving the efficiency of dynamic data allocation. The specific method used in this embodiment of the application to divide the second data into a first part and a second part is not limited; the data can be divided equally or flexibly according to task granularity or application requirements.

[0126] Through the aforementioned methods one and two, the slave processing unit can accelerate the transmission of the second data using the corresponding transmission link. In this embodiment, the second master processing unit and the slave processing unit can independently transmit different parts of the second data, or the second master processing unit and the slave processing unit can negotiate the transmission of different parts of the second data through information synchronization.

[0127] In one possible implementation, the method further includes: issuing a processing task to the slave processing unit through the second main processing unit, the processing task being used to instruct the slave processing unit to transmit corresponding data from the second data. Thus, by issuing tasks, the main processing unit controls the slave processing unit, thereby dynamically controlling the amount of data from the second data that the slave processing unit needs to transmit. In this embodiment, the first processing device includes a memory region used for task synchronization between the second main processing unit and the slave processing unit. For example, the memory region is a region within the HBM of the first processing device.

[0128] Optionally, the contents stored in the memory area may vary depending on the information required for task synchronization. For example, the memory area may store at least one of the following: task flag, address flag, granularity flag, link flag, completion flag, or end flag. The task flag indicates whether the second main processing unit has issued a processing task; the address flag indicates the address offset of the data corresponding to the issued processing task; the granularity flag indicates the length of the unit data corresponding to the issued processing task; the link flag indicates whether the transmission link corresponding to the slave processing unit is idle; the completion flag indicates whether the slave processing unit has completed the issued processing task; and the end flag indicates whether the second data transmission is complete.

[0129] For the transmission scenario described above, after the second main processing unit completes the transmission of the first unit of data, it checks whether the slave processing unit has completed the transmission of the second unit of data. If the slave processing unit has completed the transmission of the second unit of data, the second main processing unit issues a new task to the slave processing unit, which indicates the next second unit of data. If the slave processing unit has not completed the transmission of the second unit of data, the main processing unit does not issue a new task. Regardless of whether a new task is issued, the second main processing unit continues to transmit the next first unit of data; after the slave processing unit completes the transmission of the second unit of data, it checks whether the second main processing unit has issued a new task. If the second main processing unit has issued a new task, the transmission of the next second unit of data continues based on the new task. If there is no new task, the system continues to check whether the second main processing unit has issued a new task.

[0130] In this scenario, a task flag may be stored in the memory area, and the second main processing unit dispatches the task by setting the task flag. Alternatively, a granularity flag may also be stored in the memory area, in which case the second main processing unit dispatches the task by setting both the task flag and the granularity flag. If the granularity flag is not stored in the memory area, the slave processing unit can determine the second unit of data through a statically configured transfer granularity, or based on the link bandwidth of the transmission link corresponding to the slave processing unit. Alternatively, a link flag may also be stored in the memory area, and the second main processing unit sets the link flag during task dispatch to indicate which relay link is idle, allowing the slave processing unit to utilize an idle relay link to transmit the second unit of data.

[0131] Therefore, the processing unit determines whether there is a new task by checking the task flag. Alternatively, the processing unit determines the length of the second unit data corresponding to the new task by checking the granularity flag. Since the processing unit transmits the second data sequentially, the start position of this transmission is the end position of the previous transmission. The end position of this transmission can be determined based on the shift granularity, thereby determining the position of the data transmitted in this transmission within the second data. The position can be represented by an offset address.

[0132] In addition, a completion flag may be stored in the memory area. When a second unit of data is transferred, the slave processing unit sets the completion flag, allowing the second master processing unit to determine whether the slave processing unit has completed its task by checking this flag. Optionally, an end flag may also be stored in the memory area. When all the second data has been transferred, the second master processing unit sets this end flag. The slave processing unit checks the end flag while checking the task flag. If the end flag indicates that all the second data has been transferred, it can stop waiting for new tasks and directly exit the execution flow of transferring the second data.

[0133] For the transmission scenario described in Method 2 above, after the second main processing unit divides the second data into a first part and a second part, it issues a processing task to the slave processing unit. This task specifies the second part of the data, allowing the slave processing unit to sequentially transmit the second part according to its granularity of data transfer. Once the slave processing unit has completed transmitting the second part, it can exit the transmission flow. Similarly, the second main processing unit sequentially transmits the first part of the data according to its granularity of data transfer. Once the second main processing unit has completed transmitting the first part, it can exit the transmission flow.

[0134] In this scenario, an address flag can be stored in the memory area. The second main processing unit sends out tasks by setting the address flag, which indicates the starting position of the second data transmission from the processing unit. For example, the address flag stores an offset address in the second data. The second main processing unit transmits data from the beginning of the second data to the offset address, and the slave processing unit transmits data from the offset address to the end of the second data.

[0135] Next, taking the first processing device as the sending end, the second processing device as the receiving end, the processing unit as a thread, and the memory area as a flag storage area as an example, we will combine... Figure 9 The process of migrating second data using dynamic resource allocation is explained. For example... Figure 9As shown, the main thread performs data transfer starting from the head of the second data with a first transfer granularity, and the slave thread performs data transfer starting from the tail of the second data with a second transfer granularity. Data transfer refers to transferring data from the first processing device to the second processing device.

[0136] The main thread issues tasks through a flag storage area. For example, it issues tasks by setting a task flag. A first value indicates a task has been issued, while a second value indicates no task has been issued. Taking a single bit as an example, the first value can be 1, and the second value can be 0. Slave threads query the flag storage area for tasks. For example, if a new task is detected by a first value, the slave thread moves the next second unit of data from the tail to the head based on this new task. After completing the data movement, the slave thread announces task completion through a flag storage area. For example, it announces task completion by setting a completion flag. A first value indicates task completion, while a second value indicates incomplete task. Optionally, the task flag and the completion flag can share the same flag. When the same flag is set to the first value, it means that the main thread has issued a new task. When the same flag is set to the second value, it means that the slave thread has completed the task.

[0137] The main thread moves data from the beginning to the end of the second data set with the first moving granularity. After completing the moving of the previous first unit of data, it checks the flag storage area to see if the slave thread has completed its task. If the slave thread has completed its task, it issues a new task through the flag storage area. After completing the previous task, the slave thread checks the flag storage area to see if there is a new task. Based on the new task, it moves the next second unit of data from the end to the beginning of the second data set. Due to the influence of the transmission link, the slave thread may have a long transmission delay. The main thread moves data from the beginning to the end of the second data set. After completing the moving of the next first unit of data, it checks the flag storage area to see if the slave thread has completed its task. If the slave thread has not completed its task, the main thread does not issue a new task but continues to move the next first unit of data from the beginning to the end of the second data set.

[0138] For scenarios where the transmission latency may be long due to the influence of the transmission link, please refer to [link to relevant documentation]. Figure 10The diagram illustrates data transmission via a host relay. In this scenario, the second data is first moved from the HBM of the first processing device to the host's double data rate (DDR) memory, and then forwarded from the host's DDR to the HBM of the second processing device. Since the slave thread uses bandwidth resources beyond the direct link (i.e., the PCIe link from the first processing device to the host and the PCIe link from the host to the second processing device), and the relay link from the first processing device to the host and then to the second processing device may need to transmit data for other services, there is a possibility of other services competing for this relay link when the slave thread uses it. Depending on the outcome of this competition, the time required for the slave thread to transmit different units of the second data varies, making the transmission time for different units of the second data uncertain. If congestion occurs on the relay link, it may lead to a longer transmission delay for the second unit of data, for example... Figure 10 The transmission of data 2 shown is delayed due to link contention.

[0139] In this embodiment, for a scenario where the transmission link corresponding to the processing unit is a relay link, the aforementioned processing unit refers to two processing units. To distinguish between the two processing units, they are referred to as a processing unit and a nested processing unit, respectively. The processing unit is responsible for data transmission from the first processing device to the intermediate device, and the nested processing unit is responsible for data transmission from the intermediate device to the second processing device. Thus, relay transmission from the first processing device to the second processing device is achieved through the collaboration of the processing unit and the nested processing unit.

[0140] This application does not limit the dynamic collaboration method between the processing unit, nested processing unit, and main processing unit. Optionally, the main processing unit may issue tasks to the processing units and nested processing units in accordance with the above-described dynamic task issuance method, so as to control the execution of tasks by the processing units and nested processing units through the main processing unit; or, the main processing unit may issue tasks to the processing units in accordance with the above-described dynamic task issuance method, so as to control the execution of tasks by the processing units through the main processing unit, and the processing units may issue tasks to the nested processing units according to the tasks issued by the main processing unit, so as to control the execution of tasks by the nested processing units through the processing units.

[0141] When the processing unit is a thread, the slave processing unit is the slave thread, and the nested processing unit is the nested thread. Figure 10Taking the host-intermediate data transfer process as an example, the slave thread can be called an H2D thread, and the nested thread can be called a D2H thread. The H2D thread is responsible for data transfer from the first processing device to the host, and the D2H thread is responsible for data transfer from the host to the second processing device. Optionally, the main thread can issue the next processing task after the slave thread has completed the previous processing task, without waiting for the nested thread to complete the previous processing task, thus achieving pipeline masking of nested threads; or, it can issue the next processing task after the nested thread has completed the previous processing task.

[0142] In this scenario, the main thread also assigns tasks to nested threads in the same way it assigns tasks to slave threads. For example, the task flag and completion flag of the H2D thread are set in the flag storage area of ​​the first processing device, and the task flag and completion flag of both the D2H thread are set in the flag storage area of ​​the host. Optionally, the host's flag storage area also sets a transit ready flag. After a slave thread completes the data transfer from the first processing device to the host, it sets the transit ready flag to indicate that the transit data is ready. Based on the transit ready flag indicating that the transit data is ready, the nested thread transfers the transit data that has been transferred to the host to the second processing device.

[0143] For example, the state transition diagram of the main thread is as follows: Figure 11 As shown. After the main thread completes the data transfer of the first unit of data, it checks the completion flag of the slave thread through the memory flag area. If the slave thread's completion flag indicates that it has not completed the data transfer of the previous second unit of data, the main thread continues to execute the data transfer of the next first unit of data. If the slave thread's completion flag indicates that it has completed the data transfer of the previous second unit of data, it determines the remaining transferable data. If the remaining transferable data is greater than the first unit of data, it sets the slave thread's task flag in the memory flag area, issues a new task to the slave thread through the task flag, and continues to execute the data transfer of the next first unit of data. If the remaining transferable data is not greater than the first unit of data, it continues to execute the data transfer of the next first unit of data. That is, when the remaining data is small, only the main thread needs to perform the transfer. After completing the data transfer of the next first unit of data, it is determined that there is no data to transfer, sets the end flag in the memory flag area, notifies the slave thread that the data transfer is complete through the end flag, and exits the data transfer process.

[0144] The diagram illustrating the state transitions of a thread is as follows: Figure 12As shown. The slave thread checks its task flag and end flag in the memory flag area. If the slave thread's task flag indicates that there is a task, the slave thread performs a data transfer of a second unit of data. If the slave thread has not completed the data transfer of a second unit of data, it continues to perform the data transfer of that second unit of data. If the slave thread completes the data transfer of a second unit of data, it sets the slave thread's completion flag in the memory flag area to indicate that the slave thread has completed the data transfer, and continues to check the slave thread's task flag and end flag. If the end flag indicates that the data transfer is complete, the data transfer process exits.

[0145] In the case of nested threads, Figure 11 When the main thread sets the task flag of the slave thread in the memory flag region, it also sets the task flag of the nested thread in the memory flag region, and issues tasks to the nested thread through the task flag of the nested thread. Furthermore, Figure 12 In addition to setting the completion flag of the slave thread in the memory flag area, the intermediate thread also sets the intermediate ready flag to indicate that the intermediate data is ready.

[0146] For example, a diagram illustrating the state transitions of nested threads is shown below. Figure 13 As shown. The nested thread checks its task flag and end flag in the memory flag area. If the task flag indicates there is a task, the nested thread checks the transit ready flag in the memory flag area. If the transit ready flag indicates the transit data is not ready, it continues to check the transit ready flag in the memory flag area. If the transit ready flag indicates the transit data is ready, it performs data transfer. If the nested thread has not completed the transit data transfer, it continues to perform the transit data transfer. If the nested thread has completed the transit data transfer, it sets the nested thread's completion flag in the memory flag area to indicate that the nested thread has completed the data transfer, and continues to check the nested thread's task flag and end flag. If the end flag indicates the data transfer is complete, the data transfer process exits.

[0147] Therefore, by utilizing the nested pattern of slave threads and nested threads, along with the cooperation of the intermediate ready flag and additional relay transmission links, data migration is completed, achieving efficient synchronization between threads across devices. Optionally, the interaction flow between the main thread, slave threads, and nested threads can be found in [reference needed]. Figure 14 or Figure 15 The diagram shown is illustrated below. Figure 14In the main thread, the completion flag is set by the slave thread, allowing the main thread to assign new tasks to the slave thread after the slave thread completes its task; while... Figure 15 In this context, the completion flag is set by the nested thread, ensuring that the main thread can only issue new tasks to the nested threads after the nested threads have completed their tasks. Furthermore, in... Figure 14 In this context, the task of a nested thread is issued by the slave thread. If the slave thread can receive the task issued by the main thread, it will issue the task to the nested thread. At this time, the slave thread also needs to set the relay ready flag. Alternatively, the slave thread can also issue the task to the nested thread after transmitting the data to the relay device. In this case, the slave thread does not need to set the relay ready flag. Figure 14 or Figure 15 Further details regarding the interactive flow shown can be found in the relevant content mentioned above, and will not be repeated here.

[0148] In summary, the method provided in this application embodiment, when transmitting first data and second data with different data volumes, transmits the first data with a smaller data volume through the first main processing unit, and transmits the second data with a larger data volume through the second main processing unit and the slave processing unit. This accelerates the transmission of the second data through the slave processing unit, shortens the transmission completion time of the second data, avoids a large difference between the transmission completion time of the second data and the transmission completion time of the first data, and thus avoids the long-tail phenomenon caused by the large data volume of the second data. As a result, the overall completion time for transmitting data with different data volumes is shortened, and the data transmission efficiency is improved.

[0149] In computing cluster scenarios, data transmission between n processing devices is handled by two sets of processing units: n main processing units and several slave main processing units. The n main processing units are responsible for transmitting a portion of the data to be accelerated, as well as other non-accelerated data. The additional slave threads are responsible for transmitting the other portion of the data to be accelerated, making full use of the idle bandwidth resources in the computing cluster and effectively reducing the completion time of long-tail data transmission.

[0150] In the above Figure 5 In the data transmission method shown, the first data and the second data acquired by the first processing device are transmitted through two main processing units, a first main processing unit and a second main processing unit, respectively. In one possible implementation, the first data and the second data acquired by the first processing device can also be transmitted through a single main processing unit. In this case, the embodiments of this application provide another data transmission method, such as... Figure 16 As shown, the data transmission method includes, but is not limited to, the following steps 601-604.

[0151] Step 601: Obtain the first data.

[0152] Step 602: Obtain the second data.

[0153] Step 603: Transmit the first data through the first main processing unit corresponding to the first processing device.

[0154] Step 604: Based on the fact that the amount of the second data is greater than the amount of the first data, the second data is transmitted through the first main processing unit and the slave processing unit.

[0155] In this embodiment, the implementation of steps 601-604 can refer to the implementation of steps 501-504. At this time, the first main processing unit performs the operations related to the first main processing unit and the second main processing unit in steps 501-504. This embodiment will not be described redundantly.

[0156] Optionally, if the first processing device also acquires data other than the first data and the second data, it can be as follows: Figure 5 The method shown involves different data being transmitted by different main processing units corresponding to the first processing device, or, as... Figure 16 All the data shown is transmitted from the same main processing unit, or it could be... Figure 5 and Figure 16 The combination of these can be such that some data is transmitted by different main processing units, while other data can be transmitted using the same main processing unit.

[0157] The data transmission method of the present application embodiments has been described above. Corresponding to the above method, the present application embodiments also provide a data transmission device. Figure 17 This is a schematic diagram of a data transmission device provided in an embodiment of this application, which is applied to a first processing device. Based on Figure 17 The following modules are shown. Figure 17 The data transmission device shown is capable of performing all or part of the operations performed by the first processing device. It should be understood that the device may include more additional modules than those shown, or may omit some of the modules shown; this application embodiment does not impose limitations in this regard. Figure 17 As shown, the device includes: a transceiver module 1601, used for performing... Figure 5 or Figure 16 In the method shown, the receiving and / or sending related operations are performed by the first processing device. The processing module 1602 is used to perform... Figure 5 or Figure 16 Other operations performed by the first processing device in the method shown, besides the receiving and / or sending related operations.

[0158] In one possible implementation, the transceiver module 1601 includes a receiving module and / or a transmitting module. The receiving module is used to perform receiving-related operations, and the transmitting module is used to perform transmitting-related operations.

[0159] In one possible implementation, the processing module 1602 is used to acquire first data; acquire second data; and the transceiver module 1601 is used to transmit the first data through the first main processing unit among the at least two main processing units corresponding to the first processing device; and to transmit the second data through the second main processing unit and the slave processing unit among the at least two main processing units, based on the fact that the amount of the second data is greater than the amount of the first data.

[0160] In one possible implementation, the amount of second data transmitted by the second main processing unit and the second processing unit is determined based on the transmission speed of the second main processing unit and the second processing unit, respectively.

[0161] In one possible implementation, the transmission links corresponding to the second master processing unit and the slave processing unit are different.

[0162] In one possible implementation, the receiving end of the second data is a second processing device, the transmission link corresponding to the second main processing unit is a direct link from the first processing device to the second processing device, and the transmission link corresponding to the processing unit is a relay link from the first processing device to the intermediate device and then to the second processing device.

[0163] In one possible implementation, the first processing device and the second processing device are processing devices in a computing cluster, and the intermediate device is a host connected to the computing cluster, or the intermediate device is a third processing device in the computing cluster connected to the first processing device and the second processing device.

[0164] In one possible implementation, the transceiver module 1601 is configured to transmit second data from the processing unit via the relay link when the relay link is idle.

[0165] In one possible implementation, the second master processing unit has the same transmission link as the slave processing unit.

[0166] In one possible implementation, the transceiver module 1601 is configured to transmit a first unit of data in the second data through the second main processing unit, and transmit a second unit of data in the second data through the processing unit; when the second main processing unit has completed transmitting the first unit of data, transmit the next first unit of data in the second data through the second main processing unit; when the processing unit has completed transmitting the second unit of data, transmit the next second unit of data in the second data through the processing unit; and so on, until the second data transmission is completed.

[0167] In one possible implementation, the processing module 1602 is used to divide the second data into a first part of data and a second part of data through the second main processing unit; the transceiver module 1601 is used to transmit the first part of data through the second main processing unit and transmit the second part of data from the processing unit.

[0168] In one possible implementation, the transceiver module 1601 is further configured to send a processing task to the slave processing unit through the second main processing unit, the processing task being used to instruct the slave processing unit to transmit the corresponding data in the second data.

[0169] In one possible implementation, the first processing device includes a memory region for task synchronization between a second main processing unit and a slave processing unit.

[0170] In one possible implementation, the memory region stores at least one of a task flag, an address flag, a granularity flag, a link flag, a completion flag, or an end flag. The task flag is used to indicate whether the second main processing unit has issued a processing task, the address flag is used to indicate the address offset of the data corresponding to the issued processing task, the granularity flag is used to indicate the length of the unit data corresponding to the issued processing task, the link flag is used to indicate whether the transmission link corresponding to the slave processing unit is idle, the completion flag is used to indicate whether the slave processing unit has completed the issued processing task, and the end flag is used to indicate whether the second data has been transmitted.

[0171] In one possible implementation, the first processing device is a processing device in a computing cluster, which includes n processing devices. The n processing devices transmit data to each other. The second data is the data that meets the acceleration conditions among the data transmitted between the n processing devices. The acceleration conditions include the data volume being ranked in the top n from largest to smallest, and the sending and receiving ends of data of the same data volume level being different. The first data is the data other than the second data.

[0172] In one possible implementation, the second data also satisfies a benefit condition, which includes the proportion of the data volume difference to the data volume of the second data being greater than a proportion threshold, where the data volume difference is the difference between the data volume of the second data and the data volume of the first data.

[0173] In one possible implementation, at least two main processing units are main threads, and the slave processing units are slave threads.

[0174] In one possible implementation, the first processing device is a server, NPU, GPU, TPU, or CPU.

[0175] In one possible implementation, the first data and the second data are data from the alltoallv set communication operation.

[0176] It should be understood that the above Figure 17 The provided device, when implementing its functions, is only illustrated by the above-described division of functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process and beneficial effects are detailed in the method embodiments, and will not be repeated here.

[0177] It should be noted that the data transmission device mentioned in the embodiments of this application can be a data transmission device such as a server or NPU, or a component of a data transmission device, such as a single board or line card on the data transmission device, or a functional module on the data transmission device, or a chip used to implement the method of this application. The embodiments of this application do not impose specific limitations. When the data transmission device is a chip, the transceiver module used to implement the method can be, for example, the chip's interface circuit, and the processing module can be a processing circuit with processing functions within the chip. The connection methods between data transmission devices include, but are not limited to, direct connection via Ethernet cable or optical fiber.

[0178] See Figure 18 , Figure 18 A schematic diagram of the structure of a data transmission device 2000 provided in an exemplary embodiment of this application is shown. Figure 18 The data transmission device 2000 shown is used to perform the above. Figure 5 or Figure 16 The data transmission method illustrated involves the following operations. The data transmission device 2000 is, for example, a switch, a router, etc., and can be implemented using a general bus architecture.

[0179] like Figure 18 As shown, the data transmission device 2000 includes at least one processor 2001, a memory 2003, and at least one communication interface 2004.

[0180] Processor 2001 may be, for example, a general-purpose central processing unit (CPU), a digital signal processor (DSP), a network processor (NP), a graphics processing unit (GPU), a neural-network processing unit (NPU), a data processing unit (DPU), a microprocessor, or one or more integrated circuits for implementing the embodiments of this application. For example, processor 2001 includes application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A PLD may be, for example, a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof. It can implement or execute the various logic blocks, modules, and circuits described in connection with the embodiments of this invention. A processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0181] Optionally, the data transmission device 2000 also includes a bus. The bus is used to transfer information between the various components of the data transmission device 2000. The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 18 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0182] Memory 2003 may be, for example, read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions; random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions; electrically erasable programmable read-only memory (EEPROM); compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.); magnetic disk storage media or other magnetic storage devices; or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 2003 may exist independently and be connected to processor 2001 via a bus. Memory 2003 may also be integrated with processor 2001.

[0183] The communication interface 2004 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), or wireless local area networks (WLAN). The communication interface 2004 can include wired and wireless communication interfaces. Specifically, the communication interface 2004 can be an Ethernet interface, a Fast Ethernet (FE) interface, a Gigabit Ethernet (GE) interface, an Asynchronous Transfer Mode (ATM) interface, a WLAN interface, a cellular network communication interface, or a combination thereof. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. In this embodiment, the communication interface 2004 can be used by the data transmission device 2000 to communicate with other devices.

[0184] In a specific implementation, as one example, the processor 2001 may include one or more CPUs, such as Figure 18The CPUs shown are CPU0 and CPU1. Each of these processors can be a single-core CPU or a multi-core CPU. A processor here can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0185] In a specific implementation, as one example, the data transmission device 2000 may include multiple processors, such as... Figure 18 The processors shown are 2001 and 2005. Each of these processors can be a single-core CPU or a multi-core CPU. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0186] In a specific implementation, as one example, the data transmission device 2000 may further include an output device and an input device. The output device communicates with the processor 2001 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 2001 and can receive user input in various ways. For example, the input device may be a mouse, a keyboard, a touchscreen device, or a sensor device, etc.

[0187] In some embodiments, the memory 2003 stores program code 2010 for executing the solution of this application, and the processor 2001 can execute the program code 2010 stored in the memory 2003. That is, the data transmission device 2000 can implement the data transmission method provided in the method embodiment through the processor 2001 and the program code 2010 in the memory 2003. The program code 2010 may include one or more software modules. Optionally, the processor 2001 itself may also store program code or instructions for executing the solution of this application.

[0188] In a specific embodiment, the data transmission device 2000 of this application embodiment can correspond to the first processing device in the above-described method embodiments. The processor 2001 in the data transmission device 2000 reads instructions from the memory 2003, causing... Figure 18 The data transmission device 2000 shown is capable of performing all or part of the operations performed by the first processing device.

[0189] In one possible implementation, the processor 2001 is configured to acquire first data; acquire second data; transmit the first data through a first main processing unit among at least two main processing units corresponding to the first processing device; and transmit the second data through a second main processing unit and a slave processing unit among at least two main processing units, based on the fact that the amount of the second data is greater than the amount of the first data.

[0190] Other alternative implementation methods are described in the above method embodiments and will not be repeated here.

[0191] Data transmission device 2000 can also correspond to the above. Figure 17 The data transmission device shown in the diagram implements each functional module using software from the data transmission device 2000. In other words, the functional modules of the data transmission device are generated by the processor 2001 of the data transmission device 2000 reading the program code 2010 stored in the memory 2003.

[0192] in, Figure 5 or Figure 16 Each step of the data transmission method shown is completed through integrated logic circuits in the hardware or software instructions in the processor of the data transmission device 2000. The steps of the method disclosed in the embodiments of this application can be directly implemented by the hardware processor, or by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. Since the storage medium is located in memory, the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method; to avoid repetition, these will not be described in detail here.

[0193] See Figure 19 , Figure 19 This illustration shows a schematic diagram of the structure of a data transmission device 2100 provided in another exemplary embodiment of this application. Figure 19 The data transmission device 2100 shown is used to perform the above. Figure 5 or Figure 16 The data transmission method shown involves all or part of the operations. The data transmission device 2100 is, for example, a switch, a router, etc., and can be implemented using a general bus architecture.

[0194] like Figure 19 As shown, the data transmission device 2100 includes a main control board 2110 and an interface board 2130.

[0195] The main control board, also known as the main processing unit (MPU) or route processor card, is used to control and manage the various components in the data transmission device 2100, including routing calculation, device management, device maintenance, and protocol processing functions. The main control board 2110 includes a central processing unit 2111 and a memory 2112.

[0196] Interface board 2130 is also called a line processing unit (LPU), linecard, or service board. Interface board 2130 provides various service interfaces and implements packet forwarding. Service interfaces include, but are not limited to, Ethernet interfaces, POS (Packet over SONET / SDH) interfaces, etc., with Ethernet interfaces including, for example, Flexible Ethernet Clients (FlexE Clients). Interface board 2130 includes: a central processing unit 2131, a network processor 2132, a forwarding table entry memory 2134, and a physical interface card (PIC) 2133.

[0197] The central processing unit 2131 on the interface board 2130 is used to control and manage the interface board 2130 and communicate with the central processing unit 2111 on the main control board 2110.

[0198] Network processor 2132 is used to implement packet forwarding processing. Network processor 2132 can be in the form of a forwarding chip. The forwarding chip can be a network processor (NP). In some embodiments, the forwarding chip can be implemented using an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). Specifically, network processor 2132 forwards received packets based on the forwarding table stored in forwarding table entry memory 2134. If the destination address of the packet is the address of data transmission device 2100, the packet is sent to the CPU (such as central processing unit 2131) for processing; if the destination address of the packet is not the address of data transmission device 2100, the next hop and outgoing interface corresponding to the destination address are looked up in the forwarding table according to the destination address, and the packet is forwarded to the outgoing interface corresponding to the destination address. Uplink packet processing may include: packet ingress interface processing, forwarding table lookup; downlink packet processing may include: forwarding table lookup, etc. In some embodiments, the central processing unit can also perform the functions of the forwarding chip, such as implementing software forwarding based on a general-purpose CPU, thus eliminating the need for a forwarding chip on the interface board.

[0199] The physical interface card 2133 is used to implement physical layer interfacing functions. Raw traffic enters the interface board 2130 through this card, and processed packets are sent out from the physical interface card 2133. The physical interface card 2133, also called a daughter card, can be installed on the interface board 2130. It is responsible for converting photoelectric signals into packets, performing validity checks on the packets, and forwarding them to the network processor 2132 for processing. In some embodiments, the central processing unit 2131 can also perform the functions of the network processor 2132, such as implementing software forwarding based on a general-purpose CPU, thus eliminating the need for the network processor 2132 in the physical interface card 2133.

[0200] Optionally, the data transmission device 2100 includes multiple interface boards. For example, the data transmission device 2100 also includes an interface board 2140, which includes a central processing unit 2141, a network processor 2142, a forwarding table entry memory 2144, and a physical interface card 2143. The functions and implementation methods of each component in the interface board 2140 are the same as or similar to those of the interface board 2130, and will not be described in detail here.

[0201] Optionally, the data transmission device 2100 also includes a switching fabric board 2120. The switching fabric board 2120 can also be referred to as a switch fabric unit (SFU). When the data transmission device 2100 has multiple interface boards, the switching fabric board 2120 is used to complete data exchange between the interface boards. For example, interface boards 2130 and 2140 can communicate through the switching fabric board 2120.

[0202] The main control board 2110 and the interface boards are coupled. For example, the main control board 2110, interface boards 2130 and 2140, and the switching network board 2120 communicate with each other via a system bus connected to the system backplane. In one possible implementation, an inter-process communication (IPC) channel is established between the main control board 2110 and interface boards 2130 and 2140, and communication between the main control board 2110 and interface boards 2130 and 2140 is achieved through the IPC channel.

[0203] Logically, the data transmission device 2100 includes a control plane and a forwarding plane. The control plane includes a main control board 2110 and a central processing unit 2111, while the forwarding plane includes various components that perform forwarding, such as a forwarding table entry memory 2134, a physical interface card 2133, and a network processor 2132. The control plane performs functions such as router operation, generating forwarding tables, processing signaling and protocol messages, and configuring and maintaining the status of the data transmission device. The control plane distributes the generated forwarding table to the forwarding plane. In the forwarding plane, the network processor 2132 forwards messages received by the physical interface card 2133 based on the forwarding table distributed by the control plane. The forwarding table distributed by the control plane can be stored in the forwarding table entry memory 2134. In some embodiments, the control plane and the forwarding plane can be completely separated and not on the same data transmission device.

[0204] It's worth noting that there may be one or more main control boards, including a primary and a backup main control board. There may also be one or more interface boards; the stronger the data processing capability of the data transmission equipment, the more interface boards it provides. Each interface board may also have one or more physical interface cards. There may be no switching network board, or one or more; multiple boards can share the load and provide redundancy. In a centralized forwarding architecture, the data transmission equipment may not need a switching network board, as the interface boards handle the entire system's business data processing. In a distributed forwarding architecture, the data transmission equipment can have at least one switching network board, which enables data exchange between multiple interface boards, providing high-capacity data exchange and processing capabilities. Therefore, the data access and processing capabilities of a distributed architecture data transmission equipment are greater than those of a centralized architecture data transmission equipment. Alternatively, the data transmission device can also be a single board, without a switching network board. The functions of the interface board and the main control board are integrated on this one board. In this case, the central processing unit (CPU) on the interface board and the CPU on the main control board can be combined into a single CPU to perform the combined functions. This type of data transmission device has lower data exchange and processing capabilities (e.g., low-end switches or routers). The specific architecture adopted depends on the specific network deployment scenario, and no restrictions are imposed here.

[0205] In a specific embodiment, the data transmission device 2100 corresponds to the above-described... Figure 17 The data transmission apparatus shown. In some embodiments, Figure 17 The transceiver module 1601 in the data transmission device shown is equivalent to the physical interface card 2133 in the data transmission device 2100, and the processing module 1602 is equivalent to the central processing unit 2111 or network processor 2132 in the data transmission device 2100.

[0206] This application provides a server, which can vary significantly due to differences in configuration or performance. It may include one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories. This at least one computer program is loaded and executed by the one or more processors to enable the server to implement the data transmission methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0207] This application also provides a data transmission system, which includes multiple processing devices. For example, any one of the multiple processing devices is... Figure 18 The data transmission device 2000 or shown Figure 19 The data transmission device 2100 shown above, and the data transmission method executed by any of the processing devices, can be found in the above description. Figure 5 or Figure 16 The relevant descriptions of the embodiments shown will not be repeated here.

[0208] It should be understood that the aforementioned processor can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0209] Furthermore, in an alternative embodiment, the memory described above may include read-only memory and random access memory, and provide instructions and data to the processor. The memory may also include non-volatile random access memory. For example, the memory may also store device type information.

[0210] The memory can be volatile or non-volatile, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0211] This application also provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to enable the computer to implement any of the data transmission methods described above.

[0212] This application also provides a computer program (product) that, when executed by a computer, causes the processor or computer to perform the corresponding steps and / or processes in the above method embodiments.

[0213] This application also provides a chip, including a processor, for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform any of the above data transmission methods.

[0214] This application embodiment also provides another chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute any of the above data transmission methods.

[0215] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to this application are generated, in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk), etc.

[0216] Those skilled in the art will recognize that the method steps and modules described in conjunction with the embodiments disclosed herein can be implemented in software, hardware, firmware, or any combination thereof. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0217] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0218] When implemented using software, it can be implemented wholly or partially as a computer program product. This computer program product includes one or more computer program instructions. As an example, the methods of this application embodiment can be described in the context of machine-executable instructions, such as program modules that execute on a device on a real or virtual processor of the target. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc., which perform specific tasks or implement specific abstract data structures. In various embodiments, the functionality of program modules can be combined or divided among the described program modules. The machine-executable instructions for the program modules can execute within a local or distributed device. In a distributed device, the program modules can reside on both local and remote storage media.

[0219] Computer program code used to implement the methods of the embodiments of this application may be written in one or more programming languages. This computer program code may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when executed by the computer or other programmable data processing apparatus, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0220] In the context of the embodiments of this application, computer program code or related data may be carried by any suitable carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, etc.

[0221] Examples of signals may include electrical, optical, radio, sound, or other forms of propagation signals, such as carrier waves, infrared signals, etc.

[0222] A machine-readable medium can be any tangible medium that contains or stores programs for or relating to an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. More detailed examples of machine-readable storage media include electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0223] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0224] In the embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through some interfaces, devices, or modules, or they may be electrical, mechanical, or other forms of connection.

[0225] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0226] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0227] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0228] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or order of execution. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the scope of various examples, a first image can be referred to as a second image, and similarly, a second image can be referred to as a first image. Both the first image and the second image can be images, and in some cases, they can be separate and distinct images.

[0229] It should also be understood that, in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0230] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple second messages refer to two or more second messages. The terms "system" and "network" are often used interchangeably in this document.

[0231] It should be understood that the terminology used in the description of the various examples herein is for the purpose of describing particular examples only and is not intended to be limiting. As used in the description of the various examples and the appended claims, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0232] It should also be understood that the term "and / or" as used herein refers to and covers any and all possible combinations of one or more of the associated listed items. The term "and / or" describes an association between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects are in an "or" relationship.

[0233] It should also be understood that the term “comprising” (also referred to as “includes”, “including”, “comprises” and / or “comprising”) as used in this specification specifies the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0234] It should also be understood that the terms “if” and “if” can be interpreted as meaning “when” or “upon”, or “in response to determination” or “in response to detection”. Similarly, depending on the context, the phrases “if determination…” or “if detection [the stated condition or event]” can be interpreted as meaning “when determination…”, or “in response to determination…”, or “when detection [the stated condition or event]” or “in response to detection [the stated condition or event]”.

[0235] It should be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information.

[0236] It should also be understood that the phrases "an embodiment," "an embodiment," and "a possible implementation" used throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment or implementation is included in at least one embodiment of this application. Therefore, the phrases "in an embodiment," "an embodiment," or "a possible implementation" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0237] The above description is only an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A data transmission method, characterized in that, Applied to a first processing device, the method includes: Get the first data; Obtain the second data; The first data is transmitted through the first main processing unit among the at least two main processing units corresponding to the first processing device; Since the amount of the second data is greater than the amount of the first data, the second data is transmitted through the second main processing unit and the slave processing unit in the at least two main processing units.

2. The method according to claim 1, characterized in that, The amount of the second data transmitted by the second main processing unit and the slave processing unit is determined based on the transmission speed of the second main processing unit and the slave processing unit, respectively.

3. The method according to claim 1 or 2, characterized in that, The transmission links corresponding to the second main processing unit and the slave processing unit are different.

4. The method according to claim 3, characterized in that, The receiving end of the second data is the second processing device. The transmission link corresponding to the second main processing unit is a direct link from the first processing device to the second processing device. The transmission link corresponding to the secondary processing unit is a relay link from the first processing device to the intermediate device and then to the second processing device.

5. The method according to claim 4, characterized in that, The first processing device and the second processing device are processing devices in a computing cluster, and the intermediate device is a host connected to the computing cluster, or the intermediate device is a third processing device in the computing cluster connected to the first processing device and the second processing device.

6. The method according to claim 4 or 5, characterized in that, The transmission of the second data from the processing unit includes: When the relay link is idle, the second data is transmitted via the processing unit based on the relay link.

7. The method according to claim 1 or 2, characterized in that, The second main processing unit and the slave processing unit share the same transmission link.

8. The method according to any one of claims 1-7, characterized in that, The transmission of the second data through the second main processing unit and the slave processing unit in the at least two main processing units includes: The first unit of data in the second data is transmitted through the second main processing unit, and the second unit of data in the second data is transmitted through the slave processing unit; After the second main processing unit has completed transmitting the first unit of data, the next first unit of data in the second data is transmitted through the second main processing unit. When the second unit of data has been transmitted from the processing unit, the next second unit of data in the second data is transmitted through the processing unit. This process continues until the second data transmission is complete.

9. The method according to any one of claims 1-7, characterized in that, The transmission of the second data through the second main processing unit and the slave processing unit in the at least two main processing units includes: The second main processing unit divides the second data into a first part of data and a second part of data. The first part of the data is transmitted through the second main processing unit, and the second part of the data is transmitted through the slave processing unit.

10. The method according to any one of claims 1-9, characterized in that, The method further includes: The second main processing unit issues a processing task to the slave processing unit, and the processing task is used to instruct the slave processing unit to transmit the corresponding data in the second data.

11. The method according to claim 10, characterized in that, The first processing device includes a memory region for task synchronization between the second main processing unit and the slave processing unit.

12. The method according to claim 11, characterized in that, The memory region stores at least one of the following: task flag, address flag, granularity flag, link flag, completion flag, or end flag. The task flag indicates whether the second main processing unit has issued a processing task. The address flag indicates the address offset of the data corresponding to the issued processing task. The granularity flag indicates the length of the unit data corresponding to the issued processing task. The link flag indicates whether the transmission link corresponding to the slave processing unit is idle. The completion flag indicates whether the slave processing unit has completed the issued processing task. The end flag indicates whether the second data transmission is complete.

13. The method according to any one of claims 1-12, characterized in that, The first processing device is a processing device in a computing cluster, which includes n processing devices. The n processing devices transmit data to each other. The second data is the data that meets the acceleration conditions among the data transmitted between the n processing devices. The acceleration conditions include the data volume being ranked in the top n from largest to smallest, and the sending and receiving ends of data of the same data volume level being different.

14. The method according to any one of claims 1-13, characterized in that, The second data satisfies the revenue condition, which includes the proportion of the data volume difference to the data volume of the second data being greater than a proportion threshold, and the data volume difference being the difference between the data volume of the second data and the data volume of the first data.

15. The method according to any one of claims 1-14, characterized in that, The at least two main processing units are main threads, and the slave processing units are slave threads.

16. The method according to any one of claims 1-15, characterized in that, The first processing device is a server, a neural network processing unit (NPU), a graphics processing unit (GPU), a tensor processing unit (TPU), or a central processing unit (CPU).

17. The method according to any one of claims 1-16, characterized in that, The first data and the second data are variable data in all-to-allv set communication operations.

18. A data transmission device, characterized in that, The data transmission device is applied to the first processing device, and the data transmission device includes: The transceiver module is used to perform the receiving and / or sending related operations in the method according to any one of claims 1-17; The processing module is configured to perform operations other than the receiving and / or sending related operations in the method described in any one of claims 1-17.

19. A data transmission device, characterized in that, The data transmission device includes a processor coupled to a memory, the memory storing at least one program instruction or code, the at least one program instruction or code being loaded and executed by the processor to enable the data transmission device to implement the data transmission method according to any one of claims 1-17.

20. A computer-readable storage medium, characterized in that, The computer storage medium stores at least one instruction, which is loaded and executed by a processor to enable the computer to implement the data transmission method as described in any one of claims 1-17.

21. A computer program product, characterized in that, The computer program product includes: computer program code, which is loaded and executed by a computer to enable the computer to implement any of the data transmission methods described in claims 1-17.

Images