Data communication method and apparatus, electronic device, and storage medium

Abstract

The present application relates to the technical field of computer network communication, and provides a data communication method and device, electronic equipment and storage medium, the method is applied to a source computing node, the method comprises the following steps: determining a target node identifier based on input data to be processed; a target expert module for processing the input data is deployed on a target computing node corresponding to the target node identifier; generating a data communication instruction based on the target node identifier, and sending the data communication instruction to a network switching device, so that the network switching device performs data interaction with the target computing node corresponding to the target node identifier based on the data communication instruction, breaking the limitation of the traditional in-network computing which depends on the static communication group configuration, enabling the network communication to adapt to the highly dynamic and random communication mode in MoE and other scenarios, realizing efficient data distribution or aggregation for any dynamically combined target node without pre-configuration, and improving the communication efficiency of distributed large-scale model training and inference.

Description

Technical Field

[0001] This invention relates to the field of computer network communication technology, and in particular to a data communication method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, Mixture-of-Experts (MoE) models have become the mainstream architecture for large-scale model training and inference due to their ability to achieve a huge parameter scale with relatively low computational cost. In distributed training or inference scenarios of the MoE architecture, in-network computing technology is introduced to improve the communication efficiency between nodes, utilizing the hardware capabilities of switches to accelerate aggregate communication operations.

[0003] In existing on-network computing technology systems, switches manage communication groups receiving data using a static configuration approach. This involves maintaining an internal static forwarding table, with each multicast identifier corresponding to a fixed list of Graphics Processing Units (GPUs). However, this approach has significant limitations when facing the complex parallel strategies of modern large-scale MoE models. It struggles to meet dynamic communication requirements, thus hindering the efficiency of large model training and inference. Summary of the Invention

[0004] This invention provides a data communication method, apparatus, electronic device, and storage medium to solve the problem that static multicast group configuration in network computing cannot adapt to the dynamic communication requirements generated by the TopK algorithm in the MoE scenario.

[0005] This invention provides a data communication method applied to a source computing node, comprising:

[0006] Based on the input data to be processed, a target node identifier is determined; a target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier.

[0007] Based on the target node identifier, a data communication instruction is generated and sent to the network switching device, so that the network switching device can interact with the target computing node corresponding to the target node identifier based on the data communication instruction.

[0008] According to a data communication method provided by the present invention, determining the target node identifier based on the input data to be processed includes:

[0009] Determine the correlation score between the input data and multiple expert modules;

[0010] Based on the relevance score, multiple target expert modules are selected from each expert module.

[0011] The computing nodes where each target expert module resides are designated as target computing nodes;

[0012] The node identifier of the target computing node is deduplicated to obtain the target node identifier.

[0013] According to a data communication method provided by the present invention, when the data communication instruction is a data reduction instruction requesting a network reduction operation, the data communication instruction carries a multicast memory address;

[0014] The multicast memory address points to a storage space on the corresponding target computing node to store local aggregated data; the local aggregated data is obtained by aggregating the processing results of the target expert module residing on the corresponding target computing node on the input data.

[0015] The step of sending the data communication command to the network switching device further includes:

[0016] Receive the global aggregation result returned by the network switching device;

[0017] The global aggregation result is obtained by the network switching device parsing the target node identifier and the multicast memory address based on the data reduction instruction, reading the local aggregation data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier, and performing network-level aggregation on the local aggregation data.

[0018] According to a data communication method provided by the present invention, generating data communication instructions based on the target node identifier includes:

[0019] If the number of target node identifiers exceeds the total number of identifiers that a single data communication instruction can carry, the target node identifiers are divided into subsets to obtain multiple identifier subsets;

[0020] Instructions are generated based on each subset of identifiers to obtain the data communication instructions corresponding to each subset of identifiers.

[0021] According to a data communication method provided by the present invention, receiving the global aggregation result returned by the network switching device includes:

[0022] The system receives reduction results returned by the network switching device based on each network reduction instruction, and stores each reduction result in a register. The reduction result is obtained by the network switching device parsing the target node identifier and the multicast memory address in the corresponding identifier subset based on the corresponding data reduction instruction, reading the local aggregated data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier in the corresponding identifier subset, and performing network-level aggregation on the local aggregated data.

[0023] The global aggregation result is obtained by accumulating the reduction results stored in the register.

[0024] The present invention also provides a data communication method applied to a network switching device, comprising:

[0025] The system receives a data communication instruction sent by a source computing node. The data communication instruction is generated based on a target node identifier, which is determined based on the input data to be processed. A target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier.

[0026] Parse the data communication command to obtain the target node identifier;

[0027] Based on the target node identifier, data interaction is performed with the corresponding target computing node.

[0028] The present invention also provides a data communication device applied to a source computing node, comprising:

[0029] The identifier determination module is used to determine the target node identifier based on the input data to be processed; the target computing node corresponding to the target node identifier is equipped with a target expert module for processing the input data;

[0030] The instruction generation module is used to generate data communication instructions based on the target node identifier and send the data communication instructions to the network switching device, so that the network switching device can perform data interaction with the target computing node corresponding to the target node identifier based on the data communication instructions.

[0031] The present invention also provides a data communication device applied to a network switching device, comprising:

[0032] A receiving module is used to receive data communication instructions sent by a source computing node. The data communication instructions are generated based on a target node identifier, which is determined based on the input data to be processed. A target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier.

[0033] The parsing module is used to parse the data communication instructions to obtain the target node identifier;

[0034] The interaction module is used to interact with the corresponding target computing node based on the target node identifier.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the data communication method as described above.

[0036] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data communication method as described above.

[0037] The data communication method, apparatus, electronic device, and storage medium provided by this invention dynamically parse the target node identifier based on the input data at the source computing node side and integrate the node identifier into the data communication command and send it to the network switching device. This breaks the limitation of relying on static communication group configuration in traditional on-network computing, enabling network communication to adapt to highly dynamic and random communication modes in scenarios such as MoE, especially in DeepEP scenarios. It achieves efficient data distribution or aggregation for any dynamically combined target computing nodes without pre-configuration, thereby greatly improving the communication efficiency of distributed large-scale model training and inference. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is one of the flowcharts illustrating the data communication method provided by the present invention;

[0040] Figure 2 This is a framework example diagram of the data communication process provided by the present invention;

[0041] Figure 3 This is the second flowchart illustrating the data communication method provided by the present invention;

[0042] Figure 4 This is one of the structural schematic diagrams of the data communication device provided by the present invention;

[0043] Figure 5 This is the second schematic diagram of the data communication device provided by the present invention;

[0044] Figure 6 This is a schematic diagram of the structure of the electronic device provided by the present invention.

[0045] Figure label:

[0046] 410: Identification module; 420: Instruction generation module; 510: Receiving module;

[0047] 520: Parsing module; 530: Interaction module; 610: Processor;

[0048] 620: Communication interface; 630: Memory; 640: Communication bus. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] With the development of large-scale AI model technology, hybrid expert models have gradually become the mainstream architecture for large-scale model training and inference due to their ability to achieve a huge parameter scale with relatively low computational cost. In distributed training or inference scenarios of the MoE architecture, in order to improve the communication efficiency between nodes, on-network computing technologies are usually introduced to leverage the hardware capabilities of switches to accelerate aggregate communication operations, such as multicast and reduction.

[0051] In existing on-network computing technology systems, switches primarily manage communication groups, i.e., GPU groups receiving data, using static configuration. Specifically, the switch maintains a static forwarding table, where each multicast ID (mc_id) corresponds to a fixed list of GPUs. When a source computing node needs to send data to a specific group of GPUs, it simply queries this table to send the data to the corresponding GPU. However, this static configuration mechanism has gradually revealed significant problems in several aspects when facing the complex parallel strategies of modern large-scale MoE models.

[0052] Specifically, on the one hand, the number of mc_ids in the switch hardware resources is limited and cannot be allocated indefinitely, and once the correspondence between mc_id and GPU Group is configured, it is difficult to change it in real time during operation.

[0053] On the other hand, in advanced MoE parallel scenarios such as Deep Expert Parallelism (DeepEP), the data communication pattern is highly dynamic. Specifically, in DeepEP scenarios, for each piece of input data to be processed, its associated expert (EP) module needs to be calculated in real time. These expert modules are usually distributed across different GPUs in the cluster. This means that the GPU combination involved in each communication operation, such as dispatch and combine, is calculated in real time based on the input data and is random and dynamically changing. However, existing static configuration methods cannot exhaustively list all possible dynamic GPU combinations, and since it is impossible to predict which nodes the data needs to be sent to, it is also impossible to pre-configure the corresponding mc_id.

[0054] Therefore, current on-network computing technologies cannot meet the dynamic communication requirements of parallel scenarios such as DeepEP, causing the powerful acceleration capabilities of switches to be unable to play a role in these key scenarios, becoming a bottleneck restricting the efficiency of large model training and inference.

[0055] To address this issue, the present invention provides a data communication method that aims to solve the problem that the static multicast group configuration in existing on-network computing cannot meet the dynamic communication requirements in the MoE scenario. It realizes the transfer of the selection of communication targets from the static configuration of network switching devices to the dynamic instructions of source computing nodes, thereby enabling efficient data interaction for any dynamic combination of nodes using network switching devices.

[0056] Figure 1 This is one of the flowcharts illustrating the data communication method provided by the present invention, which is applied to a source computing node. In a real distributed computing cluster, the source computing node typically refers to the hardware processing unit in the server that initiates the data communication request. It can be a GPU, Tensor Processing Unit (TPU), Neural Processing Unit (NPU), or other acceleration chips used to carry out artificial intelligence (AI) model training or inference tasks. The embodiments of the present invention do not specifically limit this. Figure 1 As shown, the method includes:

[0057] Step 110: Based on the input data to be processed, determine the target node identifier; the target computing node corresponding to the target node identifier has a target expert module deployed for processing the input data;

[0058] Step 120: Based on the target node identifier, generate a data communication command and send the data communication command to the network switching device so that the network switching device can interact with the target computing node corresponding to the target node identifier based on the data communication command.

[0059] Specifically, in the large-scale training or inference process of hybrid expert models, not all computing resources are activated simultaneously. Instead, a subset of expert modules are dynamically selected for processing based on the characteristics of the input data. Specifically, the source computing node can first acquire the input data to be processed. This input data is the data that needs further processing during model training or inference, such as activation values ​​of intermediate layers in a neural network, token data from text sequences, etc.

[0060] To process this input data, the source compute node needs to select target expert modules from numerous expert modules based on the characteristics of the input data, such as weights or matching degrees calculated through a gating network. Here, the target expert module refers to the sub-model or functional module selected to participate in the current computation task. Due to the large scale of the models, target expert modules are typically deployed across different physical devices in the cluster. Therefore, after determining the target expert modules participating in the current computation task, the source compute node needs to further determine the physical locations of these target expert modules based on the cluster's topology mapping, and accordingly identify the corresponding physical devices, i.e., the target compute nodes, and their node identifiers, i.e., the target node identifiers, such as physical GPU ID, network address index, and port number.

[0061] It should be noted that, due to the sparse activation characteristic of the MoE architecture, the target expert module matched for different input data is dynamically changing. This directly leads to the target node identifier changing dynamically in real time with different input data, rather than a static combination that is fixed in advance in the network configuration.

[0062] After identifying the destination of the data communication, i.e., the target node identifier, the source compute node initiates network operations by generating data communication commands. That is, the source compute node generates corresponding data communication commands based on the target node identifier and performs data interaction accordingly. Unlike traditional solutions that rely on static switch configuration tables, such as static multicast groups, the data communication commands in this embodiment of the invention, such as specific opcodes and network request packets, directly carry or are associated with the target node identifier. In other words, the source compute node directly encodes the dynamic information of which compute nodes are involved in this communication into the command or the communication message triggered by that command.

[0063] After generating the data communication command, the source compute node sends it to the network switching device for data forwarding. This network switching device can be an Ethernet switch or an InfiniBand switch that supports Remote Direct Memory Access (RDMA). Upon receiving the command, the network switching device parses it to identify the target object of the communication, i.e., the target compute node corresponding to the target node identifier, and then interacts with it through data exchange, such as multicast distribution or reduction aggregation. That is, it distributes the input data to the corresponding target compute node, or retrieves data from the corresponding target compute node, performs reduction on the network side, and returns the data to the source compute node.

[0064] The data communication method provided by this invention dynamically parses the target node identifier based on the input data on the source computing node side and integrates the node identifier into the data communication command and sends it to the network switching device. This breaks the limitation of relying on static communication group configuration in traditional on-network computing, enabling network communication to adapt to highly dynamic and random communication modes in scenarios such as MoE, especially in DeepEP scenarios. It achieves efficient data distribution or aggregation for any dynamically combined target computing nodes without pre-configuration, thereby greatly improving the communication efficiency of distributed large-scale model training and inference.

[0065] Based on the above embodiments, step 110 includes:

[0066] Determine the correlation scores between the input data and multiple expert modules;

[0067] Based on the relevance score, multiple target expert modules are selected from each expert module.

[0068] The computing nodes where each target expert module resides are designated as target computing nodes;

[0069] The node identifier of the target computing node is deduplicated to obtain the target node identifier.

[0070] Specifically, in complex MoE scenarios such as deep expert parallelism, the process of determining the target node identifier based on the input data to be processed is not just a simple address lookup, but a mapping and transformation from the algorithm logic layer to the physical hardware layer.

[0071] In detail, during practical application, the source computing node first needs to perform correlation calculation to determine the correlation score between the input data and each expert module. Specifically, in a hybrid expert model, a gating network or routing module is set up to evaluate the matching degree between the current input data, such as a token, and multiple pre-set expert modules in the model. The source computing node runs this gating algorithm or routing module to calculate the similarity between the feature vector of the input data and the feature vectors of each expert module, and determines the correlation score accordingly. This score intuitively reflects the appropriateness of each expert module in processing the current input data.

[0072] Next, the source computing node selects multiple target expert modules from among the expert modules based on this correlation score. That is, due to limited computing resources and to maintain the characteristic of sparse activation, not all expert modules are typically activated. Instead, a Top K strategy is adopted, which sorts the expert modules according to their calculated correlation scores and selects the K expert modules with the highest scores, such as Top 2 or Top 4. These selected expert modules with the highest correlation scores are the target expert modules that need to receive input data and execute the current computing task.

[0073] After this, the source compute node can determine the compute node where each target expert module resides, i.e., the target compute node. Here, "residence" refers to the mapping between software entities and hardware devices. In a distributed cluster, a compute node, such as a server or a GPU card, often has strong computing power; therefore, multiple expert modules are deployed / run on it simultaneously. By querying the cluster's topology mapping table or configuration information, the source compute node can directly map each of the selected target expert modules to a specific physical device; these physical devices are the target compute nodes.

[0074] Since multiple expert modules may be deployed on a single computing node, and among the top K target expert modules selected above, it is highly likely that several expert modules, such as expert module A and expert module B, are deployed on the same computing node. In this case, if the node identifier of the target computing node is not processed, the source computing node may repeatedly send the same data packets to the same target computing node, resulting in a waste of network bandwidth.

[0075] Therefore, in this embodiment of the invention, after determining the target computing node, it is also necessary to perform deduplication processing on the node identifier of the target computing node, such as removing duplicate node identifiers, to ensure that for the same target computing node, regardless of how many target expert modules are running on it, the source computing node generates only one target node identifier in this communication. All node identifiers obtained after this processing are the final target node identifiers.

[0076] In this embodiment of the invention, by using scoring and filtering, physical mapping and deduplication, not only is dynamic routing based on data content realized in the MoE architecture, but more importantly, node identifier deduplication is performed at the source computing node, realizing an efficient communication mode of one transmission, multiple uses, avoiding sending redundant data to the same computing node, greatly saving network bandwidth resources, and laying the foundation for the subsequent generation of efficient data communication instructions.

[0077] Based on the above embodiments, when the data communication instruction is a data reduction instruction requesting a network reduction operation, the data communication instruction carries a multicast memory address.

[0078] The storage space pointed to by the multicast memory address on the corresponding target computing node is used to store local aggregated data;

[0079] Locally aggregated data is obtained by aggregating the processing results of the input data by the target expert module residing on the corresponding target computing node.

[0080] In step 120, the data communication command is sent to the network switching device, and the process further includes:

[0081] Receive the global aggregation results returned by the network switching device;

[0082] The global aggregation result is obtained by the network switching device based on the data reduction instruction to parse the target node identifier and multicast memory address, read the local aggregation data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier, and perform network-level aggregation on the local aggregation data.

[0083] Specifically, during the training or inference process of the MoE model, in addition to distributing data to each expert module for processing / computation (i.e., the Dispatch process), the source computing node also needs to collect the processing results of each expert module (i.e., the Combine process).

[0084] In detail, in practical applications, when the source computing node needs to collect and merge the processing results of various expert modules, the generated data communication command can be a data reduction command requesting a network reduction operation. Unlike ordinary data sending commands, this data reduction command, such as the mm.load_reduce command, not only includes the target node identifier but also carries the multicast memory address (mc_addr) in its command payload or a specific field.

[0085] Figure 2 This is a framework example diagram of the data communication process provided by the present invention, such as... Figure 2As shown, on the target computing node, such as the target GPU, the target expert modules obtain their respective processing results after processing the input data. Since multiple target expert modules may run on a single target computing node, to reduce the amount of data transmitted over the network, these modules first undergo a pre-merging process within the node. That is, their respective processing results are accumulated or aggregated into a local buffer on the target computing node. This buffer stores the local aggregated data of the target computing node; and the location of this buffer in the node's memory is the location pointed to by the multicast memory address on the target computing node. In other words, the storage space pointed to by the multicast memory address on the corresponding target computing node is used to store local aggregated data; the local aggregated data is obtained by aggregating the processing results of the target expert modules residing on the corresponding target computing node on the input data. Here, the aggregation of the processing results of each target expert module achieves physical-level reduction within the target computing node.

[0086] After the source compute node issues a data reduction instruction carrying the multicast memory address and the target node identifier, it will also receive a global aggregation result returned by the network switching device based on this instruction. Specifically, after receiving the data reduction instruction, the network switching device will first parse out the target node identifier (i.e., who to communicate with) and the multicast memory address (i.e., where to read data) from it. Then, it can use RDMA Read or a similar remote read mechanism to directly locate the storage space pointed to by the multicast memory address on the target compute node corresponding to the parsed target node identifier through the network, and read the local aggregated data from it.

[0087] After obtaining the local aggregated data of the target computing node, the network switching device will use its own computing units, such as arithmetic logic units, to perform network-level aggregation on the local aggregated data, such as summation, averaging, and finding the maximum value, and finally obtain the global aggregation result, which will be returned to the source computing node.

[0088] In this embodiment of the invention, by carrying multicast memory addresses in the data reduction instructions and combining a two-level reduction mechanism of local aggregation within nodes and network-level aggregation, efficient recycling of MoE processing results is achieved. This enables network switching devices to directly retrieve pre-processed data from the memory addresses of multiple target computing nodes for final merging. This avoids congestion caused by a massive influx of raw processing results into the network and utilizes the on-network computing capabilities of the network switching devices, greatly reducing communication latency and the occupation of source computing node resources.

[0089] Based on the above embodiments, in step 120, a data communication instruction is generated based on the target node identifier, including:

[0090] When the number of target node identifiers exceeds the total number of identifiers that a single data communication command can carry, the target node identifiers are partitioned into multiple identifier subsets.

[0091] Instructions are generated based on each subset of identifiers to obtain the data communication instructions corresponding to each subset of identifiers.

[0092] Specifically, in the process of generating data communication instructions based on target node identifiers, considering the physical constraints of hardware implementation—that is, in the hardware design of actual high-performance network protocols or network switching equipment—the length of an instruction is usually fixed, and the field specifically used to carry the target node identifier has a limited bit width, such as 64 bits. This means that there is an upper limit to the total number of identifiers that a single data communication instruction can carry, such as only being able to carry 8 node identifiers (e.g., GPU ID 0, GPU ID 1, GPU ID 2, GPU ID 3, GPU ID 4, GPU ID 5, GPU ID 6, GPU ID 7). However, with the development of MoE parallel strategies such as DeepEP, the target expert modules selected by the Top K strategy may be distributed across a large number of target computing nodes, such as 16 or even more. This would cause the number of identified target node identifiers to exceed the physical carrying capacity of a single data communication instruction. To solve this problem, the following processing strategy is adopted in this embodiment of the invention:

[0093] In other words, when the number of target node identifiers exceeds the total number of identifiers that a single data communication instruction can carry, the target node identifiers are partitioned into subsets, resulting in multiple identifier subsets. Specifically, the source compute node first determines whether the total number of currently identified target node identifiers exceeds the maximum capacity allowed by the hardware instruction format. If this capacity is exceeded, a batch processing mechanism is initiated, i.e., the target node identifiers are partitioned into subsets. This process is similar to splitting an oversized batch of goods into multiple packages. The source compute node divides all target node identifiers into several groups according to the instruction's capacity limit; each group is an identifier subset. For example, if there are 16 target node identifiers, but a single data communication instruction can only accommodate 8 target node identifiers, then it is divided into two identifier subsets containing 8 target node identifiers each.

[0094] Next, instructions can be generated based on each subset of identifiers, thus obtaining the corresponding data communication instructions for each subset. That is, the source compute node no longer attempts to complete all communication with a single instruction, but instead generates instructions based on each subset of identifiers. For each defined subset of identifiers, the source compute node constructs an independent data communication instruction. Ultimately, the source compute node will receive multiple data communication instructions equal to the number of identifier subsets, and will sequentially send these instructions to the network switching device. Through this strategy of breaking down the whole into parts, the network switching device will receive multiple instruction sequences and perform data interaction based on the target node identifier carried in each instruction, i.e., multicast distribution or reduction aggregation.

[0095] In this embodiment of the invention, by performing subset partitioning and batch instruction generation on the excessive number of target node identifiers, the limitation imposed by the fixed length of the underlying hardware instruction format on large-scale distributed communication at the upper layer is effectively overcome, greatly enhancing scalability. It can be compatible with larger-scale MoE models or larger Top K parameter settings that may appear in the future, ensuring the flexibility and universality of the software application layer under the premise that the hardware specifications remain unchanged.

[0096] Based on the above embodiments, receiving the global aggregation result returned by the network switching device includes:

[0097] The system receives reduction results returned by the network switching device based on each network reduction instruction and stores each reduction result in a register. The reduction result is obtained by the network switching device based on the corresponding data reduction instruction, which parses the target node identifier and multicast memory address in the corresponding identifier subset, reads the local aggregated data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier in the corresponding identifier subset, and performs network-level aggregation on the local aggregated data.

[0098] The global aggregation result is obtained by accumulating the reduction results stored in the register.

[0099] Specifically, when the source computing node is limited by the bit width of a single data communication instruction, and the communication task for a large-scale target computing node is split into multiple subsets and multiple data reduction instructions are sent, the source computing node also needs to process the reduction results returned in batches from the network switching device. In this embodiment of the invention, a flexible global result synthesis mechanism is designed by utilizing the high-speed storage resources within the source computing node chip.

[0100] In detail, after the network switching device processes the reduction requests for each subset of identifiers, it generates multiple reduction results in sequence. These reduction results are essentially network-level intermediate states of locally aggregated data. That is, after the network switching device parses the target node identifier and multicast memory address of the corresponding subset of identifiers based on the corresponding data reduction instruction, it reads the data from the storage space pointed to by the multicast memory address on the multiple target computing nodes corresponding to the subset of identifiers and performs network-level aggregation.

[0101] To match the high throughput of network transmissions and reduce latency, the source compute node uses on-chip registers as temporary buffers. Registers are the fastest storage units in a compute node for reading and writing. When reduction results arrive from the network side, the source compute node temporarily stores these batches of reduction results into designated registers via DMA or instruction write-back mechanisms.

[0102] After this, the source computing node performs an accumulation calculation based on the reduction results stored in the registers to obtain the global aggregation result. It should be noted that the specific timing coordination of storage and accumulation is not uniquely limited in this embodiment of the invention. To adapt to different hardware resource conditions, the following two implementation methods can be adopted:

[0103] The iterative accumulation mode is suitable for scenarios with limited register resources. In this mode, the source compute node adopts a streaming processing logic. That is, when it receives the reduction result of the first subset of identifiers, it stores it in an accumulation register; when it receives the reduction result of subsequent subsets of identifiers, the source compute node immediately reads the old value from the accumulation register, adds it to the newly arrived reduction result or performs the corresponding aggregation operation, and writes the calculated new value to the accumulation register, overwriting the old value. In this way, storage and accumulation are performed alternately, which reduces the use of registers, requiring only a small number of registers to complete the processing.

[0104] Batch accumulation mode is suitable for high-performance vector register scenarios. In this mode, the source compute node uses a vector register file to first store the reduction results of all received identifier subsets in different register addresses, i.e., completing the write-to-deployment of all results. Once the reduction results of all identifier subsets are ready, single-instruction multiple-data or dedicated adder instructions are used to sum or aggregate the reduction results in these registers all at once. This fully utilizes the parallel computing capabilities of the GPU and reduces pipeline pauses.

[0105] Regardless of the implementation method, the essence is to use registers to temporarily store the fragmentation results returned by the network side, and finally complete the logical merging of all fragments on the source computing unit side, thereby obtaining a global aggregation result that covers all target node identifiers.

[0106] In this embodiment of the invention, the problem of being unable to complete large-scale node reduction in one go due to the limited capacity of a single instruction is solved by the collaborative mechanism of network-side batch reduction and final accumulation of source computing unit registers. The high-speed registers of the source computing node are used to perform lightweight calculations in the last stage, which not only ensures high performance of data processing, but also realizes unlimited expansion of communication scale, so that large-scale parallel strategies such as DeepEP are no longer limited by instruction bit width.

[0107] The present invention also provides a data communication method. Figure 3 This is the second flowchart illustrating the data communication method provided by this invention. This method is applied to network switching equipment, which refers to network interconnection equipment deployed at the core of a distributed computing cluster. For example, it may support a programmable data plane, such as a P4 (Programming Protocol-Independent Packet Processors) language programmable switch or an RDMA switch with specific on-network computing functions. Figure 3 As shown, the method includes:

[0108] Step 310: Receive data communication instructions sent by the source computing node. The data communication instructions are generated based on the target node identifier, which is determined based on the input data to be processed. The target computing node corresponding to the target node identifier has a target expert module deployed on it for processing the input data.

[0109] Step 320: Parse the data communication command to obtain the target node identifier;

[0110] Step 330: Based on the target node identifier, perform data interaction with the corresponding target computing node.

[0111] Specifically, in actual operation, the network switching equipment is in a passive receiving state, ready to process data communication instructions from the source computing node at any time. Here, the data communication instructions received by the network switching equipment are not ordinary point-to-point transmission instructions, but special instructions carrying dynamic group information. This instruction is generated by the source computing node based on the target node identifier. The target node identifier is determined in real time by the source computing node based on the input data to be processed, such as calculating the correlation score using the Top K algorithm. The target node identifier points to the target computing node, and the target computing node deploys the target expert modules required for the current computing task. Therefore, the network switching equipment receives not just an instruction, but a dynamic business requirement characterizing which expert targets the current input data needs to be distributed to, or from which expert modules the processing results need to be collected.

[0112] Upon receiving a data communication command, the network switching device immediately enters its processing logic. That is, it first parses the data communication command. Unlike traditional network devices that simply look up a table based on the fixed IP address in the packet header for forwarding, in this embodiment of the invention, the network switching device deeply parses specific fields of the data communication command, such as the immediate value after the opcode, specific header fields, and bitmap information in the payload. Through this deep parsing, the network switching device can directly extract the target node identifier required for this communication from the command, for example, a list of GPU IDs or an encoded multicast bitmap.

[0113] After successfully resolving the target node identifier, the network switching device will interact with the corresponding target computing node based on this identifier. Specifically, the data interaction varies depending on the command type, and may include multicast distribution and reduction / aggregation. Multicast distribution refers to the network switching device sending the input data in parallel to all target computing nodes in the case of a send command, such as `mm.st`, achieving rapid distribution of a single set of data to multiple target computing nodes. Reduction / aggregation refers to the network switching device initiating read requests to all target computing nodes in the case of a reduction command, such as `mm.load_reduce`, collecting the results returned by each target computing node, performing accumulation or aggregation operations within its internal computing unit, and then returning the final result to the source computing node.

[0114] The data communication method provided by this invention enables network switching devices to parse dynamically carried target node identifiers, thereby supporting dynamic communication modes. It can closely cooperate with the expert selection logic of the MoE model and efficiently complete data distribution and aggregation for any random combination of computing nodes without relying on frequent reconfiguration, greatly improving the network's support capability for dynamic computing tasks.

[0115] The data communication device provided by the present invention is described below. The data communication device described below and the data communication method described above can be referred to in correspondence.

[0116] Figure 4 This is one of the structural schematic diagrams of the data communication device provided by the present invention, such as... Figure 4 As shown, this device is applied to the source computing node, and the device includes:

[0117] The identifier determination module 410 is used to determine the target node identifier based on the input data to be processed; the target computing node corresponding to the target node identifier is equipped with a target expert module for processing the input data.

[0118] The instruction generation module 420 is used to generate a data communication instruction based on the target node identifier and send the data communication instruction to the network switching device so that the network switching device can perform data interaction with the target computing node corresponding to the target node identifier based on the data communication instruction.

[0119] The data communication device provided by this invention dynamically parses the target node identifier based on the input data on the source computing node side and integrates the node identifier into the data communication command and sends it to the network switching device. This breaks the limitation of relying on static communication group configuration in traditional on-network computing, enabling network communication to adapt to highly dynamic and random communication modes in scenarios such as MoE, especially in DeepEP scenarios. It realizes efficient data distribution or aggregation of any dynamically combined target computing nodes without pre-configuration, thereby greatly improving the communication efficiency of distributed large-scale model training and inference.

[0120] Based on the above embodiments, the identifier determination module 410 is used for:

[0121] Determine the correlation score between the input data and multiple expert modules;

[0122] Based on the relevance score, multiple target expert modules are selected from each expert module.

[0123] The computing nodes where each target expert module resides are designated as target computing nodes;

[0124] The node identifier of the target computing node is deduplicated to obtain the target node identifier.

[0125] Based on the above embodiments, when the data communication instruction is a data reduction instruction requesting a network reduction operation, the data communication instruction carries a multicast memory address;

[0126] The multicast memory address points to a storage space on the corresponding target computing node to store local aggregated data; the local aggregated data is obtained by aggregating the processing results of the target expert module residing on the corresponding target computing node on the input data.

[0127] The device also includes a result receiving module for:

[0128] Receive the global aggregation result returned by the network switching device;

[0129] The global aggregation result is obtained by the network switching device parsing the target node identifier and the multicast memory address based on the data reduction instruction, reading the local aggregation data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier, and performing network-level aggregation on the local aggregation data.

[0130] Based on the above embodiments, the instruction generation module 420 is used for:

[0131] If the number of target node identifiers exceeds the total number of identifiers that a single data communication instruction can carry, the target node identifiers are divided into subsets to obtain multiple identifier subsets;

[0132] Instructions are generated based on each subset of identifiers to obtain the data communication instructions corresponding to each subset of identifiers.

[0133] Based on the above embodiments, the result receiving module is used for:

[0134] The system receives reduction results returned by the network switching device based on each network reduction instruction, and stores each reduction result in a register. The reduction result is obtained by the network switching device parsing the target node identifier and the multicast memory address in the corresponding identifier subset based on the corresponding data reduction instruction, reading the local aggregated data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier in the corresponding identifier subset, and performing network-level aggregation on the local aggregated data.

[0135] The global aggregation result is obtained by accumulating the reduction results stored in the register.

[0136] The data communication device provided by the present invention is described below. The data communication device described below and the data communication method described above can be referred to in correspondence.

[0137] Figure 5 This is a second schematic diagram of the data communication device provided by the present invention, as shown below. Figure 5 As shown, this device is used in network switching equipment, and the device includes:

[0138] The receiving module 510 is used to receive data communication instructions sent by the source computing node. The data communication instructions are generated based on the target node identifier, which is determined based on the input data to be processed. A target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier.

[0139] Parsing module 520 is used to parse the data communication command to obtain the target node identifier;

[0140] The interaction module 530 is used to perform data interaction with the corresponding target computing node based on the target node identifier.

[0141] The data communication device provided by this invention enables network switching equipment to parse dynamically carried target node identifiers, thereby supporting dynamic communication modes. It can closely cooperate with the expert selection logic of the MoE model and efficiently complete data distribution and aggregation for any random combination of computing nodes without relying on frequent reconfiguration, greatly improving the network side's support capability for dynamic computing tasks.

[0142] Figure 6 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 6 As shown, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640, wherein the processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a data communication method. This method can be applied to a source computing node and includes: determining a target node identifier based on input data to be processed; deploying a target expert module for processing the input data on the target computing node corresponding to the target node identifier; generating a data communication instruction based on the target node identifier and sending the data communication instruction to a network switching device, so that the network switching device can interact with the target computing node corresponding to the target node identifier based on the data communication instruction. This method can also be applied to network switching equipment. The method includes: receiving a data communication instruction sent by a source computing node, the data communication instruction being generated based on a target node identifier, the target node identifier being determined based on input data to be processed; a target expert module for processing the input data being deployed on the target computing node corresponding to the target node identifier; parsing the data communication instruction to obtain the target node identifier; and performing data interaction with the corresponding target computing node based on the target node identifier.

[0143] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, which, when executed by a computer, enable the computer to execute the data communication methods provided by the above methods. This method can be applied to a source computing node, and includes: determining a target node identifier based on input data to be processed; deploying a target expert module for processing the input data on the target computing node corresponding to the target node identifier; generating a data communication instruction based on the target node identifier, and sending the data communication instruction to a network switching device, so that the network switching device interacts with the target computing node corresponding to the target node identifier based on the data communication instruction. This method can also be applied to a network switching device, and includes: receiving a data communication instruction sent by a source computing node, the data communication instruction being generated based on a target node identifier, the target node identifier being determined based on the input data to be processed; deploying a target expert module for processing the input data on the target computing node corresponding to the target node identifier; parsing the data communication instruction to obtain the target node identifier; and interacting with the corresponding target computing node based on the target node identifier.

[0145] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program is implemented to perform the data communication methods provided by the methods described above. This method can be applied to a source computing node and includes: determining a target node identifier based on input data to be processed; deploying a target expert module for processing the input data on the target computing node corresponding to the target node identifier; generating a data communication instruction based on the target node identifier and sending the data communication instruction to a network switching device, so that the network switching device interacts with the target computing node corresponding to the target node identifier based on the data communication instruction. This method can also be applied to a network switching device, which includes: receiving a data communication instruction sent by a source computing node, the data communication instruction being generated based on a target node identifier, the target node identifier being determined based on the input data to be processed; deploying a target expert module for processing the input data on the target computing node corresponding to the target node identifier; parsing the data communication instruction to obtain the target node identifier; and interacting with the corresponding target computing node based on the target node identifier.

[0146] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data communication method, characterized in that, Applied to the source compute node, including: Based on the input data to be processed, a target node identifier is determined; a target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier. Based on the target node identifier, a data communication instruction is generated and sent to the network switching device, so that the network switching device can perform data interaction with the target computing node corresponding to the target node identifier based on the data communication instruction; When the data communication instruction is a data reduction instruction requesting a network reduction operation, the step of sending the data communication instruction to the network switching device further includes: The system receives a global aggregation result returned by the network switching device. The global aggregation result is obtained by the network switching device performing network-level aggregation on local aggregation data read from the storage space pointed to by the multicast memory address on the target computing node. The local aggregation data is obtained by performing intra-node aggregation on the processing result of the target expert module on the input data. The multicast memory address is obtained by parsing based on the data reduction instruction.

2. The data communication method according to claim 1, characterized in that, The process of determining the target node identifier based on the input data to be processed includes: Determine the correlation score between the input data and multiple expert modules; Based on the relevance score, multiple target expert modules are selected from each expert module. The computing nodes where each target expert module resides are designated as target computing nodes; The node identifier of the target computing node is deduplicated to obtain the target node identifier.

3. The data communication method according to claim 1, characterized in that, When the data communication instruction is a data reduction instruction requesting a network reduction operation, the data communication instruction carries a multicast memory address; The multicast memory address points to a storage space on the corresponding target computing node for storing local aggregated data; the local aggregated data is obtained by the corresponding target computing node aggregating the processing results of the target expert module residing on it for the input data.

4. The data communication method according to claim 3, characterized in that, The step of generating data communication instructions based on the target node identifier includes: If the number of target node identifiers exceeds the total number of identifiers that a single data communication instruction can carry, the target node identifiers are divided into subsets to obtain multiple identifier subsets; Instructions are generated based on each subset of identifiers to obtain the data communication instructions corresponding to each subset of identifiers.

5. The data communication method according to claim 4, characterized in that, The receipt of the global aggregation result returned by the network switching device includes: The system receives reduction results returned by the network switching device based on each network reduction instruction, and stores each reduction result in a register. The reduction result is obtained by the network switching device parsing the target node identifier and the multicast memory address in the corresponding identifier subset based on the corresponding data reduction instruction, reading the local aggregated data from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier in the corresponding identifier subset, and performing network-level aggregation on the local aggregated data. The global aggregation result is obtained by accumulating the reduction results stored in the register.

6. A data communication method, characterized in that, Applied to network switching equipment, including: The system receives a data communication instruction sent by a source computing node. The data communication instruction is generated based on a target node identifier, which is determined based on the input data to be processed. A target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier. Parse the data communication command to obtain the target node identifier; Based on the target node identifier, data interaction is performed with the corresponding target computing node; When the data communication instruction is a data reduction instruction requesting a network reduction operation, the step of interacting with the corresponding target computing node based on the target node identifier further includes: The target node identifier and multicast memory address are obtained by parsing the data reduction instruction. Local aggregated data is read from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier; the local aggregated data is obtained by aggregating the processing results of the target expert module on the input data within the node. The local aggregated data is then subjected to network-level aggregation to obtain the global aggregation result; The global aggregation result is returned to the source computing node.

7. A data communication device, characterized in that, Applied to the source compute node, including: The identifier determination module is used to determine the target node identifier based on the input data to be processed; the target computing node corresponding to the target node identifier is equipped with a target expert module for processing the input data; The instruction generation module is used to generate a data communication instruction based on the target node identifier and send the data communication instruction to the network switching device, so that the network switching device can perform data interaction with the target computing node corresponding to the target node identifier based on the data communication instruction; When the data communication instruction is a data reduction instruction requesting a network reduction operation, the step of sending the data communication instruction to the network switching device further includes: The system receives a global aggregation result returned by the network switching device. The global aggregation result is obtained by the network switching device performing network-level aggregation on local aggregation data read from the storage space pointed to by the multicast memory address on the target computing node. The local aggregation data is obtained by performing intra-node aggregation on the processing result of the target expert module on the input data. The multicast memory address is obtained by parsing based on the data reduction instruction.

8. A data communication device, characterized in that, Applied to network switching equipment, including: A receiving module is used to receive data communication instructions sent by a source computing node. The data communication instructions are generated based on a target node identifier, which is determined based on the input data to be processed. A target expert module for processing the input data is deployed on the target computing node corresponding to the target node identifier. The parsing module is used to parse the data communication instructions to obtain the target node identifier; The interaction module is used to interact with the corresponding target computing node based on the target node identifier; When the data communication instruction is a data reduction instruction requesting a network reduction operation, the step of interacting with the corresponding target computing node based on the target node identifier further includes: The target node identifier and multicast memory address are obtained by parsing the data reduction instruction. Local aggregated data is read from the storage space pointed to by the multicast memory address on the target computing node corresponding to the target node identifier; the local aggregated data is obtained by aggregating the processing results of the target expert module on the input data within the node. The local aggregated data is then subjected to network-level aggregation to obtain the global aggregation result; The global aggregation result is returned to the source computing node.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data communication method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the data communication method as described in any one of claims 1 to 6.

Images