A data transmission method and related device

Abstract

The application discloses a data transmission method applied to a computing device cluster comprising a plurality of data centers, wherein a first network device in a first data center and a second network device in a second data center are connected, and the method comprises the following steps: the first network device acquires to-be-transmitted data of a current iteration round in a distributed computing process, and a codebook determined according to a probability distribution of global data of historical iteration rounds in the distributed computing process, then compresses the to-be-transmitted data according to the codebook, obtains compressed data, and sends the compressed data to the second network device. The method estimates the probability distribution of the global data of the historical iteration rounds, generates a codebook by using an algorithm based on the probability distribution, compresses the to-be-transmitted data of the current iteration round by using the codebook, transmits the compressed data across DC, and reduces the bandwidth requirement of the cross-DC communication. In addition, the data compression is performed by using a lossless compression algorithm, lossless recovery can be realized, and precision loss is avoided.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data transmission method, data transmission device, network equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rise of new computing paradigms such as artificial intelligence (AI) computing and high-performance computing (HPC), the scale of computing is constantly increasing. Traditional single-machine, single-card computing architectures can no longer meet the requirements of large-scale computing. For example, in AI computing scenarios, single-machine, single-card architectures cannot meet the requirements of storing large-scale datasets and the requirements of complex model inference / training. As a result, computing architectures are gradually evolving towards distributed computing.

[0003] Distributed computing distributes computational tasks across multiple computing cards. In some cases, it is necessary to use ultra-large clusters with more than 100,000 cards to achieve distributed computing. Deploying ultra-large clusters with more than 100,000 cards in a single region poses a huge challenge to the local power supply system. Therefore, the industry usually uses multiple data centers (DCs) in combination to achieve large-scale computing.

[0004] In distributed computing architectures, multiple data centers (DCs) typically need to transmit data, such as gradient data when training AI models in parallel across multiple DCs. To ensure performance, dedicated network connections can be used between multiple DCs. For models with trillions of parameters or more, the bandwidth requirement for cross-DC communication is 50 terabit-per-second (Tbps). Since different DCs are often thousands of kilometers apart, maintaining dedicated network communication speeds of over 50 Tbps over long distances is costly in terms of construction and maintenance. Therefore, providing a solution to reduce the bandwidth requirements for cross-DC communication has become a key focus in the industry. Summary of the Invention

[0005] This application provides a data transmission method that estimates the probability distribution of global data in historical iterations during distributed computing, compresses the data to be transmitted in the current iteration using a codebook determined based on this probability distribution, and then transmits the compressed data across data centers (DCs). This significantly reduces the bandwidth requirements for cross-DC communication. Furthermore, this method supports lossless compression and lossless recovery without affecting computational accuracy. This application also provides a data transmission device, network equipment, computer-readable storage medium, and computer program product corresponding to the above method.

[0006] Firstly, this application provides a data transmission method. This method is applied to a cluster of computing devices comprising multiple data centers. The multiple data centers include a first data center and a second data center. A first network device in the first data center and a second network device in the second data center are connected. The first network device may be a top-level switch or a top-level router in the first data center, and the second network device may be a top-level switch or a top-level router in the second data center.

[0007] In specific implementation, the first network device acquires the data to be transmitted in the current iteration of the distributed computing process, as well as a codebook determined based on the probability distribution of global data from previous iterations. The global data from previous iterations is obtained from the data transmitted by the first network device in those previous iterations. Then, the first network device compresses the data to be transmitted according to the codebook, obtaining compressed data. Finally, the first network device sends the compressed data to the second network device.

[0008] This method estimates the probability distribution of global data from historical iterations, generates a codebook using an algorithm based on this probability distribution, compresses the data to be transmitted in the current iteration using this codebook, and then transmits the compressed data across data centers, reducing the bandwidth requirements for cross-data center communication. Furthermore, this method uses the codebook to compress the data to be transmitted using a lossless compression algorithm, achieving lossless recovery and avoiding accuracy loss.

[0009] In some possible implementations, when the data to be transmitted is mapped to the uplink port of the first network device, it indicates that the data to be transmitted is cross-DC traffic. The first network device can compress the data to be transmitted according to the codebook to obtain the compressed data.

[0010] This method takes into account that the uplink port traffic of top-level network devices such as top-level switches or top-level routers is cross-DC traffic. Therefore, it can identify whether the data to be transmitted is cross-DC traffic by detecting whether the data to be transmitted is mapped to the uplink port. This simplifies the identification process of cross-DC compressed data and allows data compression to be achieved on top-level network devices such as top-level switches or top-level routers, greatly reducing the difficulty and efficiency of data compression.

[0011] In some possible implementations, the global data from previous iterations comprises multiple subsets. Accordingly, the first network device can receive frequency distribution information of these subsets from multiple first computing nodes in the first data center, as well as the number of parameters processed by each of the multiple first computing nodes. The first network device performs a weighted calculation on the frequency distribution information of the multiple subsets, using the number of parameters processed by each of the multiple first computing nodes as weights, to obtain the probability distribution of the global data. Then, the first network device can determine the codebook for compressing the data to be transmitted based on the probability distribution of the global data.

[0012] In this method, the first network device estimates the global probability distribution by acquiring the frequency distribution information and parameter quantities sent by different computing nodes, thereby determining the codebook. Compression based on this codebook has good performance.

[0013] In some possible implementations, the global data for historical iterations includes multiple subsets. The first network device can obtain a codebook from the network controller of the first data center. The codebook is determined by the network controller based on the frequency distribution information of multiple subsets sent by multiple first computing nodes in the first data center and the amount of parameters processed by each of the multiple first computing nodes, which determines the probability distribution of the global data, and then determines the network device based on the probability distribution.

[0014] In this method, the network controller estimates the global probability distribution based on the frequency distribution information and parameter quantities sent by different computing nodes, and determines the codebook accordingly. The first network device obtains the codebook from the network controller, which can significantly reduce the computing pressure on the first network device.

[0015] In some possible implementations, the first network device can compress the data to be transmitted using a lossless compression algorithm based on a codebook to obtain compressed data. Lossless compression algorithms include Huffman coding, arithmetic coding, run-length encoding, or dictionary coding.

[0016] This method uses lossless compression algorithms based on probability distribution, such as Huffman coding, arithmetic coding, run-length coding, or dictionary coding, to compress the data to be transmitted. This allows the receiving end to perform lossless recovery of the compressed data, avoiding any impact on computational accuracy.

[0017] In some possible implementations, the first network device can also determine the uniformity of the data to be transmitted based on the probability distribution of the data. When the distribution of the data to be transmitted is uneven, the first network device can compress the data to be transmitted based on the codebook to obtain compressed data.

[0018] This method identifies the uniformity of the data to be transmitted. When the distribution is uneven, it uses a codebook for compression, which can achieve a high compression ratio.

[0019] In some possible implementations, the distributed computing process includes the distributed training process of an artificial intelligence (AI) model. The first data center comprises one or more parallel groups, each containing multiple first computing nodes within the first data center. The data to be transmitted includes gradient data generated by the parallel groups in the first data center during AI model training.

[0020] In this method, network devices connected in parallel groups can compress gradient data based on the codebook, thereby enabling the transmission of compressed gradient data across data centers and reducing the bandwidth requirements for cross-data center communication.

[0021] In some possible implementations, the first and second network devices are connected via an optical transport network (OTN) or a wireless communication network. This method reduces the resource consumption and bandwidth requirements of the OTN or wireless communication network by compressing cross-DC traffic.

[0022] Secondly, this application provides a data transmission apparatus. Applied to a computing device cluster comprising multiple data centers, the multiple data centers including a first data center and a second data center, a first network device in the first data center and a second network device in the second data center being connected, the apparatus being deployed on the first network device, the apparatus comprising:

[0023] The acquisition module is used to acquire the data to be transmitted in the current iteration round during the distributed computing process, and the codebook determined according to the probability distribution of the global data of the historical iteration rounds in the distributed computing process. The global data of the historical iteration rounds is obtained based on the data transmitted by the first network device in the historical iteration rounds.

[0024] A compression module is used to compress the data to be transmitted according to the codebook to obtain compressed data;

[0025] The sending module is used to send the compressed data to the second network device.

[0026] In some possible implementations, the compression module is specifically used for:

[0027] When the data to be transmitted is mapped to the uplink port of the first network device, the data to be transmitted is compressed according to the codebook to obtain compressed data.

[0028] In some possible implementations, the global data for the historical iteration rounds includes multiple subsets, and the acquisition module is specifically used for:

[0029] Receive frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center, as well as the amount of parameters processed by each of the multiple first computing nodes;

[0030] Using the number of parameters processed by each of the plurality of first computing nodes as the weight, the frequency distribution information of the plurality of subsets is weighted and calculated to obtain the probability distribution of the global data.

[0031] Based on the probability distribution of the global data, a codebook for compressing the data to be transmitted is determined.

[0032] In some possible implementations, the global data for the historical iteration rounds includes multiple subsets, and the acquisition module is specifically used for:

[0033] The codebook is obtained from the network controller of the first data center. The codebook is determined by the network controller based on the frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center and the amount of parameters processed by each of the multiple first computing nodes. The codebook is determined based on the probability distribution.

[0034] In some possible implementations, the compression module is specifically used for:

[0035] Based on the codebook, the data to be transmitted is compressed using a lossless compression algorithm to obtain compressed data. The lossless compression algorithm includes Huffman coding, arithmetic coding, run-length encoding, or dictionary coding.

[0036] In some possible implementations, the device further includes:

[0037] The determining module is used to determine the uniformity of the data to be transmitted based on the probability distribution of the data to be transmitted;

[0038] The compression module is specifically used for:

[0039] When the distribution of the data to be transmitted is uneven, the first network device compresses the data to be transmitted according to the codebook to obtain compressed data.

[0040] In some possible implementations, the distributed computing process includes a distributed training process for an artificial intelligence (AI) model, the first data center includes one or more parallel groups, the parallel groups include multiple first computing nodes in the first data center, and the data to be transmitted includes gradient data generated by the parallel groups in the first data center training the AI ​​model.

[0041] In some possible implementations, the first network device and the second network device are connected via an optical transport network (OTN) or a wireless communication network.

[0042] Thirdly, this application provides a network device. The network device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions to cause the network device to perform the data transmission method described in the first aspect or any implementation thereof.

[0043] Fourthly, this application provides a computer-readable storage medium storing instructions that instruct a network device to perform the data transmission method described in the first aspect or any implementation thereof.

[0044] Fifthly, this application provides a computer program product containing instructions that, when run on a network device, causes the network device to perform the data transmission method described in the first aspect or any implementation thereof.

[0045] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0046] To more clearly illustrate the technical methods of this application, the accompanying drawings used will be briefly described below.

[0047] Figure 1 A schematic diagram illustrating gradient data transmission during the training of an AI model, as provided in this application;

[0048] Figure 2 A schematic diagram of the architecture of a computing device cluster provided in this application;

[0049] Figure 3 A flowchart of a data transmission method provided in this application;

[0050] Figure 4 A schematic diagram illustrating the gradient data transmission process during the training of an AI model, as provided in this application;

[0051] Figure 5 A schematic diagram illustrating the gradient data distribution of an AI model under different quantizations, as provided in this application;

[0052] Figure 6 A schematic diagram of a data transmission device provided in this application;

[0053] Figure 7 A schematic diagram of the structure of a network device provided in this application;

[0054] Figure 8 A schematic diagram of another network device provided in this application. Detailed Implementation

[0055] The terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0056] First, some technical terms involved in the embodiments of this application will be introduced.

[0057] A data center (DC) is a complex facility used for the centralized processing, storage, transmission, exchange, and management of data. A data center includes computing devices (or computing components), network devices (or network components), and management devices (or management components). Computing devices may include servers, network devices may include routers and switches, and management devices may include network controllers and firewalls. Some network devices also support on-network computing, enabling computational functions beyond transmission. Network controllers are the control and management modules for network devices, typically used to issue configuration commands for devices such as switches. Firewalls monitor and control network traffic, preventing malicious attacks, unauthorized access, and data leaks, thus protecting the internal security of the data center.

[0058] As computing scale continues to grow, traditional single-machine, single-card computing architectures are struggling to meet the demands of large-scale computing, making distributed computing (DCC) in data centers a growing trend. Distributed computing, also known as distributed processing, involves distributing computing tasks across multiple computing cards.

[0059] A computing card refers to a processor or processing unit used to perform computing tasks, including but not limited to a graphics processing unit (GPU), a neural network processing unit (NPU), and a tensor processing unit (TPU). Multiple computing cards can be multiple computing cards on a single server (e.g., multiple cards on a single machine) or computing cards on multiple servers (e.g., multiple cards across multiple machines).

[0060] For example, in artificial intelligence (AI) computing or high-performance computing (HPC) scenarios, AI tasks such as training AI models or HPC tasks such as weather prediction can typically be distributed across a cluster of computing devices (or simply a cluster) to be executed. Here, an AI model is a mathematical model with reasoning capabilities built based on AI algorithms such as machine learning (ML) and deep learning. The computing device cluster that trains the AI ​​model is also called an intelligent computing cluster, and the computing device cluster that executes the HPC task is also called an HPC cluster. For ease of description, this application uses the scenario of training an AI model as an example.

[0061] The training of AI models follows the scaling law, meaning that the accuracy of an AI model increases with the number of parameters and the amount of training data. Therefore, future AI models are expected to have over ten trillion parameters, requiring clusters with over one hundred thousand GPUs to complete training in 100 days. For models with fewer than ten trillion parameters, the required computing power is relatively small, and efficient training can typically be conducted within a single data center; all communication traffic involved in AI model training can be closed-loop within a single data center.

[0062] Deploying ultra-large clusters with over 100,000 cards in a single region poses a significant challenge to the power supply system. Therefore, for AI models with over 10 trillion parameters, multiple data centers are typically used in conjunction to achieve efficient training. Conversely, for models with fewer than 10 trillion parameters, if the customer has already deployed medium-sized data centers (e.g., training clusters with thousands or tens of thousands of cards) in different regions, multiple medium-sized data centers in different regions can be combined to form an ultra-large cluster for training large-scale AI models. In both scenarios, a cross-data center approach can be used for AI model training.

[0063] Cross-DC traffic for training AI models is typically data parallelism (DP) traffic. DP is a parallel approach that partitions tasks based on data dimensions (e.g., training data). DP traffic is characterized by high throughput. To ensure training performance loss remains within a reasonable range, AI models with trillions of parameters require cross-DC communication bandwidth exceeding 50 terabit-per-second (Tbps). Different DCs are often thousands of kilometers apart, and maintaining communication speeds above 50 Tbps over long distances requires dedicated optical transport networks (OTNs), which incur high construction and maintenance costs for OTN equipment and fiber optic cable deployment.

[0064] Currently, training AI models across data centers (DCs) is still in its initial exploratory stage. In research on training AI models across DCs, multi-DC communication typically follows the communication methods used within a single DC. For example... Figure 1 As shown, mixed precision training is typically used when training AI models such as large language models (LLMs) to improve training speed and reduce memory usage. For example, the weights of the AI ​​model are converted to half-precision floating-point (FP16) during training to save memory and accelerate computation, but when saving the AI ​​model weights, they are converted to single-precision floating-point (FP32) to obtain higher numerical precision and wider hardware support. For data parallel groups of different data centers, such as data parallel group 1 (DP1) of data center 1 (DC 1) and data parallel group 2 (DP2) of DC 2, gradient data in FP16 format can be directly transmitted.

[0065] Transmitting gradient data in FP16 format can reduce the bandwidth requirements for cross-DC communication compared to transmitting FP32 format gradient data, but it still requires relatively high bandwidth. For example, for AI models with more than a trillion parameters, the bandwidth requirement for cross-DC DP communication is over 50Tbps.

[0066] In view of this, this application provides a data transmission method. This method is applied to a computing device cluster comprising multiple data centers. The multiple data centers include a first data center and a second data center. A first network device in the first data center and a second network device in the second data center are connected. The first network device can be a top-level switch or a top-level router in the first data center, and the second network device can be a top-level switch or a top-level router in the second data center.

[0067] Specifically, the first network device acquires the data to be transmitted in the current iteration of the distributed computing process, as well as a codebook determined based on the probability distribution of global data from previous iterations. The global data from previous iterations is obtained from the data transmitted by the first network device in those iterations. For example, in a scenario of training an AI model, top-level switches in different data centers can transmit local gradient data generated by parallel training groups through aggregated communication, thereby obtaining global gradient data. The probability distribution of the global gradient data determines the codebook used to compress the data to be transmitted. The codebook includes typical patterns or codewords representing the characteristics of the original data. The first network device then compresses the data to be transmitted according to the codebook, obtaining compressed data. Finally, the first network device sends the compressed data to the second network device.

[0068] This method estimates the probability distribution of global data from historical iterations, generates a codebook using an algorithm based on this probability distribution, compresses the data to be transmitted in the current iteration using this codebook, and then transmits the compressed data across data centers, reducing the bandwidth requirements for cross-data center communication. Furthermore, this method uses the codebook to compress the data to be transmitted using a lossless compression algorithm, achieving lossless recovery and avoiding accuracy loss.

[0069] To make the technical solution of this application clearer and easier to understand, the system architecture of this application will be described below with reference to the accompanying drawings.

[0070] See Figure 2The diagram illustrates an architecture of a computing device cluster 10, which includes multiple data centers 100. Each data center 100 can be an intelligent computing cluster, and the multiple data centers 100 can jointly train AI models. Each data center 100 includes multiple computing nodes 102. A computing node 102 can be a computing device, such as a server containing one or more computing cards. Multiple computing nodes 102 can be connected via network devices. In this example, the network devices may include leaf switches 104, spine switches 106, and routers 108. Leaf switches 104, also known as leaf switches, are responsible for connecting computing nodes (such as servers) and aggregating and forwarding data generated by the computing nodes. Spine switches 106, also known as spine switches, serve as the network backbone, interconnecting all leaf switches 104 in a full mesh topology, and are responsible for the rapid transmission of data between leaf switches 104, enabling efficient communication between computing nodes 102 within the data center. Router 108 may include a top-level router located at the network core or boundary, serving as a key device connecting different network areas or network types. The primary function of router 108 is to implement routing and forwarding between networks. Data centers 100 can be connected via a dedicated optical transmission network 200. For example, router 108 in one data center 100 and router 108 in another data center 100 can be connected via optical transmission network 200. Based on this, cross-DC traffic can be... Figure 1 After being aggregated by router 108, the data is transmitted to other data centers 100 via optical transmission network 200.

[0071] In AI model training scenarios, intra-DC traffic can include Tensor Parallelism (TP), Context Parallelism (CP), Expert Parallelism (EP), and Pipeline Parallelism (PP). Cross-DC traffic includes Data Parallelism (DP). Tensor Parallelism refers to partitioning tensors in the AI ​​model to achieve parallel training. Context Parallelism involves partitioning the complete context (e.g., a complete input text sequence) to achieve parallel training. Expert Parallelism involves inputting different training data into different expert modules (sub-networks) for parallel training. Pipeline Parallelism divides the AI ​​model's computation process into multiple stages, each handled by a different computing node 102. The input training data flows sequentially through the corresponding computing nodes 102 in each stage for processing, achieving parallel computation and improving efficiency. Data Parallelism involves partitioning the training data in the dataset, for example, into multiple data batches for parallel training. Accordingly, DP traffic can be gradient data generated from processing different batches of data. Figure 1 As illustrated in the example, DC 1 can transmit the gradient data generated from processing Batch 1 via the optical transport network 200, denoted as gradient of Batch 1. It can also receive gradient data transmitted from DC 2 to DC N via the optical transport network 200, such as gradient of Batch 2...gradient of Batch N. The data transmission process for other DCs can be referenced from DC 1.

[0072] In this application, router 108 of DC 1 acquires the data to be transmitted in the current iteration of the distributed computing process, such as the gradient data to be transmitted in the current iteration, and a codebook determined based on the probability distribution of global data (e.g., global gradient data, which can be the average of gradient data calculated in different data centers) from previous iterations in the distributed computing process. The global data from previous iterations is obtained from the data transmitted by router 108 in previous iterations. A codebook is a collection of codes or vectors with specific meanings and structures, typically used for tasks such as data representation, compression, retrieval, and generation. In this application, the codebook is primarily used for compression tasks. Specifically, the codebook can be viewed as a dictionary, where each entry (code or vector) can represent a certain feature or pattern in the data. Data compression can be achieved by processing the data to be transmitted using the entries in the codebook. Specifically, router 108 compresses the data to be transmitted according to the codebook to obtain compressed data (e.g., compressed gradient data), and then sends the compressed data to routers 108 in other data centers 100 (e.g., DC 2 to DC N).

[0073] It should be noted that, Figure 2 This is merely one exemplary architecture of the computing device cluster 10. In other possible implementations of this application, the computing device cluster 10 can also have other architectures. For example, when the parameter plane network of the data center 100 is a two-layer fat tree network, the spine switch (top-level switch) in the fat tree network can compress the data to be transmitted in the current iteration (such as gradient data) according to the codebook, and then transmit the compressed gradient data to the spine switch of other data centers. Here, the parameter plane network refers to the dedicated network through which the parameters (such as weights, biases), gradient data, and optimizer state of different computing nodes interact when the intelligent computing cluster trains the AI ​​model.

[0074] based on Figure 2 The computing device cluster 10 shown in this application also provides a data transmission method. The data transmission method of this application will be described in detail below with reference to embodiments.

[0075] See Figure 3The flowchart illustrates a data transmission method applied to a computing device cluster 10 comprising multiple data centers 100, including a first data center and a second data center, wherein the first data center and the second data center are different data centers among the multiple data centers 100. For example, the first data center can be DC 1, and the second data center can be DC 2 to DC N. A first network device in the first data center (e.g., router 108 in DC 1) and a second network device in the second data center (e.g., router 108 in DC 2 to DC N) are connected. The method includes the following steps:

[0076] S302, The first network device acquires the data to be transmitted in the current iteration round during the distributed computing process.

[0077] In distributed computing scenarios, the computing device cluster 10 can complete computing tasks through multiple iterations. For example, in AI computing scenarios, the computing device cluster 10 can train AI models through multiple iterations. Similarly, in HPC scenarios, the computing device cluster 10 can complete HPC tasks through multiple iterations. HPC tasks include, but are not limited to, simulating physical phenomena (such as simulating fluid flow in complex pipes) or optimizing engineering design schemes.

[0078] For each iteration round, multiple data centers 100 in the computing device cluster 10 can communicate across data centers to proceed to the next iteration round. An example of communication between a first data center and a second data center is provided. The first data center includes a first network device, which can be a top-level network device in the parametric plane network of the first data center, such as a top-level switch or a top-level router. The first network device can connect to a second network device in the second data center. Similarly, the second network device can be a top-level network device in the parametric plane network of the second data center, such as a top-level switch. The first and second network devices can be connected via an optical transport network (OTN). The optical transport network can include a dedicated optical transport network. In some examples, the first and second network devices can also be connected via a wireless communication network.

[0079] The first network device acquires the data to be transmitted in the current iteration of the distributed computing process. The data to be transmitted can be data to be transmitted to other data centers 100 (such as the second data center). The data to be transmitted can be transmitted to other data centers using a collective communication method. For ease of understanding, the following example illustrates the distributed training of an AI model in an AI computing scenario. The distributed computing process includes the distributed training process of the AI ​​model. The first data center includes one or more parallel groups, and each parallel group includes multiple first computing nodes within the first data center. The parallel group can be a data parallel group; based on this, the first computing nodes in the parallel group can store the complete parameters of the AI ​​model. When the parallel group includes multiple first computing nodes, each of the multiple first computing nodes can store a portion of the AI ​​model's parameters. The first computing nodes in the parallel group can perform forward and backward computation based on the input training data to obtain the gradient data of the parameters. Accordingly, the data to be transmitted includes the gradient data generated by the parallel groups in the first data center training the AI ​​model.

[0080] S304. The first network device obtains the codebook determined based on the probability distribution of global data from historical iterations in the distributed computing process.

[0081] The global data for historical iteration rounds is obtained from the data transmitted by the first network device in those rounds. Historical iteration rounds include all iteration rounds preceding the current one. In some possible implementations, the first network device may obtain the most recently updated codebook based on the codebook update cycle.

[0082] A codebook includes typical patterns or codewords representing the characteristics of the original data, typically determined based on the probability distribution of global data. In practice, a codebook can be represented as a collection of codes or vectors with specific meanings and structures. This collection can be a set of character sequences and their corresponding codes. A character sequence can include one or more characters, and the character sequence and its code can be stored as key-value pairs. Therefore, the codebook can be a collection of key-value pairs formed by character sequences and their codes. The codebook can be stored as key-value pairs in an array, hash table, or binary tree. Each element of the array corresponds to the encoding information of a character, and the corresponding code is accessed through a certain index of the character (such as an ASCII code value). A hash table can use characters as keys and their corresponding codes as values, allowing for quick lookup of a character's code or vice versa, efficiently implementing encoding and decoding operations. Storing the codebook in a binary tree allows for direct traversal and lookup operations on the binary tree during encoding and decoding. In some cases, this can reduce additional data structure overhead and provide a clear visual representation of the encoding generation process.

[0083] To facilitate understanding, we'll use the Huffman coding codebook as an example. The Huffman coding codebook is a data structure used to store characters and their corresponding Huffman codes; it's a key element in implementing Huffman coding and decoding. The Huffman coding codebook assigns a unique binary code to each character. When encoding data to be transmitted, the codebook allows for rapid conversion of each character into its corresponding Huffman code, thus achieving compressed data representation. For example, for the character set {a, b, c, d}, if the codebook specifies that a is encoded as 0, b as 10, c as 110, and d as 111, then the data to be transmitted, "abcd", can be encoded as "010110111". During decoding, the codebook serves as the basis for restoring the compressed data (e.g., binary Huffman code) to the original characters. The receiver, based on the received compressed data, refers to the codebook to look up the compressed data (e.g., binary Huffman code) and convert it back to the original characters, thus restoring the data.

[0084] If the codebook update cycle is configured to T (meaning the codebook is updated every T iterations), and the current iteration in the distributed computing process is i (e.g., the i-th iteration), where T and i are positive integers, then the historical iterations in the distributed computing process can be... For example, if T is 5, then when i takes values ​​from 1 to 5, the historical iteration round in the distributed computing process can be 1 (such as the 1st iteration); when i takes values ​​from 6 to 10, the historical iteration round in the distributed computing process can be 6 (such as the 6th iteration). Alternatively, if the codebook is updated in each iteration round, then the historical iteration round in the distributed computing process can be the previous iteration round.

[0085] For each historical iteration in the distributed computing process, the first network device communicates with network devices in other data centers to obtain global data on the historical iterations. This global data can be used to determine the codebook for subsequent compression of the data to be transmitted.

[0086] Using gradient data as an example, for the first... In each iteration round, the first network device can perform all reduce with network devices in other data centers (such as the second network device in the second data center) to obtain the average gradient data, which is then stored in the parallel group of the first data center. The average gradient data can be considered as global gradient data. The codebook can be determined based on the probability distribution of the global gradient data from historical iteration rounds in the distributed computing process.

[0087] In some possible implementations, the global data of historical iteration rounds includes multiple subsets. A first network device receives frequency distribution information of these subsets from multiple first computing nodes in a first data center, as well as the number of parameters processed by each of the multiple first computing nodes. These multiple first computing nodes can be the first computing nodes in a parallel group. The number of parameters processed by each first computing node can include the number of weights processed by that first computing node. In some cases, the number of parameters processed by each first computing node can be determined based on the amount of gradient data stored by that first computing node. The first network device can perform a weighted operation on the frequency distribution information of the multiple subsets using the number of parameters processed by each of the multiple first computing nodes as weights to obtain the probability distribution of the global data. Based on the probability distribution of the global data in historical iteration rounds, the first network device determines the codebook for compressing the data to be transmitted.

[0088] For example, the first network device can determine the codebook for compressing the data to be transmitted based on the probability distribution of global data using a lossless compression algorithm. The lossless compression algorithm can be a probability-based algorithm, including but not limited to Huffman coding, arithmetic coding, run-length coding (RLC), or dictionary coding.

[0089] Huffman coding is a statistical lossless coding method. It first statistically analyzes the probability of source symbols (such as gradient data) and then constructs a Huffman tree based on these probabilities. Source symbols with higher probabilities have shorter code lengths, while those with lower probabilities have longer code lengths. During Huffman tree construction, the two source symbols with the lowest probabilities are merged into a new node, with the probability of the new node being the sum of the probabilities of the two source symbols. This process is repeated until a tree is formed. The leaf nodes of the tree represent the individual source symbols, and the path from the root node to a leaf node (represented by 0s and 1s) is the Huffman code for that source symbol. This Huffman coding can be used to create a codebook.

[0090] Let's continue with the example of the character set {a,b,c,d}. Assume the probabilities (represented by frequency) of characters a, b, c, and d are 40%, 30%, 20%, and 10%, respectively. Characters with higher frequencies are closer to the root, and characters with lower frequencies are further away from the root. By continuously merging the nodes with the lowest frequencies, a Huffman tree is eventually constructed. Starting from the root node of the Huffman tree, code 0 is assigned to the left branch, and code 1 is assigned to the right branch. A unique code is assigned to each leaf node (i.e., each character) along the path of the tree. In this example, if 'a' is in the left subtree closer to the root node, it might be assigned the shorter code 0, while 'd' is in the right subtree deeper, it might be assigned the longer code 111. Organizing each character and its corresponding code into key-value pairs forms a codebook. In this example, the codebook can be represented as {a:0,b:10,c:110,d:111}.

[0091] It should be noted that Huffman coding can make the average length of the codewords (also known as the average code length) approach the Shannon limit. In source coding theory, the Shannon limit refers to the theoretical minimum value of the average length of the compressed codewords.

[0092] Arithmetic coding encodes the entire source symbol sequence as a whole, unlike Huffman coding which encodes individual source symbols. Arithmetic coding maps the source symbol sequence to a real number within the interval [0,1). This interval is then subdivided based on the probability of each symbol's occurrence, ultimately using a decimal within that interval to represent the entire source symbol sequence. For example, given a set of source symbols {A, B, C} with probabilities of 0.5, 0.3, and 0.2 respectively, to encode the symbol sequence "AB", the interval [0,1) is first divided into sub-intervals corresponding to the three symbols based on their probabilities. Then, the interval is subdivided sequentially according to the order of symbol occurrence, resulting in a sub-interval representing "AB". A decimal within this sub-interval is then used as the encoding result. Compared to Huffman coding, arithmetic coding can better approximate the Shannon limit in practical applications, thus achieving better compression performance, such as a high compression ratio.

[0093] Run-length encoding is primarily used for data containing consecutively repeated symbols or values. It works by counting the number of consecutive identical symbols (also called the run length) and then using the length value and the symbol itself to represent the repeated data. For example, a binary sequence "000011100" can be encoded as "(4,0)(3,1)(2,0)", representing four consecutive 0s, three consecutive 1s, and two consecutive 0s.

[0094] Dictionary coding involves building a dictionary based on the probability distribution of elements in the source data. During the encoding process, the parts of the original data that match dictionary entries are replaced with indices from the dictionary, thus reducing the amount of data. The dictionary used in dictionary coding can be at least one of a static dictionary or a dynamic dictionary. A static dictionary is built based on the probability distribution of elements in a pre-selected source data, typically adding elements with high probabilities of occurrence (also known as high-frequency elements). A dynamic dictionary is updated in real-time during the encoding process based on the encoded data; this update process is closely related to the probability distribution.

[0095] In some other possible implementations, the global data from previous iterations comprises multiple subsets. The network controller of the first data center can determine the probability distribution of the global data based on the frequency distribution information of the multiple subsets sent by multiple first computing nodes in the first data center and the amount of parameters processed by each of the multiple first computing nodes. The network controller can then determine the codebook for compressing the data to be transmitted based on the probability distribution of the global data. For example, the network controller can determine the codebook for compressing the data to be transmitted using a probability-based lossless compression algorithm, such as Huffman coding or arithmetic coding, based on the probability distribution of the data. Accordingly, the first network device can obtain the codebook from the network controller of the first data center.

[0096] The specific implementation of the network controller determining the probability distribution of global data and determining the codebook based on the probability distribution of global data can be found in the description of the first network device determining the probability distribution of global data and determining the codebook based on the probability distribution, and will not be repeated here.

[0097] S306. The first network device compresses the data to be transmitted according to the codebook to obtain the compressed data.

[0098] The first network device can compress the data to be transmitted using a lossless compression algorithm based on a codebook, thereby obtaining compressed data. Lossless compression algorithms include, but are not limited to, Huffman coding, arithmetic coding, run-length encoding, or dictionary coding. Specifically, the first network device can match the data to be transmitted with the codebook using a lossless compression algorithm, and then compress the data based on the matching result to obtain compressed data.

[0099] In some possible implementations, the first network device can identify whether the data to be transmitted is cross-DC traffic. When the first network device is a top-level switch or top-level router in a parametric plane network, it can determine whether the data to be transmitted is cross-DC traffic by the port mapped to it. When the data to be transmitted is mapped to the uplink port of the first network device, it indicates that the data to be transmitted is cross-DC traffic, and the first network device can compress the data to be transmitted according to the codebook to obtain compressed data.

[0100] It should be noted that compression algorithms typically perform well on non-uniform data. Based on this, the first network device can also identify the uniformity of the data to be transmitted. Specifically, the first network device can determine the uniformity of the data to be transmitted based on its probability distribution. For example, the first network device can determine the maximum difference or variance of the data to be transmitted based on its probability distribution, and then determine the uniformity of the data based on this maximum difference or variance. For instance, if the variance is greater than a threshold, it indicates that the data to be transmitted is not uniformly distributed. The first network device can then compress the data to be transmitted according to the codebook to obtain compressed data.

[0101] S308, The first network device sends compressed data to the second network device.

[0102] Specifically, the first network device can send compressed data to the second network device via an optical transport network (OTN) or a wireless communication network. Taking the example of distributed training of an AI model in an AI computing scenario, the first network device can send compressed gradient data to the second network device via an optical transport network. The first and second network devices can send their respective compressed gradient data via aggregated communication and determine the average value of the gradient data to obtain global gradient data.

[0103] Based on the above description, this application provides a data transmission method. In this method, when transmitting data, a first network device in a first data center can acquire a codebook determined based on the probability distribution of global data from estimated historical iteration rounds. This codebook is then used to compress the data to be transmitted in the current iteration round, and the compressed data is transmitted across data centers, thereby reducing the bandwidth requirements for cross-data center communication. Furthermore, this method supports the use of a lossless compression algorithm to compress the data to be transmitted, thus achieving lossless recovery and avoiding accuracy loss.

[0104] Next, the data transmission method of this application will be described in detail using the scenario of training an AI model.

[0105] See Figure 4The diagram illustrates a gradient data transmission process during AI model training. The AI ​​model is trained by a cluster of computing devices 10 comprising multiple data centers 100. Each data center includes one or more parallel groups. In this example, each parallel group includes n computing nodes, such as computing nodes 1 to n. Each computing node includes a management plane network interface. The management plane network is the network through which the computing nodes communicate with management devices such as network controllers.

[0106] The network controller can send compute node control commands to multiple compute nodes in a parallel group through the management plane network interface. These compute node control commands include control commands for calculating the frequency distribution of gradient data. It should be noted that the network controller can send these compute node control commands to multiple compute nodes in a parallel group in each iteration of the distributed training process of the AI ​​model, or periodically. Specifically, periodically sending these compute node control commands to multiple compute nodes in a parallel group could mean that the network controller sends the commands to multiple compute nodes in a parallel group every T iterations.

[0107] The parallel group can be a data parallel group, where each of the computing nodes 1 to n stores partial parameters of the AI ​​model, and the remaining nodes store the complete parameters of the AI ​​model. Nodes 1 to n can respond to computing node control commands and calculate the frequency distribution of gradient data in each iteration or every T iterations, specifically during the iterative update of the optimizer state. Each computing node can include a local frequency distribution estimation module, which calculates the frequency distribution of the gradient data of the parameters stored on that node.

[0108] All computing nodes within a parallel group can send the frequency distribution of their locally computed gradient data, along with the number of parameters processed by that node (also known as the local AI model parameters), to the network controller. The global frequency distribution estimation module in the network controller uses the number of parameters corresponding to each frequency distribution as weights to perform a weighted average of the frequency distributions of the gradient data sent by the computing nodes within the parallel group, thus obtaining the probability distribution of the global gradient data. The communication overhead of each computing node transmitting the frequency distribution (e.g., a frequency distribution table) is relatively small; assuming a single frequency value is represented by 64 decimal places with a precision of 10-1... -20 This can cover a single node with 10 20 In scenarios with a large number of parameters, the maximum number of values ​​that can be quantized by FP16 is 64K. Therefore, the frequency distribution table of a single node occupies a maximum of 512 kilobytes (KB) of storage space, which can meet business needs.

[0109] The network controller, based on the probability distribution of the global gradient data, invokes the Huffman coding algorithm to generate a codebook for compressing the gradient data. For example, the codebook generation module in the network controller uses the Huffman coding algorithm to generate a codebook for compressed gradient data. The network controller can then send the codebook to all top-level switches in the parametric plane network. For example, in the case of a two-layer fat-tree network, the network controller can send the codebook to all spine switches. Figure 4 As shown, the network controller can send codebooks to all top-level switches of the parameter plane network within a data center through the management plane network port. The network controller transmits relatively little codebook data. Assuming the original gradient data is in FP16 format, the longest codeword generated by Huffman coding is 16 bits, therefore the maximum codebook size is 256KB.

[0110] Each top-level switch acquires the gradient data to be transmitted for the current iteration during the distributed training of the AI ​​model. It determines whether the gradient data is cross-DC traffic or intra-DC traffic. If the gradient data is mapped to an uplink port, it indicates cross-DC traffic; otherwise, it's intra-DC traffic. For cross-DC traffic, the compression module in the top-level switch can compress the gradient data for the current iteration using Huffman coding, based on the codebook, to obtain compressed gradient data. The top-level switch can then send the compressed gradient data to other data centers (e.g., top-level switches within other data centers).

[0111] This method provides a lossless gradient compression technique based on the collaboration of computing, network, and management components (computing, network, and management components). Through collaboration between computing nodes within a data center (DC), top-level switches in the parameter plane network, and the network controller, the probability distribution of global gradient data is estimated. A codebook is then generated using a lossless compression encoding method based on the probability distribution, and cross-DC gradient data compression is performed at the top-level switch. This significantly reduces cross-DC DP traffic when training AI models across data centers, thereby reducing the bandwidth requirements for cross-DC private network communication. In a typical AI model training scenario, the gradient data distribution of the AI ​​model under different quantizations is as follows: Figure 5As shown, the horizontal axis represents the logarithm of the gradient values ​​(log2 magnitude), and the vertical axis represents the percentage of gradient values. The figure illustrates the gradient data distribution under FP32 and FP8 quantization. For FP8 quantization, the E5M2 notation is used, where E represents the exponent and M represents the mantissa. Compared to FP32 quantization, the percentage of the logarithm of some gradient values ​​becomes 0 in FP8 quantization. Furthermore, some minimum values ​​in the gradient data after FP8 quantization can be represented as denormalized floating-point numbers (denorms). Figure 5 Under the gradient data distribution shown, the lossless compression coding algorithm based on the data distribution can achieve an average code length of 4.2 bits; compared with FP16 transmission, the communication bandwidth requirement is reduced by approximately 74%. Furthermore, the lossless compression algorithm does not compromise the accuracy of the gradient data, thus the training accuracy of the AI ​​model remains unaffected. In addition, this application considers that the uplink port traffic of the top-level switch is cross-DC traffic. Therefore, by detecting whether the gradient data to be transmitted is mapped to the uplink port, it can identify whether the gradient data to be transmitted is cross-DC traffic, simplifying the discrimination process for cross-DC compressed gradient data. Gradient data compression can be implemented on the top-level switch, greatly reducing the difficulty and efficiency of gradient data compression.

[0112] Based on the aforementioned data transmission method, this application also provides a data transmission apparatus. The data transmission apparatus may include software, which can be deployed on hardware, such as network devices like switches. In some possible implementations, the data transmission apparatus may also include hardware, such as network devices. The structure of the data transmission apparatus will be described below from a functional modular perspective, using the data transmission apparatus as a software device.

[0113] See Figure 6 The diagram shows a data transmission device applied to a computing device cluster comprising multiple data centers, including a first data center and a second data center. A first network device in the first data center and a second network device in the second data center are connected. The data transmission device 600 is deployed on the first network device and includes:

[0114] The acquisition module 602 is used to acquire the data to be transmitted in the current iteration round during the distributed computing process, and the codebook determined according to the probability distribution of the global data of the historical iteration rounds in the distributed computing process. The global data of the historical iteration rounds is obtained based on the data transmitted by the first network device in the historical iteration rounds.

[0115] Compression module 604 is used to compress the data to be transmitted according to the codebook to obtain compressed data;

[0116] The sending module 606 is used to send the compressed data to the second network device.

[0117] When implemented in software, the acquisition module 602, compression module 604, and sending module 606 can be applications running on a network device (such as the first network device). These applications can also be virtualized and provided to users as virtualization services. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. Specifically, a VM service can be a service that uses virtualization technology to create a pool of virtual machine (VM) resources on multiple physical hosts, providing VMs for users to use on demand. A BMS service is a service that uses virtualization technology to create a pool of BMS resources on multiple physical hosts, providing BMS for users to use on demand. A container service is a service that uses virtualization technology to create a pool of container resources on multiple physical hosts, providing containers for users to use on demand. A VM can be a simulated virtual network device. A BMS is a scalable, high-performance computing service with computing performance indistinguishable from traditional physical machines, featuring secure physical isolation. A container is a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely specific examples. In practical applications, virtualization services can also include other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0118] In some possible implementations, the compression module 604 is specifically used for:

[0119] When the data to be transmitted is mapped to the uplink port of the first network device, the data to be transmitted is compressed according to the codebook to obtain compressed data.

[0120] In some possible implementations, the global data of the historical iteration rounds includes multiple subsets, and the acquisition module 602 is specifically used for:

[0121] Receive frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center, as well as the amount of parameters processed by each of the multiple first computing nodes;

[0122] Using the number of parameters processed by each of the plurality of first computing nodes as the weight, the frequency distribution information of the plurality of subsets is weighted and calculated to obtain the probability distribution of the global data.

[0123] Based on the probability distribution of the global data, a codebook for compressing the data to be transmitted is determined.

[0124] In some possible implementations, the global data of the historical iteration rounds includes multiple subsets, and the acquisition module 602 is specifically used for:

[0125] The codebook is obtained from the network controller of the first data center. The codebook is determined by the network controller based on the frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center and the amount of parameters processed by each of the multiple first computing nodes. The codebook is determined based on the probability distribution.

[0126] In some possible implementations, the compression module 604 is specifically used for:

[0127] Based on the codebook, the data to be transmitted is compressed using a lossless compression algorithm to obtain compressed data. The lossless compression algorithm includes Huffman coding, arithmetic coding, run-length encoding, or dictionary coding.

[0128] In some possible implementations, the data transmission device 600 further includes:

[0129] The determining module 608 is used to determine the uniformity of the data to be transmitted based on the probability distribution of the data to be transmitted.

[0130] The compression module 604 is specifically used for:

[0131] When the distribution of the data to be transmitted is uneven, the first network device compresses the data to be transmitted according to the codebook to obtain compressed data.

[0132] The determination module 608 can be an application running on a network device. This application can also be virtualized and provided to users as virtualization services such as BMS services, VM services, or container services.

[0133] In some possible implementations, the distributed computing process includes a distributed training process for an artificial intelligence (AI) model, the first data center includes one or more parallel groups, the parallel groups include multiple first computing nodes in the first data center, and the data to be transmitted includes gradient data generated by the parallel groups in the first data center training the AI ​​model.

[0134] In some possible implementations, the first network device and the second network device are connected via an optical transport network (OTN) or a wireless communication network.

[0135] Corresponding to the method embodiments and virtual device embodiments provided in this application, this application also provides a network device, and the hardware structure of the network device is described below.

[0136] Network device 700 or network device 800 corresponds to the first network device or the second network device in the above method embodiments. The hardware, modules, and other operations and / or functions in network device 700 or network device 800 respectively implement the various steps and methods performed by the first network device or the second network device in the method embodiments. For details on how network device 700 or network device 800 performs data transmission, please refer to the above method embodiments; for brevity, they will not be repeated here. The steps of the above method are completed by the integrated logic circuits of the hardware in the processor of network device 700 or network device 800 or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0137] See Figure 7 , Figure 7 This is a schematic diagram of the structure of a network device 700 provided in an embodiment of this application. The network device 700 can be configured as a first network device or a second network device. The network device 700 includes at least one processor 701, a communication bus 702, a memory 703, and at least one physical interface 704.

[0138] The processor 701 can be a general-purpose central processing unit (CPU), network processor (NP), microprocessor, or one or more integrated circuits for implementing the scheme of this application, such as application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0139] The communication bus 702 is used to transmit information between the aforementioned components. The communication bus 702 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0140] The memory 703 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions; it may also be a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions; it may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices; or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 703 may exist independently and be connected to the processor 701 via a communication bus 702. The memory 703 may also be integrated with the processor 701.

[0141] Physical interface 704 uses any transceiver-like device for communicating with other devices or communication networks. Physical interface 704 includes wired communication interfaces and may also include wireless communication interfaces. The wired communication interface can be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface can be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof. Physical interface 704 is also called a physical port, such as a Flexible Ethernet (FlexE) physical interface.

[0142] In a specific implementation, as one example, network device 700 may include multiple processors, such as... Figure 7 The processors 701 and 705 are shown. Each of these processors can be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0143] In some embodiments, memory 703 is used to store program code 710 for executing the scheme of this application, and processor 701 can execute the program code 710 stored in memory 703. That is, network device 700 can implement the data transmission method provided in the method embodiment through processor 701 and program code 710 in memory 703.

[0144] See Figure 8 , Figure 8 This is a schematic diagram of the structure of a network device 800 provided in an embodiment of this application. The network device 800 can be configured as a first network device or a second network device. The network device 800 includes: a main control board 810 and an interface board 830.

[0145] The main control board 810, also known as the main processing unit (MPU) or route processor card, is used to control and manage the various components in the network device 800, including route calculation, device management, device maintenance, and protocol processing functions. The main control board 810 includes a central processing unit 811 and a memory 812.

[0146] Interface board 830, also known as a line processing unit (LPU), linecard, or service board, provides various service interfaces and enables packet forwarding. Service interfaces include, but are not limited to, Ethernet interfaces, and interfaces for transmitting IP data services using synchronous fiber optic networks or synchronous digital hierarchies (Packet over SONET / SDH, POS). Ethernet interfaces may include, for example, Flexible Ethernet Clients (FlexE Clients). Interface board 830 includes: a central processing unit 831, a network processor 832, a forwarding table entry memory 834, and a physical interface card (PIC) 833.

[0147] The central processing unit 831 on the interface board 830 is used to control and manage the interface board 830 and communicate with the central processing unit 811 on the main control board 810.

[0148] The network processor 832 is used to implement packet forwarding processing. The network processor 832 can be in the form of a forwarding chip. Specifically, uplink packet processing includes: processing of the packet ingress interface, forwarding table lookup; downlink packet processing includes: forwarding table lookup, etc.

[0149] The physical interface card 833 is used to implement physical layer interfacing functions. Raw traffic enters the interface board 830 through this card, and processed packets are sent out from the physical interface card 833. The physical interface card 833 includes at least one physical interface, also called a physical port. The physical interface card 833 can be a FlexE physical interface. The physical interface card 833, also called a daughter card, can be installed on the interface board 830. It is responsible for converting photoelectric signals into packets, performing validity checks on the packets, and forwarding them to the network processor 832 for processing. In some embodiments, the central processing unit 831 of the interface board 803 can also perform the functions of the network processor 832, such as implementing software forwarding based on a general-purpose CPU. In this case, the network processor 832 is not required in the physical interface card 833.

[0150] Optionally, the network device 800 includes multiple interface boards. For example, the network device 800 also includes an interface board 840, which includes a central processing unit 841, a network processor 842, a forwarding table entry memory 844, and a physical interface card 843.

[0151] Optionally, the network device 800 also includes a switching fabric board 820. The switching fabric board 820 can also be called a switch fabric unit (SFU). When the network device has multiple interface boards 830, the switching fabric board 820 is used to complete data exchange between the interface boards. For example, interface boards 830 and 840 can communicate through the switching fabric board 820.

[0152] The main control board 810 and the interface board 830 are coupled. For example, the main control board 810, interface boards 830 and 840, and the switching network board 820 are interconnected via a system bus and the system backplane. In one possible implementation, an inter-process communication (IPC) channel is established between the main control board 810 and the interface board 830, and the main control board 810 and the interface board 830 communicate with each other through the IPC channel.

[0153] Logically, network device 800 includes a control plane and a forwarding plane. The control plane includes a main control board 810 and a central processing unit 831, while the forwarding plane includes various components that perform forwarding, such as a forwarding table entry memory 834, a physical interface card 833, and a network processor 832. The control plane performs functions such as router operation, generating forwarding tables, processing signaling and protocol messages, and configuring and maintaining the device's status. The control plane distributes the generated forwarding table to the forwarding plane. In the forwarding plane, the network processor 832 looks up the packets received by the physical interface card 833 based on the forwarding table distributed by the control plane and forwards them. The forwarding table distributed by the control plane can be stored in the forwarding table entry memory 834. In some embodiments, the control plane and the forwarding plane can be completely separated and not on the same device.

[0154] If network device 800 is configured as the first network device, the central processing unit 811 obtains the data to be transmitted in the current iteration round of the distributed computing process, as well as the codebook determined according to the probability distribution of global data in the historical iteration rounds of the distributed computing process. Then, it compresses the data to be transmitted according to the codebook to obtain the compressed data, and then sends the compressed data to the second network device.

[0155] If network device 800 is configured as a second network device, the physical interface card 833 in interface board 830 or the physical interface card 843 in interface board 840 receives the photoelectric signal corresponding to the compressed data, converts the photoelectric signal into a message, performs a validity check on the message, and then forwards it to network processor 832 / network processor 842 for processing. The central processing unit 811 in main control board 810 can decompress the received compressed data using a codebook.

[0156] It should be understood that the operation on interface board 840 in this embodiment is consistent with the operation on interface board 830, and will not be described again for the sake of simplicity. It should be understood that the network device 800 in this embodiment can correspond to the first network device or the second network device in the above-described method embodiments. The main control board 810, interface board 830 and / or interface board 840 in the network device 800 can implement the functions and / or various steps implemented by the first network device or the second network device in the above-described method embodiments, and will not be described again for the sake of simplicity.

[0157] It's worth noting that a network device may have one or more main control boards, including a primary and a backup main control board. It may also have one or more interface boards; the more powerful the network device's data processing capabilities, the more interface boards it provides. Each interface board may also have one or more physical interface cards. A switching board may or may not exist; multiple boards can share the load and provide redundancy. In a centralized forwarding architecture, the network device may not need a switching board, as the interface boards handle the entire system's business data processing. In a distributed forwarding architecture, the network device can have at least one switching board, which enables data exchange between multiple interface boards, providing high-capacity data exchange and processing capabilities. Therefore, the data access and processing capabilities of a distributed architecture network device are greater than those of a centralized architecture device. Alternatively, the network device can also be a single board, without a switching board. The functions of the interface board and the main control board are integrated on this one board. In this case, the central processing unit (CPU) on the interface board and the CPU on the main control board can be combined into a single CPU to perform the combined functions. This type of device has lower data exchange and processing capabilities (e.g., low-end switches or routers). The specific architecture used depends on the specific network deployment scenario, and no restrictions are imposed here.

[0158] In some possible embodiments, the first or second network device described above can be implemented as a virtualized device. For example, a virtualized device can be a virtual machine (VM) running a program for sending messages, and the VM is deployed on a hardware device (e.g., a physical server). A virtual machine refers to a complete computer system with full hardware system functionality simulated by software, running in a completely isolated environment. The virtual machine can be configured as the first or second network device. For example, the first or second network device can be implemented based on a general-purpose physical server combined with Network Functions Virtualization (NFV) technology. The first or second network device can be a virtual host, a virtual router, or a virtual switch. Those skilled in the art can virtualize the first or second network device with the above-mentioned functions on a general-purpose physical server by combining NFV technology by reading this application. Further details are omitted here.

[0159] It should be understood that the network devices of the various product forms described above each have any of the functions of the first network device or the second network device in the above method embodiments, which will not be elaborated here.

[0160] This application provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a network device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the network device to perform the aforementioned data transmission method.

[0161] This application also provides a computer program product that, when run on a network device, causes the network device to execute the method executed by the first network device in the above method embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data transmission method, characterized in that, A method applicable to a computing device cluster comprising multiple data centers, wherein the multiple data centers include a first data center and a second data center, wherein a first network device in the first data center and a second network device in the second data center are connected, the method comprising: The first network device acquires the data to be transmitted in the current iteration round during the distributed computing process, and the codebook determined according to the probability distribution of the global data of the historical iteration rounds during the distributed computing process. The global data of the historical iteration rounds is obtained based on the data transmitted by the first network device in the historical iteration rounds. The first network device compresses the data to be transmitted according to the codebook to obtain compressed data; The first network device sends the compressed data to the second network device.

2. The method according to claim 1, characterized in that, The first network device compresses the data to be transmitted according to the codebook to obtain compressed data, including: When the data to be transmitted is mapped to the uplink port of the first network device, the first network device compresses the data to be transmitted according to the codebook to obtain compressed data.

3. The method according to claim 1 or 2, characterized in that, The global data for the historical iteration rounds includes multiple subsets, and the method further includes: The first network device receives frequency distribution information of the plurality of subsets sent by the plurality of first computing nodes of the first data center, as well as the amount of parameters processed by each of the plurality of first computing nodes; The first network device uses the amount of parameters processed by each of the plurality of first computing nodes as the weight to perform a weighted operation on the frequency distribution information of the plurality of subsets to obtain the probability distribution of the global data. The first network device acquires a codebook determined based on the probability distribution of global data from historical iterations in the distributed computing process, including: The first network device determines the codebook for compressing the data to be transmitted based on the probability distribution of the global data.

4. The method according to claim 1 or 2, characterized in that, The global data of the historical iteration rounds includes multiple subsets. The first network device acquires a codebook determined based on the probability distribution of the global data of the historical iteration rounds in the distributed computing process, including: The first network device obtains a codebook from the network controller of the first data center. The codebook is determined by the network controller based on the frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center and the amount of parameters processed by each of the multiple first computing nodes. The codebook is determined based on the probability distribution of the global data.

5. The method according to any one of claims 1 to 4, characterized in that, The first network device compresses the data to be transmitted according to the codebook to obtain compressed data, including: The first network device compresses the data to be transmitted using a lossless compression algorithm based on the codebook to obtain compressed data. The lossless compression algorithm includes Huffman coding, arithmetic coding, run-length encoding, or dictionary coding.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: The first network device determines the uniformity of the data to be transmitted based on the probability distribution of the data to be transmitted; The first network device compresses the data to be transmitted according to the codebook to obtain compressed data, including: When the distribution of the data to be transmitted is uneven, the first network device compresses the data to be transmitted according to the codebook to obtain compressed data.

7. The method according to any one of claims 1 to 6, characterized in that, The distributed computing process includes a distributed training process for an artificial intelligence (AI) model. The first data center includes one or more parallel groups, each of which includes multiple first computing nodes within the first data center. The data to be transmitted includes gradient data generated by the parallel groups in the first data center during training of the AI ​​model.

8. The method according to any one of claims 1 to 7, characterized in that, The first network device and the second network device are connected via an optical transmission network (OTN) or a wireless communication network.

9. A data transmission device, characterized in that, An apparatus for use with a computing device cluster comprising multiple data centers, the multiple data centers including a first data center and a second data center, wherein a first network device in the first data center and a second network device in the second data center are connected, the apparatus being deployed on the first network device, the apparatus comprising: The acquisition module is used to acquire the data to be transmitted in the current iteration round during the distributed computing process, and the codebook determined according to the probability distribution of the global data of the historical iteration rounds in the distributed computing process. The global data of the historical iteration rounds is obtained based on the data transmitted by the first network device in the historical iteration rounds. A compression module is used to compress the data to be transmitted according to the codebook to obtain compressed data; The sending module is used to send the compressed data to the second network device.

10. The apparatus according to claim 9, characterized in that, The compression module is specifically used for: When the data to be transmitted is mapped to the uplink port of the first network device, the data to be transmitted is compressed according to the codebook to obtain compressed data.

11. The apparatus according to claim 9 or 10, characterized in that, The global data for the historical iteration rounds includes multiple subsets, and the acquisition module is specifically used for: Receive frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center, as well as the amount of parameters processed by each of the multiple first computing nodes; Using the number of parameters processed by each of the plurality of first computing nodes as the weight, the frequency distribution information of the plurality of subsets is weighted and calculated to obtain the probability distribution of the global data. Based on the probability distribution of the global data, a codebook for compressing the data to be transmitted is determined.

12. The apparatus according to claim 9 or 10, characterized in that, The global data for the historical iteration rounds includes multiple subsets, and the acquisition module is specifically used for: The codebook is obtained from the network controller of the first data center. The codebook is determined by the network controller based on the frequency distribution information of the multiple subsets sent by the multiple first computing nodes of the first data center and the amount of parameters processed by each of the multiple first computing nodes. The codebook is determined based on the probability distribution.

13. The apparatus according to any one of claims 9 to 12, characterized in that, The compression module is specifically used for: Based on the codebook, the data to be transmitted is compressed using a lossless compression algorithm to obtain compressed data. The lossless compression algorithm includes Huffman coding, arithmetic coding, run-length encoding, or dictionary coding.

14. The apparatus according to any one of claims 9 to 13, characterized in that, The device further includes: The determining module is used to determine the uniformity of the data to be transmitted based on the probability distribution of the data to be transmitted; The compression module is specifically used for: When the distribution of the data to be transmitted is uneven, the first network device compresses the data to be transmitted according to the codebook to obtain compressed data.

15. The apparatus according to any one of claims 9 to 14, characterized in that, The distributed computing process includes a distributed training process for an artificial intelligence (AI) model. The first data center includes one or more parallel groups, each of which includes multiple first computing nodes within the first data center. The data to be transmitted includes gradient data generated by the parallel groups in the first data center during training of the AI ​​model.

16. The apparatus according to any one of claims 9 to 15, characterized in that, The first network device and the second network device are connected via an optical transmission network (OTN) or a wireless communication network.

17. A network device, characterized in that, The network device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions to cause the network device to perform the data transmission method as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, Includes computer-readable instructions; the computer-readable instructions are used to implement the data transmission method according to any one of claims 1 to 8.

19. A computer program product, characterized in that, Includes computer-readable instructions; the computer-readable instructions are used to implement the data transmission method according to any one of claims 1 to 8.

Images