Data transmission method, device, equipment and computer readable storage medium
By quantizing and inversely quantizing data on the intelligent network card, the accuracy and type of data transmission is dynamically determined, which solves the problem of low data transmission efficiency in model training, and realizes efficient data transmission and training process.
Patent Information
- Application Number
- CN202411187773.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2044-08-28
AI Technical Summary
During the model training process, the slow data quantization speed and the long data transmission have become bottlenecks in communication between computing nodes, affecting the training efficiency.
By receiving floating-point model training data on the sending intelligent network card, quantization operations and calculating the loss degree, determining the data transmission type based on the preset loss degree range, sending quantized data or original data, and performing inverse quantization reconciliation on the receiving end Compression to realize real-time data transmission and restoration.
It improves the data quantization speed, reduces transmission time, reduces network communication, saves network bandwidth, and improves training speed.
Smart Images
Figure CN118694765B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer application technology, and in particular to a data transmission method, device, equipment and computer-readable storage medium. Background Art
[0002] A large amount of memory and computing resources are required during model training. In some model training processes, the communication volume required to synchronize gradients within a single computational iteration is as high as hundreds of GB. The communication speed and efficiency between computing nodes have become the biggest bottleneck in model training.
[0003] In mathematics and digital signal processing, quantization refers to the process of mapping input values from a large set to a smaller set. In the field of machine learning, this mapping is usually from floating point values to integer values.
[0004] Automatic Mixed Precision (AMP) training is to reduce the precision of some operators when training the model, while keeping the precision of other operators unchanged. This can shorten the training time while ensuring that the model training accuracy meets the requirements, and thus can support larger models and larger inputs for training. The current quantization method is to judge whether to quantize based on the convergence state of the neural network at the software level, and then decide whether to quantize based on the judgment result, and then transmit the corresponding data based on whether to quantize. The data quantization speed is slow and the data transmission takes a long time.
[0005] In summary, how to effectively solve the problems of slow data quantization speed and long data transmission time is a problem that technicians in this field need to solve urgently. Summary of the invention
[0006] The object of the present invention is to provide a data transmission method, which greatly improves the data quantization speed and reduces the transmission time; another object of the present invention is to provide a data transmission device, equipment and computer-readable storage medium.
[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0008] A data transmission method, comprising:
[0009] Utilize the sending end smart network card to receive the floating point model training data set to be transmitted;
[0010] Quantizing each floating-point model training data in the floating-point model training data set using the transmitting-end intelligent network card to obtain each quantized model training data;
[0011] Calculate the loss degree of each quantized model training data respectively to obtain the loss degree calculation result;
[0012] The sending-end intelligent network card sends the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end intelligent network card, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving-end intelligent network card;
[0013] The quantized model training data is dequantized using the receiving end intelligent network card to obtain restored floating-point model training data, and the restored floating-point model training data and the received original floating-point model training data are uploaded to the receiving end.
[0014] In a specific implementation of the present invention, the sending-end intelligent network card is used to perform a quantization operation on each floating-point model training data in the floating-point model training data set, including:
[0015] Obtaining maximum floating-point model training data and minimum floating-point model training data in the floating-point model training data set;
[0016] Get the maximum quantization value and the minimum quantization value in the quantization range;
[0017] Calculating a scaling factor according to the maximum floating-point model training data, the minimum floating-point model training data, the maximum quantization value, and the minimum quantization value;
[0018] Calculate a zero point value according to the maximum floating point model training data and the minimum floating point model training data;
[0019] The sending-end intelligent network card is used to perform a quantization operation on each floating-point model training data in the floating-point model training data set according to the scaling factor and the zero point value.
[0020] In a specific implementation of the present invention, obtaining the maximum quantization value and the minimum quantization value in the quantization range includes:
[0021] Determine a target quantization range according to the maximum floating-point model training data and the minimum floating-point model training data;
[0022] The maximum quantization value and the minimum quantization value in the target quantization range are obtained.
[0023] In a specific implementation of the present invention, the sending-end intelligent network card sends the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end intelligent network card, including:
[0024] Using the transmitting end intelligent network card, the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range, the scaling factor and the zero point value are sent to the receiving end intelligent network card;
[0025] Accordingly, the quantized model training data is dequantized using the receiving-end intelligent network card, including:
[0026] The receiving-end intelligent network card is used to dequantize the quantized model training data according to the scaling factor and the zero point value.
[0027] In a specific implementation of the present invention, dequantizing the quantized model training data using the receiving-end intelligent network card includes:
[0028] Using the receiving end intelligent network card to obtain the flag bit of each received data;
[0029] When the quantized model training data is determined according to the flag bit, the quantized model training data is dequantized using the receiving-end intelligent network card.
[0030] In a specific implementation of the present invention, the sending-end intelligent network card sends the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end intelligent network card, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving-end intelligent network card, including:
[0031] Using the transmitting end intelligent network card, compressing the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to obtain compressed quantized model training data;
[0032] Using the transmitting end intelligent network card, compressing the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range, to obtain compressed floating-point model training data;
[0033] The compressed and quantized model training data and the compressed floating-point model training data are sent to the receiving-end intelligent network card.
[0034] In a specific implementation of the present invention, dequantizing the quantized model training data using the receiving-end intelligent network card includes:
[0035] Decompressing the compressed and quantized model training data using the receiving-end intelligent network card to obtain decompressed and quantized model training data;
[0036] The receiving-end intelligent network card is used to dequantize the decompressed quantized model training data.
[0037] In a specific implementation of the present invention, uploading both the restored floating-point model training data and the received original floating-point model training data to the receiving end includes:
[0038] Decompressing the compressed floating-point model training data using the receiving-end intelligent network card to obtain decompressed floating-point model training data;
[0039] The restored floating-point model training data and the decompressed floating-point model training data are both uploaded to the receiving end.
[0040] In a specific implementation of the present invention, receiving a floating-point model training data set to be transmitted using a transmitting end intelligent network card includes:
[0041] Utilize the transmitting end intelligent network card to receive the data transmission request, and parse the data transmission request to obtain the floating point model training data set and the target address to be transmitted;
[0042] The receiving-end intelligent network card is determined according to the target address.
[0043] In a specific implementation of the present invention, parsing the data transmission request to obtain the floating-point model training data set and the target address to be transmitted includes:
[0044] Parsing the data transmission request to obtain the floating-point model training data set to be transmitted, the target address and the data batch;
[0045] Correspondingly, the sending-end intelligent network card sends the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end intelligent network card, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving-end intelligent network card, including:
[0046] The sending-end smart network card sends the quantized model training data whose loss degree calculation results satisfy the preset loss degree allowable range to the receiving-end smart network card in batches according to the data batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range to the receiving-end smart network card in batches.
[0047] In a specific implementation of the present invention, the sending-end intelligent network card is used to perform a quantization operation on each floating-point model training data in the floating-point model training data set, including:
[0048] The sending-end intelligent network card is used to perform parallel quantization operations on each floating-point model training data in the floating-point model training data set.
[0049] In a specific implementation of the present invention, dequantizing the quantized model training data using the receiving-end intelligent network card includes:
[0050] The receiving-end intelligent network card is used to perform parallel dequantization on the quantized model training data.
[0051] A data transmission device, comprising:
[0052] Memory for storing computer programs;
[0053] A processor is used to implement the steps of the data transmission method as described above when executing the computer program.
[0054] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data transmission method described above are implemented.
[0055] The data transmission method provided by the present invention uses a sending-end smart network card to receive a floating-point model training data set to be transmitted; uses the sending-end smart network card to perform a quantization operation on each floating-point model training data in the floating-point model training data set to obtain each quantized model training data; respectively performs loss degree calculation on each quantized model training data to obtain a loss degree calculation result; uses the sending-end smart network card to send the quantized model training data whose loss degree calculation result meets a preset loss degree allowable range to the receiving-end smart network card, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving-end smart network card; uses the receiving-end smart network card to dequantize the quantized model training data to obtain restored floating-point model training data, and uploads both the restored floating-point model training data and the received original floating-point model training data to the receiving end.
[0056] It can be seen from the above technical scheme that, in order to solve the problem of frequent and large-scale communication between computing nodes during the training of large-scale distributed neural networks, it is proposed to dynamically determine the actual transmission data accuracy in the network by quantifying the loss degree, first quantify and then judge the loss degree, determine the transmission data type according to the loss degree, define the transmission content according to the transmission data type, and then determine the data processing and transmission logic of the smart network card at the sending and receiving ends. It not only ensures the accuracy of network training, but also reduces the network communication volume and saves network bandwidth. The loss degree calculation and data transmission are implemented in the integrated hardware of the smart network card. Realizing real-time data transmission without the upper-layer software's perception greatly improves the data quantization speed and reduces the transmission time.
[0057] Correspondingly, the present invention also provides a data transmission device, equipment and computer-readable storage medium corresponding to the above-mentioned data transmission method, which have the above-mentioned technical effects and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0059] Figure 1 It is a functional implementation block diagram of a traditional network card in the related technology;
[0060] Figure 2 A flowchart of a data transmission method according to an embodiment of the present invention;
[0061] Figure 3 A functional implementation block diagram of a smart network card in an embodiment of the present invention;
[0062] Figure 4 Another implementation flow chart of the data transmission method in the embodiment of the present invention;
[0063] Figure 5 This is a structural block diagram of a data transmission device in an embodiment of the present invention;
[0064] Figure 6 This is a structural block diagram of a data transmission device in an embodiment of the present invention;
[0065] Figure 7 A schematic diagram of the specific structure of a data transmission device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0066] The current quantization method is to judge whether to quantize based on the convergence state of the neural network at the software level, and then decide whether to quantize based on the judgment result, and then transmit the corresponding data based on whether to quantize. The data quantization speed is slow and the data transmission takes a long time.
[0067] See also Figure 1 , Figure 1 This is a functional block diagram of a traditional network card in the related art. In the process of model training data transmission in the related art, the traditional network card used in the host, data compression and decompression, storage, and network are all provided by the host central processor.
[0068] To this end, the data transmission method provided in this application greatly improves the data quantization speed and reduces the transmission time.
[0069] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0070] See also Figure 2 , Figure 2 This is a flowchart of an implementation of a data transmission method in an embodiment of the present invention. The method may include the following steps:
[0071] S201: Receive a floating-point model training data set to be transmitted using a transmitting end smart network card.
[0072] A smart network card is pre-set in each host. When the floating-point model training data set in the current host (i.e., the sending end) needs to be transmitted to another host, the floating-point model training data set in the current host is sent to the smart network card in the current host (i.e., the sending end smart network card), and the sending end smart network card is used to receive the floating-point model training data set to be transmitted.
[0073] S202: Quantize each floating-point model training data in the floating-point model training data set by using the transmitting end intelligent network card to obtain each quantized model training data.
[0074] After receiving the floating-point model training data set to be transmitted, the transmitting end smart network card uses the transmitting end smart network card to quantize each floating-point model training data in the floating-point model training data set, such as quantizing the FP32 (Floating Point 32, 32-bit floating point number) value to the INT8 (Integer 8, 8-bit integer) value to obtain each quantized model training data.
[0075] In a specific implementation of the present invention, using a transmitting end intelligent network card to quantize each floating-point model training data in a floating-point model training data set may include the following steps:
[0076] The sending-end intelligent network card is used to perform parallel quantization operations on each floating-point model training data in the floating-point model training data set.
[0077] When performing data quantization, the sending end smart network card is used to perform parallel quantization operations on each floating-point model training data in the floating-point model training data set. In the quantization process, the smart network card hardware is used to realize parallel computing, which greatly improves the data quantization speed.
[0078] S203: Calculate the loss degree of each quantized model training data to obtain the loss degree calculation result.
[0079] After quantizing each floating-point model training data in the floating-point model training data set to obtain each quantized model training data, the loss degree is calculated for each quantized model training data to obtain the loss degree calculation result. The loss degree calculation formula is as follows:
[0080] ;
[0081] Among them, d is the loss degree, F is the floating-point model training data, and Q is the quantized model training data.
[0082] S204: Use the transmitting end smart network card to send the quantized model training data whose loss degree calculation results meet the preset loss degree allowable range to the receiving end smart network card, and send the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range to the receiving end smart network card.
[0083] Preset the allowable range of loss degree, and after calculating the loss degree result, divide each quantized model training data into data that meets the preset allowable range of loss degree and data that does not meet the preset allowable range of loss degree. Use the sending end smart network card to send the quantized model training data whose loss degree calculation results meet the preset allowable range of loss degree to the receiving end smart network card, and send the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset allowable range of loss degree to the receiving end smart network card. In this way, only the quantized model training data whose quantized accuracy meets the requirements is output to the receiving end smart network card in the quantized data format, and the quantized model training data whose quantized accuracy does not meet the requirements is output to the receiving end smart network card in the original floating-point data format. This not only ensures the accuracy of network training, but also reduces the network communication volume and saves network bandwidth occupancy.
[0084] It should be noted that the preset allowable range of loss degree can be set and adjusted according to actual conditions, and the embodiment of the present invention does not limit this. For example, it can be set to be less than or equal to 5%.
[0085] In a specific implementation of the present invention, step S201 may include the following steps:
[0086] Step 1: Use the sending end smart network card to receive the data transmission request, and parse the data transmission request to obtain the floating-point model training data set and the target address to be transmitted;
[0087] Step 2: Determine the receiving end smart network card according to the target address.
[0088] For the convenience of description, the above two steps can be combined for explanation.
[0089] There may be multiple hosts in the node cluster used for model training. When the floating-point model training data set in the current host needs to be transferred to another host, a data transmission request is first sent to the sending end smart network card. The data transmission request includes the floating-point model training data set to be transmitted and the target address. The sending end smart network card receives the data transmission request and parses the data transmission request to obtain the floating-point model training data set to be transmitted and the target address. The receiving end smart network card is determined according to the target address, thereby determining which host the model data needs to be transferred to for further processing. By using the target address to determine the receiving end smart network card, the determination efficiency of the receiving end smart network card is greatly improved.
[0090] In a specific implementation of the present invention, parsing a data transmission request to obtain a floating-point model training data set and a target address to be transmitted may include the following steps:
[0091] Parse the data transmission request to obtain the floating-point model training data set, target address and data batch to be transmitted;
[0092] Accordingly, step S204 may include the following steps:
[0093] The sending-end smart network card sends the quantized model training data whose loss calculation results meet the preset loss allowable range to the receiving-end smart network card in batches according to data batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss calculation results exceed the preset loss allowable range to the receiving-end smart network card in batches.
[0094] In addition to the floating-point model training data set to be transmitted and the target address for marking the receiving-end smart network card, the data transmission request may also include a data batch corresponding to the floating-point model training data set to be transmitted. The amount of model training data to be transmitted may be relatively large. After receiving the data transmission request, the sending-end smart network card parses the data transmission request to obtain the floating-point model training data set to be transmitted, the target address and the data batch. The sending-end smart network card sends the quantized model training data whose loss calculation results meet the preset loss allowable range to the receiving-end smart network card in batches according to the data batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss calculation results exceed the preset loss allowable range in batches to the receiving-end smart network card.
[0095] After obtaining the data batches, the data batches can be set to be proportional to the generation time sequence of the model training data, that is, the smaller the data batches, the earlier the generation time of the model training data. By sending each model training data in batches according to the data batches, when the amount of model training data to be transmitted is relatively large and exceeds the amount of data that can be transmitted at a single time, the sending end smart network card uses the data batches from small to large to send the quantized model training data whose loss calculation results meet the preset loss allowable range to the receiving end smart network card in batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss calculation results exceed the preset loss allowable range to the receiving end smart network card in batches. In addition, the data transmission accuracy can be dynamically determined according to the data transmitted in different batches. When the quantization loss is large, in order to ensure the accuracy of data transmission, the data before quantization is used for transmission, and the convergence of the network is guaranteed. When the quantization loss is acceptable, the amount of network transmission data is appropriately reduced on the premise of reducing the convergence impact, thereby effectively reducing the network communication volume. By sending data in order from small to large batches, it is ensured that model training data generated earlier is sent first, which improves the orderliness of data sending and avoids bandwidth competition among different data.
[0096] S205: Dequantize the quantized model training data using the receiving end intelligent network card to obtain restored floating-point model training data, and upload the restored floating-point model training data and the received original floating-point model training data to the receiving end.
[0097] After sending the model training data to the receiving end smart network card, the receiving end smart network card is used to dequantize the quantized model training data to obtain the restored floating-point model training data, and the restored floating-point model training data and the received original floating-point model training data are uploaded to the receiving end, thereby realizing data restoration. The automatic quantization and transmission of distributed floating-point data in software and hardware collaboration are realized through the smart network card, and the judgment index of automatic quantization is proposed to determine the data transmission accuracy in real time. The numerical accuracy processing and data transmission algorithm are integrated into the hardware chips in the smart network card communication equipment, such as FPGA (Field Programmable Gate Array), etc., to realize the real-time parallel conversion and transmission of network transmission data accuracy, which not only ensures data transmission, but also effectively reduces the network communication volume and improves communication efficiency, thereby speeding up training.
[0098] In a specific implementation of the present invention, dequantizing the quantized model training data using a receiving-end intelligent network card may include the following steps:
[0099] Step 1: Use the receiving end smart network card to obtain the flag bit of each received data;
[0100] Step 2: When the data is determined to be quantized model training data according to the flag bit, the quantized model training data is dequantized using the receiving end smart network card.
[0101] For the convenience of description, the above two steps can be combined for explanation.
[0102] Each model training data has a flag bit. After the model training data is sent to the receiving end smart network card, the receiving end smart network card is used to obtain the flag bit of each received data. When it is determined to be quantized model training data according to the flag bit, the receiving end smart network card is used to dequantize the quantized model training data. By setting the flag bit to mark whether the data is quantized data or original floating-point data, the efficiency of the receiving end smart network card in identifying the type of model training data received is improved, thereby improving the efficiency of data dequantization.
[0103] In a specific implementation of the present invention, step S204 may include the following steps:
[0104] Step 1: Using the intelligent network card at the transmitting end, compress the quantized model training data whose loss calculation result meets the preset loss allowable range to obtain compressed quantized model training data;
[0105] Step 2: Using the intelligent network card at the transmitting end, compress the floating-point model training data corresponding to the quantized model training data whose loss calculation result exceeds the preset loss allowable range, to obtain compressed floating-point model training data;
[0106] Step 3: Send the compressed quantized model training data and the compressed floating-point model training data to the receiving end smart network card.
[0107] For the convenience of description, the above three steps can be combined for explanation.
[0108] See also Figure 3 , Figure 3 The functional implementation block diagram of a smart network card in an embodiment of the present invention. After the model training data to be sent is divided into quantized model training data that meets the preset loss degree allowable range and floating-point model training data corresponding to the quantized model training data that exceeds the preset loss degree allowable range, the sending-end smart network card is used to compress the quantized model training data whose loss degree calculation results meet the preset loss degree allowable range to obtain compressed quantized model training data, and the sending-end smart network card is used to compress the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range to obtain compressed floating-point model training data, and the compressed quantized model training data and the compressed floating-point model training data are sent to the receiving-end smart network card. By compressing the data before transmission, the network communication volume is further reduced, saving network bandwidth occupancy. In addition, data compression is completed by the receiving-end smart network card, which greatly improves the data compression efficiency, so that the resources of the host central processor are effectively released. In addition, the storage and network of model training data are also provided by the Smart NIC. Due to its integrated internal processing unit, the Smart NIC can offload the data processing work usually completed by the host CPU to itself. In addition to using the Smart NIC to complete data processing and transmission, the software-level judgment on whether data is quantized and transmitted is offloaded to the Smart NIC. At the same time, the Smart NIC completes data quantization and recovery operations in real time, thereby further releasing the resources of the host CPU.
[0109] In a specific implementation of the present invention, dequantizing the quantized model training data using a receiving-end intelligent network card may include the following steps:
[0110] Step 1: Decompress the compressed and quantized model training data using the receiving end smart network card to obtain decompressed and quantized model training data;
[0111] Step 2: Use the receiving end smart network card to dequantize the decompressed quantized model training data.
[0112] For the convenience of description, the above two steps can be combined for explanation.
[0113] After receiving the compressed and quantized model training data, the receiving end smart network card first decompresses the compressed and quantized model training data to obtain the decompressed quantized model training data, and then dequantizes the decompressed quantized model training data using the receiving end smart network card. By using the receiving end smart network card to decompress the data, the resources of the host central processor are further released.
[0114] In a specific implementation of the present invention, uploading both the restored floating-point model training data and the received original floating-point model training data to the receiving end may include the following steps:
[0115] Step 1: Decompress the compressed floating-point model training data using the receiving end smart network card to obtain the decompressed floating-point model training data;
[0116] Step 2: Upload both the restored floating-point model training data and the decompressed floating-point model training data to the receiving end.
[0117] For the convenience of description, the above two steps can be combined for explanation.
[0118] After receiving the compressed floating-point model training data, the receiving end smart network card decompresses the compressed floating-point model training data using the receiving end smart network card to obtain the decompressed floating-point model training data, and uploads both the restored floating-point model training data and the decompressed floating-point model training data to the receiving end. By using the receiving end smart network card to decompress the data, the resources of the host central processor are further released.
[0119] In a specific implementation of the present invention, dequantizing the quantized model training data using a receiving-end intelligent network card may include the following steps:
[0120] The receiving-end smart network card is used to perform parallel dequantization on the quantized model training data.
[0121] When dequantizing the quantized model training data, the receiving end smart network card is used to perform parallel dequantization on the quantized model training data. By using the smart network card hardware to realize parallel computing during the dequantization process, the data dequantization speed is greatly improved.
[0122] It can be seen from the above technical scheme that, in order to solve the problem of frequent and large-scale communication between computing nodes during the training of large-scale distributed neural networks, it is proposed to dynamically determine the actual transmission data accuracy in the network by quantifying the loss degree, first quantify and then judge the loss degree, determine the transmission data type according to the loss degree, define the transmission content according to the transmission data type, and then determine the data processing and transmission logic of the smart network card at the sending and receiving ends. It not only ensures the accuracy of network training, but also reduces the network communication volume and saves network bandwidth. The loss degree calculation and data transmission are implemented in the integrated hardware of the smart network card. Realizing real-time data transmission without the upper-layer software's perception greatly improves the data quantization speed and reduces the transmission time.
[0123] It should be noted that, based on the above embodiment, the embodiment of the present invention also provides corresponding improved solutions. In the subsequent embodiments, the same steps or corresponding steps as those in the above embodiment can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, which will not be repeated one by one in the following improved embodiments.
[0124] See also Figure 4 , Figure 4 Another implementation flow chart of the data transmission method in the embodiment of the present invention is as follows. The method may include the following steps:
[0125] S401: Receive a floating-point model training data set to be transmitted using a transmitting end smart network card.
[0126] S402: Obtain maximum floating-point model training data and minimum floating-point model training data in the floating-point model training data set.
[0127] After receiving the floating-point model training data set to be transmitted, the transmitting end intelligent network card obtains the maximum floating-point model training data and the minimum floating-point model training data in the floating-point model training data set.
[0128] S403: Obtain the maximum quantization value and the minimum quantization value in the quantization range.
[0129] The quantization range may be preset, or may be determined according to the range of data in the floating-point model training data set to be transmitted. After the quantization range is determined, the maximum quantization value and the minimum quantization value in the quantization range are obtained.
[0130] In a specific implementation of the present invention, step S403 may include the following steps:
[0131] Step 1: Determine the target quantization range based on the maximum floating-point model training data and the minimum floating-point model training data;
[0132] Step 2: Obtain the maximum quantization value and the minimum quantization value in the target quantization range.
[0133] For the convenience of description, the above two steps can be combined for explanation.
[0134] After obtaining the maximum floating-point model training data and the minimum floating-point model training data in the floating-point model training data set, the target quantization range is determined according to the maximum floating-point model training data and the minimum floating-point model training data. For example, the target quantization range of the quantized model training data obtained by quantizing all the data in the floating-point model training data set can be determined according to the maximum floating-point model training data and the minimum floating-point model training data, and the maximum quantization value and the minimum quantization value in the target quantization range are obtained. By determining the target quantization range according to the maximum floating-point model training data and the minimum floating-point model training data, the accuracy of determining the quantization range is improved.
[0135] S404: Calculate a scaling factor according to the maximum floating-point model training data, the minimum floating-point model training data, the maximum quantization value, and the minimum quantization value.
[0136] After obtaining the maximum quantization value and the minimum quantization value in the quantization range, the scaling factor is calculated according to the maximum floating-point model training data, the minimum floating-point model training data, the maximum quantization value and the minimum quantization value. For example, the scaling factor can be calculated according to the following formula:
[0137] ;
[0138] Where S is the scaling factor, For the maximum floating point model training data, For the minimum floating point model training data, is the maximum quantization value, is the minimum quantized value, such as in an 8-bit integer, is 255, is 0.
[0139] S405: Calculate a zero point value according to the maximum floating point model training data and the minimum floating point model training data.
[0140] After the scaling factor is calculated, the zero point value is calculated according to the maximum floating point model training data and the minimum floating point model training data. For example, the zero point value can be calculated according to the following formula:
[0141] ;
[0142] Where Z is the zero value.
[0143] S406: Quantize each floating-point model training data in the floating-point model training data set according to the scaling factor and the zero point value by using the transmitting end intelligent network card to obtain each quantized model training data.
[0144] After the zero point value is calculated, the transmitting end smart network card performs quantization operation on each floating-point model training data in the floating-point model training data set according to the scaling factor and the zero point value to obtain each quantized model training data. For example, data quantization can be performed according to the following formula:
[0145] ;
[0146] in, is the quantized model training data, and r is the floating-point model training data.
[0147] By calculating the scaling factor and the zero point value in advance and performing data quantization according to the scaling factor and the zero point value, an effective parameter basis is provided for data quantization, thereby improving data quantization efficiency.
[0148] S407: Calculate the loss degree of each quantized model training data to obtain the loss degree calculation result.
[0149] S408: Use the transmitting end smart network card to send the quantized model training data whose loss degree calculation results meet the preset loss degree allowable range to the receiving end smart network card, and send the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range to the receiving end smart network card.
[0150] S409: Dequantize the quantized model training data using the receiving end intelligent network card to obtain restored floating-point model training data, and upload the restored floating-point model training data and the received original floating-point model training data to the receiving end.
[0151] In a specific implementation of the present invention, using a transmitting end smart network card to send quantized model training data whose loss degree calculation result satisfies a preset loss degree allowable range to a receiving end smart network card may include the following steps:
[0152] The quantized model training data, the scaling factor and the zero point value whose loss degree calculation result satisfies the preset loss degree allowable range are sent to the receiving end smart network card by using the sending end smart network card;
[0153] Accordingly, dequantizing the quantized model training data using the receiving end smart network card may include the following steps:
[0154] The receiving end smart network card is used to dequantize the quantized model training data according to the scaling factor and the zero point value.
[0155] When the quantized model training data is sent to the receiving-end smart network card, the quantized model training data, the scaling factor and the zero-point value whose loss calculation results meet the preset loss allowable range are sent to the receiving-end smart network card by the sending-end smart network card, and the receiving-end smart network card is used to dequantize the quantized model training data according to the scaling factor and the zero-point value. By sending the scaling factor, the zero-point value and the quantized model training data together to the receiving-end smart network card, effective dequantization parameters are provided for the receiving-end smart network card to dequantize the quantized model training data, thereby greatly improving the dequantization efficiency.
[0156] Corresponding to the above method embodiment, the present invention further provides a data transmission device. The data transmission device described below and the data transmission method described above can be referenced to each other.
[0157] See also Figure 5 , Figure 5 This is a structural block diagram of a data transmission device in an embodiment of the present invention, and the device may include:
[0158] The training data receiving module 51 is used to receive the floating-point model training data set to be transmitted by using the transmitting end intelligent network card;
[0159] The data quantization module 52 is used to perform a quantization operation on each floating-point model training data in the floating-point model training data set by using the transmitting end intelligent network card to obtain each quantized model training data;
[0160] The loss degree calculation module 53 is used to calculate the loss degree of each quantized model training data to obtain the loss degree calculation result;
[0161] The data sending module 54 is used to send the quantized model training data whose loss degree calculation results meet the preset loss degree allowable range to the receiving end smart network card by using the sending end smart network card, and send the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range to the receiving end smart network card;
[0162] The data upload module 55 is used to use the receiving end intelligent network card to dequantize the quantized model training data to obtain restored floating-point model training data, and upload the restored floating-point model training data and the received original floating-point model training data to the receiving end.
[0163] It can be seen from the above technical scheme that, in order to solve the problem of frequent and large-scale communication between computing nodes during the training of large-scale distributed neural networks, it is proposed to dynamically determine the actual transmission data accuracy in the network by quantifying the loss degree, first quantify and then judge the loss degree, determine the transmission data type according to the loss degree, define the transmission content according to the transmission data type, and then determine the data processing and transmission logic of the smart network card at the sending and receiving ends. It not only ensures the accuracy of network training, but also reduces the network communication volume and saves network bandwidth. The loss degree calculation and data transmission are implemented in the integrated hardware of the smart network card. Realizing real-time data transmission without the upper-layer software's perception greatly improves the data quantization speed and reduces the transmission time.
[0164] In a specific embodiment of the present invention, the data quantization module 52 includes:
[0165] A floating-point data acquisition submodule is used to acquire the maximum floating-point model training data and the minimum floating-point model training data in the floating-point model training data set;
[0166] A quantization value acquisition submodule is used to obtain the maximum quantization value and the minimum quantization value in the quantization range;
[0167] A scaling factor calculation submodule, used to calculate a scaling factor according to the maximum floating point model training data, the minimum floating point model training data, the maximum quantization value and the minimum quantization value;
[0168] A zero-point value calculation submodule, used to calculate the zero-point value according to the maximum floating-point model training data and the minimum floating-point model training data;
[0169] The data quantization submodule is used to quantize each floating-point model training data in the floating-point model training data set according to the scaling factor and the zero point value using the transmitter intelligent network card.
[0170] In a specific implementation of the present invention, the quantization value acquisition submodule includes:
[0171] A quantization range determination unit, used to determine a target quantization range according to maximum floating-point model training data and minimum floating-point model training data;
[0172] The quantization value acquisition unit is used to acquire the maximum quantization value and the minimum quantization value in the target quantization range.
[0173] In a specific embodiment of the present invention, the data sending module 54 is specifically a module that uses the sending-end smart network card to send the quantized model training data, scaling factor and zero point value whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end smart network card;
[0174] The data upload module 55 is specifically a module that utilizes the receiving end intelligent network card to dequantize the quantized model training data according to the scaling factor and the zero point value.
[0175] In a specific embodiment of the present invention, the data upload module 55 includes:
[0176] The flag bit acquisition submodule is used to obtain the flag bit of each received data by using the receiving end intelligent network card;
[0177] The first dequantization submodule is used to dequantize the quantized model training data using the receiving-end intelligent network card when the quantized model training data is determined to be quantized model training data according to the flag bit.
[0178] In a specific implementation of the present invention, the data sending module includes:
[0179] A first data compression submodule is used to compress the quantized model training data whose loss degree calculation result meets the preset loss degree allowable range by using the sending end intelligent network card to obtain compressed quantized model training data;
[0180] The second data compression submodule is used to compress the floating-point model training data corresponding to the quantized model training data whose loss calculation result exceeds the preset loss allowable range by using the transmitting end intelligent network card to obtain compressed floating-point model training data;
[0181] The data sending submodule is used to send the compressed and quantized model training data and the compressed floating-point model training data to the receiving-end smart network card.
[0182] In a specific embodiment of the present invention, the data upload module 55 includes:
[0183] A first decompression submodule is used to decompress the compressed and quantized model training data using a receiving-end intelligent network card to obtain decompressed and quantized model training data;
[0184] The second dequantization submodule is used to dequantize the decompressed quantized model training data using the receiving end intelligent network card.
[0185] In a specific embodiment of the present invention, the data upload module 55 includes:
[0186] The second decompression submodule is used to decompress the compressed floating-point model training data using the receiving-end intelligent network card to obtain the decompressed floating-point model training data;
[0187] The data upload submodule is used to upload both the restored floating-point model training data and the decompressed floating-point model training data to the receiving end.
[0188] In a specific embodiment of the present invention, the training data receiving module 51 includes:
[0189] The request parsing submodule is used to receive the data transmission request by using the sending end intelligent network card, and parse the data transmission request to obtain the floating point model training data set and the target address to be transmitted;
[0190] The receiving end intelligent network card determination submodule is used to determine the receiving end intelligent network card according to the target address.
[0191] In a specific embodiment of the present invention, the request parsing submodule is specifically a module for parsing the data transmission request to obtain the floating-point model training data set, the target address and the data batch to be transmitted;
[0192] The data sending module 54 specifically uses the sending-end smart network card to send the quantized model training data whose loss degree calculation results meet the preset loss degree allowable range in batches to the receiving-end smart network card according to data batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range in batches to the module of the receiving-end smart network card.
[0193] In a specific implementation of the present invention, the data quantization module 52 is specifically a module that uses a transmitting-end intelligent network card to perform parallel quantization operations on each floating-point model training data in a floating-point model training data set.
[0194] In a specific implementation of the present invention, the data upload module 55 is specifically a module that uses a receiving-end intelligent network card to perform parallel dequantization on the quantized model training data.
[0195] Corresponding to the above method embodiment, see Figure 6 , Figure 6 This is a schematic diagram of a data transmission device provided by the present invention, and the device may include:
[0196] A memory 332, for storing computer programs;
[0197] The processor 322 is configured to implement the steps of the data transmission method in the above method embodiment when executing a computer program.
[0198] For details, please refer to Figure 7 , Figure 7A specific structural diagram of a data transmission device provided for this embodiment, the data transmission device may have relatively large differences due to different configurations or performances, and may include a processor (central processing units, CPU) 322 (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be a temporary storage or a permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 can be configured to communicate with the memory 332 to execute a series of instruction operations in the memory 332 on the data transmission device 301.
[0199] The data transmission device 301 may further include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input and output interfaces 358 , and / or one or more operating systems 341 .
[0200] The steps in the data transmission method described above can be implemented by the structure of a data transmission device.
[0201] Corresponding to the above method embodiment, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps can be implemented:
[0202] A floating-point model training data set to be transmitted is received by using a transmitting end smart network card; each floating-point model training data in the floating-point model training data set is quantized by using the transmitting end smart network card to obtain each quantized model training data; a loss degree is calculated for each quantized model training data to obtain a loss degree calculation result; the quantized model training data whose loss degree calculation result satisfies a preset loss degree allowable range is sent to the receiving end smart network card by using the transmitting end smart network card, and the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range is sent to the receiving end smart network card; the quantized model training data is dequantized by using the receiving end smart network card to obtain restored floating-point model training data, and the restored floating-point model training data and the received original floating-point model training data are uploaded to the receiving end.
[0203] The computer-readable storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.
[0204] For an introduction to the computer-readable storage medium provided by the present invention, please refer to the above method embodiment, and the present invention will not be elaborated here.
[0205] Corresponding to the above method embodiments, the present invention further provides a computer program product, including a computer program, which implements the steps of the above data transmission method when executed by a processor.
[0206] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the devices, equipment and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.
[0207] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the technical solution and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the present invention.
Claims
1. A data transmission method, characterized in that: include: Utilize the sending end smart network card to receive the floating point model training data set to be transmitted; Quantizing each floating-point model training data in the floating-point model training data set using the transmitting-end intelligent network card to obtain each quantized model training data; Calculate the loss degree of each quantized model training data respectively to obtain the loss degree calculation result; The sending-end intelligent network card sends the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end intelligent network card, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving-end intelligent network card; Dequantizing the quantized model training data using the receiving end intelligent network card to obtain restored floating-point model training data, and uploading the restored floating-point model training data and the received original floating-point model training data to the receiving end; The step of using the transmitting end intelligent network card to quantize each floating-point model training data in the floating-point model training data set includes: Obtaining maximum floating-point model training data and minimum floating-point model training data in the floating-point model training data set; Get the maximum quantization value and the minimum quantization value in the quantization range; Calculating a scaling factor according to the maximum floating-point model training data, the minimum floating-point model training data, the maximum quantization value, and the minimum quantization value; Calculate a zero point value according to the maximum floating point model training data and the minimum floating point model training data; Using the transmitting end intelligent network card to perform a quantization operation on each floating-point model training data in the floating-point model training data set according to the scaling factor and the zero point value; The method uses the transmitting end intelligent network card to send the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving end intelligent network card, including: Using the transmitting end intelligent network card, the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range, the scaling factor and the zero point value are sent to the receiving end intelligent network card; Accordingly, the quantized model training data is dequantized using the receiving-end intelligent network card, including: Dequantizing the quantized model training data using the receiving-end intelligent network card according to the scaling factor and the zero point value; Dequantizing the quantized model training data using the receiving end intelligent network card includes: Using the receiving end intelligent network card to obtain the flag bit of each received data; When the quantized model training data is determined according to the flag bit, dequantizing the quantized model training data using the receiving-end intelligent network card; The sending-end intelligent network card is used to perform a quantization operation on each floating-point model training data in the floating-point model training data set, including: Using the transmitting end intelligent network card to perform parallel quantization operation on each floating-point model training data in the floating-point model training data set; The method comprises: using the transmitting end intelligent network card to send the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving end intelligent network card, and sending the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving end intelligent network card, including: When the amount of model training data to be transmitted exceeds the amount of data that can be transmitted at a single time, the sending-end smart network card uses the data batches in ascending order to send the quantized model training data whose loss calculation results satisfy the preset loss tolerance range to the receiving-end smart network card in batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss calculation results exceed the preset loss tolerance range to the receiving-end smart network card in batches.
2. The data transmission method according to claim 1, characterized in that: Get the maximum and minimum quantization values in the quantization range, including: Determine a target quantization range according to the maximum floating-point model training data and the minimum floating-point model training data; The maximum quantization value and the minimum quantization value in the target quantization range are obtained.
3. The data transmission method according to claim 1, characterized in that: The method comprises: using the transmitting end intelligent network card to send the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving end intelligent network card, and sending the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving end intelligent network card, including: Using the transmitting end intelligent network card, compressing the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to obtain compressed quantized model training data; Using the transmitting end intelligent network card, compressing the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range, to obtain compressed floating-point model training data; The compressed and quantized model training data and the compressed floating-point model training data are sent to the receiving-end intelligent network card.
4. The data transmission method according to claim 3, characterized in that: Dequantizing the quantized model training data using the receiving end intelligent network card includes: Decompressing the compressed and quantized model training data using the receiving-end intelligent network card to obtain decompressed and quantized model training data; The receiving-end intelligent network card is used to dequantize the decompressed quantized model training data.
5. The data transmission method according to claim 3, characterized in that: Upload the restored floating-point model training data and the received original floating-point model training data to the receiving end, including: Decompressing the compressed floating-point model training data using the receiving-end intelligent network card to obtain decompressed floating-point model training data; The restored floating-point model training data and the decompressed floating-point model training data are both uploaded to the receiving end.
6. The data transmission method according to claim 1, characterized in that: The floating-point model training data set to be transmitted is received by using the transmitter smart network card, including: Utilize the transmitting end intelligent network card to receive the data transmission request, and parse the data transmission request to obtain the floating point model training data set and the target address to be transmitted; The receiving-end intelligent network card is determined according to the target address.
7. The data transmission method according to claim 6, characterized in that: Parsing the data transmission request to obtain the floating-point model training data set and the target address to be transmitted includes: Parsing the data transmission request to obtain the floating-point model training data set to be transmitted, the target address and the data batch; Correspondingly, the sending-end intelligent network card sends the quantized model training data whose loss degree calculation result satisfies the preset loss degree allowable range to the receiving-end intelligent network card, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation result exceeds the preset loss degree allowable range to the receiving-end intelligent network card, including: The sending-end smart network card sends the quantized model training data whose loss degree calculation results satisfy the preset loss degree allowable range to the receiving-end smart network card in batches according to the data batches, and sends the floating-point model training data corresponding to the quantized model training data whose loss degree calculation results exceed the preset loss degree allowable range to the receiving-end smart network card in batches.
8. The data transmission method according to claim 1, characterized in that: Dequantizing the quantized model training data using the receiving end intelligent network card includes: The receiving-end intelligent network card is used to perform parallel dequantization on the quantized model training data.
9. A data transmission device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the data transmission method according to any one of claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 8 are implemented.
11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the data transmission method according to any one of claims 1 to 8 are implemented.