Data processing method, device and related equipment

Abstract

The present application provides a data processing method, apparatus and related equipment, the method comprising: a computing device determines a first reference value based on a plurality of first data to be compressed; then calculates the difference between each first data and the first reference value; determines the code corresponding to each difference; and obtains the compressed data of the corresponding first data according to each code. Wherein, the length of part or all of the codes corresponding to each first data is less than the length of the original first data. The above-mentioned reference value is the data that appears more frequently in the plurality of data to be compressed. When the plurality of data to be compressed are respectively subtracted from the reference value, the number of 0s in the plurality of differences obtained is the largest. After the difference is encoded, the data corresponding to the difference can be compressed to a smaller number of bits. Since the bits occupied by the code are smaller than those of each data, data compression can be achieved, the amount of data exchanged between different computing devices can be reduced, and the efficiency of data transmission can be improved.

Description

[0001] This application is a divisional application. The application number of the original application is 202211289095.4, and the original application date is October 20, 2022. The entire content of the original application is incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, and related equipment. Background Art

[0003] With the development of information technology, the demand for application data and multimedia communication services has increased dramatically. Large amounts of data need to be transmitted or exchanged between different devices over the network. For example, in applications such as high-performance computing (HPC), artificial intelligence (AI) model training, and multimedia data transmission, in order to improve computing efficiency and user experience, it is necessary to improve the efficiency of data exchange between different devices and reduce the latency during data transmission. Reducing the amount of data transmitted between different devices is an effective way to improve data exchange efficiency and reduce latency. Therefore, how to reduce the amount of data exchanged between different devices is a technical problem that needs to be solved. Summary of the Invention

[0004] The present application provides a data processing method, apparatus and related equipment, which can reduce the amount of data to be transmitted, improve data exchange efficiency and reduce data transmission delay by compressing the data to be transmitted.

[0005] In a first aspect, the present application provides a data processing method, comprising: determining a first reference value based on a plurality of first data to be compressed by a computing device; then calculating the difference between each first data and the first reference value; determining a code corresponding to each difference; and obtaining compressed data corresponding to each code, wherein the length of some or all of the codes corresponding to each first data is less than the length of the original first data.

[0006] The above-mentioned reference value is the data that appears more frequently among the multiple data to be compressed. After the multiple data to be compressed are respectively subtracted from the reference value and the code corresponding to each data is determined based on the difference, the length of the code corresponding to some or all of the data is less than the length of the original first data, and the number of 0s in the difference between the multiple data and the reference value is the largest. After encoding the difference of 0 in the difference, the data corresponding to the difference of 0 can be compressed to a smaller number of bits; since the bits occupied by the code are smaller than those of each data, data compression can be achieved, the amount of data exchanged between different computing devices can be reduced, and data transmission efficiency can be improved.

[0007] In a possible implementation, the plurality of first data are exponent values ​​of respective floating-point numbers.

[0008] In applications like AI model training, the data types for model weights, activation values, gradients, and other data generated during model training are typically floating point (FP) types, such as double-precision floating point (FP64), single-precision floating point (FP32), and half-precision floating point (FP16). By compressing the exponent values ​​in floating point numbers, the amount of floating point data required for transmission can be reduced, improving data transmission efficiency and reducing data latency. For AI model training, this can reduce the amount of data exchanged during training, reduce data transmission latency, and improve model training efficiency.

[0009] In one possible implementation, determining the code corresponding to each difference value includes determining the code corresponding to each difference value based on each difference value and a coding table, wherein the coding table records codes corresponding to a plurality of preset difference values. The coding table is stored in a computing device, and after calculating the difference between a data value and a reference value, the corresponding code can be determined based on the coding table.

[0010] In one possible implementation, in the above-mentioned coding table, the code corresponding to the difference whose absolute value is greater than the threshold is a fixed value, and the compressed data of the corresponding first data obtained according to each code includes: when the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as the compressed data of the first data corresponding to the first difference.

[0011] Optionally, if multiple first data are the exponent values ​​of multiple floating-point numbers, for any one of the first exponent values, the code corresponding to the first difference between the first exponent value and the reference value is determined according to the coding table; when the absolute value of the first difference is less than or equal to the first threshold, the code corresponding to the first difference is used as the code corresponding to the first exponent value; when the absolute value of the first difference is greater than the first threshold, the code corresponding to the first difference and the first exponent value are used as the code corresponding to the first exponent value; wherein the above-mentioned reference value is an exponent value that appears more times, and when the multiple exponent values ​​to be compressed are respectively subtracted from the reference value, the number of 0s in the multiple differences obtained is the largest, and the 0s in the differences are used as the codes. After the difference is encoded according to the above encoding method, the exponential value corresponding to the difference of 0 can be compressed to a smaller number of bits; in model training, the distribution of the exponential value of the floating-point number is usually close to the normal distribution, and the proportion of the differences whose absolute values ​​are less than the threshold in the above multiple differences is also relatively large. Therefore, the differences whose absolute values ​​are less than or equal to the threshold in the differences are also encoded according to the above encoding method. Since the bits occupied by the encoding are smaller than the bits of each exponent value, the number of differences whose absolute values ​​are less than the first threshold in the differences accounts for a large proportion, so that most of the exponential values ​​can be compressed by the above method, thereby achieving compressed floating-point numbers, reducing the amount of data exchanged between different computing devices, and improving data transmission efficiency.

[0012] In one possible implementation, the above-mentioned determining a first reference value based on multiple first data to be compressed includes: sampling the multiple first data to obtain multiple sampling values; dividing the multiple sampling values ​​into t1 groups, determining the median of multiple sampling values ​​in each of the t1 groups, and obtaining t1 medians; dividing the t1 medians into t2 groups, determining the median of each of the t2 groups, and obtaining t2 medians, and using the median corresponding to the t2 medians as the first reference value corresponding to the multiple first data.

[0013] When the amount of first data is large, sampling the first data can reduce the amount of data processed when determining the reference value, thereby improving the efficiency of determining the reference value.

[0014] In one possible implementation, the above-mentioned multiple floating-point numbers also include mantissa values; the data processing method also includes: the computing device compresses the mantissa values ​​of the multiple floating-point numbers respectively to obtain compressed data of each mantissa value; and obtains compressed data of the multiple floating-point numbers based on the compressed data of the exponent values ​​of the multiple floating-point numbers and the compressed data of the mantissa values ​​of the multiple floating-point numbers.

[0015] The exponent value of the floating point number is also compressed, which can further reduce the amount of data that needs to be transmitted during data transmission and improve the efficiency of data transmission.

[0016] In a possible implementation, the data processing method further includes: generating first metadata, where the first metadata includes a first reference value, and the first reference value is used to decode compressed data of the plurality of first data to obtain the plurality of first data.

[0017] By generating metadata including a reference value, the compressed data and the metadata are sent to the receiving end, so that the computing device receiving the compressed data can decode the received compressed data according to the metadata to obtain the data before compression.

[0018] In one possible implementation, when the multiple first data are respectively the exponent values ​​of multiple floating-point numbers, the first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed.

[0019] When compressing floating-point data, you can choose whether to compress the mantissa value of the floating-point number, and choose the degree of compression of the mantissa value. Therefore, when compressing the floating-point number, the above metadata also includes the compression type and the mantissa compression level, so that the computing device that receives the compressed data can decode the exponent part and the mantissa part of the received compressed data according to the metadata respectively to obtain the floating-point data before compression.

[0020] In one possible implementation, the method further includes: receiving data to be decompressed, the data to be decompressed including an encoded portion and second metadata; wherein the second metadata includes a second reference value, and the encoded portion includes multiple codes; and decoding the multiple codes respectively according to the second reference value and the multiple codes to obtain multiple second data.

[0021] After the computing device receives the data, when there is a reference value in the received data, the computing device can determine that the received data is the data to be decompressed. Through the encoding in the data to be decompressed and the encoding table stored in the computing device, the difference corresponding to each encoding can be determined, and then the original data can be restored based on the difference and the reference value.

[0022] In one possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; the above-mentioned decoding of the data to be decompressed according to the second reference value and multiple codes to obtain multiple second data includes: determining the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; decoding the multiple codes respectively according to the length of the exponent value, the second reference value and the multiple codes to obtain multiple exponent values; when it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the compressed data of the mantissa part is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values; and restoring the floating-point number before compression according to the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0023] In a second aspect, the present application provides a data processing device, comprising a communication unit and a processing unit, wherein the communication unit is configured to obtain a plurality of first data; the processing unit is configured to determine a first reference value based on the plurality of first data to be compressed; then calculate the difference between each first data and the first reference value; determine a code corresponding to each difference; and obtain compressed data corresponding to each code. The length of some or all of the codes corresponding to each first data is less than the length of the exponent value.

[0024] In a possible implementation, the plurality of first data are respectively exponent values ​​of a plurality of floating-point numbers.

[0025] In a possible implementation, the processing unit is specifically configured to determine a code corresponding to each difference value according to each difference value and a coding table, wherein the coding table records codes corresponding to a plurality of preset difference values.

[0026] In one possible implementation, in the above-mentioned coding table, the code corresponding to the difference whose absolute value is greater than the threshold is a fixed value, and the above-mentioned processing unit is specifically used to: when the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as the compressed data of the first data corresponding to the first difference.

[0027] In a possible implementation, when determining a first reference value based on multiple first data to be compressed, the processing unit is specifically used to: sample multiple first data to obtain multiple sampling values; divide the multiple sampling values ​​into t1 groups, determine the median of multiple sampling values ​​in each of the t1 groups, and obtain t1 medians; divide the t1 medians into t2 groups, determine the median of each of the t2 groups, and obtain t2 medians, and use the median corresponding to the t2 medians as the first reference value corresponding to the multiple first data.

[0028] In one possible implementation, the above-mentioned multiple floating-point numbers also include mantissa values; the processing unit is further used to compress the mantissa values ​​of the multiple floating-point numbers respectively to obtain compressed data of each mantissa value; and obtain compressed data of the multiple floating-point numbers based on the compressed data of the exponent values ​​of the multiple floating-point numbers and the compressed data of the mantissa values ​​of the multiple floating-point numbers.

[0029] In a possible implementation, the processing unit is further configured to generate first metadata, where the first metadata includes a first reference value, and the first reference value is used to decode compressed data of the plurality of first data to obtain the plurality of first data.

[0030] In one possible implementation, when the multiple first data are respectively the exponent values ​​of multiple floating-point numbers, the above-mentioned first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed.

[0031] In one possible implementation, the communication unit is further used to: receive data to be decompressed, the data to be decompressed including an encoded part and second metadata; wherein the second metadata includes a second reference value, and the encoded part includes multiple codes; and decode the multiple codes respectively according to the second reference value and the multiple codes to obtain multiple second data.

[0032] In one possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; the above-mentioned processing unit is specifically used to: determine the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; decode the multiple codes respectively according to the length of the exponent value, the second reference value and the multiple codes to obtain multiple exponent values; when it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the compressed data of the mantissa part is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values; and restore the floating-point number before compression according to the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0033] In a third aspect, the present application provides a computing device comprising one or more processors, a memory, and a compression engine, wherein the memory stores computer program instructions, the processor executes the computer program instructions to generate data to be compressed, and the compression engine is used to implement the data processing method described in the first aspect or any possible implementation of the first aspect to compress the data to be compressed.

[0034] In one possible implementation, the computing device also includes a communication engine and a decompression engine. The communication engine is used to receive data to be decompressed, and the decompression engine is used to implement the data processing method described in the first aspect or any possible implementation of the first aspect to decode the received data to be decompressed.

[0035] In a possible implementation, the compression engine and decompression engine are located in the communication engine.

[0036] In a possible implementation, the compression engine is located in each processor, and the decompression engine is located in the communication engine.

[0037] In a possible implementation, the compression engine and decompression engine are modules independent of the processor, memory, and communication engine.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium storing instructions. When the instructions are executed on a server, the server executes the data processing method in the first aspect or any possible implementation of the first aspect.

[0039] In a fifth aspect, the present application provides a computer program product, which, when running on a server, enables the server to execute the data processing method in the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0041] Figure 1 This is a schematic diagram of the architecture of a distributed training cluster provided in an embodiment of the present application;

[0042] Figure 2 This is a flow chart of a data processing method provided in an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a floating-point number provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a computing device provided in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0047] Figure 7 is a schematic diagram of a compression engine provided in an embodiment of the present application;

[0048] Figure 8 is a schematic diagram of another compression engine provided in an embodiment of the present application;

[0049] Figure 9 is a schematic diagram of another compression engine provided in an embodiment of the present application;

[0050] Figure 10 This is a schematic diagram of determining a reference value provided in an embodiment of the present application;

[0051] Figure 11 This is another schematic diagram of determining a reference value provided in an embodiment of the present application;

[0052] Figure 12 is a schematic diagram of a payload provided in an embodiment of the present application;

[0053] Figure 13 is a schematic diagram of another payload provided in an embodiment of the present application;

[0054] Figure 14 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0055] Figure 15 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0056] Figure 16 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0057] Figure 17 is a schematic diagram of a decompression engine provided in an embodiment of the present application;

[0058] Figure 18 is a schematic diagram of a decompression engine provided in an embodiment of the present application;

[0059] Figure 19 It is a schematic diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solution of the data processing method provided by this application will be described below in conjunction with the accompanying drawings.

[0061] With the development of information technology, large amounts of data need to be transmitted or exchanged between different devices over the network. For example, applications such as high-performance computing (HPC), AI model training, and multimedia data transmission involve large amounts of data transmission. By improving the efficiency of data exchange between different computing devices and reducing data transmission latency, data processing efficiency can be improved and the user experience can be enhanced. To address the issue of how to improve the efficiency of data exchange between different computing devices, reducing the amount of data transmitted between different computing devices is an effective way to improve data transmission efficiency and reduce data transmission latency.

[0062] For example, Figure 1 As shown, Figure 1This is an architectural diagram of a distributed training cluster provided in an embodiment of the present application. The distributed training cluster is used to train AI models in a distributed training scenario. The distributed training cluster includes one or more servers, and each server includes one or more processors. When multiple servers are included, the multiple servers are connected through a network. During the training of the AI ​​model, multiple servers participating in the training or different computing devices in the same server can exchange data through aggregated communication methods such as all-to-all, allgather, or allreduce to complete the training of the AI ​​model. As the scale of AI models becomes larger and larger, the amount of data exchanged between different computing devices is also increasing, and the delay required for the data interaction process in model training is also increasing. It should be noted that the above-mentioned computing device can be a server in a cluster or a processor in a server, and the embodiments of the present application do not impose specific restrictions. Therefore, how to reduce the amount of data exchanged during model training is an effective way to reduce the delay of data interaction and improve the efficiency of model training.

[0063] This application provides a data processing method that can compress data, thereby reducing the amount of data that needs to be transmitted during the data interaction process, improving the efficiency of data transmission, and reducing latency. Figure 2 As shown, Figure 2 1 is a flow chart of a data processing method provided in an embodiment of the present application. The method includes the following steps S201 to S203.

[0064] S201: Determine a first reference value according to a plurality of first data to be compressed.

[0065] The plurality of first data to be compressed are data that the computing device needs to transmit. Before transmitting the plurality of first data, the computing device needs to compress the plurality of first data to reduce the amount of data during the transmission process.

[0066] Before compressing the multiple first data, the computing device first determines a first reference value based on the multiple first data. When determining the first reference value, the computing device may count the first data that appears the most times among the multiple first data, and use the first data that appears the most times as the first reference value. The computing device may also divide the multiple first data into t1 groups, determine the median in each group, and obtain t1 medians; then divide the t1 medians into t2 groups, and then determine the median in each of the t2 groups, and obtain t2 medians. This process continues in this manner until the multiple medians obtained ultimately cannot be further grouped. For example, if each group contains fewer than three values ​​after grouping, the median corresponding to the multiple medians is used as the reference value corresponding to the set. The computing device may also first sample the multiple first data to obtain multiple sampled values, then divide the sampled values ​​into t1 groups, and then determine the first reference value according to the above method.

[0067] For example, the first set includes 100 exponent values. The exponent encoding module 320 samples these 100 exponent values ​​to obtain 27 exponent values. The exponent encoding module 320 first groups the 27 exponent values ​​into groups of three, obtaining nine groups of data, and determines the median of each group of data to obtain nine medians. The module then further groups the nine medians into groups of three, obtaining three groups of data, and determines the median of each group of data to obtain three medians. Finally, the median of these three medians is used as the reference value corresponding to the first set.

[0068] S202: Determine the difference between each first data and a first reference value.

[0069] After determining the first reference value, the computing device calculates the difference between each of the plurality of first data and the first reference value. It should be understood that the difference between the plurality of first data and the first reference value includes positive numbers, 0, and negative numbers.

[0070] S203: Determine the code corresponding to each difference value, and obtain compressed data of the corresponding first data according to each code.

[0071] A coding table is stored in the computing device, which records the codes corresponding to multiple preset difference values. After determining the difference between each first data and the first reference value, the computing device searches for the code corresponding to each difference value according to the coding table. The code corresponding to each difference value is the code corresponding to the first data corresponding to the difference, and the code corresponding to each first data is the data after each first data is compressed.

[0072] It should be noted that the correspondence between the difference and the encoding in this application and the form of the compressed data corresponding to each first data will be introduced in detail later and will not be elaborated here.

[0073] In an embodiment of the present application, after compressing the plurality of first data, the computing device sends the compressed data of each first data to other computing devices, which then need to decode the compressed data. When compressing the plurality of first data, the computing device generates corresponding first metadata, which includes the first reference value. The computing device also sends the metadata to the other computing devices, so that the other computing devices decode the compressed data based on the compressed data of each first data and the first metadata to obtain the plurality of first data.

[0074] In a possible embodiment, the first data may be an exponent value in a floating-point number. A floating-point number is a number represented by scientific notation. A floating-point number includes a sign bit, a mantissa, a base, and an exponent. A floating-point number can be represented in the following form:

[0075] V=(-1) s *M*R E

[0076] Among them, S represents the sign bit, which takes a value of 0 or 1, 0 represents a positive floating point number, and 1 represents a negative floating point number; M represents the mantissa; R represents the base, and the value of R in binary numbers is 2; E represents the exponent.

[0077] In a computer, when a floating-point number is used to represent a number, it is only necessary to determine the sign bit, exponent, and mantissa variables mentioned above. The lengths of the exponent and mantissa are different in floating-point numbers of different precisions. For example, FP64 uses 64 bits (bits) to store a floating-point number, and the 64 bits include a 1-bit sign bit, an 11-bit exponent, and a 52-bit mantissa; FP32 uses 32 bits to store a floating-point number, including a 1-bit sign bit, an 8-bit exponent, and a 23-bit mantissa; FP16 uses 16 bits to store a floating-point number, including a 1-bit sign bit, a 5-bit exponent, and a 10-bit mantissa. Figure 3 As shown, Figure 3 is a schematic diagram of a floating-point number provided in an embodiment of the present application. Figure 3 The figure shows an example of the storage diagram of FP32 data. In the 32-bit data, the 1st to 23rd bits store the mantissa, the 24th to 31st bits store the exponent, and the 32nd bit stores the sign bit.

[0078] In applications like AI model training, the data types for model weights, activation values, gradients, and other data generated during model training are typically floating point (FP) types, such as double-precision floating point (FP64), single-precision floating point (FP32), and half-precision floating point (FP16). By compressing the exponent values ​​in floating point numbers, the amount of floating point data required for transmission can be reduced, improving data transmission efficiency and reducing data latency. For AI model training, this can reduce the amount of data exchanged during training, reduce data transmission latency, and improve model training efficiency.

[0079] In one possible implementation, if the first data is an exponent value in a floating-point number, that is, the computing device compresses the floating-point number, and each floating-point number also includes a mantissa value; the computing device can also compress the mantissa values ​​of multiple floating-point numbers separately to obtain compressed data of each mantissa value; and obtain compressed data of multiple floating-point numbers based on the compressed data of the exponent values ​​of multiple floating-point numbers and the compressed data of the mantissa values ​​of multiple floating-point numbers.

[0080] It should be understood that when the plurality of first data are exponent values ​​of a plurality of floating-point numbers, the first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the plurality of floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the compressed mantissa values ​​when the mantissa values ​​are compressed. When compressing the floating-point data, the user can select whether to compress the mantissa values ​​of the floating-point numbers and the degree of compression of the mantissa values. Therefore, when compressing the floating-point numbers, the first metadata also includes the compression type and mantissa compression level, so that a computing device that receives the compressed data can decode the exponent portion and the mantissa portion of the received compressed data according to the first metadata to obtain the floating-point data before compression.

[0081] In one possible implementation, the computing device can also receive compressed data sent by another computing device, that is, receive data to be decompressed, the data to be decompressed including an encoded portion and second metadata; wherein the second metadata includes a second reference value, and the encoded portion includes multiple codes. The computing device can decode the multiple codes based on the second reference value and the multiple codes to obtain multiple second data. For example, the computing device can determine a difference value corresponding to a code and a code table stored in the computing device, and then add the difference value to the second reference value to obtain the original data corresponding to the code.

[0082] In one possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa portion, and the second metadata also includes the data type, compression type, and mantissa compression level of the compressed floating-point number. The compression type is used to indicate whether the mantissa value of the floating-point number is compressed; the mantissa compression level is used to indicate the length of the compressed mantissa value after the mantissa value is compressed. After receiving the data to be decompressed after the floating-point number is compressed, the computing device determines the length of the exponent value and the length of the mantissa value of the floating-point number before compression based on the data type; decodes the multiple codes based on the length of the exponent value, the second reference value, and the multiple codes to obtain multiple exponent values. When it is determined based on the compression type that the mantissa portion in the data to be decompressed is data after the mantissa value is compressed, the data after the mantissa portion is compressed is decoded based on the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values. For example, if the original mantissa value is 10 bits long and the mantissa compression level indicates that the compressed mantissa value is 6 bits long, the computing device will add 4 bits to each compressed mantissa value to obtain a 10-bit mantissa value. Finally, the computing device recovers the original floating-point number based on the decoded exponent values ​​and mantissa values.

[0083] The data processing method provided in this application is described in detail below, taking the case where the first data is the exponential value of a floating-point number, i.e., compressing and decoding the floating-point number, as an example, and combining the structure of the computing device provided in this application.

[0084] The embodiment of the present application provides a computing device, which includes a compression engine. The compression engine can compress floating point numbers, thereby reducing the amount of data exchanged between computing devices during data exchange, improving data transmission efficiency, and reducing data transmission delay. Figure 4 As shown, Figure 4: This is a schematic diagram of a computing device provided by an embodiment of the present application. The computing device 10 includes one or more processors 100, a communication engine 200, a compression engine 300 and a memory 400, and the compression engine 300 is located in the communication engine 200; wherein, only one processor is shown in the figure, and each processor 100 includes one or more cores; the processor 100, the communication engine 200 and the memory 400 are connected through an on-chip bus (not shown in the figure) and exchange data through the on-chip bus. The multiple cores of the computing device 10 respectively calculate a certain operator in the AI ​​model through parallel computing, and each generates a slice of the original tensor corresponding to the operator. Each core writes the generated slice into the memory 400 to obtain the complete original tensor corresponding to the above operator. When the computing device calls the communication engine 200 to transmit data between computing devices, the communication engine 200, after obtaining the data in the memory 400, calls the compression engine 300 to compress the data to obtain compressed floating-point data. The compression engine 300 sends the compressed floating point data to the communication engine 200, which then encapsulates the compressed floating point data and sends it to other computing devices. The structure of the compression engine 300 and the method by which the compression engine 300 compresses floating point numbers will be described in detail later and will not be repeated here.

[0085] like Figure 5 As shown, Figure 5 This is a schematic diagram of another computing device provided by an embodiment of the present application. In the computing device 20, the compression engine 300 is deployed in the core. After a core generates a slice of the original tensor, the core calls the compression engine 300 to compress the data to obtain compressed floating-point data. When each core writes the compressed floating-point data into the memory 400, it will be stored in the storage space corresponding to multiple discontinuous addresses, that is, the original tensor is a plurality of discontinuous data fragments in the memory, and the memory 400 records the address information of the floating-point data written by each core through a scatter gather list (SGL). The communication engine 200 reads the SGL, reads the floating-point data in the memory 400 according to the address information in the SGL, encapsulates it, and sends it to other computing devices. The structure of the compression engine 300 and the method for compressing floating-point numbers by the compression engine 300 are described in detail later and will not be repeated here.

[0086] like Figure 6 As shown, Figure 6: This is a schematic diagram of another computing device provided by an embodiment of the present application. In the computing device 30, the compression engine 300 is deployed in the computing device 30 and is a module in the computing device 30 that is independent of the processor 100, the communication engine 200, and the memory 400. One or more processors 100, the communication engine 200, the compression engine 300, and the memory 400 are connected via an on-chip bus (not shown in the figure) and exchange data via the on-chip bus. The multiple cores of the computing device 10 each calculate an operator in the AI ​​model and each generates a slice of the original tensor corresponding to the operator. Each core writes the generated slice into the memory 400 to obtain the complete original tensor corresponding to the above operator. The computing device 30 calls the compression engine 300 to compress the floating-point numbers in the original tensor to obtain compressed floating-point data, and writes the compressed floating-point data into the memory 400 again. When the computing device 30 calls the communication engine 200 for data transmission between computing devices, the communication engine 200 obtains the compressed floating-point data from the memory 400, encapsulates the compressed floating-point data, and sends it to other computing devices. The structure of the compression engine 300 and the method for compressing floating-point numbers by the compression engine 300 will be described in detail later and will not be repeated here.

[0087] It should be noted that the raw tensor described above refers to data generated by a computing device that needs to be sent to other computing devices. The compression engine 300 obtains data in a data packet at a time. Due to data packet limitations, this data packet includes a portion of the data in the raw tensor. For example, if the raw tensor is a 1-megabyte floating-point number, the data packet only includes 512 bytes of floating-point numbers. After obtaining the floating-point numbers in the data packet, the compression engine 300 compresses the multiple floating-point numbers included in the data packet to obtain compressed floating-point data.

[0088] It should be noted that the structure of the above-mentioned computing device is only an example and cannot be understood as a specific limitation. The computing device can also have other structures, which are not specifically limited in the embodiments of the present application. The above-mentioned communication engine 200 can interact with other computing devices through technologies such as direct memory access (DMA), remote direct memory access (RDMA), RDMA over converged ethernet (RoCE) and system direct memory access (SDMA). The above-mentioned other computing devices may be other processors in the same server or processors in other servers, which are not specifically limited in the embodiments of the present application.

[0089] The structure of the compression engine 300 and the method by which the compression engine 300 compresses floating-point data are introduced below.

[0090] like Figure 7 As shown, Figure 7 Schematic diagram of a compression engine provided by an embodiment of the present application. Compression engine 300 can compress floating-point numbers. When computing devices need to exchange floating-point data with each other, it can reduce the amount of data exchanged between computing devices and improve data transmission efficiency. For example, when using a distributed cluster to train an AI model, it can reduce the amount of data exchanged between computing devices used for training, thereby increasing model training speed. Compression engine 300 includes an extraction module 310, an exponent encoding module 320, and a data merging module 330. Extraction module 310 is used to obtain multiple floating-point numbers, extract the exponents of each floating-point number from the multiple floating-point numbers, and obtain the exponent values ​​corresponding to the exponents of each floating-point number. Each floating-point number corresponds to an exponent, and each exponent has a corresponding exponent value, that is, each floating-point number corresponds to an exponent value; the exponent values ​​corresponding to different floating-point numbers may be the same or different. For example, if compression engine 300 obtains 100 floating-point numbers and extracts 100 exponent values ​​corresponding to these 100 floating-point numbers, some of these 100 exponent values ​​may be the same. For example, the exponent value corresponding to three floating-point numbers is 4, and the exponent value corresponding to four floating-point numbers is 6.

[0091] The exponent encoding module 320 is used to encode the exponent values ​​of each floating-point number extracted by the extraction module 310, respectively, to obtain the codes corresponding to each exponent value. Among them, in the codes corresponding to each exponent value, the length of the codes corresponding to some or all of the exponent values ​​is less than the length of the exponent value, thereby achieving compression of the floating-point number. For example, for the 11-bit exponent in the above-mentioned FP64 floating-point number, the length of the codes corresponding to some exponent values ​​is less than 11 bits. The data merging module 330 is used to merge the codes corresponding to each exponent value and the mantissa corresponding to each floating-point number to obtain compressed floating-point data. It should be understood that each floating-point number also includes a sign bit, and the above-mentioned data merging module 330 is used to merge the codes corresponding to the exponent values ​​of each floating-point number, the mantissa corresponding to each floating-point number and the sign bit of each floating-point number.

[0092] In the embodiment of the present application, after the extraction module 310 extracts the exponent values ​​of each floating-point number, the exponent encoding module 320 first determines the reference value corresponding to each exponent value, and then encodes each exponent value based on each exponent value and the reference value corresponding to each exponent value to obtain a code corresponding to each exponent value. It should be noted that the method for determining the reference value based on each exponent value and the method for encoding the exponent value based on the exponent value and the reference value will be described later and will not be repeated here.

[0093] In one possible implementation, Figure 8 As shown, Figure 8 It is a schematic diagram of another compression engine provided by an embodiment of the present application. The above-mentioned compression engine 300 also includes a metadata generation module 340, which is used to generate metadata. The metadata includes the data types of the above-mentioned multiple floating-point numbers and the reference values ​​corresponding to each exponent value. Among them, the data types include types corresponding to floating-point numbers such as FP128, FP64, FP32 or FP16; the reference values ​​in the metadata are used for the computing device that receives the compressed floating-point data to decode the floating-point data, so as to restore the respective exponent values ​​according to the codes and reference values ​​corresponding to the respective exponent values. The data merging module 330 is used to merge the codes corresponding to the above-mentioned exponent values, the mantissas of the respective floating-point numbers and the metadata to obtain compressed floating-point data.

[0094] In one possible implementation, Figure 9 As shown, Figure 9 : is a schematic diagram of another compression engine provided by an embodiment of the present application. The compression engine 300 also includes a mantissa compression module 350, which is used to compress the mantissa values ​​corresponding to the mantissas of each floating-point number to obtain compressed values ​​corresponding to each mantissa value. When the compression engine 300 includes the mantissa compression module 350, the above-mentioned extraction module 310 is also used to extract the mantissas of each floating-point number from the above-mentioned multiple floating-point numbers to obtain multiple mantissa values. Among them, each floating-point number corresponds to a mantissa, and one mantissa corresponds to one mantissa value, that is, one floating-point number corresponds to one mantissa value. When compressing the mantissa, the mantissa compression module 350 can achieve compression of the mantissa by truncating the mantissa. For the mantissa of any type of floating-point data, the mantissa compression module 350 can intercept a part of a preset length or a preset proportion in the mantissa value as the compressed value corresponding to the mantissa value. For example, the mantissa compression module 350 intercepts the first half of the characters in the mantissa value as a compressed value corresponding to the mantissa value, or the mantissa compression module 350 intercepts the first 5 bits of the mantissa value as the compressed value corresponding to each mantissa value. For example, for FP32 floating-point numbers, the mantissa portion includes 23 bits, and the first 12 bits of each mantissa value can be intercepted as the compressed value corresponding to the mantissa value; for FP16 floating-point numbers, the mantissa portion includes 10 bits, and the first 5 bits of each mantissa value can be intercepted as the compressed value corresponding to the mantissa value.

[0095] Optionally, the mantissa compression module 350 can compress the mantissa according to the configured mantissa compression level. The mantissa compression level is used to indicate the compression engine's compression of the mantissa and the length of the compressed value obtained after compressing the mantissa value. Different levels correspond to different lengths of the compressed value after compression. For example, the mantissa compression level includes four levels, 0 to 3, where level 0 means that the mantissa is not compressed, level 1 means that the first 60% of the data in the mantissa value is retained; level 2 means that the first 50% of the data in the mantissa value is retained; and level 3 means that the first 40% of the data in the mantissa value is retained. For FP16 floating-point data, the mantissa actually includes 10 bits. Level 0 means that the mantissa is not compressed, level 1 means that the compressed value length obtained by compressing the mantissa value is 6 bits, level 2 means that the compressed value length obtained by compressing the mantissa value is 5 bits, and level 3 means that the compressed value length obtained by compressing the mantissa value is 4 bits. For example, if the mantissa compression level is 3, and the mantissa value corresponding to an FP16 floating-point number is 0011101011, then the compressed value corresponding to the mantissa value after compression is 0011.

[0096] It should be understood that when the compression engine 300 includes the mantissa compression module 350, the metadata generated by the metadata generation module 340 also includes a compression type and a mantissa compression level. As shown in Table 1 below, the metadata may include the fields shown in Table 1. The compression type includes four types: no compression of floating-point numbers, exponent compression only, mantissa compression only, and both exponent and mantissa compression. The mantissa compression level indicates the length of the compressed value obtained after the mantissa value is compressed.

[0097] The data merging module 330 is configured to merge the codes corresponding to the exponent values, the compressed values ​​corresponding to the compressed mantissa values, and the metadata to obtain compressed floating-point data. The metadata is used by a computing device that receives the compressed floating-point data to recover the floating-point number based on the various fields included in the metadata. The method by which the computing device recovers the floating-point number based on the metadata is described later and is not further elaborated here.

[0098] Table 1

[0099]

[0100] It should be understood that the above-mentioned compression type and mantissa compression level can be configured by the user according to needs. The user can configure the compression engine 300 to compress only the exponent of the floating-point number, or to compress both the exponent and the mantissa of the floating-point number. When configuring the compression of the mantissa, it is also necessary to configure the mantissa compression level so that the compression engine 300 can determine the length of the mantissa after compression. For example, when training an AI model, when a shorter training time is required, the compression type can be set to compressed exponent and mantissa, and a higher mantissa compression level can be set to make the length of the compressed value after the mantissa is compressed shorter, thereby minimizing the amount of data that needs to be transmitted and reducing the delay caused by communication between computing devices.

[0101] The following describes a method in which the index encoding module 320 determines the reference value corresponding to each index value.

[0102] When determining the reference value based on the exponent values, the exponent encoding module 320 divides the multiple exponent values ​​into n sets. Then, based on the different exponent values ​​in each set, a reference value corresponding to the multiple exponent values ​​in each set is determined, resulting in m reference values. Here, n is a positive integer greater than or equal to 1, and m is a positive integer less than or equal to n.

[0103] After the multiple index values ​​are divided into n sets, when determining the reference values ​​corresponding to the multiple index values ​​in each set, the reference value corresponding to each set may be determined in the following three ways.

[0104] The first way, such as Figure 10 As shown, Figure 10 This is a schematic diagram of determining a reference value provided by an embodiment of the present application. For multiple index values ​​in each set, the index encoding module 320 determines a reference value through the multiple index values ​​in each set, that is, each set in the n sets corresponds to a reference value E bi , where i is a positive integer less than or equal to n. For the first set of n sets, the index encoding module 320 counts the index value that appears the most times in the first set, and uses the index value that appears the most times as the reference value E corresponding to the K index values ​​in the first set. b1 Alternatively, the exponential encoding module 320 samples the K exponential values ​​in the first set to obtain a plurality of sampled values; then determines the three sampled values ​​that appear the most times among the plurality of sampled values, and uses the average of the three sampled values ​​that appear the most times as the reference value E corresponding to the K exponential values ​​in the first set. b1 It should be noted that the first set is one of the n sets, and for any one of the n sets, the reference values ​​corresponding to the multiple index values ​​in each set are determined by the above method.

[0105] The second method is to determine the reference value corresponding to each set. For the first set of n sets, first sample the K exponential values ​​in the first set to obtain K1 sample values. For these K1 sample values, first divide the K1 sample values ​​into t1 groups, determine the median of the sample values ​​in each group, and obtain t1 medians; then divide the t1 medians into t2 groups, and then determine the median of each of the t2 groups to obtain t2 medians. This process is repeated until the multiple medians obtained can no longer be grouped. For example, if each group contains fewer than three values ​​after grouping, the median of these multiple medians is used as the reference value corresponding to the set.

[0106] For example, the first set includes 100 exponent values. The exponent encoding module 320 samples these 100 exponent values ​​to obtain 27 exponent values. The exponent encoding module 320 first groups the 27 exponent values ​​into groups of three, obtaining nine groups of data, and determines the median of each group of data to obtain nine medians. The module then further groups the nine medians into groups of three, obtaining three groups of data, and determines the median of each group of data to obtain three medians. Finally, the median of these three medians is used as the reference value corresponding to the first set.

[0107] The third way, such as Figure 11 As shown, Figure 11 This is another schematic diagram of determining a reference value provided by an embodiment of the present application. For multiple index values ​​in each set, the index encoding module 320 uses the reference value calculated based on the multiple index values ​​in the previous set as the reference value corresponding to the multiple index values ​​in the next set. That is, based on the multiple index values ​​in the i-th set, a reference value E is calculated. bi Then, the reference value E bi As the reference value corresponding to the multiple exponent values ​​in the i+1th set, when encoding the multiple exponent values ​​in the i+1th set, based on the reference value E bi For encoding. For the first set among the n sets, since there is no corresponding benchmark value, the benchmark value corresponding to the set can be considered to be 0; for the last set among the n sets, the benchmark value is not calculated based on the multiple index values ​​in the set. It should be noted that when determining a benchmark value based on multiple index values ​​in a set, the method in the first or second manner described above can be used. Among them, the method for determining a benchmark value based on multiple index values ​​in each set can refer to the method for determining the benchmark value in the first manner described above.

[0108] It should be understood that when n is equal to 1, that is, the index encoding module 320 does not divide the multiple index values ​​into multiple sets, the index encoding module 320 can only determine the reference values ​​corresponding to the multiple index values ​​through the first or second method mentioned above.

[0109] It should be noted that when the index encoding module 320 determines the reference value corresponding to each set according to any of the above methods, when sampling, it may not sample the value with an exponent value of 0.

[0110] The following describes the operation process of encoding the exponent value by the exponent encoding module 320. After the exponent encoding module 320 determines the reference value corresponding to each exponent value, it can encode each exponent value using the following three methods based on each exponent value and the reference value corresponding to each exponent value.

[0111] In the first method, the exponent coding module 320 encodes the exponent value according to the difference between the exponent value and the reference value corresponding to the exponent value. The exponent coding module 320 encodes the difference and then uses the code corresponding to the difference as the code corresponding to the exponent value. As shown in Table 2 below, Table 2 is a coding table provided in an embodiment of the present application, which includes codes corresponding to different differences between the exponent value and the reference value. If the absolute value of the difference between an exponent value and the reference value corresponding to the exponent value is less than a first threshold value (the first threshold value in Table 2 is 4), the code corresponding to the difference is determined according to the coding table, and the code of the difference is the code corresponding to the exponent value; if the absolute value of the difference between an exponent value and the reference value corresponding to the exponent value is greater than or equal to the first threshold value, the code corresponding to the difference is determined according to the coding table, and the code corresponding to the exponent value includes the code corresponding to the difference and the actual value of the exponent value, that is, the code corresponding to the exponent value is a combination of the code corresponding to the difference and the actual value of the exponent value.

[0112] It should be noted that the encoding method for encoding the difference can also be other methods. The correspondence between the difference and the encoding shown in Table 2 is only an example, and the difference and the encoding can also have other correspondences; in addition, for differences whose absolute values ​​are greater than or equal to the first threshold, they are collectively referred to as "other differences" in Table 2, and the encoding corresponding to all other differences is the same; the above-mentioned first threshold is only used as an example and cannot be understood as a specific limitation. The first threshold can also be other values, such as 3, 5 or 6, etc., which are not specifically limited in the embodiments of the present application.

[0113] Table 2

[0114]

[0115]

[0116] When the index encoding module 320 encodes the index value in each set and the reference value corresponding to the index value in each set, for the first set of the n sets, the index encoding module 320 determines the index value E corresponding to each index value in the first set and the reference value E corresponding to the first set. b1, and then, based on the difference between each index value and the reference value, in combination with Table 2, encode each index value in the first set to obtain a code corresponding to each index value in the first set. It should be understood that the first set is any one of the n sets, and for each of the n sets, the index encoding module 320 performs the above-mentioned operation on the first set to obtain a code corresponding to the index value in each set.

[0117] For example, if it is necessary to compress an FP32 floating-point number, the exponent value includes 8 bits and the reference value is 3. For an exponent with an exponent value of 2, the difference between the exponent value and the reference value is -1, the absolute value of the difference is less than the first threshold value of 4, and the code corresponding to the difference is determined to be 10 according to the coding table. In this case, the code corresponding to the exponent value of 2 is 10. For an exponent with an exponent value of 6, the difference between the exponent value and the reference value is 3, and the code corresponding to the exponent value of 6 is 11110. For an exponent with an exponent value of 8, it is represented in binary as 00001000, the difference between the exponent value and the reference value is 5, the absolute value of the difference is greater than the first threshold value of 4, the code corresponding to the difference is 111111, and the code corresponding to the exponent value of 8 is 11111100001000.

[0118] Through the above compression method, since the reference value is an exponential value that appears more frequently in a set, when multiple exponential values ​​in a set are respectively subtracted from the reference value, the number of 0s in the multiple differences obtained is the largest. After encoding the differences that are 0 according to the above encoding method, the exponential value corresponding to the difference of 0 can be compressed to 2 bits; and because multiple exponential values ​​usually satisfy a normal distribution, the proportion of differences whose absolute values ​​are less than the first threshold in the above multiple differences is also relatively large. Therefore, the differences whose absolute values ​​are less than the first threshold in the differences are also encoded according to the above encoding method. Since the bits occupied by the encoding are smaller than those of each exponent value, the number of differences whose absolute values ​​are less than the first threshold in the differences accounts for a relatively large proportion, so that most of the exponential values ​​can be compressed through the above method, thereby achieving compressed floating-point numbers, reducing the amount of data exchanged between different computing devices, and improving data transmission efficiency.

[0119] The second method is that when the exponent coding module 320 determines the reference value corresponding to each set, if when sampling the exponent values ​​in each set, the exponent with an exponent value of 0 is not sampled, when encoding the exponent value, the exponent coding module 320 encodes the absolute value of the difference between the exponent value and the reference value for non-zero exponent values, and adds a sign bit to indicate the sign of the difference when encoding; for the exponent with an exponent value of 0, it is directly encoded as 0. Specifically, as shown in Table 3 below, Table 3 is a coding table provided in an embodiment of the present application. In Table 3, for the two cases where the difference is 0 and the exponent value is 0, the corresponding code is 0, and the sign bit is used to distinguish whether the code 0 indicates that the difference is 0 or the exponent value is 0. When the symbol position is 1 and the code is 0, it indicates a difference of 0; when the symbol position is 0 and the code is 0, it indicates an exponent value of 0. If the absolute value of the difference between an index value and the reference value corresponding to the index value is less than the second threshold value (the second threshold value is 7 in Table 3), the code corresponding to the index value is the code of the corresponding difference; if the absolute value of the difference between an index value and the reference value corresponding to the index value is greater than or equal to the second threshold value, the code corresponding to the difference is determined according to the coding table, and the code corresponding to the index value includes the code corresponding to the difference and the actual value of the index value, that is, the code corresponding to the index value is a combination of the code corresponding to the difference and the actual value of the index value.

[0120] For example, if the base value is 2, a sign bit of 0 indicates that the difference between the exponent and the base value is positive, while a sign bit of 1 indicates that the difference between the exponent and the base value is negative. For an exponent with an exponent value of 2, that is, a difference of 0, the code corresponding to the difference is 0, and the sign bit corresponding to the code is assigned to 1; for an exponent with an exponent value of 0, the code corresponding to the exponent value is 0, and the sign bit corresponding to the code is assigned to 0. For an exponent with an exponent value of 4 or -4, the absolute value of the difference is 2. The code corresponding to the exponent value of 4 or -4 is 110, but the sign bit corresponding to the code of the exponent value of 4 is assigned to 0; the sign bit corresponding to the code of the exponent value of -4 is assigned to 1. For an exponent with an exponent value of 9, the absolute value of the difference is 7, and the code corresponding to the difference is 11111110. The exponent value of 9 is represented in binary as 00001001, so the code corresponding to the exponent value of 9 is 1111111000001001, and the sign bit corresponding to the code is assigned to 0.

[0121] Table 3

[0122]

[0123]

[0124] It should be noted that the encoding method for encoding the absolute value of the difference can also be other methods. The correspondence between the absolute value of the difference and the code shown in Table 3 is only an example, and other correspondences between the difference and the code can also exist. In addition, for differences whose absolute value is greater than or equal to the second threshold, they are collectively referred to as "other differences" in Table 3, and the codes corresponding to the absolute values ​​of all other differences are the same. It should be understood that the above-mentioned second threshold is for example only and should not be understood as a specific limitation. The second threshold can also be other values, such as 3, 5, or 6, and is not specifically limited in the embodiments of this application.

[0125] In scenarios such as AI training, the data that needs to be transmitted between computing devices is usually sparse, that is, the data to be transmitted usually includes a large number of zero values. By compressing the exponential value of 0 into a one-bit code, the amount of data that needs to be transmitted can be greatly reduced. At the same time, a sign bit is used to indicate the sign of the difference between the exponential value and the reference value, so that differences with the same absolute value can use the same code, so that more differences can be compressed through the above code, that is, more exponential values ​​can be compressed, thereby better reducing the amount of compressed floating-point data.

[0126] In a third method, after determining the reference values ​​corresponding to each of the above-mentioned sets, the exponent coding module 320, for any set, such as the above-mentioned first set, adds integers whose absolute values ​​are less than a third threshold to the reference values ​​corresponding to the first set, thereby obtaining multiple numerical values. For example, if the third threshold is 4, the reference value is added with ±1, ±2, and ±3, etc. These multiple numerical values ​​are then encoded to obtain multiple codes. When encoding the multiple exponent values ​​in the first set, the multiple exponent values ​​in the first set are matched with the above-mentioned multiple numerical values. If an exponent value is the same as one of the numerical values, the code corresponding to the value is used as the code corresponding to the exponent value. If an exponent value does not match any of the above-mentioned multiple numerical values, the code corresponding to the exponent value includes the codes corresponding to the other numerical values ​​and the exponent value, where the other numerical values ​​refer to numerical values ​​other than the above-mentioned multiple numerical values. It should be understood that the above-mentioned third threshold is used as an example only and should not be understood as a specific limitation.

[0127] As shown in Table 4, Table 4 is a coding table between multiple numerical values ​​and codes. In Table 4, the reference value is 6, that is, when the reference value corresponding to the multiple exponent values ​​in the first set is 6, the exponent coding module can generate the coding table shown in Table 4. When an exponent value in the first set is 6, the code corresponding to the exponent value is 00. If an exponent value in the first set is 8, the code corresponding to the exponent is 110. If an exponent value is 10, and the exponent value occupies 8 bits for the above-mentioned multiple floating-point FP32 type data, then the exponent value 10 is represented in binary as 00001010, and the code corresponding to the exponent value 10 is 11111100001010.

[0128] Table 4

[0129] Numerical coding 6 00 7 01 5 10 8 110 4 1110 9 11110 3 111110 Other values 111111

[0130] It should be noted that when the above-mentioned multiple exponents are divided into multiple sets, each set will generate a coding table between the numerical value and the code based on the reference value and the third threshold value after determining a reference value, and when the reference values ​​corresponding to the two sets are different, the mapping relationship between the numerical value and the code is also different. For example, if the corresponding reference value in the second set is 4, the corresponding coding table may be as shown in Table 5 below. When an exponent value in the second set is 4, the code corresponding to the exponent value is 00, and when an exponent value in the second set is 6, the code corresponding to the exponent value is 110. If an exponent value is 8, and the above-mentioned multiple floating-point numbers are FP32 type data, the exponent value 8 is represented in binary as 00001000, then the code corresponding to the exponent value 8 is 11111100001000.

[0131] Table 5

[0132] Numerical coding 4 00 5 01 3 10 6 110 2 1110 7 11110 1 111110 Other values 111111

[0133] As shown in Tables 4 and 5 above, for different sets, when the reference values ​​corresponding to the two sets are different, the codes corresponding to different exponent values ​​are different, and the same code represents different exponent values. In the embodiment of the present application, the code corresponding to the reference value is used as the reference code, and the distance between each code and the reference code is defined, for example, the reference code is 00. The distance between a code and code 00 indicates the difference between the numerical value corresponding to the code and the reference value corresponding to code 00. For example, the distance between code 01 and code 00 is defined as 1, and the distance between code 10 and code 00 is -1; the distance between code 110 and code 00 is 2, and the distance between code 1110 and code 00 is -2; the distance between code 11110 and code 00 is 3, and the distance between code 111110 and code 00 is -3. Through the above definition, when recovering the corresponding exponent value according to the code, first determine the distance between the code and 00, and then add the distance to the reference value to obtain the corresponding numerical value, which is the exponent value corresponding to the code.

[0134] It should be noted that the embodiment of the present application does not specifically limit the number of exponent values ​​in each of the above sets. Preferably, the number of exponent values ​​in each set is the number of data that can be processed by the compression engine in one clock cycle. For example, if each data packet acquired by the compression engine includes 512 bytes of FP16 floating-point numbers, that is, 256 floating-point numbers, and the compression engine can process 128 bytes of data in one clock cycle, that is, 64 FP16 floating-point numbers, then the floating-point numbers in the data packet can be divided into 4 sets, with each set including 64 floating-point numbers.

[0135] Through the above compression method, since the reference value is an exponential value that appears more frequently in a set, and since multiple exponential values ​​generally satisfy a normal distribution, when the reference value corresponding to a set is added to integers whose absolute values ​​are less than the second threshold to obtain multiple values, these multiple values ​​are the same as the exponential values ​​that account for a larger proportion in the set. Then, encoding these multiple values ​​is equivalent to encoding a portion of the exponential values ​​that account for a larger proportion in the set. Since the length of the code is smaller than each exponential value, a portion of the exponential values ​​that account for a larger proportion in the set can be compressed after being represented by a shorter-length code. Therefore, most of the exponential values ​​can be compressed in the above manner to achieve compressed floating-point numbers, reduce the amount of data exchanged between different computing devices, and improve data transmission efficiency. For example, reducing the amount of data exchanged between different computing devices during AI model training improves model training efficiency.

[0136] In an embodiment of the present application, after the compression engine 300 compresses multiple floating-point numbers, the computing device needs to send the floating-point data including metadata obtained after compression to the communication engine 200, or write the floating-point data into the memory 400, and the communication engine 200 obtains the floating-point data corresponding to these multiple floating-point numbers from the memory 400, and then the communication engine 200 encapsulates the floating-point data into a data packet and sends it to other computing devices. When the communication engine 200 sends a data packet, the length of the payload in a data packet is usually fixed. For example, the payload of an Ethernet data packet is 1500 bytes. In an embodiment of the present application, the payload of a data packet is 512 bytes as an example. If a data packet obtained by the compression engine 300 includes 512 bytes of FP16 floating-point numbers, that is, it includes 256 floating-point numbers. After compressing the 256 floating-point numbers in this data packet, the compression engine 300 obtains a set of corresponding floating-point data. After the communication engine 200 obtains this set of compressed floating-point data, since the floating-point data is less than 512 bytes, for example, the size of a 512-byte floating-point number after compression is 394 bytes, the communication engine 200 will fill the 394 bytes to 512 bytes when encapsulating, and then send it out after encapsulation.

[0137] In one possible implementation, when encapsulating floating-point data, the communication engine 200 may also pad the floating-point data in a preset manner. For example, the payload may be aligned to 10 bytes or 20 bytes. Taking 20-byte padding as an example, the payload in the data packet sent by the communication engine 200 is an integer multiple of 20 bytes. If the length of a group of floating-point data obtained by the compression engine after compressing the floating-point numbers in a data packet is 235 bytes, after the communication engine 200 obtains this group of 235-byte floating-point data, the communication engine 200 will pad 5 bytes to pad the payload to 240 bytes, and then add a message header to the 240-byte data for encapsulation and send it to other computing devices.

[0138] In one possible implementation, when the communication engine 200 pads the data in the data packet, as shown in Table 6 below, the metadata generated by the compression engine 300 also includes the data length, which is used to indicate the length of the floating-point data obtained by the communication engine 200, so that the communication engine 200 determines the length of the data that needs to be filled.

[0139] It should be noted that each time the compression engine 300 obtains a floating-point number from a data packet, it compresses the multiple floating-point numbers included in the data packet to obtain a set of floating-point data. Because the reference value in the metadata generated during the compression process is only applicable to recovering this set of floating-point data, the communication engine 200 obtains a set of floating-point data from the compression engine 300 or memory 400 each time to complete and encapsulate it.

[0140] Table 6

[0141]

[0142] In one possible implementation, if the payload of each data packet sent by the communication engine 200 needs to be of a fixed length, for example, 512 bytes, the compression engine 300 may compress the floating-point numbers in the received data packets by compressing the exponent values ​​of the floating-point numbers in multiple data packets using the same base value to obtain multiple groups of floating-point numbers. The communication engine 200 then pads these multiple groups of floating-point data to 512 bytes before encapsulating them.

[0143] Exemplarily, after obtaining the first data packet, the compression engine 300 determines the reference values ​​corresponding to the multiple floating-point numbers in the data packet, and compresses the exponent values ​​to obtain a set of floating-point data. If the length of the set of floating-point data is less than 512 bytes, the compression engine 300 continues to compress the multiple floating-point numbers in the second data packet using the reference values ​​corresponding to the first data packet after obtaining the multiple floating-point numbers in the second data packet, so that the metadata of the first data packet and the second data packet are the same. If the data length of the first data packet after compression is 215 bytes, the data length of the second data packet after compression is 230 bytes, and the length of the metadata is 10 bytes, then when the communication engine 200 pads the two sets of floating-point data according to 512 bytes, it needs to add 57 bytes before encapsulating and sending them together.

[0144] The above describes the computing device provided by the embodiment of the present application and the method for compressing data using the computing device. After the compression engine 300 compresses the floating-point number to obtain the compressed floating-point data, the communication engine 200 obtains the compressed floating-point data, uses the compressed floating-point data as a payload, encapsulates it into a message, and sends it to other computing devices. According to the compression method of the compression engine 300 for compressing the floating-point number, the compressed floating-point data includes a metadata part, a coding part, a mantissa part, and an exponent value part. Figure 12 As shown, Figure 12 This is a schematic diagram of a payload provided in an embodiment of the present application. Figure 12 In the above, the metadata part is used to store the fields shown in Table 1 or Table 6; when the compression engine 300 does not compress the mantissa of the floating-point number, the mantissa part is used to store the sign bit and mantissa value of each floating-point number; when the compression engine 300 compresses the mantissa part of the floating-point number, the mantissa part is used to store the compressed value of the sign bit and mantissa value of the floating-point number; the encoding part is used to store the encoding corresponding to each exponent value; the exponent value part is used to store the true exponent value of some exponents. For example, when the exponent value is encoded by the first method mentioned above, the exponent value part is used to store the exponent value whose absolute value of the difference from the reference value is greater than or equal to the first threshold; when the exponent value is encoded by the second method mentioned above, the exponent value part is used to store the exponent value whose absolute value of the difference from the reference value is greater than or equal to the second threshold; when the exponent value is encoded by the third method mentioned above, the exponent value part is used to store the exponent value greater than or equal to the third threshold.

[0145] Among them, the sign bit of each floating-point number in the compressed multiple floating-point numbers is stored together with the mantissa value of the floating-point number or the compressed value corresponding to the mantissa value of the floating-point number, that is, when the mantissa part is stored, the sign bit of the first floating-point number and the compressed value corresponding to the mantissa value of the first floating-point number are stored first, and then the sign bit of the second floating-point number and the compressed value corresponding to the mantissa value of the second floating-point number are stored, and so on, until the compression engine 300 stores the compressed values ​​corresponding to the sign bits and mantissa values ​​corresponding to the above-mentioned multiple floating-point numbers in the above-mentioned mantissa part.

[0146] It should be understood that when the compression engine 300 uses the first method mentioned above to compress the exponent value, that is, when the difference is encoded through the coding table shown in Table 2 above, the coding part is used to store the code corresponding to the difference between each exponent value and the reference value; if the compression engine adopts the third method mentioned above to compress the exponent value, that is, when the difference is encoded through the coding table shown in Table 4 or Table 5 above, the coding part is used to store the code corresponding to each numerical value.

[0147] When the compression engine 300 compresses the exponential value using the second method, that is, when the absolute value of the difference between the exponential value and the reference value is encoded using the encoding table shown in Table 3, the format of the payload portion generated by the compression engine 300 is as follows: Figure 13 As shown, Figure 13 This is a schematic diagram of another payload provided in an embodiment of the present application. Figure 13 The payload portion in also includes a sign bit corresponding to the encoding, and the sign bit corresponding to the encoding includes a sign corresponding to the difference between a non-zero exponent value and a reference value and a sign of an exponent whose exponent value is 0.

[0148] It should be noted that when the compression engine 300 generates a payload in the above format, there is a one-to-one correspondence between the data stored in the encoding part and the data stored in the mantissa part. If the compression engine compresses a total of 256 floating-point numbers, and compresses both the exponent value and the mantissa value, the mantissa part first stores the compressed value corresponding to the sign bit of the first floating-point number and the mantissa value of the first floating-point number, and then stores the compressed value corresponding to the sign bit of the second floating-point number and the mantissa value of the second floating-point number; the encoding part also first stores the code corresponding to the first floating-point number, and then stores the code corresponding to the second floating-point number. If the payload also includes a sign bit corresponding to the code, the sign bit part corresponding to the code first stores the sign of the difference between the exponent value of the first floating-point number and the reference value; and then stores the sign of the difference between the exponent value of the second floating-point number and the reference value.

[0149] After the computing device uses the above compression engine to compress the floating point number to reduce the amount of data transmitted, the computing device that receives the compressed floating point data needs to decompress the floating point data to obtain the complete floating point number. The following describes the decompression engine provided by the embodiment of the present application.

[0150] The embodiment of the present application also provides another computing device, which also includes a decompression engine. When the computing device receives compressed floating-point data sent by other computing devices, it can call the decompression engine to decompress the compressed floating-point data and restore multiple floating-point numbers. The computing device 40 includes one or more processors 100, a communication engine 200, a compression engine 300, a decompression engine 500 and a memory 400. Among them, the compression engine 300 can be deployed in the communication engine 200 or in the core. The compression engine 300 can also be an independent module outside the processor 100, the communication engine 200 and the memory 400; the decompression engine 500 can be deployed in the communication engine 200 or an independent module outside the processor 100, the communication engine 200 and the memory 400. The structure of the compression engine 300 and the method for compressing floating-point numbers can refer to the above Figures 7 to 13 The operations described in the related description, the structure of the decompression engine 500, and the method for the decompression engine 500 to decompress the compressed floating-point data will be explained later and will not be repeated here.

[0151] For example, Figure 14 As shown, Figure 14 Schematic diagram of another computing device provided by an embodiment of the present application. The compression engine 300 and the decompression engine 500 are both deployed in the communication engine 200. Figure 15 As shown, Figure 15 Schematic diagram of another computing device provided by an embodiment of the present application. The compression engine 300 is deployed in the core, and the decompression engine 500 is an independent module outside the processor 100, the communication engine 200 and the memory 400. Figure 16 As shown, Figure 16 is a schematic diagram of another computing device provided by an embodiment of the present application. The compression engine 300 and decompression engine 500 are both independent modules outside of the processor 100, communication engine 200, and memory 400. It should be understood that the above is merely an example, and the compression engine 300 and decompression engine 500 can also be deployed in other forms, such as the compression engine 300 being deployed in the core and the decompression engine 500 being deployed in the communication engine.

[0152] The structure of the decompression engine 500 and the method by which the decompression engine 500 decompresses floating-point data are introduced below.

[0153] like Figure 17 As shown, Figure 17 This is a schematic diagram of a decompression engine provided by an embodiment of the present application. Figure 8 As shown, the compression engine 300 can only compress the exponential value, so the decompression engine 500 includes a metadata parsing module 510, a separation module 520, an exponential decoding module 530, and a data recovery module 540. The metadata parsing module 510 is used to obtain the data type and reference value, and send the data type to the separation module 520, so that the separation module 520 determines the length of each mantissa value based on the data type and then separates the mantissa and encoding parts from the payload. The metadata parsing module 510 is also used to send the data type and reference value to the exponential decoding module 530, so that the exponential decoding module 530 determines the length of the exponential value based on the data type and recovers the exponential value based on the reference value and encoding part. The separation module 520 is used to extract the mantissa, encoding, and exponential value parts from the payload.

[0154] For floating-point data obtained by encoding the exponent value using the first method described above, after the exponent decoding module 530 obtains one or more reference values ​​from the encoded portion and metadata, for any code in the encoded portion, for example, the tth code, the exponent decoding module 530 first determines the reference value corresponding to the code and the difference value corresponding to the code after obtaining the tth code. The difference value is then added to the reference value to obtain the true exponent value corresponding to the code. It should be noted that if the code obtained by the exponent decoding module 530 is 111111, the exponent decoding module 530 obtains the true exponent value corresponding to the code from the exponent value portion.

[0155] For example, if the exponent decoding module 530 receives the code 111111 for the jth time, the exponent decoding module 530 obtains the jth exponent value from the exponent value portion, which is the actual exponent value corresponding to the code. Starting from the (j-1)p+1th bit, the exponent decoding module 530 obtains p consecutive bits. The value composed of these p bits is the jth exponent value. Where p is the length of the actual exponent value. For example, if the compressed floating-point number is a floating-point number of FP32 type, that is, the exponent value includes 8 bits, the reference value corresponding to a set is 4, and for the encoding corresponding to the set, if the encoding is 00, the reference value corresponding to the encoding is 4, and the difference corresponding to the encoding is 0, then the exponent value corresponding to the encoding 00 is 4; if the encoding is 10, the difference corresponding to the encoding is -1, then the exponent value corresponding to the encoding 10 is 3; if the encoding is 11110, the difference corresponding to the encoding is 3, then the exponent value corresponding to the encoding 11110 is 7; if the encoding is 111111, the exponent decoding module 530 obtains the encoding 111111 for the third time, then the exponent decoding module 530 obtains 8-bit data starting from the 17th bit of the exponent value part, and if the obtained 8-bit data is 00001001, then the exponent value corresponding to the encoding is 9.

[0156] For floating-point data whose exponent value is encoded using the second method described above, the exponent decoding module 530 obtains one or more reference values ​​from the encoding portion and metadata. For any code in the encoding portion, for example, the tth code, the exponent decoding module first determines the reference value corresponding to that code, the absolute value of the difference corresponding to that code, and the sign bit of the difference corresponding to that code. The difference is then added to the reference value to obtain the true exponent value corresponding to that code. It should be noted that if the code is 11111110, the exponent decoding module 530 obtains the true exponent value corresponding to that code from the exponent value portion. For example, if the compressed floating-point number is a floating-point number of FP32 type, that is, the exponent value includes 8 bits, and the base value corresponding to a set is 4, for the encoding corresponding to the set, if the encoding is 0, the sign bit corresponding to the encoding is 1, indicating that the encoding is the encoding corresponding to the difference value 0, that is, the exponent value corresponding to the encoding is 4; if the encoding is 0, and the sign bit corresponding to the encoding is 0, it means that the exponent value corresponding to the encoding is 0; if the encoding is 10, the sign bit corresponding to the encoding is 1, indicating that the difference value corresponding to the encoding is -1, then the encoding is 0. The exponent value corresponding to code 10 is 3; if the code is 11110, the sign bit corresponding to the code is 0, indicating that the difference value corresponding to the code is 4, then the exponent value corresponding to the code 11110 is 8; if the code is 11111110, the code is the second time that the exponent decoding module 530 obtains the code 11111110, then the exponent decoding module 530 starts from the 9th bit of the exponent value part and obtains 8-bit data. If the obtained 8-bit data is 00001000, then the exponent value corresponding to the code is 8.

[0157] For floating-point data whose exponent values ​​are encoded using the third method described above, the exponent decoding module 530 obtains one or more reference values ​​from the encoding portion and metadata. For any code in the encoding portion, for example, the tth code, the corresponding reference value is determined. If the code is 00, the exponent value corresponding to that code is the reference value. For other codes, the exponent value corresponding to that code is calculated by adding the distance between the code and 00 to the reference value. It should be noted that if the code is 111111, the exponent decoding module 530 obtains the actual exponent value corresponding to that code from the exponent value portion.

[0158] For example, the base value corresponding to a set is 4. For the code corresponding to the set, if the code is 00, the exponent value corresponding to the code is 4; if the code is 10, the distance between the code and the code 00 is -1, then the exponent value corresponding to the code 10 is 3; if the code is 11110, the distance between the code and the code 00 is 3, then the exponent value corresponding to the code 11110 is 7.

[0159] The data recovery module 540 is used to recover each floating-point number based on the exponent value corresponding to the exponent of each floating-point number and the sign bit and mantissa value of the floating-point number in the mantissa portion. For example, the compressed floating-point numbers are FP32 data. For one of the floating-point numbers, the exponent value of the floating-point number is determined to be 8 according to the encoding portion. The exponent is converted to an 8-bit binary number, and the sign bit and 23-bit mantissa of the floating-point number are obtained from the mantissa portion to obtain the true value of the floating-point number.

[0160] In one possible implementation, Figure 18 As shown, Figure 18 It is a schematic diagram of a decompression engine provided by an embodiment of the present application. If the compression engine 300 includes a mantissa compression module 350, the decompression engine 500 also includes a mantissa recovery module 550. The metadata also includes a compression type and a mantissa compression level. The data type indicates the data type of the multiple floating-point numbers; the compression type indicates whether the compression engine 300 compresses the floating-point numbers, and whether the compression engine 300 compresses the exponent or the mantissa, or both the exponent and the mantissa; the mantissa compression level indicates the length of the compressed value obtained after the compression engine 300 compresses the mantissa value. If the compression engine 300 compresses the mantissa value, the mantissa part of the payload includes the compressed values ​​corresponding to the sign bits of the multiple floating-point numbers and the mantissa values ​​of the multiple floating-point numbers.

[0161] If the separation module 520 determines that the mantissa portion includes compressed values ​​based on the compression type, it can determine the length of each compressed value based on the mantissa compression level and then extract each compressed value from the mantissa portion. The mantissa recovery module 550 can determine the length of the mantissa value of each floating-point number based on the data type, and based on the length of the mantissa value and the length of the compressed value, determine the length to be deleted by the compression engine 300 when compressing the mantissa value. Finally, it compensates each compressed value with a random number of the corresponding length to obtain the mantissa value corresponding to each mantissa.

[0162] For example, if the separation module 520 determines based on the compression type that the compression engine 300 compressed the exponent and mantissa, and determines based on the data type in the metadata that the compressed data is FP16 data, i.e., the mantissa value includes 10 bits, and the mantissa compression level determines that the compressed value of the mantissa value is 4 bits, then the compression engine 300 deletes 6 bits of characters when compressing the mantissa. After obtaining the encoded portion, the separation module 520 extracts a compressed value for every 4 bits, and then the mantissa recovery module 550 randomly generates a 6-bit random number, which is concatenated with the 4-bit compressed value to obtain a 10-bit mantissa value.

[0163] The data recovery module 540 is used to match and concatenate the exponent values ​​corresponding to the floating-point numbers output by the exponent decoding module 530 and the mantissa values ​​corresponding to the floating-point numbers output by the mantissa recovery module 550 to obtain the floating-point numbers.

[0164] For the above method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited to the order of the actions described. In addition, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0165] Other reasonable step combinations or reasonable computing device structures that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required for the present invention.

[0166] The above description is based on the accompanying drawings to introduce in detail the data processing method provided by the present application and the computing device used to implement the data processing method. The following description is based on the accompanying drawings to introduce the relevant devices and equipment provided by the present application.

[0167] like Figure 19 As shown, Figure 19 1 is a schematic diagram of a data processing device provided in an embodiment of the present application. The data processing device 190 includes a communication unit 191 and a processing unit 192. The communication unit 191 is used to obtain a plurality of first data; the processing unit 192 is used to determine a first reference value based on the plurality of first data to be compressed; then calculate the difference between each first data and the first reference value; determine the code corresponding to each difference; and obtain the compressed data of the corresponding first data according to each code. The length of some or all of the codes corresponding to each first data is less than the length of the exponent value. It should be understood that the plurality of first data can be the exponent values ​​of a plurality of floating-point numbers.

[0168] When determining the code corresponding to each difference, the processing unit 192 is specifically configured to determine the code corresponding to each difference based on each difference and a code table, wherein the code table records the codes corresponding to a plurality of preset difference values. In the code table, the codes corresponding to differences whose absolute values ​​are greater than a threshold are fixed values. The processing unit is specifically configured to, when the absolute value of the first difference is greater than the threshold, use the fixed value and the first data corresponding to the first difference as the compressed data corresponding to the first difference.

[0169] Optionally, the floating-point number also includes a mantissa value; the processing unit 192 can also be used to compress the mantissa values ​​of multiple floating-point numbers respectively to obtain compressed data of each mantissa value; based on the compressed data of the exponent values ​​of multiple floating-point numbers and the compressed data of the mantissa values ​​of multiple floating-point numbers, the compressed data of multiple floating-point numbers is obtained.

[0170] In this embodiment of the present application, after compressing the plurality of first data, the computing device sends the compressed data of each first data to other computing devices, which then need to decode the compressed data. When compressing the plurality of first data, processing unit 192 generates corresponding first metadata, which includes the first reference value. The computing device also sends the metadata to the other computing devices, so that the other computing devices decode the compressed data based on the compressed data of each first data and the first metadata to obtain the plurality of first data.

[0171] Optionally, when compressing multiple floating-point numbers, the above-mentioned first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa values ​​when the mantissa values ​​are compressed.

[0172] In one possible implementation, the communication unit 191 is also used to receive data to be decompressed sent by other devices, where the data to be decompressed includes an encoded part and second metadata; wherein the second metadata includes a second reference value, and the encoded part includes multiple codes; and according to the second reference value and the multiple codes, the multiple codes are decoded respectively to obtain multiple second data.

[0173] In one possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; the processing unit 192 determines the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; decodes the multiple codes respectively according to the length of the exponent value, the second reference value and the multiple codes to obtain multiple exponent values; when it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the compressed data of the mantissa part is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values; and restores the floating-point number before compression according to the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0174] It should be noted that the data processing device 190 is capable of compressing and decompressing data. The operation of the data processing device 190 to compress data can refer to the operation performed by the compression engine in the above embodiment, and the operation of decompressing data can refer to the operation performed by the compression engine in the above embodiment, which will not be repeated here.

[0175] The present application also provides a computing device. The structural diagram of the computing device can refer to the above Figures 4 to 6 or Figures 14 to 16 Schematic diagram of the corresponding computing device.

[0176] It should be noted that the computing devices include multiple processors 100, and each processor 100 can exchange data through aggregate communication methods such as all-to-all, allgather or allreduce. Among them, the above-mentioned processor 100 can be a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processing unit (NPU), a tensor processing unit (TPU), a data processing unit (DPU), an accelerated processing unit (APU), a floating-point processing unit (FPU) or an application-specific integrated circuit (ASIC), etc. The processor can also be a single-core processor or a multi-core processor. The processor can be a combination of a CPU and a hardware chip. The above-mentioned hardware chip can be an ASIC, a PLD or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), an FPGA, a generic array logic (GAL) or any combination thereof. The processor may also be implemented solely using a logic device with built-in processing logic, such as an FPGA or a digital signal processor (DSP).

[0177] The communication engine 200 may be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface may be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface may be a cellular network interface or a wireless local area network interface. In the embodiment of the present application, the communication engine 200 may be used to obtain data to be decompressed sent by other computing devices, etc.

[0178] The memory 400 may be a non-volatile memory, such as a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The memory 400 may also be a volatile memory, which may be a random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0179] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0180] Optionally, the computing device may further include an input / output interface connected to an input / output device for receiving information input by a user, such as a compression type and a mantissa compression level input by the user.

[0181] Specifically, the specific implementation of the various operations performed by the computing device can refer to the specific operations of data compression and data decoding performed by the computing device in the above method embodiment, which will not be repeated here.

[0182] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a processor, it can implement the method steps performed by the above-mentioned computing device. The specific implementation of the processor of the computer-readable storage medium in executing the above-mentioned method steps can refer to the specific operations of the above-mentioned embodiment, which will not be repeated here.

[0183] An embodiment of the present application further provides a computer program product, which includes computer instructions. When executed by a computing device, the computing device executes the method executed by the computing device in the above embodiment.

[0184] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0185] The above embodiments can be implemented in whole or in part through software, hardware, firmware, or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium, or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0186] The steps in the method of the embodiment of the present application can be adjusted in order, combined or deleted according to actual needs; the modules in the device of the embodiment of the present application can be divided, combined or deleted according to actual needs.

[0187] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: The method comprises: determining a first reference value according to a plurality of first data to be compressed, wherein the plurality of first data are data to be transmitted by a computing device; Calculating the difference between each first data and the first reference value; Determine the code corresponding to each difference; Obtaining compressed data of the corresponding first data according to each of the codes; The step of determining a first reference value according to the plurality of first data to be compressed includes: Sampling the plurality of first data to obtain a plurality of sampling values; Divide the plurality of sample values ​​into t1 groups, determine the median of the plurality of sample values ​​in each of the t1 groups, and obtain t1 medians; The t1 medians are divided into t2 groups, the median in each of the t2 groups is determined to obtain t2 medians, and the median corresponding to the t2 medians is used as the first reference value corresponding to the multiple first data.

2. The method according to claim 1, wherein The plurality of first data are respectively exponent values ​​of a plurality of floating-point numbers.

3. The method according to claim 1, wherein Determining the code corresponding to each difference includes: The code corresponding to each difference is determined according to each difference and a coding table, where the coding table records codes corresponding to a plurality of preset difference values.

4. The method according to claim 3, wherein In the coding table, the code corresponding to the difference whose absolute value is greater than the threshold is a fixed value, and the compressed data of the corresponding first data obtained according to each of the codes includes: When the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as compressed data of the first data corresponding to the first difference.

5. The method according to claim 2, wherein The plurality of floating point numbers also includes a mantissa value; The method further comprises: Compressing the mantissa values ​​of the plurality of floating-point numbers respectively to obtain compressed data of the mantissa values; The compressed data of the plurality of floating-point numbers is obtained according to the compressed data of the exponent values ​​of the plurality of floating-point numbers and the compressed data of the mantissa values ​​of the plurality of floating-point numbers.

6. The method according to any one of claims 2 to 5, wherein: The method further comprises: First metadata is generated, where the first metadata includes the first reference value, and the first reference value is used to decode compressed data of the plurality of first data to obtain the plurality of first data.

7. The method according to claim 6, wherein When the multiple first data are respectively exponent values ​​of multiple floating-point numbers, the first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa values ​​when the mantissa values ​​are compressed.

8. The method according to any one of claims 1 to 5 or 7, wherein: The method further comprises: Receive data to be decompressed, the data to be decompressed including a coding portion and second metadata; wherein the second metadata includes a second reference value, and the coding portion includes a plurality of codes; The plurality of codes are decoded respectively according to the second reference value and the plurality of codes to obtain a plurality of second data.

9. The method according to claim 8, wherein When the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed further includes a mantissa portion, and the second metadata further includes a data type, a compression type, and a mantissa compression level of the compressed floating-point number, wherein the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; The decoding of the to-be-decompressed data according to the second reference value and the plurality of codes to obtain a plurality of second data includes: Determine the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; Decoding the multiple codes respectively according to the length of the exponent value, the second reference value, and the multiple codes to obtain multiple exponent values; When it is determined according to the compression type that the mantissa portion in the to-be-decompressed data is data obtained by compressing the mantissa value, decoding the compressed mantissa portion data according to the mantissa compression level and the length of the mantissa value to obtain a plurality of mantissa values; The floating-point number before compression is restored according to the multiple exponent values ​​and the multiple mantissa values.

10. A data processing device, characterized in that: The device comprises: A communication unit, configured to obtain a plurality of first data, wherein the plurality of first data are data that the computing device needs to transmit; a processing unit, configured to determine a first reference value according to a plurality of first data to be compressed; Calculating the difference between each first data and the first reference value; Determine the code corresponding to each difference; Obtaining compressed data of the corresponding first data according to each of the codes; The step of determining a first reference value according to the plurality of first data to be compressed includes: Sampling the plurality of first data to obtain a plurality of sampling values; Divide the plurality of sample values ​​into t1 groups, determine the median of the plurality of sample values ​​in each of the t1 groups, and obtain t1 medians; The t1 medians are divided into t2 groups, the median in each of the t2 groups is determined to obtain t2 medians, and the median corresponding to the t2 medians is used as the first reference value corresponding to the multiple first data.

11. The device according to claim 10, wherein The plurality of first data are respectively exponent values ​​of a plurality of floating-point numbers.

12. The device according to claim 10, wherein The processing unit is specifically configured to: The code corresponding to each difference is determined according to each difference and a coding table, where the coding table records codes corresponding to a plurality of preset difference values.

13. The device according to claim 11, wherein The plurality of floating point numbers also includes a mantissa value; The processing unit is further configured to: Compressing the mantissa values ​​of the plurality of floating-point numbers respectively to obtain compressed data of the mantissa values; The compressed data of the plurality of floating-point numbers is obtained according to the compressed data of the exponent values ​​of the plurality of floating-point numbers and the compressed data of the mantissa values ​​of the plurality of floating-point numbers.

14. The device according to any one of claims 10 to 13, characterized in that The communication unit is further configured to receive data to be decompressed, the data to be decompressed comprising a coding portion and second metadata; wherein the metadata comprises a second reference value, and the coding portion comprises a plurality of codes; The plurality of codes are decoded respectively according to the second reference value and the plurality of codes to obtain a plurality of second data.

15. The device according to claim 14, wherein When the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed further includes a mantissa portion, and the second metadata further includes a data type, a compression type, and a mantissa compression level of the compressed floating-point number, wherein the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; The decoding of the to-be-decompressed data according to the second reference value and the plurality of codes to obtain a plurality of second data includes: Determine the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; Decoding the multiple codes respectively according to the length of the exponent value, the second reference value, and the multiple codes to obtain multiple exponent values; When it is determined according to the compression type that the mantissa portion in the to-be-decompressed data is data obtained by compressing the mantissa value, decoding the compressed mantissa portion data according to the mantissa compression level and the length of the mantissa value to obtain a plurality of mantissa values; The floating-point number before compression is restored according to the multiple exponent values ​​and the multiple mantissa values.

16. A computing device, characterized in that The computing device includes one or more processors, a memory and a compression engine, wherein the memory stores computer program instructions, the processor executes the computer program instructions to generate data to be compressed, and the compression engine is used to implement the data processing method described in any one of claims 1 to 9 to compress the data to be compressed.

17. The computing device of claim 16, wherein: The computing device further includes a communication engine and a decompression engine. The communication engine is used to receive data to be decompressed, and the decompression engine is used to decode the data to be decompressed.

18. The computing device of claim 17, wherein: The compression engine and the decompression engine are located in the communication engine.

19. The computing device of claim 17, wherein: The compression engine is located in the processor, and the decompression engine is located in the communication engine.

20. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed, the operation steps of the method according to any one of claims 1 to 9 are performed.

Images