Data processing method and device and related equipment

Abstract

The invention provides a data processing method and device and related equipment. The method comprises the steps that computing equipment determines a first reference value according to multiple pieces of first data to be compressed; then calculating a difference value between each piece of first data and a first reference value; determining a code corresponding to each difference value; and obtaining data after the corresponding first data is compressed according to each code. Wherein the length of partial or all codes in the codes corresponding to the first data is smaller than the length of the original first data. The reference value is the data with more occurrence times in the plurality of data to be compressed, when the plurality of data to be compressed are respectively subtracted from the reference value, the number of 0 in the plurality of obtained difference values is the maximum, and after the difference values are coded, the data corresponding to the difference values can be compressed to less digits; as the bits occupied by coding are smaller than those occupied by each piece of data, the data compression can be realized, the data volume interacted between different computing devices is reduced, and the data transmission efficiency is improved.

Description

[0001] This application is a divisional application. The application number of the original application is 202211289095.4, and the original application date is October 20, 2022. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus and related equipment. Background Art

[0003] With the development of information technology, the demand for application data and multimedia communication services has greatly increased. A large amount of data needs to be transmitted or interacted between different devices through the network. For example, in applications such as high performance computing (HPC), artificial intelligence (AI) model training, and multimedia data transmission, in order to improve computing efficiency and user experience, it is necessary to improve the efficiency of data interaction between different devices and reduce the latency during data transmission. Reducing the amount of data transmitted between different devices is an effective way to improve data interaction efficiency and reduce latency. Therefore, how to reduce the amount of data interacting between different devices is a technical problem that needs to be solved. Summary of the invention

[0004] The present application provides a data processing method, an apparatus and related equipment, which can reduce the amount of data to be transmitted, improve data exchange efficiency and reduce the delay of data transmission by compressing the data to be transmitted.

[0005] In a first aspect, the present application provides a data processing method, the method comprising: a computing device determines a first reference value according to a plurality of first data to be compressed; then calculates the difference between each first data and the first reference value; determines the code corresponding to each difference; and obtains the compressed data of the corresponding first data according to each code. The length of some or all of the codes corresponding to each first data is less than the length of the original first data.

[0006] The above-mentioned benchmark value is the data that appears more frequently among the multiple data to be compressed. After the multiple data to be compressed are respectively subtracted from the benchmark value and the code corresponding to each data is determined according to the difference, the length of the code corresponding to part or all of the data is less than the length of the original first data, and the number of 0s in the difference between the multiple data and the benchmark value is the largest. After encoding the difference of 0 in the difference, the data corresponding to the difference of 0 can be compressed to a smaller number of bits; since the bits occupied by the code are smaller than those of each data, data compression can be achieved, the amount of data exchanged between different computing devices can be reduced, and data transmission efficiency can be improved.

[0007] In a possible implementation, the plurality of first data are exponent values ​​of respective floating-point numbers.

[0008] In applications such as AI model training, the data types of the weights in the model and the activation values, gradients, and other data generated during the model training process are usually floating point (FP) types, such as double-precision floating point (FP64), single-precision floating point (FP32), and half-precision floating point (FP16). By compressing the exponent value in the floating point number, the amount of floating point data in the transmission process can be reduced, the data transmission efficiency can be improved, and the data latency can be reduced. For AI model training, the amount of data interacted during the AI ​​model training process can be reduced, the data transmission latency can be reduced, and the model training efficiency can be improved.

[0009] In a possible implementation, the determining of the code corresponding to each difference includes: determining the code corresponding to each difference according to each difference and a coding table, wherein the coding table records codes corresponding to a plurality of preset difference values. The coding table is stored in the computing device, and after calculating the difference between a data and a reference value, the corresponding code can be determined according to the coding table.

[0010] In a possible implementation, in the above-mentioned coding table, the code corresponding to the difference whose absolute value is greater than a threshold is a fixed value, and the compressed data of the corresponding first data obtained according to each code includes: when the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as the compressed data of the first data corresponding to the first difference.

[0011] Optionally, if multiple first data are exponent values ​​of multiple floating-point numbers, for any one of the first exponent values, the code corresponding to the first difference between the first exponent value and the reference value is determined according to the coding table; when the absolute value of the first difference is less than or equal to the first threshold, the code corresponding to the first difference is used as the code corresponding to the first exponent value; when the absolute value of the first difference is greater than the first threshold, the code corresponding to the first difference and the first exponent value are used as the code corresponding to the first exponent value; wherein the above-mentioned reference value is an exponent value that appears more often, and when the multiple exponent values ​​to be compressed are respectively subtracted from the reference value, the number of 0s in the multiple differences obtained is the largest, and the 0s in the differences are used as the codes. After the difference is encoded according to the above encoding method, the exponential value corresponding to the difference of 0 can be compressed to a smaller number of bits; in model training, the distribution of the exponential value of floating-point numbers is usually close to a normal distribution, and the proportion of the differences whose absolute values ​​are less than the threshold in the above multiple differences is also relatively large. Therefore, the differences whose absolute values ​​are less than or equal to the threshold in the differences are also encoded according to the above encoding method. Since the bits occupied by the encoding are smaller than the respective exponential values, the number of differences whose absolute values ​​are less than the first threshold in the differences accounts for a large proportion, so that most of the exponential values ​​can be compressed by the above method, thereby achieving compressed floating-point numbers, reducing the amount of data exchanged between different computing devices, and improving data transmission efficiency.

[0012] In a possible implementation, the above-mentioned determining a first reference value based on multiple first data to be compressed includes: sampling the multiple first data to obtain multiple sampling values; dividing the multiple sampling values ​​into t1 groups, determining the median of the multiple sampling values ​​in each of the t1 groups, and obtaining t1 medians; dividing the t1 medians into t2 groups, determining the median of each of the t2 groups, and obtaining t2 medians, and using the median corresponding to the t2 medians as the first reference value corresponding to the multiple first data.

[0013] When the amount of first data is large, sampling the first data can reduce the amount of data to be processed when determining the reference value, thereby improving the efficiency of determining the reference value.

[0014] In a possible implementation, the above-mentioned multiple floating-point numbers also include mantissa values; the data processing method also includes: a computing device compresses the mantissa values ​​of the multiple floating-point numbers respectively to obtain compressed data of each mantissa value; and obtains compressed data of the multiple floating-point numbers based on the compressed data of the exponent values ​​of the multiple floating-point numbers and the compressed data of the mantissa values ​​of the multiple floating-point numbers.

[0015] The exponent value of the floating point number is also compressed, which can further reduce the amount of data that needs to be transmitted during data transmission and improve the efficiency of data transmission.

[0016] In a possible implementation, the data processing method further includes: generating first metadata, the first metadata including a first reference value, the first reference value being used to decode compressed data of the plurality of first data to obtain the plurality of first data.

[0017] By generating metadata including a reference value, the compressed data and the metadata are sent to a receiving end, so that a computing device receiving the compressed data can decode the received compressed data according to the metadata to obtain the data before compression.

[0018] In one possible implementation, when the multiple first data are respectively exponent values ​​of multiple floating-point numbers, the first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed.

[0019] When compressing floating-point data, you can choose whether to compress the mantissa value of the floating-point number, and choose the degree of compression of the mantissa value. Therefore, when compressing the floating-point number, the above metadata also includes the compression type and the mantissa compression level, so that the computing device that receives the compressed data can decode the exponent part and the mantissa part of the received compressed data according to the metadata respectively to obtain the floating-point data before compression.

[0020] In a possible implementation, the method further includes: receiving data to be decompressed, the data to be decompressed including a coding part and second metadata; wherein the second metadata includes a second reference value, and the coding part includes multiple codes; and decoding the multiple codes respectively according to the second reference value and the multiple codes to obtain multiple second data.

[0021] After receiving the data, when there is a reference value in the received data, the computing device can determine that the received data is data to be decompressed. Through the codes in the data to be decompressed and the coding table stored in the computing device, the difference corresponding to each code can be determined, and then the original data can be restored based on the difference and the reference value.

[0022] In a possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; the above-mentioned decoding of the data to be decompressed according to the second reference value and multiple codes to obtain multiple second data includes: determining the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; decoding the multiple codes respectively according to the length of the exponent value, the second reference value and the multiple codes to obtain multiple exponent values; when it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the compressed data of the mantissa part is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values; and restoring the floating-point number before compression according to the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0023] In a second aspect, the present application provides a data processing device, the device comprising a communication unit and a processing unit, the communication unit is used to obtain a plurality of first data; the processing unit is used to determine a first reference value according to the plurality of first data to be compressed; then calculate the difference between each first data and the first reference value; determine the code corresponding to each difference; and obtain the compressed data of the corresponding first data according to each code. The length of some or all of the codes corresponding to each first data is less than the length of the exponent value.

[0024] In a possible implementation, the plurality of first data are respectively exponent values ​​of a plurality of floating-point numbers.

[0025] In a possible implementation, the processing unit is specifically configured to: determine the code corresponding to each difference according to each difference and a coding table, wherein the coding table records codes corresponding to a plurality of preset differences.

[0026] In a possible implementation, in the above-mentioned coding table, the code corresponding to the difference whose absolute value is greater than the threshold is a fixed value, and the above-mentioned processing unit is specifically used to: when the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as the compressed data of the first data corresponding to the first difference.

[0027] In a possible implementation, when determining a first reference value based on multiple first data to be compressed, the processing unit is specifically used to: sample multiple first data to obtain multiple sampling values; divide the multiple sampling values ​​into t1 groups, determine the median of multiple sampling values ​​in each of the t1 groups, and obtain t1 medians; divide the t1 medians into t2 groups, determine the median of each of the t2 groups, and obtain t2 medians, and use the median corresponding to the t2 medians as the first reference value corresponding to the multiple first data.

[0028] In a possible implementation, the above-mentioned multiple floating-point numbers also include mantissa values; the processing unit is also used to compress the mantissa values ​​of the multiple floating-point numbers respectively to obtain compressed data of each mantissa value; based on the compressed data of the exponent values ​​of the multiple floating-point numbers and the compressed data of the mantissa values ​​of the multiple floating-point numbers, the compressed data of the multiple floating-point numbers is obtained.

[0029] In a possible implementation, the processing unit is further used to: generate first metadata, where the first metadata includes a first reference value, and the first reference value is used to decode compressed data of the plurality of first data to obtain the plurality of first data.

[0030] In a possible implementation, when the multiple first data are respectively exponent values ​​of multiple floating-point numbers, the above-mentioned first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed.

[0031] In a possible implementation, the communication unit is also used to: receive data to be decompressed, the data to be decompressed includes a coding part and second metadata; wherein the second metadata includes a second reference value, and the coding part includes multiple codes; and according to the second reference value and the multiple codes, respectively decode the multiple codes to obtain multiple second data.

[0032] In a possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; the above-mentioned processing unit is specifically used to: determine the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; decode the multiple codes respectively according to the length of the exponent value, the second reference value and the multiple codes to obtain multiple exponent values; when it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the compressed data of the mantissa part is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values; restore the floating-point number before compression according to the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0033] In a third aspect, the present application provides a computing device comprising one or more processors, a memory and a compression engine, wherein the memory stores computer program instructions, the processor executes the computer program instructions to generate data to be compressed, and the compression engine is used to implement the data processing method described in the first aspect or any possible implementation of the first aspect above, to compress the data to be compressed.

[0034] In one possible implementation, the computing device also includes a communication engine and a decompression engine. The communication engine is used to receive data to be decompressed, and the decompression engine is used to implement the data processing method described in the first aspect or any possible implementation of the first aspect above, and decode the received data to be decompressed.

[0035] In a possible implementation, the compression engine and the decompression engine are located in the communication engine.

[0036] In a possible implementation, the compression engine is located in each processor, and the decompression engine is located in the communication engine.

[0037] In a possible implementation, the compression engine and decompression engine are modules independent of the processor, memory, and communication engine.

[0038] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a server, the server executes the data processing method in the first aspect or any possible implementation of the first aspect.

[0039] In a fifth aspect, the present application provides a computer program product, which, when running on a server, enables the server to execute the data processing method in the second aspect or any possible implementation of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0041] Figure 1 It is a schematic diagram of the architecture of a distributed training cluster provided in an embodiment of the present application;

[0042] Figure 2 It is a flowchart of a data processing method provided in an embodiment of the present application;

[0043] Figure 3 is a schematic diagram of a floating point number provided in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a computing device provided in an embodiment of the present application;

[0045] Figure 5 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0046] Figure 6 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0047] Figure 7 is a schematic diagram of a compression engine provided in an embodiment of the present application;

[0048] Figure 8 is a schematic diagram of another compression engine provided in an embodiment of the present application;

[0049] Fig. 9 is a schematic diagram of another compression engine provided in an embodiment of the present application;

[0050] Fig.10 is a schematic diagram of determining a reference value provided in an embodiment of the present application;

[0051] Fig.11 is another schematic diagram for determining a reference value provided in an embodiment of the present application;

[0052] Fig.12 is a schematic diagram of a payload provided in an embodiment of the present application;

[0053] Fig.13 is a schematic diagram of another effective load provided in an embodiment of the present application;

[0054] Fig.14 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0055] Fig.15 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0056] Fig.16 is a schematic diagram of another computing device provided in an embodiment of the present application;

[0057] Fig.17 is a schematic diagram of a decompression engine provided in an embodiment of the present application;

[0058] Fig.18 is a schematic diagram of a decompression engine provided in an embodiment of the present application;

[0059] Fig.19 It is a schematic diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solution of the data processing method provided by the present application will be described below in conjunction with the accompanying drawings.

[0061] With the development of information technology, a large amount of data needs to be transmitted or interacted between different devices through the network. For example, high performance computing (HPC), AI model training, multimedia data transmission and other applications involve a large amount of data transmission. By improving the efficiency of data interaction between different computing devices and reducing the latency of data transmission, the efficiency of data processing can be improved and the user experience can be improved. Regarding the issue of how to improve the efficiency of data interaction between different computing devices, reducing the amount of data transmitted between different computing devices is an effective way to improve data transmission efficiency and reduce data transmission latency.

[0062] For example, Figure 1 As shown, Figure 1It is a schematic diagram of the architecture of a distributed training cluster provided in an embodiment of the present application. The distributed training cluster is used to train the AI ​​model in a distributed training scenario. The distributed training cluster includes one or more servers, and each server includes one or more processors. When multiple servers are included, the multiple servers are connected through a network. During the training process of the AI ​​model, multiple servers participating in the training or different computing devices in the same server can exchange data through aggregated communication methods such as all-to-all, allgather or allreduce, thereby completing the training of the AI ​​model. As the scale of AI models becomes larger and larger, the amount of data exchanged between different computing devices is also increasing, and the delay required for the data interaction process accounts for an increasing proportion of model training. It should be noted that the above-mentioned computing device can be a server in a cluster or a processor in a server, and the embodiment of the present application does not make specific restrictions. Therefore, how to reduce the amount of data exchanged during model training is an effective way to reduce the delay of data interaction and improve the efficiency of model training.

[0063] The present application provides a data processing method that can compress data, thereby reducing the amount of data that needs to be transmitted during the data interaction process, improving the efficiency of data transmission, and reducing latency. Figure 2 As shown, Figure 2 1 is a flow chart of a data processing method provided in an embodiment of the present application. The method includes the following S201 to S203.

[0064] S201. Determine a first reference value according to a plurality of first data to be compressed.

[0065] The multiple first data to be compressed are data that the computing device needs to transmit. Before transmitting the multiple first data, the computing device needs to compress the multiple first data to reduce the amount of data during the transmission process.

[0066] Before compressing the multiple first data, the computing device first determines a first reference value based on the multiple first data. When determining the first reference value, the computing device can count the first data with the most occurrences among the multiple first data, and use the first data with the most occurrences as the first reference value. The computing device can also divide the multiple first data into t1 groups, determine the median in each group, and obtain t1 medians; then divide the t1 medians into t2 groups, and then determine the median in each of the t2 groups to obtain t2 medians. And so on, until the multiple medians finally obtained cannot be further grouped, for example, after grouping, the values ​​in each group are less than three, and the median corresponding to the multiple medians is used as the reference value corresponding to the set. The computing device can also first sample the multiple first data to obtain multiple sampled values, and then divide the sampled values ​​into t1 groups, and then determine the first reference value according to the above method.

[0067] Exemplarily, the first set includes 100 index values, and the index encoding module 320 samples the 100 index values ​​to obtain 27 index values. The index encoding module 320 first divides the 27 index values ​​into groups of 3 to obtain 9 groups of data, determines the median in each group of data, and obtains 9 medians; then divides the 9 medians into groups of 3 to obtain 3 groups of data, determines the median in each group of data, and obtains 3 medians, and finally uses the median corresponding to the 3 medians as the reference value corresponding to the first set.

[0068] S202: Determine the difference between each first data and a first reference value.

[0069] After determining the first reference value, the computing device calculates the difference between each of the plurality of first data and the first reference value. It should be understood that the difference between the plurality of first data and the first reference value includes positive numbers, 0, and negative numbers.

[0070] S203: Determine the code corresponding to each difference value, and obtain compressed data of the corresponding first data according to each code.

[0071] A coding table is stored in the computing device, in which codes corresponding to multiple preset difference values ​​are recorded. After determining the difference between each first data and the first reference value, the computing device searches for the code corresponding to each difference value according to the coding table. The code corresponding to each difference value is the code corresponding to the first data corresponding to the difference. The code corresponding to each first data is the data after each first data is compressed.

[0072] It should be noted that the corresponding relationship between the difference and the encoding in the present application and the form of the compressed data corresponding to each first data will be introduced in detail later and will not be elaborated here.

[0073] In the embodiment of the present application, after compressing the above-mentioned multiple first data, the computing device will send the compressed data of each first data to other computing devices, and the other computing devices need to decode the compressed data. When the computing device compresses the multiple first data, it will generate corresponding first metadata, and the first metadata includes the above-mentioned first reference value. The computing device will also send the metadata to other computing devices, so that the other computing devices decode the compressed data according to the compressed data of each first data and the first metadata to obtain the above-mentioned multiple first data.

[0074] In a possible embodiment, the first data may be an exponent value in a floating point number. A floating point number is a number represented by scientific notation. A floating point number includes a sign bit, a mantissa, a base, and an exponent. A floating point number can be represented in the following form:

[0075] V=(-1) s *M*R E

[0076] Among them, S represents the sign bit, which takes a value of 0 or 1, 0 represents a positive floating point number, and 1 represents a negative floating point number; M represents the mantissa; R represents the base, and the value of R in binary numbers is 2; E represents the exponent.

[0077] In a computer, when a floating point number is used to represent a number, it is only necessary to determine the sign bit, exponent, and mantissa variables mentioned above. The lengths of the exponent and mantissa are different in floating point numbers of different precisions. For example, FP64 uses 64 bits (bits) to store a floating point number, and the 64 bits include a 1-bit sign bit, an 11-bit exponent, and a 52-bit mantissa; FP32 uses 32 bits to store a floating point number, including a 1-bit sign bit, an 8-bit exponent, and a 23-bit mantissa; FP16 uses 16 bits to store a floating point number, including a 1-bit sign bit, a 5-bit exponent, and a 10-bit mantissa. Figure 3 As shown, Figure 3 is a schematic diagram of a floating point number provided in an embodiment of the present application, Figure 3 The storage diagram of FP32 data is shown as an example. In the 32-bit data, the 1st to 23rd bits store the mantissa, the 24th to 31st bits store the exponent, and the 32nd bit stores the sign bit.

[0078] In applications such as AI model training, the data types of the weights in the model and the activation values, gradients, and other data generated during the model training process are usually floating point (FP) types, such as double-precision floating point (FP64), single-precision floating point (FP32), and half-precision floating point (FP16). By compressing the exponent value in the floating point number, the amount of floating point data in the transmission process can be reduced, the data transmission efficiency can be improved, and the data latency can be reduced. For AI model training, the amount of data interacted during the AI ​​model training process can be reduced, the data transmission latency can be reduced, and the model training efficiency can be improved.

[0079] In a possible implementation, if the first data is an exponent value in a floating-point number, that is, the computing device compresses the floating-point number, and each floating-point number also includes a mantissa value; the computing device may also compress the mantissa values ​​of multiple floating-point numbers respectively to obtain compressed data of each mantissa value; and obtain compressed data of multiple floating-point numbers based on the compressed data of the exponent values ​​of multiple floating-point numbers and the compressed data of the mantissa values ​​of multiple floating-point numbers.

[0080] It should be understood that when the plurality of first data are respectively exponent values ​​of a plurality of floating-point numbers, the first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the plurality of floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed. When compressing the floating-point data, the user can choose whether to compress the mantissa value of the floating-point number, and select the degree of compression of the mantissa value. Therefore, when compressing the floating-point number, the first metadata also includes a compression type and a mantissa compression level, so that the computing device that receives the compressed data can decode the exponential part and the mantissa part of the received compressed data according to the first metadata, respectively, to obtain the floating-point data before compression.

[0081] In a possible implementation, the computing device can also receive compressed data sent by other computing devices, that is, receive data to be decompressed, the data to be decompressed includes a coding part and second metadata; wherein the second metadata includes a second reference value, and the coding part includes multiple codes. The computing device can decode multiple codes respectively according to the second reference value and the multiple codes to obtain multiple second data. For example, the computing device determines the difference corresponding to a code and a coding table stored in the computing device, and then adds the difference to the second reference value to obtain the original data corresponding to the code.

[0082] In a possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type, and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed; the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed. After receiving the data to be decompressed after the floating-point number is compressed, the computing device determines the length of the exponential value and the length of the mantissa value of the floating-point number before compression according to the data type; according to the length of the exponential value, the second reference value, and the multiple codes, the multiple codes are respectively decoded to obtain multiple exponential values. When it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the data after the mantissa part is compressed is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values. For example, if the length of the original mantissa value is 10 bits, and the mantissa compression level indicates that the length of the compressed mantissa value is 6 bits, the computing device adds 4 bits to each compressed data of the mantissa part to obtain a 10-bit mantissa value. Finally, the computing device restores the floating point number before compression based on the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0083] The following takes the above-mentioned first data being the exponential value of a floating-point number, that is, compressing and decoding the floating-point number as an example, and combines the structure of the computing device provided by the present application to introduce the data processing method provided by the present application in detail.

[0084] The present application provides a computing device including a compression engine. The compression engine can compress floating point numbers, thereby reducing the amount of data exchanged between computing devices during data exchange, improving data transmission efficiency, and reducing data transmission delay. Figure 4 As shown, Figure 4It is a schematic diagram of a computing device provided by an embodiment of the present application. The computing device 10 includes one or more processors 100, a communication engine 200, a compression engine 300 and a memory 400, and the compression engine 300 is located in the communication engine 200; wherein, only one processor is shown in the figure, and each processor 100 includes one or more cores; the processor 100, the communication engine 200 and the memory 400 are connected through an on-chip bus (not shown in the figure), and data is exchanged through the on-chip bus. The multiple cores of the computing device 10 respectively calculate a certain operator in the AI ​​model through parallel computing, and each generates a slice in the original tensor corresponding to the operator, and each core writes the generated slice into the memory 400 to obtain the complete original tensor corresponding to the above operator. When the computing device calls the communication engine 200 to transmit data between computing devices, the communication engine 200, after obtaining the data in the memory 400, calls the compression engine 300 to compress the data to obtain compressed floating-point data. The compression engine 300 sends the compressed floating point data to the communication engine 200, which encapsulates the compressed floating point data and sends it to other computing devices. The structure of the compression engine 300 and the method for compressing floating point numbers by the compression engine 300 are described in detail below and will not be repeated here.

[0085] like Figure 5 As shown, Figure 5 It is a schematic diagram of another computing device provided by an embodiment of the present application. In the computing device 20, the compression engine 300 is deployed in the core. After a core generates a slice of the original tensor, the core calls the compression engine 300 to compress the data to obtain compressed floating-point data. When each core writes the compressed floating-point data into the memory 400, it will be stored in the storage space corresponding to multiple discontinuous addresses, that is, the original tensor is a plurality of discontinuous data fragments in the memory, and the memory 400 records the address information of the floating-point data written by each core through a scatter gather list (SGL). The communication engine 200 reads the SGL, reads the floating-point data in the memory 400 according to the address information in the SGL, encapsulates it, and sends it to other computing devices. The structure of the compression engine 300 and the method for compressing floating-point numbers by the compression engine 300 are described in detail later, and will not be repeated here.

[0086] like Figure 6 As shown, Figure 6It is a schematic diagram of another computing device provided by an embodiment of the present application. In the computing device 30, the compression engine 300 is deployed in the computing device 30, which is a module in the computing device 30 that is independent of the processor 100, the communication engine 200 and the memory 400, wherein one or more processors 100, the communication engine 200, the compression engine 300 and the memory 400 are connected through an on-chip bus (not shown in the figure), and data is exchanged through the on-chip bus. The multiple cores of the computing device 10 respectively calculate a certain operator in the AI ​​model, and each generates a slice in the original tensor corresponding to the operator, and each core writes the generated slice into the memory 400 to obtain the complete original tensor corresponding to the above operator. The computing device 30 calls the compression engine 300 to compress the floating-point numbers in the original tensor, obtains the compressed floating-point data, and writes the compressed floating-point data into the memory 400 again. When the computing device 30 calls the communication engine 200 for data transmission between computing devices, the communication engine 200 obtains the compressed floating-point data from the memory 400, encapsulates the compressed floating-point data and sends it to other computing devices. The structure of the compression engine 300 and the method by which the compression engine 300 compresses floating-point numbers will be described in detail later and will not be repeated here.

[0087] It should be noted that the above-mentioned original tensor refers to the data generated by the computing device that needs to be sent to other computing devices. The compression engine 300 obtains the data of a data packet each time. Due to the limitation of the data packet, the data packet includes part of the data in the original tensor. For example, the original tensor is a 1-megabyte floating point number, and the data packet only includes 512 bytes of floating point numbers. After obtaining the floating point number of the above-mentioned data packet, the compression engine 300 compresses the multiple floating point numbers included in the data packet to obtain compressed floating point data.

[0088] It should be noted that the structure of the above-mentioned computing device is only an example and cannot be understood as a specific limitation. The computing device can also have other structures, which are not specifically limited in the embodiments of the present application. The above-mentioned communication engine 200 can interact with other computing devices through technologies such as direct memory access (DMA), remote direct memory access (RDMA), RDMA over converged ethernet (RoCE) and system direct memory access (SDMA). The above-mentioned other computing devices may be other processors in the same server or processors in other servers, which are not specifically limited in the embodiments of the present application.

[0089] The structure of the compression engine 300 and the method by which the compression engine 300 compresses floating-point data are introduced below.

[0090] like Figure 7 As shown, Figure 7 It is a schematic diagram of a compression engine provided by an embodiment of the present application. The compression engine 300 can realize the compression of floating-point numbers. When the computing devices need to exchange floating-point data with each other, the amount of data exchanged between the computing devices can be reduced, and the data transmission efficiency can be improved. For example, in the process of using a distributed cluster to train an AI model, the amount of data exchanged between the computing devices used for training can be reduced, and the model training speed can be improved. The compression engine 300 includes an extraction module 310, an exponent encoding module 320, and a data merging module 330. The extraction module 310 is used to obtain multiple floating-point numbers, extract the exponents of each floating-point number from the multiple floating-point numbers, and obtain the exponent values ​​corresponding to the exponents of each floating-point number. Among them, each floating-point number corresponds to an exponent, and each exponent has a corresponding exponent value, that is, each floating-point number corresponds to an exponent value; the exponent values ​​corresponding to different floating-point numbers may be the same or different. For example, the compression engine 300 obtains 100 floating-point numbers, and 100 floating-point numbers correspond to extracting 100 exponent values. Among these 100 exponent values, some exponent values ​​may be the same. For example, the exponent value corresponding to three floating-point numbers is 4, and the exponent value corresponding to four floating-point numbers is 6.

[0091] The exponent encoding module 320 is used to encode the exponent values ​​of each floating-point number extracted by the extraction module 310, respectively, to obtain the codes corresponding to each exponent value. Among them, in the codes corresponding to each exponent value, the length of the codes corresponding to some or all of the exponent values ​​is less than the length of the exponent value, thereby achieving compression of the floating-point number. For example, for the 11-bit exponent in the above-mentioned FP64 floating-point number, the length of the codes corresponding to some exponent values ​​is less than 11 bits. The data merging module 330 is used to merge the codes corresponding to each exponent value and the mantissa corresponding to each floating-point number to obtain compressed floating-point data. It should be understood that each floating-point number also includes a sign bit, and the above-mentioned data merging module 330 is used to merge the codes corresponding to the exponent values ​​of each floating-point number, the mantissa corresponding to each floating-point number and the sign bit of each floating-point number.

[0092] In the embodiment of the present application, after the extraction module 310 extracts the exponent value of each floating point number, the exponent encoding module 320 first determines the reference value corresponding to each exponent value, and then encodes each exponent value according to each exponent value and the reference value corresponding to each exponent value to obtain the code corresponding to each exponent value. It should be noted that the method for determining the reference value of each exponent value and the method for encoding the exponent value according to the exponent value and the reference value will be described later, and will not be repeated here.

[0093] In one possible implementation, Figure 8 As shown, Figure 8 It is a schematic diagram of another compression engine provided by an embodiment of the present application. The above-mentioned compression engine 300 also includes a metadata generation module 340, which is used to generate metadata, and the metadata includes the data types of the above-mentioned multiple floating-point numbers and the reference values ​​corresponding to each exponent value. Among them, the data type includes the type corresponding to floating-point numbers such as FP128, FP64, FP32 or FP16; the reference value in the metadata is used for the computing device that receives the compressed floating-point data to decode the floating-point data, so as to restore each exponent value according to the encoding and reference value corresponding to each exponent value. The data merging module 330 is used to merge the encoding corresponding to each exponent value, the mantissa of each floating-point number and the metadata to obtain compressed floating-point data.

[0094] In one possible implementation, Fig. 9 As shown, Fig. 9 It is a schematic diagram of another compression engine provided by an embodiment of the present application. The compression engine 300 also includes a mantissa compression module 350, which is used to compress the mantissa values ​​corresponding to the mantissas of each floating-point number to obtain compressed values ​​corresponding to each mantissa value. When the compression engine 300 includes the mantissa compression module 350, the above-mentioned extraction module 310 is also used to extract the mantissas of each floating-point number from the above-mentioned multiple floating-point numbers to obtain multiple mantissa values. Among them, each floating-point number corresponds to a mantissa, and a mantissa corresponds to a mantissa value, that is, a floating-point number corresponds to a mantissa value. When compressing the mantissa, the mantissa compression module 350 can achieve compression of the mantissa by intercepting the mantissa. For the mantissa of any type of floating-point data, the mantissa compression module 350 can intercept a portion of a preset length or a preset proportion in the mantissa value as the compressed value corresponding to the mantissa value. For example, the mantissa compression module 350 intercepts the first half of the characters in the mantissa value as a compressed value corresponding to a mantissa value, or the mantissa compression module 350 intercepts the first 5 bits of the mantissa value as the compressed value corresponding to each mantissa. For example, for FP32 floating point numbers, the mantissa part includes 23 bits, and the first 12 bits of each mantissa value can be intercepted as the compressed value corresponding to the mantissa value; for FP16 floating point numbers, the mantissa part includes 10 bits, and the first 5 bits of each mantissa value can be intercepted as the compressed value corresponding to the mantissa value.

[0095] Optionally, the mantissa compression module 350 can compress the mantissa according to the configured mantissa compression level. The mantissa compression level is used to indicate the compression engine to compress the mantissa and the length of the compressed value obtained after compressing the mantissa value. Different levels correspond to different lengths of the compressed value after compression. For example, the mantissa compression level includes 4 levels, 0 to 3, where level 0 means that the mantissa is not compressed, level 1 means that the first 60% of the data in the mantissa value is retained; level 2 means that the first 50% of the data in the mantissa value is retained; level 3 means that the first 40% of the data in the mantissa value is retained. For FP16 floating-point data, the mantissa actually includes 10 bits, level 0 means that the mantissa is not compressed, level 1 means that the length of the compressed value obtained by compressing the mantissa value is 6 bits, level 2 means that the length of the compressed value obtained by compressing the mantissa value is 5 bits, and level 3 means that the length of the compressed value obtained by compressing the mantissa value is 4 bits. For example, if the mantissa compression level is 3, and the mantissa value corresponding to an FP16 floating-point number is 0011101011, then the compressed value corresponding to the mantissa value after compression is 0011.

[0096] It should be understood that when the compression engine 300 includes the mantissa compression module 350, the metadata generated by the metadata generation module 340 also includes a compression type and a mantissa compression level. As shown in Table 1 below, the metadata may include the fields shown in Table 1 below. The compression type includes four compression types: no compression of floating point numbers, compression of exponents only, compression of mantissas only, and compression of both exponents and mantissas; the mantissa compression level is used to indicate the length of the compressed value obtained after the mantissa value is compressed.

[0097] The data merging module 330 is used to merge the codes corresponding to the exponent values, the compressed values ​​corresponding to the compressed mantissa values, and the metadata to obtain the compressed floating point data. The metadata is used for the computing device that receives the compressed floating point data to restore the floating point number according to the various domains included in the metadata. The method for the computing device to restore the floating point number according to the metadata will be described later and will not be described here.

[0098] Table 1

[0099]

[0100] It should be understood that the above-mentioned compression type and mantissa compression level can be configured by the user according to needs. The user can configure the compression engine 300 to compress only the exponent of the floating point number, or to compress both the exponent and the mantissa of the floating point number. When configuring the compression of the mantissa, it is also necessary to configure the mantissa compression level so that the compression engine 300 determines the length of the mantissa after compression. For example, when training an AI model, when a shorter training time is required, the compression type can be set to compressed exponent and mantissa, and set to a higher mantissa compression level, so that the length of the compressed value after the mantissa is compressed is shorter, thereby minimizing the amount of data that needs to be transmitted and reducing the delay caused by communication between computing devices.

[0101] The following describes a method in which the index encoding module 320 determines the reference value corresponding to each index value.

[0102] When the index coding module 320 determines the reference value according to each index value, the index coding module 320 divides the above multiple index values ​​into n sets. Then, according to the different index values ​​in each set, a reference value corresponding to the multiple index values ​​in each set is determined to obtain m reference values; wherein n is a positive integer greater than or equal to 1, and m is a positive integer less than or equal to n.

[0103] After the above-mentioned multiple index values ​​are divided into n sets, when determining the reference values ​​corresponding to the multiple index values ​​in each set, the reference value corresponding to each set may be determined in the following three ways.

[0104] The first way, such as Fig.10 As shown, Fig.10 : is a schematic diagram of determining a reference value provided by an embodiment of the present application. For multiple index values ​​in each set, the index encoding module 320 determines a reference value through the multiple index values ​​in each set, that is, each set in the n sets corresponds to a reference value E bi , where i is a positive integer less than or equal to n. For the first set of the n sets, the index encoding module 320 counts the index value with the largest number of occurrences in the first set, and uses the index value with the largest number of occurrences as the reference value E corresponding to the K index values ​​in the first set. b1 Alternatively, the index encoding module 320 samples the K index values ​​in the first set to obtain multiple sample values; then determines the three sample values ​​that appear the most times among the multiple sample values, and uses the average value of the three sample values ​​that appear the most times as the reference value E corresponding to the K index values ​​in the first set. b1 It should be noted that the first set is one of the n sets, and for any one of the n sets, the reference values ​​corresponding to the multiple index values ​​in each set are determined by the above method.

[0105] The second method is to determine the reference value corresponding to each set. For the first set among the n sets, first sample the K index values ​​in the first set to obtain K1 sample values. For these K1 sample values, first divide the K1 sample values ​​into t1 groups, determine the median of the sample values ​​in each group, and obtain t1 medians; then divide the t1 medians into t2 groups, and then determine the median in each of the t2 groups to obtain t2 medians. And so on, until the multiple medians finally obtained cannot be further grouped. For example, after grouping, there are less than three values ​​in each group. The median corresponding to these multiple medians is used as the reference value corresponding to the set.

[0106] Exemplarily, the first set includes 100 index values, and the index encoding module 320 samples the 100 index values ​​to obtain 27 index values. The index encoding module 320 first divides the 27 index values ​​into groups of 3 to obtain 9 groups of data, determines the median in each group of data, and obtains 9 medians; then divides the 9 medians into groups of 3 to obtain 3 groups of data, determines the median in each group of data, and obtains 3 medians, and finally uses the median corresponding to the 3 medians as the reference value corresponding to the first set.

[0107] The third method, such as Fig.11 As shown, Fig.11 3 is another schematic diagram of determining a reference value provided by an embodiment of the present application. For multiple index values ​​in each set, the index encoding module 320 uses the reference value calculated based on the multiple index values ​​in the previous set as the reference value corresponding to the multiple index values ​​in the next set. That is, a reference value E is calculated based on the multiple index values ​​in the i-th set. bi Then, the reference value E bi As the reference value corresponding to the multiple exponent values ​​in the i+1th set, when encoding the multiple exponent values ​​in the i+1th set, based on the reference value E bi Encoding. For the first set among the n sets, since there is no corresponding benchmark value, the benchmark value corresponding to the set can be considered to be 0; for the last set among the n sets, the benchmark value is not calculated based on the multiple index values ​​in the set. It should be noted that when determining a benchmark value based on multiple index values ​​in a set, the method in the first method or the second method mentioned above can be used. Among them, the method for determining a benchmark value based on multiple index values ​​in each set can refer to the method for determining the benchmark value in the first method mentioned above.

[0108] It should be understood that when n is equal to 1, that is, the index encoding module 320 does not divide the multiple index values ​​into multiple sets, the index encoding module 320 can only determine the reference values ​​corresponding to the multiple index values ​​by the first method or the second method.

[0109] It should be noted that when the index encoding module 320 determines the reference value corresponding to each set according to any of the above methods, when sampling, it may not sample the value with an exponent value of 0.

[0110] The following describes the operation process of encoding the index value by the index encoding module 320. After the index encoding module 320 determines the reference value corresponding to each index value, each index value can be encoded by the following three methods based on each index value and the reference value corresponding to each index value.

[0111] In the first method, the index coding module 320 encodes the index value according to the difference between the index value and the reference value corresponding to the index value. The index coding module 320 encodes the difference and then uses the code corresponding to the difference as the code corresponding to the index value. As shown in Table 2 below, Table 2 is a coding table provided in an embodiment of the present application, and the coding table includes codes corresponding to different differences between the index value and the reference value. If the absolute value of the difference between an index value and the reference value corresponding to the index value is less than a first threshold value (the first threshold value in Table 2 is 4), the code corresponding to the difference is determined according to the coding table, and the code of the difference is the code corresponding to the index value; if the absolute value of the difference between an index value and the reference value corresponding to the index value is greater than or equal to the first threshold value, the code corresponding to the difference is determined according to the coding table, and the code corresponding to the index value includes the code corresponding to the difference and the actual value of the index value, that is, the code corresponding to the index value is a combination of the code corresponding to the difference and the actual value of the index value.

[0112] It should be noted that the encoding method for encoding the difference may also be other methods. The correspondence between the difference and the encoding shown in Table 2 is only an example, and the difference and the encoding may also be other correspondences; in addition, for differences whose absolute values ​​are greater than or equal to the first threshold, they are collectively referred to as "other differences" in Table 2, and the encodings corresponding to all other differences are the same; the above-mentioned first threshold is only used as an example and cannot be understood as a specific limitation. The first threshold may also be other values, such as 3, 5 or 6, etc., and the embodiments of the present application do not make specific limitations.

[0113] Table 2

[0114]

[0115]

[0116] When the index encoding module 320 performs encoding according to the index value in each set and the reference value corresponding to the index value in each set, for a first set among the n sets, the index encoding module 320 determines the index value E corresponding to each index value in the first set and the reference value E corresponding to the first set. b1Then, according to the difference between each index value and the reference value, the index values ​​in the first set are encoded respectively in combination with the above Table 2 to obtain the codes corresponding to the index values ​​in the first set. It should be understood that the above first set is any one of the above n sets, and for each of the n sets, the index encoding module 320 will perform the above operation on the first set to obtain the codes corresponding to the index values ​​in each set.

[0117] Exemplarily, if it is necessary to compress the floating point number of FP32, the exponent value includes 8 bits, and the reference value is 3. For an exponent with an exponent value of 2, the difference between the exponent value and the reference value is -1, and the absolute value of the difference is less than the first threshold value 4. According to the coding table, the code corresponding to the difference is 10, and the code corresponding to the exponent value 2 is 10. For an exponent with an exponent value of 6, the difference between the exponent value and the reference value is 3, and the code corresponding to the exponent value 6 is 11110. For an exponent with an exponent value of 8, it is represented in binary as 00001000, the difference between the exponent value and the reference value is 5, and the absolute value of the difference is greater than the first threshold value 4. The code corresponding to the difference is 111111, and the code corresponding to the exponent value 8 is 11111100001000.

[0118] Through the above compression method, since the reference value is an exponential value that appears more frequently in a set, when multiple exponential values ​​in a set are respectively subtracted from the reference value, the number of 0s in the multiple differences obtained is the largest. After encoding the differences that are 0 according to the above encoding method, the exponential value corresponding to the difference of 0 can be compressed to 2 bits; and because multiple exponential values ​​usually satisfy a normal distribution, the proportion of differences whose absolute values ​​are less than the first threshold in the above multiple differences is also relatively large. Therefore, the differences whose absolute values ​​are less than the first threshold in the differences are also encoded according to the above encoding method. Since the bits occupied by the encoding are smaller than those of each exponential value, the number of differences whose absolute values ​​are less than the first threshold in the differences accounts for a relatively large proportion, so that most of the exponential values ​​can be compressed through the above method, thereby achieving compressed floating-point numbers, reducing the amount of data exchanged between different computing devices, and improving data transmission efficiency.

[0119] The second method is that when the exponent coding module 320 determines the reference value corresponding to each set, if when sampling the exponent values ​​in each set, the exponent with an exponent value of 0 is not sampled, when encoding the exponent value, the exponent coding module 320 encodes the absolute value of the difference between the exponent value and the reference value for non-0 exponent values, and adds a sign bit to indicate the sign of the difference when encoding; for the exponent with an exponent value of 0, it is directly encoded as 0. Specifically, as shown in Table 3 below, Table 3 is a coding table provided in an embodiment of the present application. In Table 3, for the two cases where the difference is 0 and the exponent value is 0, the corresponding codes are all 0, and the sign bit is used to distinguish whether the code 0 indicates that the difference is 0 or the exponent value is 0. When the sign position is 1 and the code is 0, it indicates a difference of 0; when the sign position is 0 and the code is 0, it indicates an exponent value of 0. If the absolute value of the difference between an index value and the reference value corresponding to the index value is less than a second threshold value (the second threshold value is 7 in Table 3), the code corresponding to the index value is the code of the corresponding difference; if the absolute value of the difference between an index value and the reference value corresponding to the index value is greater than or equal to the second threshold value, the code corresponding to the difference is determined according to the coding table, and the code corresponding to the index value includes the code corresponding to the difference and the actual value of the index value, that is, the code corresponding to the index value is a combination of the code corresponding to the difference and the actual value of the index value.

[0120] For example, if the reference value is 2, when the sign bit is 0, it indicates that the difference between the exponent value and the reference value is positive, and when the sign bit is 1, it indicates that the difference between the exponent value and the reference value is negative. For an exponent with an exponent value of 2, that is, when the difference is 0, the code corresponding to the difference is 0, and the sign bit corresponding to the code is assigned to 1; for an exponent with an exponent value of 0, the code corresponding to the exponent value is 0, and the sign bit corresponding to the code is assigned to 0. For an exponent with an exponent value of 4 or -4, the absolute value of the difference is 2, and the codes corresponding to the exponent values ​​4 or -4 are both 110, but for the exponent value 4, the sign bit corresponding to the code of the exponent value is assigned to 0; for the exponent value -4, the sign bit corresponding to the code of the exponent value is assigned to 1. For an exponent with an exponent value of 9, the absolute value of the difference is 7, and the code of the difference is 11111110. The binary representation of the exponent value 9 is 00001001, so the code of the exponent value 9 is 1111111000001001, and the sign bit corresponding to the code is assigned to 0.

[0121] Table 3

[0122]

[0123]

[0124] It should be noted that the encoding method for encoding the absolute value of the difference may also be other methods. The correspondence between the absolute value of the difference and the encoding shown in Table 3 is only an example, and there may be other correspondences between the difference and the encoding; in addition, for the difference whose absolute value is greater than or equal to the second threshold, it is collectively referred to as "other differences" in Table 3, and the encoding corresponding to the absolute values ​​of all other differences is the same. It should be understood that the above second threshold is only used for example and cannot be understood as a specific limitation. The second threshold may also be other values, such as 3, 5 or 6, etc., which are not specifically limited in the embodiments of the present application.

[0125] In scenarios such as AI training, the data that needs to be transmitted between computing devices is usually sparse, that is, the data to be transmitted usually includes more 0 values. By compressing the exponential value of 0 into a one-bit code, the amount of data that needs to be transmitted can be greatly reduced. At the same time, a sign bit is used to indicate the sign of the difference between the exponential value and the reference value, so that differences with the same absolute value can use the same encoding, so that more differences can be compressed through the above encoding, that is, more exponential values ​​can be compressed, thereby better reducing the amount of compressed floating-point data.

[0126] The third method is that after the index coding module 320 determines the reference values ​​corresponding to the above-mentioned sets, for any set, such as the above-mentioned first set, the index coding module 320 adds integers whose absolute values ​​are less than the third threshold to the reference values ​​corresponding to the first set, and obtains multiple values. For example, if the third threshold is 4, the reference value is added with ±1, ±2, and ±3, etc. Then, these multiple values ​​are encoded to obtain multiple codes. When encoding multiple index values ​​in the first set, the multiple index values ​​in the first set are matched with the above-mentioned multiple values ​​respectively. If an index value is the same as one of the values, the code corresponding to the value is used as the code corresponding to the index value; if an index value does not match any of the above-mentioned multiple values, the code corresponding to the index value includes the codes corresponding to other values ​​and the index value, and the other values ​​refer to values ​​other than the above-mentioned multiple values. It should be understood that the above-mentioned third threshold is only used as an example and cannot be understood as a specific limitation.

[0127] As shown in Table 4, Table 4 is a coding table between multiple numerical values ​​and codes. In Table 4, the reference value is 6, that is, when the reference value corresponding to the multiple exponent values ​​in the first set is 6, the exponent coding module can generate a coding table as shown in Table 4. When an exponent value in the first set is 6, the code corresponding to the exponent value is 00. If an exponent value in the first set is 8, the code corresponding to the exponent is 110. If an exponent value is 10, and the exponent value occupies 8 bits for the above-mentioned multiple floating-point FP32 type data, the exponent value 10 is represented in binary as 00001010, and the code corresponding to the exponent value 10 is 11111100001010.

[0128] Table 4

[0129] Numeric coding 6 00 7 01 5 10 8 110 4 1110 9 11110 3 111110 Other values 111111

[0130] It should be noted that when the above-mentioned multiple exponents are divided into multiple sets, each set will generate a coding table between a numerical value and a code according to the reference value and the third threshold value after determining a reference value, and when the reference values ​​corresponding to the two sets are different, the mapping relationship between the numerical value and the code is also different. Exemplarily, if the corresponding reference value in the second set is 4, the corresponding coding table may be as shown in Table 5 below. When an exponential value in the second set is 4, the code corresponding to the exponential value is 00, when an exponential value in the second set is 6, the code corresponding to the exponential value is 110, if an exponent value is 8, and the above-mentioned multiple floating-point numbers are FP32 type data, the exponent value 8 is represented in binary as 00001000, then the code corresponding to the exponent value 8 is 11111100001000.

[0131] Table 5

[0132] Numeric coding 4 00 5 01 3 10 6 110 2 1110 7 11110 1 111110 Other values 111111

[0133] As shown in Tables 4 and 5 above, for different sets, when the reference values ​​corresponding to the two sets are different, the codes corresponding to different exponent values ​​are different, and the same code represents different exponent values. In the embodiment of the present application, the code corresponding to the reference value is used as the reference code, and the distance between each code and the reference code is defined, for example, the reference code is 00. The distance between a code and code 00 indicates the difference between the numerical value corresponding to the code and the reference value corresponding to code 00. For example, the distance between code 01 and code 00 is defined as 1, and the distance between code 10 and code 00 is -1; the distance between code 110 and code 00 is 2, and the distance between code 1110 and code 00 is -2; the distance between code 11110 and code 00 is 3, and the distance between code 111110 and code 00 is -3. Through the above definition, when the corresponding exponent value is restored according to the code, the distance between the code and 00 is first determined, and then the distance is added to the reference value to obtain the corresponding numerical value, which is the exponent value corresponding to the code.

[0134] It should be noted that the embodiment of the present application does not specifically limit the number of exponent values ​​in each of the above sets. Preferably, the number of exponent values ​​in each set is the number of data that can be processed by the compression engine in one clock cycle. For example, if a data packet acquired by the compression engine each time includes 512 bytes of FP16 floating-point numbers, that is, 256 floating-point numbers, and the compression engine can process 128 bytes of data in one clock cycle, that is, 64 FP16 floating-point numbers, then the floating-point numbers in the data packet can be divided into 4 sets, and one set includes 64 floating-point numbers.

[0135] Through the above compression method, since the reference value is an exponential value that appears more often in a set, and because multiple exponential values ​​usually satisfy a normal distribution, when the reference value corresponding to a set is added to integers whose absolute values ​​are less than the second threshold to obtain multiple values, these multiple values ​​are the same as the exponential values ​​that account for a large proportion in the set, and then these multiple values ​​are encoded, which is equivalent to encoding a part of the exponential values ​​that account for a large proportion in the set. Since the length of the code is less than each exponential value, a part of the exponential values ​​that account for a large proportion in the set can be compressed after being represented by a shorter length code. Therefore, most of the exponential values ​​can be compressed in the above manner to achieve compressed floating-point numbers, reduce the amount of data exchanged between different computing devices, and improve data transmission efficiency. For example, reduce the amount of data exchanged between different computing devices during AI model training and improve model training efficiency.

[0136] In the embodiment of the present application, after the compression engine 300 compresses multiple floating-point numbers, the computing device needs to send the floating-point data including metadata obtained after compression to the communication engine 200, or write the floating-point data into the memory 400, and the communication engine 200 obtains the floating-point data corresponding to the multiple floating-point numbers from the memory 400, and then the communication engine 200 encapsulates the floating-point data into a data packet and sends it to other computing devices. When the communication engine 200 sends a data packet, the length of the payload in a data packet is usually fixed. For example, the payload of an Ethernet data packet is 1500 bytes. In the embodiment of the present application, the payload of a data packet is 512 bytes as an example. If a data packet obtained by the compression engine 300 includes 512 bytes of FP16 floating-point numbers, that is, 256 floating-point numbers. After the compression engine 300 compresses the 256 floating-point numbers in this data packet, a set of corresponding floating-point data is obtained. After the communication engine 200 obtains this group of compressed floating-point data, since the floating-point data is less than 512 bytes, for example, the size of a 512-byte floating-point number after compression is 394 bytes, the communication engine 200 will fill the 394 bytes into 512 bytes when encapsulating, and then send it out after encapsulation.

[0137] In a possible implementation, the communication engine 200 may pad the floating-point data in a preset manner when encapsulating the floating-point data. For example, the payload is aligned to 10 bytes or 20 bytes. Taking 20-byte padding as an example, the payload in the data packet sent by the communication engine 200 is an integer multiple of 20 bytes. If the length of a group of floating-point data obtained by the compression engine after compressing the floating-point numbers in a data packet is 235 bytes, after the communication engine 200 obtains this group of 235 bytes of floating-point data, the communication engine 200 will fill 5 bytes to pad the payload to 240 bytes, and then add a message header to the 240-byte data for encapsulation and send it to other computing devices.

[0138] In one possible implementation, when the communication engine 200 pads the data in the data packet, as shown in Table 6 below, the metadata generated by the compression engine 300 also includes a data length, and the data length is used to indicate the length of the floating-point data obtained by the communication engine 200, so that the communication engine 200 determines the length of the data that needs to be filled.

[0139] It should be noted that each time the compression engine 300 obtains a floating point number in a data packet, it compresses the multiple floating point numbers included in the data packet to obtain a group of floating point data. Since the reference value in the metadata generated during the compression process is only applicable to recovering the group of floating point data, the communication engine 200 obtains a group of floating point data from the compression engine 300 or the memory 400 each time to complete and encapsulate.

[0140] Table 6

[0141]

[0142] In a possible implementation, if the payload of each data packet sent by the communication engine 200 needs to be of a fixed length, for example, the fixed length of the payload is 512 bytes, the compression engine 300 may compress the exponent values ​​of the floating point numbers in multiple data packets with the same reference value when compressing the floating point numbers in the received data packets, to obtain multiple groups of floating point numbers, and then the communication engine 200 pads the multiple groups of floating point data with 512 bytes and encapsulates them.

[0143] Exemplarily, after obtaining the first data packet, the compression engine 300 determines the reference values ​​corresponding to the multiple floating-point numbers in the data packet, and compresses the exponent value to obtain a group of floating-point data. If the length of the group of floating-point data is less than 512 bytes, after obtaining the multiple floating-point numbers in the second data packet, the compression engine 300 continues to compress the multiple floating-point numbers in the second data packet using the reference value corresponding to the first data packet, so that the metadata of the first data packet and the second data packet are the same. If the data length of the first data packet after compression is 215 bytes, the data length of the second data packet after compression is 230 bytes, and the length of the metadata is 10 bytes, then when the communication engine 200 fills the two groups of floating-point data with 512 bytes, it needs to supplement 57 bytes before encapsulating and sending them together.

[0144] The above describes the computing device provided by the embodiment of the present application and the method for compressing data using the computing device. After the compression engine 300 compresses the floating point number to obtain the compressed floating point data, the communication engine 200 obtains the compressed floating point data, and uses the compressed floating point data as a payload, encapsulates it into a message, and sends it to other computing devices. According to the compression method of the compression engine 300 for compressing the floating point number, the compressed floating point data includes a metadata part, a coding part, a mantissa part, and an exponent value part. Fig.12 As shown, Fig.12 It is a schematic diagram of a payload provided in an embodiment of the present application. Fig.12 In the above, the metadata part is used to store the domain segments shown in Table 1 or Table 6; when the compression engine 300 does not compress the mantissa of the floating-point number, the mantissa part is used to store the sign bit and mantissa value of each floating-point number; when the compression engine 300 compresses the mantissa part of the floating-point number, the mantissa part is used to store the compressed value of the sign bit and mantissa value of the floating-point number; the encoding part is used to store the encoding corresponding to each exponent value; the exponent value part is used to store the real exponent value of some exponents. For example, when the exponent value is encoded by the first method, the exponent value part is used to store the exponent value whose absolute value of the difference with the reference value is greater than or equal to the first threshold; when the exponent value is encoded by the second method, the exponent value part is used to store the exponent value whose absolute value of the difference with the reference value is greater than or equal to the second threshold; when the exponent value is encoded by the third method, the exponent value part is used to store the exponent value greater than or equal to the third threshold.

[0145] Among them, the sign bit of each floating-point number in the compressed multiple floating-point numbers is stored together with the mantissa value of the floating-point number or the compressed value corresponding to the mantissa value of the floating-point number, that is, when storing the mantissa part, the sign bit of the first floating-point number and the compressed value corresponding to the mantissa value of the first floating-point number are stored first, and then the sign bit of the second floating-point number and the compressed value corresponding to the mantissa value of the second floating-point number are stored, and so on, until the compression engine 300 stores the compressed values ​​corresponding to the sign bits and mantissa values ​​corresponding to the above-mentioned multiple floating-point numbers in the above-mentioned mantissa part.

[0146] It should be understood that when the compression engine 300 uses the above-mentioned first method to compress the exponent value, that is, when the difference is encoded through the coding table shown in the above Table 2, the encoding part is used to store the code corresponding to the difference between each exponent value and the reference value; if the compression engine adopts the above-mentioned third method to compress the exponent value, that is, when the difference is encoded through the coding table shown in the above Table 4 or Table 5, the above-mentioned encoding part is used to store the code corresponding to each numerical value.

[0147] When the compression engine 300 compresses the exponent value by the second method, that is, encodes the absolute value of the difference between the exponent value and the reference value by the encoding table shown in Table 3, the format of the payload part generated by the compression engine 300 is as follows: Fig.13 As shown, Fig.13 It is a schematic diagram of another effective load provided in an embodiment of the present application. Fig.13 The payload portion in also includes a sign bit corresponding to the encoding, and the sign bit corresponding to the encoding includes a sign corresponding to the difference between a non-zero exponent value and a reference value and a sign of an exponent whose exponent value is 0.

[0148] It should be noted that when the compression engine 300 generates a payload in the above format, there is a one-to-one correspondence between the data stored in the encoding part and the data stored in the mantissa part. If the compression engine compresses a total of 256 floating-point numbers, and compresses both the exponent value and the mantissa value, the mantissa part first stores the compressed value corresponding to the sign bit of the first floating-point number and the mantissa value of the first floating-point number, and then stores the compressed value corresponding to the sign bit of the second floating-point number and the mantissa value of the second floating-point number; the encoding part also first stores the code corresponding to the first floating-point number, and then stores the code corresponding to the second floating-point number. If the payload also includes a sign bit corresponding to the code, the sign bit part corresponding to the code first stores the sign of the difference between the exponent value of the first floating-point number and the reference value; and then stores the sign of the difference between the exponent value of the second floating-point number and the reference value.

[0149] After the computing device uses the above compression engine to compress the floating point number to reduce the amount of data transmitted, the computing device that receives the compressed floating point data needs to decompress the floating point data to obtain a complete floating point number. The decompression engine provided by the embodiment of the present application is introduced below.

[0150] The embodiment of the present application also provides another computing device, which also includes a decompression engine. When the computing device receives compressed floating-point data sent by other computing devices, it can call the decompression engine to decompress the compressed floating-point data and restore multiple floating-point numbers. The computing device 40 includes one or more processors 100, a communication engine 200, a compression engine 300, a decompression engine 500 and a memory 400. Among them, the compression engine 300 can be deployed in the communication engine 200, or it can be deployed in the core. The compression engine 300 can also be an independent module outside the processor 100, the communication engine 200 and the memory 400; the decompression engine 500 can be deployed in the communication engine 200, or it can be an independent module outside the processor 100, the communication engine 200 and the memory 400. The structure of the compression engine 300 and the method for compressing floating-point numbers can refer to the above Figures 7 to 13 The operations in the related description, the structure of the decompression engine 500 and the method by which the decompression engine 500 decompresses the compressed floating-point data will be described later and will not be repeated here.

[0151] For example, Fig.14 As shown, Fig.14 2 is a schematic diagram of another computing device provided in an embodiment of the present application. The compression engine 300 and the decompression engine 500 are both deployed in the communication engine 200. Fig.15 As shown, Fig.15 Schematic diagram of another computing device provided by an embodiment of the present application. The compression engine 300 is deployed in the core, and the decompression engine 500 is an independent module outside the processor 100, the communication engine 200 and the memory 400. Fig.16 As shown, Fig.16 is a schematic diagram of another computing device provided by an embodiment of the present application. The above compression engine 300 and decompression engine 500 are both independent modules outside the processor 100, the communication engine 200 and the memory 400. It should be understood that the above is only an example, and the compression engine 300 and the decompression engine 500 can also be deployed in other forms, such as the compression engine 300 is deployed in the core, and the decompression engine 500 is deployed in the communication engine.

[0152] The structure of the decompression engine 500 and the method by which the decompression engine 500 decompresses floating-point data are introduced below.

[0153] like Fig.17 As shown, Fig.17 is a schematic diagram of a decompression engine provided by an embodiment of the present application. Figure 8 As shown, the compression engine 300 can only compress the exponential value, and the decompression engine 500 includes a metadata parsing module 510, a separation module 520, an exponential decoding module 530 and a data recovery module 540. The metadata parsing module 510 is used to obtain the above data type and reference value, and send the data type to the separation module 520, so that the separation module 520 determines the length of each mantissa value according to the data type, and then separates the above mantissa part and the coding part from the payload; the metadata parsing module 510 is also used to send the data type and reference value to the exponential decoding module 530, so that the exponential decoding module 530 determines the length of the exponential value according to the data type, and recovers the exponential value according to the reference value and the coding part. The separation module 520 is used to extract the mantissa part, the coding part and the exponential value part in the payload.

[0154] For the floating-point data obtained by encoding the exponent value by the first method described above, after the exponent decoding module 530 obtains one or more reference values ​​in the encoding part and the metadata, for any one code of the encoding part, for example, the t-th code, after obtaining the t-th code, the exponent decoding module 530 first determines the reference value corresponding to the code and the difference value corresponding to the code, and then adds the difference value to the reference value to obtain the true exponent value corresponding to the code. It should be noted that if the code obtained by the exponent decoding module 530 is 111111, the exponent decoding module 530 obtains the true exponent value corresponding to the code from the exponent value part.

[0155] For example, if the exponent decoding module 530 obtains the code 111111 for the jth time, the exponent decoding module 530 obtains the jth exponent value from the exponent value part, which is the real exponent value corresponding to the code. The exponent decoding module 530 starts from the (j-1)p+1th bit and obtains p consecutive bits, and the value composed of these p bits is the jth exponent value. Wherein, p is the length of the real exponent value. Exemplarily, if the compressed floating-point number is a floating-point number of the FP32 type, that is, the exponent value includes 8 bits, and the reference value corresponding to a set is 4, for the encoding corresponding to the set, if the encoding is 00, the reference value corresponding to the encoding is 4, and the difference value corresponding to the encoding is 0, then the exponent value corresponding to the encoding 00 is 4; if the encoding is 10, and the difference value corresponding to the encoding is -1, then the exponent value corresponding to the encoding 10 is 3; if the encoding is 11110, and the difference value corresponding to the encoding is 3, then the exponent value corresponding to the encoding 11110 is 7; if the encoding is 111111, and the encoding is the third time that the exponent decoding module 530 obtains the encoding 111111, then the exponent decoding module 530 obtains 8 bits of data starting from the 17th bit of the exponent value part, and if the obtained 8 bits of data are 00001001, then the exponent value corresponding to the encoding is 9.

[0156] For floating-point data for which the exponent value is encoded by the second method, after the exponent decoding module 530 obtains one or more reference values ​​in the above-mentioned encoding part and metadata, for any encoding of the encoding part, for example, the t-th encoding, first determine the reference value corresponding to the encoding, the absolute value of the difference corresponding to the encoding, and the sign bit of the difference corresponding to the encoding, and add the difference to the above-mentioned reference value to obtain the true exponent value corresponding to the encoding. It should be noted that if the encoding is 11111110, the exponent decoding module 530 obtains the true exponent value corresponding to the encoding from the exponent value part. For example, if the compressed floating-point number is a floating-point number of the FP32 type, that is, the exponent value includes 8 bits, and the base value corresponding to a set is 4, for the encoding corresponding to the set, if the encoding is 0, the sign bit corresponding to the encoding is 1, indicating that the encoding is the encoding corresponding to the difference value 0, that is, the exponent value corresponding to the encoding is 4; if the encoding is 0, and the sign bit corresponding to the encoding is 0, it indicates that the exponent value corresponding to the encoding is 0; if the encoding is 10, the sign bit corresponding to the encoding is 1, indicating that the difference value corresponding to the encoding is -1, then the The exponent value corresponding to code 10 is 3; if the code is 11110, the sign bit corresponding to the code is 0, indicating that the difference corresponding to the code is 4, and the exponent value corresponding to the code 11110 is 8; if the code is 11111110, the code is the second time that the exponent decoding module 530 obtains the code 11111110, then the exponent decoding module 530 starts from the 9th bit of the exponent value part and obtains 8 bits of data. If the obtained 8 bits of data are 00001000, the exponent value corresponding to the code is 8.

[0157] For floating point data that encodes the exponent value using the third method, the exponent decoding module 530 obtains one or more reference values ​​in the above-mentioned encoding part and metadata. For any code in the encoding part, such as the tth code, the reference value corresponding to the set is determined. If the code is 00, the exponent value corresponding to the code is the reference value. For other codes, based on the distance between the code and code 00, the distance between the code and code 00 is added to the reference value to obtain the exponent value corresponding to the code. It should be noted that if the code is 111111, the exponent decoding module 530 obtains the real exponent value corresponding to the code from the exponent value part.

[0158] Exemplarily, the reference value corresponding to a set is 4. For the code corresponding to the set, if the code is 00, the exponent value corresponding to the code is 4; if the code is 10, the distance between the code and the code 00 is -1, then the exponent value corresponding to the code 10 is 3; if the code is 11110, the distance between the code and the code 00 is 3, then the exponent value corresponding to the code 11110 is 7.

[0159] The data recovery module 540 is used to recover each floating point number according to the exponent value corresponding to the exponent of each floating point number and the sign bit and mantissa value of the floating point number in the mantissa part. For example, the compressed multiple floating point numbers are FP32 type data. For one of the floating point numbers, the value of the exponent of the floating point number is 8 according to the encoding part, and the exponent is converted into an 8-bit binary number. The sign bit of the floating point number and the 23-bit mantissa are obtained from the mantissa part, and the true value of the floating point number can be obtained.

[0160] In one possible implementation, Fig.18 As shown, Fig.18 It is a schematic diagram of a decompression engine provided by an embodiment of the present application. If the compression engine 300 includes a mantissa compression module 350, the decompression engine 500 also includes a mantissa recovery module 550. The metadata also includes a compression type and a mantissa compression level. The data type indicates the data type of the multiple floating-point numbers; the compression type indicates whether the compression engine 300 compresses the floating-point numbers, and whether the compression engine 300 compresses the exponent or the mantissa, or both the exponent and the mantissa; the mantissa compression level indicates the length of the compressed value obtained after the compression engine 300 compresses the mantissa value. If the compression engine 300 compresses the mantissa value, the mantissa part of the payload includes the compressed values ​​corresponding to the sign bits of the multiple floating-point numbers and the mantissa values ​​of the multiple floating-point numbers.

[0161] If the separation module 520 determines that the mantissa part includes compressed values ​​according to the compression type, the length of each compressed value can be determined according to the mantissa compression level, and then each compressed value can be extracted from the mantissa part. The mantissa recovery module 550 can determine the length of the mantissa value of each floating point number according to the data type, and determine the length deleted by the compression engine 300 when compressing the mantissa value according to the length of the mantissa value and the length of the compressed value, and finally compensate each compressed value with a random number of corresponding length to obtain the mantissa value corresponding to each mantissa.

[0162] For example, if the separation module 520 determines that the compression engine 300 compresses the exponent and mantissa according to the compression type; determines that the compressed data is FP16 type data according to the data type in the metadata, that is, the mantissa value includes 10 bits; determines that the length of the compressed value after the mantissa value is compressed is 4 bits according to the mantissa compression level, the compression engine 300 deletes 6 bits of characters when compressing the mantissa. After obtaining the encoding part, the separation module 520 extracts a compressed value for every 4 bits, and then the mantissa recovery module 550 randomly generates a 6-bit random number, which is spliced ​​with the 4-bit compressed value to obtain a 10-bit mantissa value.

[0163] The data recovery module 540 is used to match and concatenate the exponent values ​​corresponding to each floating point number output by the exponent decoding module 530 and the mantissa values ​​corresponding to each floating point number output by the mantissa recovery module 550 one by one to obtain each floating point number.

[0164] For the above method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited by the order of the actions described. In addition, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0165] Other reasonable step combinations or reasonable computing device structures that can be thought of by those skilled in the art based on the above description also fall within the scope of protection of this application. Secondly, those skilled in the art should also be familiar with the fact that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0166] The above description is combined with the accompanying drawings to introduce in detail the data processing method provided by the present application and the computing device used to implement the data processing method. The following description is combined with the accompanying drawings to introduce the relevant devices and equipment provided by the present application.

[0167] like Fig.19 As shown, Fig.19 It is a schematic diagram of a data processing device provided in an embodiment of the present application. The data processing device 190 includes a communication unit 191 and a processing unit 192. Among them, the communication unit 191 is used to obtain multiple first data; the processing unit 192 is used to determine a first reference value according to the multiple first data to be compressed; then calculate the difference between each first data and the first reference value; determine the code corresponding to each difference; and obtain the compressed data of the corresponding first data according to each code. The length of some or all of the codes corresponding to each first data is less than the length of the exponent value. It should be understood that the above-mentioned multiple first data can be the exponent values ​​of multiple floating-point numbers.

[0168] When determining the code corresponding to each difference, the processing unit 192 is specifically used to: determine the code corresponding to each difference according to each difference and a coding table, wherein the coding table records the codes corresponding to a plurality of preset difference values. In the coding table, the code corresponding to the difference whose absolute value is greater than the threshold is a fixed value, and the processing unit is specifically used to: when the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as the compressed data of the first data corresponding to the first difference.

[0169] Optionally, the floating-point number also includes a mantissa value; the processing unit 192 can also be used to compress the mantissa values ​​of multiple floating-point numbers respectively to obtain compressed data of each mantissa value; based on the compressed data of the exponent values ​​of multiple floating-point numbers and the compressed data of the mantissa values ​​of multiple floating-point numbers, the compressed data of multiple floating-point numbers is obtained.

[0170] In the embodiment of the present application, after compressing the above-mentioned multiple first data, the computing device will send the compressed data of each first data to other computing devices, and the other computing devices need to decode the compressed data. When the processing unit 192 compresses the multiple first data, it will generate corresponding first metadata, and the first metadata includes the above-mentioned first reference value. The computing device will also send the metadata to other computing devices, so that the other computing devices decode the compressed data according to the compressed data of each first data and the first metadata to obtain the above-mentioned multiple first data.

[0171] Optionally, when compressing multiple floating-point numbers, the above-mentioned first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed.

[0172] In a possible implementation, the communication unit 191 is also used to receive data to be decompressed sent by other devices, where the data to be decompressed includes a coding part and second metadata; wherein the second metadata includes a second reference value, and the coding part includes multiple codes; according to the second reference value and the multiple codes, the multiple codes are decoded respectively to obtain multiple second data.

[0173] In a possible implementation, when the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed also includes a mantissa part, and the second metadata also includes the data type, compression type and mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the compressed data of the mantissa value after the mantissa value is compressed; the processing unit 192 determines the length of the exponent value and the length of the mantissa value of the floating-point number before compression according to the data type; decodes the multiple codes respectively according to the length of the exponent value, the second reference value and the multiple codes to obtain multiple exponent values; when it is determined according to the compression type that the mantissa part in the data to be decompressed is data after the mantissa value is compressed, the compressed data of the mantissa part is decoded according to the mantissa compression level and the length of the mantissa value to obtain multiple mantissa values; restores the floating-point number before compression according to the multiple exponent values ​​and the multiple mantissa values ​​obtained by decoding.

[0174] It should be noted that the data processing device 190 is capable of compressing and decompressing data. The data processing device 190 can implement operations of compressing data by referring to the operations performed by the compression engine in the above embodiment, and can implement operations of decompressing data by referring to the operations performed by the compression engine in the above embodiment, which will not be repeated here.

[0175] The present application also provides a computing device. The structural diagram of the computing device can refer to the above Figures 4 to 6 or Figures 14 to 16 Schematic diagram of the corresponding computing device.

[0176] It should be noted that the computing devices include multiple processors 100, and each processor 100 can exchange data through aggregate communication methods such as all-to-all, allgather or allreduce. Among them, the above-mentioned processor 100 can be a central processing unit (CPU), a graphics processing unit (GPU), an embedded neural network processor (neural-network processing units, NPU), a tensor processor (tensor processing unit, TPU), a data processing unit (data processing units, DPU), an accelerated processing unit (accelerated processing unit, APU), a floating point computing unit (floating processing units, FPU) or an application-specific integrated circuit (application-specific integrated circuit, ASIC), etc. The processor can also be a single-core processor or a multi-core processor. The processor can be a combination of a CPU and a hardware chip. The above-mentioned hardware chip can be an ASIC, a PLD or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (complex programmable logic device, CPLD), an FPGA, a generic array logic (generic array logic, GAL) or any combination thereof. The processor may also be implemented using a logic device with built-in processing logic, such as an FPGA or a digital signal processor (DSP).

[0177] The communication engine 200 may be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface may be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface may be a cellular network interface or a wireless local area network interface, etc. In the embodiment of the present application, the communication engine 200 may be specifically used for operations such as obtaining data to be decompressed sent by other computing devices.

[0178] The memory 400 may be a non-volatile memory, such as a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The memory 400 may also be a volatile memory, which may be a random access memory (RAM) used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0179] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0180] Optionally, the computing device may further include an input / output interface, to which an input / output device is connected, for receiving information input by a user, such as a compression type and a mantissa compression level input by the user.

[0181] Specifically, the specific implementation of the various operations performed by the computing device can refer to the specific operations of data compression and data decoding performed by the computing device in the above method embodiment, which will not be repeated here.

[0182] An embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is executed on a processor, it can implement the method steps performed by the above-mentioned computing device. The specific implementation of the processor of the computer-readable storage medium in executing the above-mentioned method steps can refer to the specific operations of the above-mentioned embodiment, which will not be repeated here.

[0183] An embodiment of the present application also provides a computer program product, which includes computer instructions. When the computer program product is executed by a computing device, the computing device executes the method executed by the computing device in the above embodiment.

[0184] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0185] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0186] The steps in the method of the embodiment of the present application can be adjusted in order, combined or deleted according to actual needs; the modules in the device of the embodiment of the present application can be divided, combined or deleted according to actual needs.

[0187] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Determine a first reference value according to a plurality of first data to be compressed, wherein the plurality of first data are data that the computing device needs to transmit; Calculate the difference between each first data and the first reference value; Determine the code corresponding to each difference; According to each of the codes, the corresponding first data is compressed.

2. The method according to claim 1, characterized in that The plurality of first data are respectively exponent values ​​of a plurality of floating point numbers.

3. The method according to claim 1, characterized in that Determining the code corresponding to each difference value includes: The code corresponding to each difference is determined according to each difference and a coding table, wherein the coding table records codes corresponding to a plurality of preset difference values.

4. The method according to claim 3, characterized in that In the coding table, the code corresponding to the difference whose absolute value is greater than the threshold is a fixed value, and the compressed data of the corresponding first data obtained according to each of the codes includes: When the absolute value of the first difference is greater than the threshold, the fixed value and the first data corresponding to the first difference are used as compressed data of the first data corresponding to the first difference.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining a first reference value according to a plurality of first data to be compressed comprises: Sampling the plurality of first data to obtain a plurality of sampling values; Divide the multiple sampling values ​​into t1 groups, determine the median of the multiple sampling values ​​in each of the t1 groups, and obtain t1 medians; The t1 medians are divided into t2 groups, the median in each of the t2 groups is determined to obtain t2 medians, and the median corresponding to the t2 medians is used as the first reference value corresponding to the multiple first data.

6. The method according to any one of claims 2 to 5, characterized in that: The plurality of floating point numbers also includes a mantissa value; The method further comprises: Compressing the mantissa values ​​of the plurality of floating-point numbers respectively to obtain compressed data of the mantissa values; The compressed data of the multiple floating-point numbers are obtained according to the compressed data of the exponent values ​​of the multiple floating-point numbers and the compressed data of the mantissa values ​​of the multiple floating-point numbers.

7. The method according to any one of claims 2 to 6, characterized in that: The method further comprises: First metadata is generated, where the first metadata includes the first reference value, and the first reference value is used to decode compressed data of the plurality of first data to obtain the plurality of first data.

8. The method according to claim 7, characterized in that When the multiple first data are respectively exponent values ​​of multiple floating-point numbers, the first metadata also includes a compression type and a mantissa compression level; the compression type is used to indicate whether the mantissa values ​​of the multiple floating-point numbers are compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa values ​​are compressed when the mantissa values ​​are compressed.

9. The method according to any one of claims 1 to 8, characterized in that The method further comprises: Receive data to be decompressed, the data to be decompressed includes a coding part and second metadata; wherein the second metadata includes a second reference value, and the coding part includes a plurality of codes; According to the second reference value and the plurality of codes, the plurality of codes are decoded respectively to obtain a plurality of second data.

10. The method according to claim 9, characterized in that When the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed further includes a mantissa part, and the second metadata further includes a data type, a compression type, and a mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa value is compressed; The step of decoding the to-be-decompressed data according to the second reference value and the plurality of codes to obtain a plurality of second data comprises: Determine the length of the exponent value and the length of the mantissa value of the floating point number before compression according to the data type; According to the length of the exponent value, the second reference value and the multiple codes, respectively decode the multiple codes to obtain multiple exponent values; When it is determined according to the compression type that the mantissa part in the to-be-decompressed data is data compressed by the mantissa value, decoding the compressed data of the mantissa part according to the mantissa compression level and the length of the mantissa value to obtain a plurality of mantissa values; The floating point number before compression is restored according to the multiple exponent values ​​and the multiple mantissa values.

11. A data processing device, characterized in that: The device comprises: A communication unit, configured to obtain a plurality of first data, wherein the plurality of first data are data that the computing device needs to transmit; A processing unit, configured to determine a first reference value according to a plurality of first data to be compressed; Calculate the difference between each first data and the first reference value; Determine the code corresponding to each difference; According to each of the codes, the corresponding first data is compressed.

12. The device according to claim 11, characterized in that The plurality of first data are respectively exponent values ​​of a plurality of floating point numbers.

13. The device according to claim 11, characterized in that The processing unit is specifically used for: The code corresponding to each difference is determined according to each difference and a coding table, wherein the coding table records codes corresponding to a plurality of preset difference values.

14. The device according to claim 12 or 13, characterized in that The plurality of floating point numbers also includes a mantissa value; The processing unit is also used for: Compressing the mantissa values ​​of the plurality of floating-point numbers respectively to obtain compressed data of the mantissa values; The compressed data of the multiple floating-point numbers are obtained according to the compressed data of the exponent values ​​of the multiple floating-point numbers and the compressed data of the mantissa values ​​of the multiple floating-point numbers.

15. The device according to any one of claims 11 to 14, characterized in that The communication unit is further used to receive data to be decompressed, wherein the data to be decompressed includes a coding part and second metadata; wherein the metadata includes a second reference value, and the coding part includes a plurality of codes; According to the second reference value and the plurality of codes, the plurality of codes are decoded respectively to obtain a plurality of second data.

16. The device according to claim 15, characterized in that When the data to be decompressed is data corresponding to a compressed floating-point number, the data to be decompressed further includes a mantissa part, and the second metadata further includes a data type, a compression type, and a mantissa compression level of the compressed floating-point number, the compression type is used to indicate whether the mantissa value of the floating-point number is compressed, and the mantissa compression level is used to indicate the length of the data after the mantissa value is compressed; The step of decoding the to-be-decompressed data according to the second reference value and the plurality of codes to obtain a plurality of second data comprises: Determine the length of the exponent value and the length of the mantissa value of the floating point number before compression according to the data type; According to the length of the exponent value, the second reference value and the multiple codes, respectively decode the multiple codes to obtain multiple exponent values; When it is determined according to the compression type that the mantissa part in the to-be-decompressed data is data compressed by the mantissa value, decoding the compressed data of the mantissa part according to the mantissa compression level and the length of the mantissa value to obtain a plurality of mantissa values; The floating point number before compression is restored according to the multiple exponent values ​​and the multiple mantissa values.

17. A computing device, characterized in that The computing device includes one or more processors, a memory and a compression engine, the memory stores computer program instructions, the processor executes the computer program instructions to generate data to be compressed, and the compression engine is used to implement the data processing method described in any one of claims 1 to 10 to compress the data to be compressed.

18. The computing device of claim 17, wherein: The computing device also includes a communication engine and a decompression engine, wherein the communication engine is used to receive data to be decompressed, and the decompression engine is used to implement the data processing method described in any one of claims 1 to 10 to decode the data to be decompressed.

19. The computing device of claim 18, wherein: The compression engine and the decompression engine are located in the communication engine.

20. The computing device of claim 17, wherein: The compression engine is located in the processor, and the decompression engine is located in the communication engine.

21. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the operation steps of the method according to any one of claims 1 to 10 are performed.

Images