Data compression and decompression method, device and system

Abstract

The embodiment of the present application discloses a data compression and decompression method, device and system, which are applied to the field of data processing technology and are used to select the optimal compressed data block length when the compression rate is different to achieve the optimization of data compression and decompression. The method includes: when performing data compression, the compression engine obtains the target data to be compressed and compresses the data blocks of L bytes in length into compressed data blocks of l bytes in length in sequence to obtain a compression result; wherein L and l are the maximum bandwidth length of the decompression engine decompression outlet and the maximum bandwidth length of the inlet respectively; then l is changed to l1 to ln in sequence, and the above compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and then the compressed data block length is stored in the compression information, and the compression information and the compression result with the smallest total data length are sent to the decompression engine.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data compression and decompression method, device and system. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, more and more AI hardware accelerators (i.e., AI hardware integrated chips) are used to accelerate operations with large amounts of data. For example, AI hardware accelerators can be used to accelerate the process of image processing using convolutional neural networks (CNN).

[0003] However, when using AI hardware accelerators to run machine learning algorithms such as convolutional neural networks, a large number of parameters are often required, and these parameters cannot be completely stored inside the chip of the AI ​​hardware accelerator. Therefore, when running machine learning algorithms, a large number of parameters need to be imported from the outside in real time to complete the calculation. Since the real-time import of parameters will occupy the input / output (I / O) bandwidth of the AI ​​hardware accelerator, if the IO bandwidth is insufficient, it will not be possible to import enough external parameters in real time, which will cause the computing unit of the AI ​​hardware accelerator to be idle, thereby reducing the overall performance of the AI ​​hardware accelerator. Therefore, it is necessary to pre-compress the external parameters offline to reduce the amount of external parameter data, and use the decompression engine in the AI ​​hardware accelerator to perform real-time online decompression to restore the original external parameters, so that more external parameters can be obtained under the same I / O bandwidth.

[0004] However, currently, when external parameters are compressed offline, the method usually adopted is to take the original data of a fixed length of L (the maximum bandwidth length of the decompression outlet) bytes as the current original data block each time, and try to compress it. If the compressed length is less than the maximum bandwidth length l of the decompression inlet, it is padded with 0 to a length of l. Otherwise, the remaining data in the L-byte original data is put into the next block for compression, and finally several compressed data blocks of length l are obtained. Correspondingly, when online decompression is performed in the AI ​​hardware accelerator, a compressed data block of length l is taken each time for decompression to obtain data of length less than or equal to L bytes, and it is spliced ​​to obtain several decompression results of length L. Although the decompression method can ensure that fixed-length data is read in each clock cycle during decompression, there is no coupling relationship between the data read in the two clock cycles before and after, and the length of the data after decompression will not be greater than the data bit width of the decompression outlet, which is convenient for hardware implementation, this compression / decompression method, when processing data with a low sparsity rate, due to the high compression rate, will cause the data length contained in the decompression result corresponding to each clock cycle to be shorter, so that the decompression results of multiple clock cycles need to be spliced ​​to make up the data of L length, which leads to low utilization rate of the bandwidth of the decompression outlet. In addition, repeated data splicing operations will also frequently shift the data, which consumes a lot of power. When processing data with a high sparsity rate, due to the low compression rate, 0 values ​​are filled in during compression, which reduces the data compression rate in disguise. It can be seen that the currently commonly used data compression / decompression method will either reduce the data compression rate or reduce the utilization rate of the decompression outlet bandwidth and consume a lot of power when processing data with different compression rates. Summary of the invention

[0005] The embodiments of the present application provide a data compression and decompression method, device and system, which are helpful to select the optimal compressed data block length when the compression rates are different, so as to achieve optimization of data compression and decompression.

[0006] In a first aspect, the present application provides a data compression method, the method comprising: when performing data compression, after the compression engine obtains the target data to be compressed, first compresses the data blocks of L bytes in the target data into compressed data blocks of l bytes in length in sequence to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; then, l is changed to l1 to ln in sequence, and the above compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and the compressed data block length can be stored in the compression information and sent to the decompression engine (such as Figure 1 The decompression engine in the AI ​​hardware accelerator 102 in the embodiment sends the compressed information and the compression result with the smallest total data length.

[0007] Compared with the traditional technology, the present application can select the compression result with the smallest total data length as the final compression result when the compression rates are different, and send the corresponding compressed data block length to the decompression engine, so that the decompression engine can decompress according to the compressed data block length. In this way, when the compression rate is high, the invalid data read at the decompression engine entrance is less, and the bandwidth utilization rate of the decompression outlet is not reduced. Moreover, when the compression rate is low, the data read at the decompression engine entrance is more (this is because the compressed data block length corresponding to the compression result is larger). After decompression, it can basically ensure that the decompression outlet can output data in each clock cycle. There is no need to repeatedly shift and splice the decompression results. Only one data splicing operation is required, which avoids power consumption and achieves the optimization of data compression and decompression.

[0008] In a possible implementation, data blocks of L bytes in the target data are sequentially compressed into compressed data blocks of l bytes in length to obtain a compression result, including: sequentially compressing data blocks of L bytes in the target data to obtain first compression results corresponding to each data block; when there is a first compression result with a byte length less than l, adding a value of 0 as first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches l, and the first compression result and the first additional data are used together as an updated first compression result; all first compression results that do not need to be updated and all the updated first compression results are sorted in order to obtain a compression result.

[0009] In this way, when the data block length reaches L during decompression, it means that the decompression can be stopped, and the remaining 0 values ​​of the compressed data can be discarded, thereby improving the decompression efficiency.

[0010] In a possible implementation, sequentially compressing data blocks of L bytes in the target data into compressed data blocks of 1 byte length to obtain a compression result includes: sequentially compressing data blocks of L bytes in the target data to obtain second compression results corresponding to each data block; when there is a second compression result with a byte length equal to 1, sorting all the second compression results in order to obtain a compression result. So that the original data with a data block length of L can be obtained after subsequent decompression, which will not be greater than the maximum bandwidth length of the decompression outlet, and is easy to implement in hardware.

[0011] In a possible implementation, data blocks of L bytes in the target data are sequentially compressed into compressed data blocks of l bytes in length to obtain a compression result, including: sequentially compressing the data blocks of L bytes in the target data to obtain a third compression result corresponding to each data block; when there is a third compression result with a byte length greater than l, removing part of the tail data in the data block of L bytes in the target data to obtain a data block with a length less than L after the tail data is removed; compressing the data block with a length less than L after the tail data is removed to obtain a fourth compression result; when the byte length of the fourth compression result is less than l and the byte length of the fourth compression result plus a coding length is greater than l, adding an invalid flag as second additional data after the fourth compression result, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches l, and using the fourth compression result and the second additional data as an updated third compression result; sorting all third compression results that do not need to be updated and all updated third compression results in order to obtain a compression result.

[0012] In this way, if an invalid flag is identified during decompression, the decompression operation can be stopped, the decompressed original data block with a length less than or equal to L is output, and the compressed data after the invalid flag is discarded, thereby improving the decompression efficiency.

[0013] In a possible implementation, obtaining the compressed data block length corresponding to the compression result with the smallest total data length among n compression results includes: obtaining the total data length after adding up all the compressed data blocks contained in each of the n compression results; and determining the compressed data block length corresponding to the compression result with the smallest total data length from the total data length after adding up all the compressed data blocks contained in each of the compression results. So that the decompression engine can decompress according to the compressed data block length, so that when the compression rate is high, the decompression engine entrance reads less invalid data, which will not reduce the bandwidth utilization rate of the decompression outlet, and when the compression rate is low, the decompression engine entrance reads more data, and after decompression, it can basically ensure that the decompression outlet can output data in each clock cycle, without repeatedly shifting and splicing the decompression results, and only one data splicing operation is required, thereby avoiding power consumption waste.

[0014] In a second aspect, the present application also provides a data decompression method, the method comprising: a decompression engine (such as Figure 1 The decompression engine in the AI ​​hardware accelerator 102 in the decompression engine can first obtain the compressed data block length in the compressed information after obtaining the data to be decompressed and the compression information corresponding to the data to be decompressed sent by the compression engine; wherein the data to be decompressed is the compression result with the smallest total data length corresponding to the original data; then, the data blocks in the data to be decompressed whose length is the compressed data block length are decompressed in turn to obtain the decompression result.

[0015] In this way, when the decompression engine decompresses according to the length of the compressed data block, when the compression rate is high, the decompression engine entrance reads less invalid data, and will not reduce the decompression outlet bandwidth utilization rate. When the compression rate is low, the decompression engine entrance reads more data. After decompression, it can basically ensure that the decompression outlet can output data in each clock cycle. There is no need to repeatedly shift and splice the decompression results. Only one data splicing operation is required, thus avoiding power waste.

[0016] In a possible implementation, sequentially decompressing data blocks of the length of the compressed data block in the data to be decompressed to obtain a decompression result includes: sequentially decompressing data blocks of the length of the compressed data block in the data to be decompressed; when the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L, stopping the decompression operation and outputting the original data block of the length of L; or, when decompressing the data block of the compressed data block length, if an invalid flag is identified, stopping the decompression operation and outputting the original data block of the length less than or equal to L; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine. Thus, the optimization of data decompression can be achieved.

[0017] In a third aspect, the present application also provides a data compression and decompression system, which includes: a compression engine and a decompression engine; wherein the compression engine is used to execute: S1: obtain target data to be compressed; S2: compress the data blocks of L bytes in the target data into compressed data blocks of l bytes in length in turn to obtain a compression result; L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; S3: change l to l1 to ln in turn, repeat step S2 to obtain n compression results; ln is a positive integer greater than 1 and less than L; n is a positive integer greater than 1; S4: obtain the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results; S5: store the compressed data block length in the compression information, and send the compression information and the compression result with the smallest total data length to the decompression engine; the decompression engine is used to obtain the compressed data block length in the compression information; decompress the data blocks with a length of the compressed data block in the data to be decompressed in turn to obtain the decompression result.

[0018] In one possible implementation, the compression engine is also used to: compress data blocks of L bytes in the target data in sequence to obtain a first compression result corresponding to each data block; when there is a first compression result with a byte length less than 1, add a value of 0 as the first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches 1, and use the first compression result and the first additional data together as the updated first compression result; sort all first compression results that do not need to be updated and all updated first compression results in chronological order to obtain a compression result.

[0019] In one possible implementation, the compression engine is also used to: compress data blocks of L bytes in length in the target data in sequence to obtain second compression results corresponding to each data block; when there is a second compression result with a byte length equal to l, sort all second compression results in order to obtain a compression result.

[0020] In a possible implementation, the compression engine is further used to: compress data blocks of L bytes in the target data in sequence to obtain a third compression result corresponding to each data block; when there is a third compression result with a byte length greater than 1, remove part of the tail data in the data block of L bytes in the target data to obtain a data block with a length less than L after removing the tail data; compress the data block with a length less than L after removing the tail data to obtain a fourth compression result; when the byte length of the fourth compression result is less than 1 and the byte length of the fourth compression result plus a coding length is greater than 1, add an invalid flag as the second additional data after the fourth compression result, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches 1, and use the fourth compression result and the second additional data as the updated third compression result; sort all third compression results that do not need to be updated and all updated third compression results in order to obtain a compression result.

[0021] In one possible implementation, the compression engine is also used to: obtain the total data length of all compressed data blocks contained in each of the n compression results; and determine the compressed data block length corresponding to the compression result with the smallest total data length from the total data length of all compressed data blocks contained in each of the n compression results.

[0022] In a possible implementation, the decompression engine is further used to: sequentially decompress data blocks of the to-be-decompressed data whose length is the length of the compressed data block; when the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L, stop the decompression operation and output the original data block of the length L; or, when decompressing the data block of the compressed data block length, if an invalid flag is identified, stop the decompression operation and output the original data block of the decompressed length less than or equal to L; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine.

[0023] In a fourth aspect, the present application also provides a data compression device, which includes: a first acquisition unit, used to acquire target data to be compressed; a first compression unit, used to compress data blocks of L bytes in the target data into compressed data blocks of l bytes in length in turn, to obtain a compression result; L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; a second compression unit, used to change l to l1 to ln in turn, repeatedly call the first compression unit to compress data blocks of L bytes in the target data into compressed data blocks of l1 to ln bytes in length in turn, to obtain n compression results; ln is a positive integer greater than 1 and less than L; n is a positive integer greater than 1; the second acquisition unit, used to obtain the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results; a sending unit, used to store the compressed data block length in the compression information, and send the compression information and the compression result with the smallest total data length to the decompression engine.

[0024] In a possible implementation, the first compression unit includes:

[0025] A first compression subunit is used to compress the data blocks of L bytes in the target data in sequence to obtain a first compression result corresponding to each data block;

[0026] a first adding subunit, configured to, when there is a first compression result whose byte length is less than 1, add a value of 0 as first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches 1, and use the first compression result and the first additional data together as an updated first compression result;

[0027] The first obtaining subunit is used to sort all the first compression results that do not need to be updated and all the updated first compression results in order to obtain a compression result.

[0028] In a possible implementation, the first compression unit includes:

[0029] A second compression subunit is used to compress the data blocks of L bytes in the target data in sequence to obtain a second compression result corresponding to each data block;

[0030] The second obtaining subunit is used to sort all the second compression results in order of precedence to obtain a compression result when there is a second compression result whose byte length is equal to 1.

[0031] In a possible implementation, the first compression unit includes:

[0032] A third compression subunit is used to compress the data blocks of L bytes in the target data in sequence to obtain a third compression result corresponding to each data block;

[0033] a removal subunit, configured to remove part of the tail data in the data block of L bytes in the target data when the byte length of the third compression result is greater than 1, so as to obtain a data block whose length after removing the tail data is less than L;

[0034] A fourth compression subunit, used for compressing the data block whose length is less than L after removing the tail data, to obtain a fourth compression result;

[0035] a second adding subunit, configured to, when the byte length of the fourth compression result is less than 1 and the byte length of the fourth compression result plus a coding length is greater than 1, add an invalid flag as second additional data after the fourth compression result, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches 1, and use the fourth compression result and the second additional data as the updated third compression result;

[0036] The third obtaining subunit is used to sort all the third compression results that do not need to be updated and all the updated third compression results in order to obtain a compression result.

[0037] In a possible implementation, the second obtaining unit includes:

[0038] An acquisition subunit, used for acquiring the total length of data after adding up all compressed data blocks contained in each of the n compression results;

[0039] The determination subunit is used to determine the compressed data block length corresponding to the compression result with the smallest total data length from the total data length of all compressed data blocks contained in each compression result.

[0040] In a fifth aspect, the present application further provides a data decompression device, the device comprising:

[0041] The first acquisition unit is used to obtain the data to be decompressed sent by the compression engine and the compression information corresponding to the data to be decompressed; the data to be decompressed is the compression result with the smallest total data length corresponding to the original data; the second acquisition unit is used to obtain the length of the compressed data block in the compression information; the decompression unit is used to decompress the data blocks in the data to be decompressed in sequence with the length of the compressed data block to obtain the decompression result.

[0042] In a possible implementation, the decompression unit includes:

[0043] A decompression subunit, used for sequentially decompressing data blocks of a length equal to the length of the compressed data block in the data to be decompressed;

[0044] The output subunit is used to stop the decompression operation and output the original data block of length L when the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L; or, when decompressing the data block of the compressed data block length, if an invalid flag is recognized, stop the decompression operation and output the original data block of length less than or equal to L; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine.

[0045] In a sixth aspect, the present application further provides a data compression device, the device comprising a memory and a processor;

[0046] A memory is used to store instructions; a processor is used to execute the instructions in the memory to perform any one of the methods of the first aspect above.

[0047] In a seventh aspect, the present application also provides a data decompression device, the device comprising a memory and a processor;

[0048] A memory is used to store instructions; a processor is used to execute the instructions in the memory to perform any one of the methods of the second aspect above.

[0049] In an eighth aspect, the present application also provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute any one of the above methods.

[0050] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0051] When performing data compression in the embodiment of the present application, the compression engine first obtains the target data to be compressed; then compresses the data blocks of L bytes in the target data into compressed data blocks of l bytes in length respectively, to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; then, l is changed to l1 to ln in sequence, and the aforementioned compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and the compressed data block length can be stored in the compression information, and the compression information and the compression result with the smallest total data length are sent to the decompression engine. It can be seen that, since the present application can select the compression result with the smallest total data length as the final compression result when the compression rates are different, and send the corresponding compressed data block length to the decompression engine, so that the decompression engine can decompress according to the compressed data block length, when the compression rate is high, the decompression engine entrance reads less invalid data, and will not reduce the decompression outlet bandwidth utilization rate, and when the compression rate is low, the decompression engine entrance reads more data (this is because the compressed data block length corresponding to the compression result is larger), after decompression, it can basically ensure that the decompression outlet can output data in each clock cycle, without repeated data shifting and splicing of the decompression result, only one data splicing operation is required, thereby avoiding power consumption waste, thereby achieving the optimization of data compression and decompression. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A structural block diagram of the AI ​​hardware accelerator provided in an embodiment of the present application;

[0053] Figure 2 A flowchart of a data compression method provided in an embodiment of the present application;

[0054] Figure 3 One of the schematic diagrams of compressing a data block of L bytes in target data into a compressed data block of l bytes in length provided in an embodiment of the present application;

[0055] Figure 4 A second schematic diagram of compressing a data block of L bytes in target data into a compressed data block of l bytes in length provided in an embodiment of the present application;

[0056] Figure 5 A third schematic diagram of compressing a data block of L bytes in target data into a compressed data block of l bytes in length provided in an embodiment of the present application;

[0057] Figure 6 A flowchart of a data decompression method provided in an embodiment of the present application;

[0058] Figure 7A schematic diagram of the overall implementation process of the data compression method provided in the embodiment of the present application;

[0059] Figure 8 A structural block diagram of a data compression and decompression system provided in an embodiment of the present application;

[0060] Fig. 9 A structural block diagram of a data compression device provided in an embodiment of the present application;

[0061] Fig.10 A structural block diagram of a data decompression device provided in an embodiment of the present application;

[0062] Fig.11 A schematic diagram of the structure of a data compression device provided in an embodiment of the present application;

[0063] Fig.12 A schematic diagram of the structure of a data decompression device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The embodiments of the present application provide a data compression method, device and system, which are helpful to select the optimal compression block length when the compression rates are different, so as to achieve optimization of data compression and decompression.

[0065] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0066] See also Figure 1 , which shows a structural block diagram of the AI ​​hardware accelerator provided in an embodiment of the present application, such as Figure 1 As shown, in this scenario, a central processing unit (CPU) 101, an AI hardware accelerator 102, and an external storage unit 103 are included. The central processing unit 101 is connected to the AI ​​hardware accelerator 102, and the AI ​​hardware accelerator 102 is connected to the external storage unit 103. The above-mentioned "connection" can be a direct connection or an indirect connection.

[0067] Among them, the central processing unit 101 can be a host central processing unit (host CPU), which is used to assign acceleration tasks to the AI ​​hardware accelerator 102 mounted thereon.

[0068] The AI ​​hardware accelerator 102 includes a decompression engine, a weight buffer, a data buffer, a cube unit, a vector unit, an accumulator, a direct memory access controller (DMAC), an on-chip unified buffer, a flow control unit, an instruction fetch buffer, and a bus interface unit (BIU).

[0069] The matrix calculation unit is used to complete the matrix-matrix calculation, such as the matrix-matrix calculation corresponding to the convolution layer and the fully connected layer in the convolutional neural network. Specifically, when performing convolutional layer or fully connected layer operations, the matrix calculation unit reads the data corresponding to the data matrix from the data cache unit and the parameter cache unit, wherein the parameter data read from the parameter cache unit is transferred to the parameter cache unit through the DMAC. During the transfer process, the parameter data needs to be decompressed by the decompression engine before the matrix multiplication calculation can be performed on the matrix calculation unit to obtain the partial result or the final result of the matrix, which is stored in the accumulator.

[0070] The vector processing unit can further process the output results of the matrix calculation unit when necessary, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for network calculations of other layers outside the non-convolutional layer and the fully connected layer in the neural network, such as the activation function (rectified linear unit, Relu) layer, the pooling layer, etc.

[0071] The on-chip unified cache unit is used to store output calculation results, input data of certain layers of the neural network, etc.

[0072] The bus interface unit is used to interact between the CPU 101 and the DMAC and instruction fetch cache unit through the bus, so that the CPU 101 can control the startup of the AI ​​hardware accelerator 102 through the bus, and the AI ​​hardware accelerator 102 can also obtain compressed data and machine learning model parameters from the external storage unit 103 through the bus, and then use the decompression engine to complete real-time online decompression and perform various calculations on the decompressed and restored data.

[0073] The DMAC is mainly used to transfer data in the external storage unit 103 to the parameter cache unit, the data cache unit or the on-chip unified cache unit, or to transfer data in the on-chip unified cache unit to the external storage unit 103 .

[0074] The instruction fetch cache unit is used to cache instructions and output instructions through the instruction control unit to control the working process of the AI ​​hardware accelerator 102.

[0075] The external storage unit 103 refers to an external memory private to the hardware architecture of the AI ​​hardware accelerator, which can be a double data rate SDRAM memory (double data rate, DDR), including a series of specifications such as DDR, DDR2, DDR3, DDR4, etc. It is used to store the data to be processed and the compressed data compressed by the compression engine (software supporting the AI ​​hardware accelerator 102) to provide corresponding services for the AI ​​hardware accelerator 102, such as providing parameter data.

[0076] In the embodiment of the present application, the compression engine first obtains the target data to be compressed; then compresses the data blocks of L bytes in the target data into compressed data blocks of l bytes in length, and obtains a compression result; then, l is changed to l1 to ln in sequence, and the aforementioned compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and then the compressed data block length can be stored in the compression information, and the compression information and the compression result with the smallest total data length are sent to the AI ​​hardware accelerator 102. On this basis, the decompression engine in the AI ​​hardware accelerator 102 can obtain the compressed data block length according to the compression information, and sequentially decompress the data blocks with the length of the compressed data block in the compression result with the smallest total data length, and obtain the decompression result, thereby achieving the optimization of data compression and decompression.

[0077] It should be noted that the above application scenarios are only shown for the purpose of facilitating the understanding of the present application, and the implementation of the present application is not limited in this respect. On the contrary, the implementation of the present application can be applied to any applicable scenario.

[0078] Based on the above application scenarios, an embodiment of the present application provides a data compression method, which can be applied to a compression engine matching the AI ​​hardware accelerator 102. The method is introduced below.

[0079] S1: Obtain target data to be compressed.

[0080] In this embodiment, the data compressed by this embodiment is defined as target data. It should be noted that the embodiment of the present application does not limit the type and acquisition method of the target data. For example, the target data may be a face image of a person R captured by a camera, or the target data may be a model parameter of a pre-built convolutional neural network model. After the compression engine obtains the target data to be compressed, it may optimize the compression of the target data through the following steps S2-S5.

[0081] S2: Compress the L-byte data blocks in the target data into l-byte compressed data blocks in sequence to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine.

[0082] In this embodiment, after the target data to be compressed is obtained through step S1, data blocks of L bytes in length can be taken out from the target data in order, and the data blocks can be compressed using existing or future compression algorithms, and each data block of L bytes in length can be compressed into a compressed data block of l bytes in length to obtain a compression result. For example, general compression algorithms such as entropy coding, Huffman coding, run-length coding, and mixed or variant compression algorithms of these compression algorithms can be used for compression. In this embodiment, a compression algorithm that can scan the characters to be compressed one by one and perform encoding and compression in sequence can be preferably selected, so that the length of the compressed data can be obtained while compressing the data, so as to perform subsequent steps.

[0083] Among them, L refers to the maximum bandwidth length of the decompression outlet of the decompression engine in the AI ​​hardware accelerator. l refers to the maximum bandwidth length of the decompression entrance of the decompression engine in the AI ​​hardware accelerator. Both can be determined according to the hardware support of the compression engine. For example, the specific values ​​of L and l can be configured in advance in the AI ​​hardware accelerator circuit implementation, or the specific values ​​of L and l can be determined by reading the relevant configuration values ​​when the program or AI hardware accelerator circuit is running. For another example, the specific values ​​of L and l can be determined by reading a configuration file stored locally or remotely that records the values ​​of L and l.

[0084] In a possible implementation of this embodiment, the specific implementation process of step S2 may include the following steps A1-A3:

[0085] Step A1: compress data blocks of L bytes in length in the target data in sequence to obtain a first compression result corresponding to each data block.

[0086] In this implementation, data blocks of L bytes in length are taken out from the target data in sequence, and each time a data block of L bytes in length is taken out, it is compressed to obtain a first compression result corresponding to the data block. Here, the data length corresponding to the first compression result is defined as x1.

[0087] Step A2: When the byte length of the first compression result is less than 1, add the value 0 as the first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches 1, and use the first compression result and the first additional data together as the updated first compression result.

[0088] In this implementation, when the byte length x1 of the first compression result corresponding to a certain data block is less than 1, that is, less than the maximum bandwidth length of the decompression entry of the decompression engine, it is necessary to add a value 0 after the first compression result as the first additional data, so that the sum of the byte length x1 of the first compression result and the byte length of all the added values ​​0 in the first additional data reaches 1. Then, the first compression result and the first additional data can be used together as the updated first compression result.

[0089] For example: Figure 3 As shown in the figure, assuming that the byte length x1 of the first compression result obtained after compressing the target data of L bytes is less than the maximum bandwidth length l of the decompression entry of the decompression engine, at this time, the value 0 can be added after the first compression result as the first additional data, as shown in the gray shaded box in the figure. Among them, the total byte length of the added value 0 is l-x1, so that the sum of the byte lengths of the first compression result and the first additional data can just reach l.

[0090] Step A3: sort all first compression results that do not need to be updated and all updated first compression results in order of precedence to obtain a compression result.

[0091] In this implementation, after adding the first additional data after the first compression result with a byte length less than l for updating through step A2, all updated first compression results and first compression results that do not need to be updated (i.e., the first compression result with a byte length equal to l) can be further sorted in order of chronological order to obtain a merged compression result with a compression step length of l corresponding to the target data.

[0092] In a possible implementation of this embodiment, the specific implementation process of step S2 may include the following steps B1-B2:

[0093] Step B1: compress the data blocks of L bytes in the target data in sequence to obtain the second compression results corresponding to the data blocks.

[0094] In this implementation, data blocks of L bytes in length are taken out from the target data in sequence, and each time a data block of L bytes in length is taken out, it is compressed to obtain a second compression result corresponding to the data block. Here, the data length corresponding to the second compression result is defined as x2.

[0095] Step B2: When there is a second compression result whose byte length is equal to the said 1, all the second compression results are sorted in order of precedence to obtain a compression result.

[0096] In this implementation, when the byte length x2 of the second compression result corresponding to the data block of L bytes in the target data obtained by step B1 is equal to l, it is equal to the maximum bandwidth length of the decompression entrance of the decompression engine. At this time, all the second compression results can be sorted in order of precedence to obtain a merged compression result with a compression step length of l corresponding to the target data.

[0097] In a possible implementation of this embodiment, the specific implementation process of step S2 may include the following steps C1-C5:

[0098] Step C1: compressing data blocks of L bytes in length in the target data in sequence to obtain a third compression result corresponding to each data block.

[0099] In this implementation, data blocks of L bytes in length are taken out from the target data in sequence, and after each data block of L bytes in length is taken out, it is compressed to obtain a third compression result corresponding to the data block. Here, the data length corresponding to the third compression result is defined as x3.

[0100] Step C2: When there is a third compression result whose byte length is greater than 1, part of the tail data in the data block with a length of L bytes in the target data is removed to obtain a data block whose length is less than L after the tail data is removed.

[0101] In this implementation, when the byte length x3 of the third compression result corresponding to a certain data block is greater than 1, that is, greater than the maximum bandwidth length of the decompression entry of the decompression engine, it is necessary to remove part of the tail data in the data block of L bytes to obtain a data block whose length after removing the tail data is less than L, so that the byte length of the compression result obtained after compressing the data block may not be greater than 1. The removed tail data will be spliced ​​with subsequent data to form the next data block of L bytes for subsequent compression.

[0102] For example: Figure 4As shown, assuming that the byte length x3 of the third compression result obtained after compressing the target data of L bytes is greater than the maximum bandwidth length l of the decompression entry of the decompression engine, at this time, part of the tail data in the data block of L bytes can be removed (as shown in the gray shaded box in the figure) to obtain a data block with a length less than L after removing the tail data, so that the byte length of the compression result obtained after compressing it can be no greater than 1. The removed tail data will be spliced ​​with the subsequent data into the next data block of L bytes for subsequent compression.

[0103] Step C3: compress the data block whose length is less than L after removing the tail data to obtain a fourth compression result.

[0104] In this implementation, part of the tail data in the data block with a length of L bytes in the target data is removed through step C2 to obtain a data block with a length less than L after removing the tail data. The data block with a length less than L can then be further compressed to obtain a fourth compression result corresponding to the data block. Here, the data length corresponding to the fourth compression result is defined as x4.

[0105] It should be noted that in order to improve the data compression rate and the utilization rate of the decompression export bandwidth and reduce power consumption, after removing part of the tail data in the data block with a length of L bytes, it is necessary to ensure that the length x4 of the fourth compression result obtained after compressing the remaining data block with a length less than L is the maximum value close to l allowed by the compression algorithm adopted, that is, it is necessary to satisfy that the byte length x4 of the fourth compression result is less than l and the byte length x4 of the fourth compression result plus a coding length will be greater than 1, where "a coding length" refers to the compressed data length obtained after compressing a fixed-length data taken out each time when the compression algorithm is used to compress the data.

[0106] For example: Assuming that the maximum bandwidth length l of the decompression engine decompression entry is 64 bits, the byte length x4 of the fourth result is 60 bits, and "one coding length" is 6 bits, at this time, it is guaranteed that the byte length x4 of the fourth compression result is less than l, that is, 60 bits < 64 bits, and it is also guaranteed that the byte length x4 of the fourth compression result plus one coding length is greater than l, that is, 60 bits + 6 bits > 64 bits. Therefore, when the data block of L bytes in the target data is compressed to 60 bits, the compression can be stopped, and the remaining tail data in the data block can be removed.

[0107] Step C4: When the byte length of the fourth compression result is less than 1 and the byte length of the fourth compression result plus a coding length is greater than 1, an invalid flag is added after the fourth compression result as the second additional data so that the sum of the byte lengths of the fourth compression result and the second additional data reaches 1, and the fourth compression result and the second additional data are used as the updated third compression result.

[0108] In this implementation, when the byte length x4 of the fourth compression result is less than 1 and the byte length x4 of the fourth compression result plus a coding length is greater than 1, an invalid flag may be added after the fourth compression result as the second additional data, so that the sum of the byte length x4 of the fourth compression result and the byte length of all invalid flags added in the second additional data reaches 1. Then, the fourth compression result and the second additional data may be used together as the updated third compression result.

[0109] Among them, the specific value of the invalid flag may vary depending on the compression algorithm adopted. For example, when a combined compression algorithm of entropy coding and run-length coding is used for compression, if the character 011 appears, it means that all characters after this character are invalid flags. For the decompression engine, when an invalid flag is recognized, the decompression can be stopped and these invalid flags will be discarded.

[0110] For example: Figure 5 As shown, assuming that the byte length x4 of the fourth compression result obtained after the data block with a length less than L after removing the tail data is compressed is less than the maximum bandwidth length l of the decompression entry of the decompression engine, at this time, an invalid flag can be added after the fourth compression result as the second additional data, as shown in the gray shaded box in the figure. The total byte length of the added invalid flag is l-x4, so that the sum of the byte lengths of the fourth compression result and the second additional data is just l.

[0111] Step C5: sort all third compression results that do not need to be updated and all updated third compression results in order of precedence to obtain a compression result.

[0112] In this implementation, after adding the second additional data after the fourth compression result as the updated third compression result through step C4, all updated third compression results and third compression results that do not need to be updated (i.e., the third compression results with a byte length equal to l) can be further sorted in order of chronological order to obtain a merged compression result with a compression step of l corresponding to the target data.

[0113] It should be noted that when sequentially compressing data blocks with a length of L bytes in the target data into compressed data blocks with a length of l bytes in the above step S2, at least one of the above three different situations of steps A1 - A3, B1 - B2, and C1 - C5 may occur. However, regardless of which situation occurs, the operations can be performed according to the corresponding above steps until all the data blocks included in the target data are compressed to obtain a final compression result corresponding to the target data.

[0114] S3: Sequentially change l to l1 to ln, and repeat step S2 to obtain n compression results; where ln is a positive integer greater than 1 and less than L; n is a positive integer greater than 1.

[0115] In this embodiment, in order to select the optimal length of the compressed data block to achieve the optimization of data compression and decompression, all positive integers from 1 to L can be traversed as the length of the compressed data block. That is, the length of the compressed data block can be successively set to l1 to ln, where l1 to ln are all positive integers greater than 1 and less than L, and n is a positive integer greater than 1, that is, 1 < l1 < ln < L. Then, l in step S2 can be successively changed to l1 to ln, and after repeating step S2, n compression results corresponding to the target data can be obtained respectively. It can be understood that l is one of the values from l1 to L, and the specific value can be determined according to the hardware specifications of the decompression engine.

[0116] For example: Assume 1 = 64bit = 8Byte, L = 64Byte, then l1 to ln can successively take all positive integers from 1 to 64, that is, 1 < l1 <... < ln < 64, and l1 to ln are all positive integers.

[0117] S4: Obtain the length of the compressed data block corresponding to the compression result with the smallest total data length among the n compression results.

[0118] In this embodiment, by successively changing l to l1 to ln in step S3 and respectively compressing the target data to obtain n compression results, the total data length included in each of these n compression results can be calculated, and the compression result with the smallest total data length and its corresponding compressed data block length can be obtained therefrom. The specific implementation process can include the following steps D1 - D2:

[0119] Step D1: Obtain the total data length after adding up all the compressed data blocks included in each compression result among the n compression results.

[0120] In order to select the optimal compressed data block length, it is first necessary to calculate the total length of all compressed data blocks contained in each of the n compression results. For example, assuming that one of the n compression results contains 1000 compressed data blocks of 1 byte in length, and the value of 1 is 64 bits, then it can be calculated that the total length of the data contained in the compression result is 64000 bits, that is, 1000*64bit=64000bit.

[0121] Step D2: From the total length of data obtained by adding up all compressed data blocks contained in each compression result, determine the length of the compressed data block corresponding to the compression result with the smallest total data length.

[0122] After obtaining the total data length of all compressed data blocks contained in each of the n compression results through step D1, the compressed data block length corresponding to the compression result with the smallest total data length can be further selected as the optimal compressed data block length, which has the lowest corresponding compression rate.

[0123] S5: Store the compressed data block length in the compression information, and send the compression information and the compression result with the smallest total data length to the decompression engine.

[0124] In this embodiment, after obtaining the compressed data block length corresponding to the compression result with the smallest total data length through step S4, the compressed data block length can be further stored in the compression information. For example, when performing data compression, the original length of the target data, the compression mode and the compressed data block length can be uniformly defined as compression information, and stored in the memory separately with the corresponding compression result with the smallest total data length, and then the compression information and the compression result with the smallest total data length are sent to the decompression engine, so that the decompression engine can perform a decompression operation on the compression result with the smallest total data length according to the compressed data block length contained therein after reading the compression information. In this way, it can be ensured that when the compression rate is different, the optimal compression block length is selected to achieve the optimization of compression and decompression of the target data.

[0125] It should be noted that when selecting the compression result to be sent to the decompression engine and its corresponding compressed data block length, it is also necessary to consider the compression range supported by the hardware specifications of the decompression engine, that is, the decompression operations of data of different lengths that the decompression engine can support, and the compression effect that can be achieved by the compression algorithm used. After executing the above steps S1-S4, the compressed data block lengths corresponding to each of the n compression results obtained are combined with the actual situation, and then the final compression result to be sent to the decompression engine and its corresponding compressed data block length are determined based on empirical values.

[0126] In summary, the present embodiment provides a data compression method. When performing data compression, the compression engine first obtains the target data to be compressed; then compresses the data blocks of L bytes in the target data into compressed data blocks of l bytes in length, to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; then, l is changed to l1 to ln in sequence, and the aforementioned compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and the compressed data block length can be stored in the compression information, and the compression information and the compression result with the smallest total data length are sent to the decompression engine. It can be seen that, since the present application can select the compression result with the smallest total data length as the final compression result when the compression rates are different, and send the corresponding compressed data block length to the decompression engine, so that the decompression engine can decompress according to the compressed data block length, when the compression rate is high, the decompression engine entrance reads less invalid data, and will not reduce the decompression outlet bandwidth utilization rate; and, when the compression rate is low, the decompression engine entrance reads more data, and after decompression, it can basically ensure that the decompression outlet can output data in each clock cycle, without the need to repeatedly shift and splice the decompression results, and only one data splicing operation is required, thereby avoiding power consumption waste and achieving the optimization of data compression and decompression.

[0127] Corresponding to the above data compression method, the embodiment of the present application also provides a data decompression method. Figure 6 , which is a flow chart of a data decompression method provided by an embodiment of the present application. The method is introduced below.

[0128] S601: Obtain the data to be decompressed and the compression information corresponding to the data to be decompressed sent by the compression engine; wherein the data to be decompressed is the compression result with the smallest total length of the data corresponding to the original data.

[0129] In the embodiment of the present application, in order to optimize data decompression, the decompression engine first needs to obtain the data to be decompressed and the compression information corresponding to the data to be decompressed sent by the compression engine, wherein the data to be decompressed is the compression result with the smallest total length of the data corresponding to the original data. The specific acquisition method can refer to the relevant introduction of the above steps S1-S5. The compression information includes the original length of the original data, the compression mode and the compressed data block length.

[0130] S602: Obtain the length of the compressed data block in the compression information.

[0131] In the embodiment of the present application, after the compression information corresponding to the data to be decompressed is obtained through step S601, the length of the compressed data block contained in the compression information can be further obtained to execute the subsequent step S603.

[0132] S603: Decompress the data blocks whose length is the length of the compressed data block in the data to be decompressed in sequence to obtain a decompression result.

[0133] In the embodiment of the present application, after obtaining the length of the compressed data block in step S601, the data blocks with a length equal to the length of the compressed data block in the data to be decompressed can be decompressed in sequence to obtain a decompression result, and then the decompression result (i.e., the original data) is output to the decompression outlet for output.

[0134] In a possible implementation of this embodiment, the specific implementation process of step S603 may include the following steps E1-E2:

[0135] Step E1: sequentially decompressing data blocks whose length is the length of the compressed data block in the data to be decompressed.

[0136] In this implementation, after the length of the compressed data block is obtained, the compressed data blocks of this length can be taken out from the data to be decompressed in sequence, and after each compressed data block of this length is taken out, it is further compressed to obtain the decompression result corresponding to the data block.

[0137] Step E2: when the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L, the decompression operation is stopped and the original data block of L length is output; or, when decompressing the data block of the compressed data block length, if an invalid flag is identified, the decompression operation is stopped and the original data block of which the decompressed length is less than or equal to L is output; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine.

[0138] In this implementation, when the data blocks to be decompressed whose length is the length of the compressed data block are decompressed in sequence in step E1 to obtain the original data blocks corresponding to each compressed data block, if the length of the original data block is equal to L, the decompression operation is stopped and the original data block of length L is output. Even if there are remaining coded parts in the compressed data block, they are not considered valid because these remaining coded parts can only be the value 0 (i.e., the first additional data) added in the above step A2, which can be discarded.

[0139] Alternatively, if when decompressing a compressed data block, an invalid flag is identified after the decompressed original data block whose length is less than L, then if it is determined that all the codes after the invalid flag are illegal codewords, the decompression operation can be stopped immediately, and these codes can be discarded, and only the decompressed original data block whose length is less than L needs to be output.

[0140] Alternatively, if when decompressing a compressed data block, an invalid flag is identified after the decompressed original data block with a length equal to L, if it is determined that the invalid flag is composed of added 0 values, the decompression operation can be stopped immediately, and these 0 values ​​can be discarded, and only the decompressed original data block with a length equal to L needs to be output.

[0141] It should be noted that, for the case where the value 0 is filled in the above step A2, other invalid characters can also be used instead. In this way, as long as the decompression engine recognizes the invalid character, it cannot decode it, and it can be determined that the decompression of the compressed data block has been completed, and the remaining invalid characters can be discarded.

[0142] In this way, the decompression engine decompresses according to the length of the compressed data block. When the compression rate is high, the decompression engine entrance reads less invalid data and does not reduce the bandwidth utilization of the decompression exit. When the compression rate is low, the decompression engine entrance reads more data (this is because the compressed data block length corresponding to the compression result is larger). After decompression, it can basically ensure that the decompression exit can output data in each clock cycle. There is no need to repeatedly shift and splice the decompression results. Only one data splicing operation is required, which avoids power waste and achieves the optimization of data decompression.

[0143] For ease of understanding, we now combine Figure 7 The overall implementation process diagram of the data compression method shown is used to introduce the data compression and decompression method provided by the present application:

[0144] S701: According to the specification of the decompression engine, determine the maximum bandwidth length L of the decompression outlet and the maximum bandwidth length l of the decompression inlet of each clock cycle of the decompression engine. It should be noted that the embodiments of the present application will be described below with l=8Byte=64bit and L=64Byte.

[0145] S702: During the compression process, the compression engine determines whether there is any uncompressed original data. If not, it proceeds to the subsequent step S706 and exits the compression process. If so, it reads the original data with a length of L for compression, and compares the length x of the compressed data with l. When x is less than l, it proceeds to the subsequent step S703; when x is greater than l, it proceeds to the subsequent step S704; when x is equal to l, it proceeds to the subsequent step S705. After all the data has been read and compressed, it proceeds to the subsequent step S706 and exits the compression process.

[0146] Specifically, during the compression process, the compression engine attempts to compress the original data in chunks of length L each time, and simultaneously records the length x of the currently compressed data during the compression process. If the length x of the compressed data after compressing L bytes of the original data is less than l, then it proceeds to the subsequent step S703; if the length x of the compressed data is exactly equal to l, then it proceeds to the subsequent step S705; if the length x of the compressed data is greater than l, then it removes the trailing data of the original data of length L to make the length x0 of the compressed data less than l, and when x0 reaches the maximum critical value that is close to l and allowed by the compression algorithm, it proceeds to the subsequent step S704. Among them, x0 needs to satisfy the condition: x0 < l and x0 plus the next encoding length is greater than l. Here, "one encoding length" refers to the length of the compressed data obtained after compressing a fixed-length data taken each time when compressing data using the compression algorithm.

[0147] S703: The compression engine pads 0 to the last l - x bits of the compressed data to make its length equal to l.

[0148] Specifically, the compression engine adds the value 0 to the end of the compressed data until the length reaches l. At this time, the new compressed data can include two parts: the previous compressed data and the added value 0, with a total length of l. Then it can continue to proceed to the subsequent step S705.

[0149] S704: After the compression engine removes the trailing data of the original data of length L to make the compressed length x0 less than l, it can continue to fill in invalid flags until the length reaches l, and incorporates the removed trailing data into the subsequent data to form the next original data block of length L for subsequent compression.

[0150] S705: The compression engine writes the compressed data into the compressed file in sequence, and then returns to execute step S702.

[0151] S706: Exit the current round of the compression process and record the total length of the compressed data.

[0152] S707: Replace l with l1 to ln in sequence and repeat the above S702 - S706.

[0153] S708: After all l values ​​are compressed, the minimum value of the total length of the compressed data and its corresponding compressed data block length (here defined as l_best) are selected, and the corresponding compression result is the current optimal compression result, and the corresponding compression rate is the lowest. Then, l_best can be stored in the compression information, and the compression result and compression information are sent to the decompression engine.

[0154] It should be noted that the implementation process of steps S701-S708 is consistent with that of steps S1-S5 above. For relevant details, please refer to the introduction of steps S1-S5 above, which will not be repeated here.

[0155] S709: The decompression engine obtains the compressed data block length l_best from the compression information.

[0156] S710: The decompression engine reads compressed data of length l_best in each clock cycle and sends the data to the decompression entry unit.

[0157] S711: The decompression engine decompresses the compressed data of the obtained l-best length according to the compression algorithm, and then stores the decompressed original data into the decompression exit buffer unit, and records the current effective decompression length (i.e., the length of the decompressed data). Due to the guarantee of the compression method used when compressing the data before, the length of the compressed data with a length of l_best=8Byte will not exceed L (i.e., 64Byte) after decompression and restoration, so the decompression exit buffer unit with a capacity of L=64Byte can definitely store these decompressed data.

[0158] S712: The decompression engine splices the contents stored in the decompression egress buffer unit according to the effective decompression length and combines them to output a final decompression result.

[0159] In summary, since the present application can select the compression result with the smallest total data length as the final compression result when the compression rates are different, and send its corresponding compressed data block length l_best to the decompression engine, the decompression engine can decompress according to l_best, so that when the compression rate is high, the decompression engine entrance reads less invalid data, and will not reduce the bandwidth utilization rate of the decompression outlet. Moreover, when the compression rate is low, the decompression engine entrance reads more data (this is because the compressed data block length corresponding to the compression result is larger). After decompression, it can basically ensure that the decompression outlet can output data in each clock cycle. There is no need to repeatedly shift and splice the decompression results. Only one data splicing operation is required, which avoids power consumption and achieves the optimization of data compression and decompression.

[0160] Next, a data compression and decompression system provided in an embodiment of the present application is introduced.

[0161] See also Figure 8 The authentication system includes a compression engine 801 and a decompression engine 802, which can be directly connected or indirectly connected.

[0162] The compression engine 801 is used to execute: S1: obtain target data to be compressed; S2: compress the data blocks of L bytes in the target data into compressed data blocks of l bytes in length in turn to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; S3: change l to l1 to ln in turn, repeat step S2, and obtain n compression results; wherein ln is a positive integer greater than 1 and less than L; n is a positive integer greater than 1; S4: obtain the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results; S5: store the compressed data block length in the compression information, and send the compression information and the compression result with the smallest total data length to the decompression engine 802; for the specific implementation process, please refer to the introduction of steps S1-S5 above, which will not be repeated here.

[0163] The decompression engine 802 is used to obtain the length of the compressed data block in the compressed information; decompress the data blocks with the length of the compressed data block in the decompressed data in turn to obtain the decompression result. The specific implementation process is described in the above steps S601-S603, which will not be repeated here.

[0164] In a possible implementation of this embodiment, the compression engine 801 is specifically used for:

[0165] Compressing the data blocks of L bytes in the target data in sequence to obtain a first compression result corresponding to each data block;

[0166] When there is a first compression result whose byte length is less than 1, adding a value of 0 after the first compression result as the first additional data so that the sum of the byte lengths of the first compression result and the first additional data reaches 1, and taking the first compression result and the first additional data together as the updated first compression result;

[0167] All first compression results that do not need to be updated and all updated first compression results are sorted in order to obtain a compression result.

[0168] In a possible implementation of this embodiment, the compression engine 801 is specifically used for:

[0169] Compressing the data blocks of L bytes in the target data in sequence to obtain a second compression result corresponding to each data block;

[0170] When there is a second compression result whose byte length is equal to 1, all the second compression results are sorted in order to obtain a compression result.

[0171] In a possible implementation of this embodiment, the compression engine 801 is specifically used for:

[0172] Compressing the data blocks of L bytes in the target data in sequence to obtain a third compression result corresponding to each data block;

[0173] When there is a third compression result whose byte length is greater than 1, part of the tail data in the data block with a length of L bytes in the target data is removed to obtain a data block whose length after removing the tail data is less than L;

[0174] Compressing the data block whose length is less than L after removing the tail data to obtain a fourth compression result;

[0175] When the byte length of the fourth compression result is less than 1 and the byte length of the fourth compression result plus a coding length is greater than 1, an invalid flag is added after the fourth compression result as the second additional data so that the sum of the byte lengths of the fourth compression result and the second additional data reaches 1, and the fourth compression result and the second additional data are used as the updated third compression result;

[0176] All third compression results that do not need to be updated and all updated third compression results are sorted in order to obtain a compression result.

[0177] In a possible implementation of this embodiment, the compression engine 801 is specifically used for:

[0178] Obtain the total length of data after adding up all compressed data blocks contained in each of the n compression results;

[0179] From the total data length of all compressed data blocks contained in each compression result, the length of the compressed data block corresponding to the compression result with the smallest total data length is determined.

[0180] In a possible implementation of this embodiment, the decompression engine 802 is specifically used for:

[0181] Decompressing data blocks whose length is the length of the compressed data block in the data to be decompressed in sequence;

[0182] When the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L, the decompression operation is stopped and the original data block of L length is output; or, when decompressing the data block of the compressed data block length, if an invalid flag is recognized, the decompression operation is stopped and the original data block of which the decompressed length is less than L is output; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine.

[0183] In order to better implement the above solution of the embodiment of the present application, a related device for implementing the above data compression solution is also provided below. Fig. 9 As shown, a data compression device 900 provided in an embodiment of the present application. The device 900 may include: a first acquisition unit 901, a first compression unit 902, a second compression unit 903, a second acquisition unit 904 and a sending unit 905. The first acquisition unit 901 is used to perform Figure 2 S1 in the illustrated embodiment. The first compression unit 902 is used to perform Figure 2 S2 in the embodiment shown. The second compression unit 903 is used to perform Figure 2 S3 in the embodiment shown. The second acquisition unit 904 is used to execute Figure 2 S4 in the illustrated embodiment. The sending unit 905 is used to execute Figure 2 S5 in the illustrated embodiment. Specifically,

[0184] A first acquisition unit 901 is used to acquire target data to be compressed;

[0185] The first compression unit 902 is used to compress the data blocks of L bytes in the target data into compressed data blocks of l bytes in sequence to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine;

[0186] The second compression unit 903 is used to change l to l1 to ln in sequence, and repeatedly call the first compression unit 902 to compress the data blocks of L bytes in the target data into compressed data blocks of l1 to ln bytes in length in sequence, to obtain n compression results; wherein ln is a positive integer greater than 1 and less than L; n is a positive integer greater than 1;

[0187] The second acquisition unit 904 is used to acquire the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results;

[0188] The sending unit 905 is used to store the compressed data block length in the compression information, and send the compression information and the compression result with the smallest total data length to the decompression engine.

[0189] In an implementation of this embodiment, the first compression unit 902 includes:

[0190] A first compression subunit is used to compress the data blocks of L bytes in the target data in sequence to obtain a first compression result corresponding to each data block;

[0191] a first adding subunit, configured to, when there is a first compression result whose byte length is less than 1, add a value of 0 as first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches 1, and use the first compression result and the first additional data together as an updated first compression result;

[0192] The first obtaining subunit is used to sort all the first compression results that do not need to be updated and all the updated first compression results in order to obtain a compression result.

[0193] In an implementation of this embodiment, the first compression unit 902 includes:

[0194] A second compression subunit is used to compress the data blocks of L bytes in the target data in sequence to obtain a second compression result corresponding to each data block;

[0195] The second obtaining subunit is used to sort all the second compression results in order of precedence to obtain a compression result when there is a second compression result whose byte length is equal to 1.

[0196] In an implementation of this embodiment, the first compression unit 902 includes:

[0197] A third compression subunit is used to compress the data blocks of L bytes in the target data in sequence to obtain a third compression result corresponding to each data block;

[0198] a removal subunit, configured to remove part of the tail data in the data block of L bytes in the target data when the byte length of the third compression result is greater than 1, so as to obtain a data block whose length after removing the tail data is less than L;

[0199] A fourth compression subunit, used for compressing the data block whose length is less than L after removing the tail data, to obtain a fourth compression result;

[0200] a second adding subunit, configured to, when the byte length of the fourth compression result is less than 1 and the byte length of the fourth compression result plus a coding length is greater than 1, add an invalid flag as second additional data after the fourth compression result, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches 1, and use the fourth compression result and the second additional data as the updated third compression result;

[0201] The third obtaining subunit is used to sort all the third compression results that do not need to be updated and all the updated third compression results in order to obtain a compression result.

[0202] In an implementation of this embodiment, the second obtaining unit 904 includes:

[0203] An acquisition subunit, used for acquiring the total length of data after adding up all compressed data blocks contained in each of the n compression results;

[0204] The determination subunit is used to determine the compressed data block length corresponding to the compression result with the smallest total data length from the total data length of all compressed data blocks contained in each compression result.

[0205] In summary, the data compression device provided by the present embodiment, when performing data compression, the compression engine first obtains the target data to be compressed; then compresses the data blocks of L bytes in the target data into compressed data blocks of l bytes in length respectively, to obtain a compression result; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine; l is the maximum bandwidth length of the decompression inlet of the decompression engine; then, l is changed to l1 to ln in sequence, and the aforementioned compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and the compressed data block length can be stored in the compression information, and the compression information and the compression result with the smallest total data length are sent to the decompression engine. It can be seen that, since the present application can select the compression result with the smallest total data length as the final compression result when the compression rates are different, and send the corresponding compressed data block length to the decompression engine, so that the decompression engine can decompress according to the compressed data block length, when the compression rate is high, the decompression engine entrance reads less invalid data, and will not reduce the decompression outlet bandwidth utilization rate; and, when the compression rate is low, the decompression engine entrance reads more data, and after decompression, it can basically ensure that the decompression outlet can output data in each clock cycle, without the need to repeatedly shift and splice the decompression results, and only one data splicing operation is required, thereby avoiding power consumption waste and achieving the optimization of data compression and decompression.

[0206] In order to better implement the above solution of the embodiment of the present application, a related device for implementing the above data compression solution is also provided below. Fig.10 As shown, an embodiment of the present application provides a data decompression device 1000. The device 1000 may include: a first acquisition unit 1001, a second acquisition unit 1002 and a decompression unit 1003. The first acquisition unit 1001 is used to perform Figure 6 S601 in the embodiment shown. The second acquisition unit 1002 is used to execute Figure 6 S602 in the illustrated embodiment. The decompression unit 1003 is used to execute Figure 6 S603 in the illustrated embodiment. Specifically,

[0207] The first acquisition unit 1001 is used to acquire the data to be decompressed sent by the compression engine and the compression information corresponding to the data to be decompressed; wherein the data to be decompressed is the compression result with the smallest total length of the data corresponding to the original data;

[0208] The second acquisition unit 1002 is used to acquire the compressed data block length in the compressed information;

[0209] The decompression unit 1003 is used to sequentially decompress data blocks whose length is the length of the compressed data block in the data to be decompressed to obtain a decompression result.

[0210] In one implementation of this embodiment, the decompression unit 1003 includes:

[0211] A decompression subunit, used for sequentially decompressing data blocks of a length equal to the length of the compressed data block in the data to be decompressed;

[0212] The output subunit is used to stop the decompression operation and output the original data block of length L when the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L; or, when decompressing the data block of the compressed data block length, if an invalid flag is recognized, stop the decompression operation and output the original data block of length less than or equal to L; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine.

[0213] In summary, the data decompression device provided by the present embodiment decompresses data according to the length of the compressed data block when decompressing the data. This ensures that when the compression rate is high, less invalid data is read at the decompression engine entrance, and the bandwidth utilization rate of the decompression exit is not reduced. When the compression rate is low, more data is read at the decompression engine entrance (this is because the compressed data block length corresponding to the compression result is larger). After decompression, it can basically ensure that the decompression exit can output data in each clock cycle. There is no need to repeatedly shift and splice the decompression results, and only one data splicing operation is required, thereby avoiding power consumption and achieving optimization of data decompression.

[0214] See also Fig.11 The embodiment of the present application provides a data compression device 1100, which includes a memory 1101, a processor 1102 and a communication interface 1103.

[0215] A memory 1101, used for storing instructions;

[0216] Processor 1102, used to execute instructions in memory 1101, and execute the above application Figure 2 The data compression method in the illustrated embodiment;

[0217] The communication interface 1103 is used for communication.

[0218] The memory 1101, the processor 1102 and the communication interface 1103 are interconnected via a bus 1104; the bus 1104 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0219] In a specific embodiment, the processor 1102 is used to compress the data blocks of L bytes in the compression engine into compressed data blocks of l bytes in length to obtain a compression result. L and l are the maximum bandwidth length of the decompression engine's decompression outlet and inlet, respectively. Then l is changed to l1 to ln in turn, and the above compression steps are repeated to obtain n compression results, and the compressed data block length corresponding to the compression result with the smallest total data length among the n compression results is obtained, and then the compressed data block length is stored in the compression information, and the compression information and the compression result with the smallest total data length are sent to the decompression engine. Please refer to the above for the detailed processing process of the processor 1102. Figure 2 The detailed description of S1, S2, S3, S4 and S5 in the illustrated embodiment will not be repeated here.

[0220] See also Fig.12 The embodiment of the present application provides a data decompression device 1200, which includes a memory 1201, a processor 1202 and a communication interface 1203.

[0221] Memory 1201, used for storing instructions;

[0222] Processor 1202, used to execute instructions in memory 1201, and execute the above application Figure 6 The data decompression method in the illustrated embodiment;

[0223] The communication interface 1203 is used for communication.

[0224] The memory 1201, the processor 1202 and the communication interface 1203 are interconnected via a bus 1204; the bus 1204 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0225] In a specific embodiment, when the processor 1202 is used to decompress data, after the decompression engine obtains the data to be decompressed and the compression information corresponding to the data to be decompressed sent by the compression engine, it can first obtain the length of the compressed data block in the compression information; wherein the data to be decompressed is the compression result with the smallest total length of the data corresponding to the original data; then, the data blocks with the length of the compressed data block in the data to be decompressed are sequentially decompressed to obtain the decompression result. Please refer to the above for the detailed processing process of the processor 1202 Figure 6 The detailed description of S601, S602 and S603 in the illustrated embodiment will not be repeated here.

[0226] The above-mentioned memory 1101 and memory 1201 can be random-access memory (RAM), flash memory, read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), register, hard disk, mobile hard disk, CD-ROM or any other form of storage medium known to those skilled in the art.

[0227] The processor 1102 and the processor 1202 may be, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. They may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0228] The communication interface 1103 and the communication interface 1303 may be, for example, an interface card, etc., and may be an Ethernet interface or an asynchronous transfer mode (ATM) interface.

[0229] An embodiment of the present application also provides a computer-readable storage medium, including instructions, which, when executed on a computer, enables the computer to execute the above-mentioned data compression method and data decompression method.

[0230] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0231] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0232] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0233] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0235] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data compression method, characterized in that: The method includes: S1: Obtain target data to be compressed; S2: Sequentially compress data blocks with a length of L bytes in the target data into compressed data blocks with a length of l bytes respectively to obtain a compression result; where L is the maximum bandwidth length of the decompression outlet of the decompression engine; and l is the maximum bandwidth length of the decompression inlet of the decompression engine; S3: Sequentially change l to l1 to ln, and repeat step S2 to obtain n compression results; l1 to ln are all positive integers greater than 1 and less than L; n is a positive integer greater than 1, 1 < l1 < ln < L, and l is one of the values from l1 to L; S4: Obtain the length of the compressed data block corresponding to the compression result with the minimum total data length among the n compression results; S5: Store the length of the compressed data block into the compression information, and send the compression information and the compression result with the minimum total data length to the decompression engine.

2. The method according to claim 1, characterized in that The step of sequentially compressing data blocks with a length of L bytes in the target data into compressed data blocks with a length of l bytes respectively to obtain a compression result includes: Sequentially compress data blocks with a length of L bytes in the target data to obtain first compression results corresponding to each data block; When the byte length of a first compression result is less than l, add the value 0 as first additional data after the first compression result so that the sum of the byte lengths of the first compression result and the first additional data reaches l, and use the first compression result and the first additional data together as the updated first compression result; Sort all the first compression results that do not need to be updated and all the updated first compression results in the order of precedence to obtain a compression result.

3. The method according to claim 1, characterized in that The step of sequentially compressing data blocks with a length of L bytes in the target data into compressed data blocks with a length of l bytes respectively to obtain a compression result includes: Sequentially compress data blocks with a length of L bytes in the target data to obtain second compression results corresponding to each data block; When the byte length of a second compression result is equal to l, sort all the second compression results in the order of precedence to obtain a compression result.

4. The method according to claim 1, characterized in that: The step of sequentially compressing data blocks with a length of L bytes in the target data into compressed data blocks with a length of l bytes respectively to obtain a compression result includes: Sequentially compress data blocks with a length of L bytes in the target data to obtain third compression results corresponding to each data block; When the byte length of a third compression result is greater than l, remove some tail data from the data blocks with a length of L bytes in the target data to obtain data blocks with a length less than L after removing the tail data; Compress the data blocks with a length less than L after removing the tail data to obtain fourth compression results; When the byte length of the fourth compression result is less than l and the byte length of the fourth compression result plus a coding length is greater than l, an invalid flag is added after the fourth compression result as the second additional data, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches l, and the fourth compression result and the second additional data are used as the updated third compression result; Sort all the third compression results that do not need to be updated and all the updated third compression results in sequence to obtain a compression result.

5. The method according to claim 1, characterized in that The obtaining the length of the compression data block corresponding to the compression result with the smallest total data length among the n compression results includes: Obtaining the total data length after adding up all the compression data blocks included in each of the n compression results; Determining the length of the compression data block corresponding to the compression result with the smallest total data length from the total data lengths after adding up all the compression data blocks included in each of the compression results.

6. A data decompression method, characterized in that: The method includes: Obtaining the data to be decompressed sent by the compression engine and the compression information corresponding to the data to be decompressed; the data to be decompressed is the compression result with the smallest total data length among the n compression results corresponding to the original data, and the n compression results are obtained by sequentially changing l to l1 to ln and repeatedly executing step S2. l1 to ln are all positive integers greater than 1 and less than L; n is a positive integer greater than 1, 1 < l1 < ln < L, and l is one of the values from l1 to L; S2 includes sequentially compressing the data blocks with a length of L bytes in the original data into compression data blocks with a length of l bytes to obtain a compression result, where L is the maximum bandwidth length of the decompression outlet of the decompression engine, and l is the maximum bandwidth length of the decompression inlet of the decompression engine; Obtaining the length of the compression data block in the compression information; Sequentially decompressing the data blocks with a length equal to the length of the compression data block in the data to be decompressed to obtain a decompression result.

7. The method according to claim 6, characterized in that The sequentially decompressing the data blocks with a length equal to the length of the compression data block in the data to be decompressed to obtain a decompression result includes: Sequentially decompressing the data blocks with a length equal to the length of the compression data block in the data to be decompressed; When the length of the original data block obtained by decompressing the data block with the length of the compression data block is equal to L, stopping the decompression operation and outputting the original data block with a length of L; or, when decompressing the data block with the length of the compression data block, if an invalid flag is recognized, stopping the decompression operation and outputting the original data block with a length less than or equal to L obtained by decompression; where L is the maximum bandwidth length of the decompression outlet of the decompression engine.

8. A data compression and decompression system, characterized in that: The system includes: a compression engine and a decompression engine; The compression engine is used to perform the following steps: S1: Obtain target data to be compressed; S2: Sequentially compress data blocks with a length of L bytes in the target data into compressed data blocks with a length of l bytes, obtaining a compression result; where L is the maximum bandwidth length at the decompression outlet of the decompression engine, and l is the maximum bandwidth length at the decompression inlet of the decompression engine; S3: Sequentially change l to l1 to ln, and repeat step S2 to obtain n compression results; l1 to ln are all positive integers greater than 1 and less than L; n is a positive integer greater than 1, 1 < l1 < ln < L, and l is one of the values from l1 to L; S4: Obtain the length of the compressed data block corresponding to the compression result with the minimum total data length among the n compression results; S5: Store the length of the compressed data block into the compression information, and send the compression information and the compression result with the minimum total data length to the decompression engine. The decompression engine is used to obtain the length of the compressed data block in the compression information; sequentially decompress data blocks with a length equal to the length of the compressed data block in the data to be decompressed, obtaining a decompression result.

9. The system according to claim 8, characterized in that The compression engine is also used to: Sequentially compress data blocks with a length of L bytes in the target data, obtaining first compression results corresponding to each data block; When the byte length of a first compression result is less than l, add the numerical value 0 as first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches l, and use the first compression result and the first additional data together as the updated first compression result; Sort all the first compression results that do not need to be updated and all the updated first compression results in sequence, obtaining a compression result.

10. The system according to claim 8, characterized in that The compression engine is also used to: Sequentially compress data blocks with a length of L bytes in the target data, obtaining second compression results corresponding to each data block; When the byte length of a second compression result is equal to l, sort all the second compression results in sequence, obtaining a compression result.

11. The system according to claim 8, characterized in that The compression engine is also used to: Sequentially compress data blocks with a length of L bytes in the target data, obtaining third compression results corresponding to each data block; When the byte length of a third compression result is greater than l, remove some tail data from the data block with a length of L bytes in the target data, obtaining a data block with a length less than L after removing the tail data; Compress the data block with a length less than L after removing the tail data, obtaining a fourth compression result; When the byte length of the fourth compression result is less than l and the byte length of the fourth compression result plus a coding length is greater than l, add an invalid flag as second additional data after the fourth compression result, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches l, and use the fourth compression result and the second additional data as the updated third compression result; Sort all the third compression results that do not need to be updated and all the updated third compression results in sequence to obtain a compression result.

12. The system according to claim 8, characterized in that The compression engine is further configured to: Obtain the total data length after adding up all the compressed data blocks included in each of the n compression results; Determine the length of the compressed data block corresponding to the compression result with the smallest total data length from the total data lengths after adding up all the compressed data blocks included in each of the compression results.

13. The system according to claim 8, characterized in that The decompression engine is further configured to: Successively decompress the data blocks with the length of the compressed data block in the data to be decompressed; When the length of the original data block obtained by decompressing the data block with the length of the compressed data block is equal to L, stop the decompression operation and output the original data block with the length of L; or, when decompressing the data block with the length of the compressed data block, if an invalid flag is recognized, stop the decompression operation and output the original data block with a length less than or equal to L obtained by decompression; wherein, L is the maximum bandwidth length of the decompression engine's decompression outlet.

14. A data compression device, characterized in that: The device includes: A first acquisition unit, configured to acquire target data to be compressed; A first compression unit, configured to successively compress the data blocks with the length of L bytes in the target data into compressed data blocks with the length of l bytes respectively to obtain a compression result; L is the maximum bandwidth length of the decompression engine's decompression outlet; l is the maximum bandwidth length of the decompression engine's decompression inlet; A second compression unit, configured to successively change l to l1 to ln, and repeatedly call the first compression unit to successively compress the data blocks with the length of L bytes in the target data into compressed data blocks with the length of l1 to ln bytes respectively to obtain n compression results; l1 to ln are all positive integers greater than 1 and less than L; n is a positive integer greater than 1, 1 < l1 < ln < L, and l is one of the values from l1 to L; A second acquisition unit, configured to acquire the length of the compressed data block corresponding to the compression result with the smallest total data length among the n compression results; A sending unit, configured to store the length of the compressed data block into the compression information, and send the compression information and the compression result with the smallest total data length to the decompression engine.

15. The device according to claim 14, characterized in that The first compression unit includes: A first compression subunit, configured to successively compress the data blocks with the length of L bytes in the target data to obtain first compression results corresponding to each data block; A first addition subunit, configured to, when the byte length of the first compression result is less than l, add the numerical value 0 as the first additional data after the first compression result, so that the sum of the byte lengths of the first compression result and the first additional data reaches l, and use the first compression result and the first additional data together as the updated first compression result; A first obtaining subunit, configured to sort all the first compression results that do not need to be updated and all the updated first compression results in sequence to obtain a compression result.

16. The device according to claim 14, characterized in that The first compression unit includes: A second compression subunit, configured to sequentially compress data blocks with a length of L bytes in the target data to obtain second compression results corresponding to the respective data blocks; A second obtaining subunit, configured to, when the byte length of the second compression result is equal to l, sort all the second compression results in the order of appearance to obtain a compression result.

17. The device according to claim 14, characterized in that The first compression unit includes: A third compression subunit, configured to sequentially compress data blocks with a length of L bytes in the target data to obtain third compression results corresponding to the respective data blocks; A removing subunit, configured to, when the byte length of the third compression result is greater than l, remove some tail data from the data blocks with a length of L bytes in the target data to obtain data blocks with a length less than L after removing the tail data; A fourth compression subunit, configured to compress the data blocks with a length less than L after removing the tail data to obtain a fourth compression result; A second adding subunit, configured to, when the byte length of the fourth compression result is less than l and the byte length of the fourth compression result plus a coding length is greater than l, add an invalid flag as second additional data after the fourth compression result, so that the sum of the byte lengths of the fourth compression result and the second additional data reaches l, and use the fourth compression result and the second additional data as an updated third compression result; A third obtaining subunit, configured to sort all the third compression results that do not need to be updated and all the updated third compression results in the order of appearance to obtain a compression result.

18. The device according to claim 14, characterized in that The second obtaining unit includes: An obtaining subunit, configured to obtain the total data length after adding up all the compressed data blocks included in each compression result among the n compression results; A determining subunit, configured to determine, from the total data lengths after adding up all the compressed data blocks included in each compression result, the length of the compressed data block corresponding to the compression result with the smallest total data length.

19. A data decompression device, characterized in that: The apparatus includes: A first obtaining unit, configured to obtain the data to be decompressed sent by the compression engine and the compression information corresponding to the data to be decompressed; the data to be decompressed is the compression result with the smallest total data length among n compression results corresponding to the original data, the n compression results are obtained by sequentially changing l to l1 to ln and repeatedly executing step S2, l1 to ln are all positive integers greater than 1 and less than L; n is a positive integer greater than 1, 1 < l1 < ln < L, and l is one of the values from l1 to L; S2 includes sequentially compressing the data blocks with a length of L bytes in the original data into compressed data blocks with a length of l bytes to obtain a compression result, L is the maximum bandwidth length of the decompression outlet of the decompression engine, and l is the maximum bandwidth length of the decompression inlet of the decompression engine; A second obtaining unit, configured to obtain the length of the compressed data block in the compression information; A decompression unit, configured to sequentially decompress the data blocks with a length of the compressed data block length in the data to be decompressed to obtain a decompression result.

20. The device according to claim 19, characterized in that The decompression unit includes: A decompression subunit, used for sequentially decompressing data blocks of the to-be-decompressed data having a length of the compressed data block; The output subunit is used to stop the decompression operation and output the original data block of the length L when the length of the original data block obtained by decompressing the data block of the compressed data block length is equal to L; or, when decompressing the data block of the compressed data block length, if an invalid flag is identified, stop the decompression operation and output the original data block of the decompressed length less than or equal to L; wherein L is the maximum bandwidth length of the decompression outlet of the decompression engine.

21. A data compression device, characterized in that: The device includes a memory and a processor; The memory is used to store instructions; The processor is used to execute the instructions in the memory and execute the method according to any one of claims 1 to 5.

22. A data decompression device, characterized in that: The device includes a memory and a processor; The memory is used to store instructions; The processor is used to execute the instructions in the memory and execute the method according to any one of claims 6 to 7.

23. A computer-readable storage medium comprising instructions, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 7.

Images