Data compression, decompression method and related device, electronic equipment, system

Abstract

The embodiment of the present application discloses a data compression and decompression method and related devices, electronic devices, and systems. The data compression method processes the original data blocks in the original data based on the compression algorithm to obtain the standard compressed data blocks with a length of L2 corresponding to the original data blocks, and then compresses the original data into one or more standard compressed data blocks with a length of L2 for parallel decompression by the decompression device, wherein the decompression device includes multiple decompression engines, each of which can decompress a standard compressed data block within a processing cycle. By executing the data compression method of the embodiment of the present invention, the data can be compressed into standard compressed data blocks with a fixed length, so that each decompression engine can decompress synchronously when decompressing the multiple standard compressed data blocks, thereby improving the decompression efficiency of the data.

Description

[0001] This application claims priority to a Chinese patent application filed with the State Intellectual Property Office of China on July 31, 2018, with application number 201810859928.3 and application name “Data compression, decompression methods and related devices, electronic equipment, and systems”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present invention relates to the technical field of data compression, and in particular to a data compression and decompression method and related devices, electronic equipment and systems. Background Art

[0003] For operations with large amounts of data, such as video, image processing, or machine learning, hardware chips are usually used for acceleration. In the process of data processing by hardware chips, it is often necessary to import a large amount of data from external memory in real time. For example, hardware acceleration of machine learning is a technology that achieves computing acceleration by executing machine learning algorithms through hardware chips. When running machine learning algorithms, a large number of parameters are often required. These parameters cannot be completely stored inside the hardware chip, so when the machine learning algorithm is running, a large number of parameters need to be imported from the outside in real time. Since the real-time import of a large number of parameters occupies the input / output (I / O) bandwidth of the chip, when the I / O bandwidth is insufficient, the computing power of the hardware chip is idle, reducing the running speed of the machine learning algorithm.

[0004] By compressing the external parameters offline to reduce their size, and then performing real-time online decompression and restoration of the external parameters in the hardware chip through the decompression engine, more external parameters can be transmitted under the same I / O bandwidth. However, the decompression rate that the decompression engine can achieve is limited, which also limits the running speed of the machine learning algorithm.

[0005] At present, the common method to improve the decompression rate of hardware chips is to use multiple engines for parallel decompression. Figure 1 The schematic diagram of the principle of decompression by multiple decompression engines in the prior art is shown. When an electronic device compresses a file to be compressed, the file to be compressed is split into multiple original data blocks of equal length, and the multiple original data blocks are compressed respectively to generate compressed data blocks corresponding to each original data block; when decompressing, multiple compressed data blocks are first obtained, and the multiple compressed data blocks are distributed to multiple decompression engines, and the multiple decompression engines restore multiple original data blocks, and then the multiple original data blocks are spliced ​​into the original file (i.e., the file to be compressed).

[0006] It can be seen that in the prior art, due to the different compression rates of the original data blocks, the lengths of the compressed data blocks obtained are different, which makes the time required for each decompression engine to decompress each compressed data block different. The decompression device can only splice the original data blocks restored by each decompression engine into the required data when the last compressed data block is decompressed, and the decompression efficiency is low. Summary of the invention

[0007] The technical problem to be solved by the embodiments of the present invention is to provide a data compression method, a data decompression method and related devices, electronic devices, and systems, which standardize the compression process of the original data to generate standardized compressed data blocks, so that each decompression engine can decompress the multiple standardized compressed data blocks synchronously, thereby improving the data decompression efficiency.

[0008] In a first aspect, an embodiment of the present invention provides a data compression method that can be applied to a first electronic device or a compression device, and is used to compress original data into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device includes multiple decompression engines, each decompression engine can decompress one of the standard compressed data blocks within one processing cycle, and L2 is an integer greater than 1. The method includes: performing one or more original data block compressions, wherein the original data block compression includes: processing the original data block based on a compression algorithm to obtain a standard compressed data block corresponding to the original data block, wherein the original data block is the data included in the original data.

[0009] By executing the data compression method of the embodiment of the present invention, data can be compressed into standard compressed data blocks with a fixed length, so that each decompression engine can decompress the multiple standard compressed data blocks synchronously, thereby improving data decompression efficiency.

[0010] In one implementation of an embodiment of the present application, an implementation in which a first electronic device or a compression device processes an original data block based on a compression algorithm to obtain a standard compressed data block corresponding to the original data block may include: a decompression device independently compresses the original data block through a compression algorithm to obtain a compressed data block corresponding to the original data block, and the length d of the compressed data block is not greater than the L2; further, additional data of a length of L2-d is added after the compressed data block, thereby generating a standard compressed data block of a length of L2, where d is an integer greater than 0.

[0011] In another implementation of the embodiment of the present application, the compression device or the processor of the first electronic device compresses each of the S continuous original data blocks in the original data by a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, the length d of each compressed data block in the S compressed data blocks is not greater than L2, and S is a positive integer; further, additional data of length L2-d is added after each compressed data block in the S compressed data blocks to generate S standard compressed data blocks of length L2, wherein the standard data block of length L2 is a data block that can be processed by a single decompression engine in the decompression device within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2. It can be understood that the execution subject of the data compression method can be the first electronic device or the compression device, and the following is explained by taking the execution subject compression device as an example.

[0012] By executing the above method, when compressing the original data, the length of the S standard compressed data blocks generated is a fixed value L2, and the length of the original data block generating the standard compressed data block is not greater than L1, which standardizes the length of the compressed data block generated by the original data compression, and each standard compressed data block is independently compressed, and can be independently decompressed, so that each decompression engine can decompress synchronously when decompressing the multiple standard compressed data blocks, and can complete the processing of data of a specific length within one processing cycle, without waiting for the output of the slowest decompression engine, thereby solving the problem of the "short board" in the Cannikin Law and improving the data decompression efficiency. Moreover, compared with compression into variable-length compressed data blocks, the present application is set to a standard compressed data block of a fixed length L2, and the compression end does not need to record the length of each compressed data block, and the decompression end does not need to record the length of the acquired data, making the design of the decompression end simpler.

[0013] Furthermore, in the prior art, the number of original data blocks into which the original data is divided is equal to the number of decompression engines, that is, the number of decompression engines limits the granularity at which the original data can be divided, so that the length of the multiple compressed data blocks compressed by the original data is relatively large. When accessing data at a random address, it is necessary to decompress the compressed data block where the data at the random address is located. During the decompression process, due to the correlation of the data in the compressed data block, the compressed data block can only be decompressed in one decompression engine, and it is impossible to achieve double-speed decompression of the compressed data. By executing the technical solution described in the above method, the granularity at which the original data can be divided is independent of the number of decompression engines, and the original data can be compressed into a large number of smaller standard compressed data blocks, and there is no correlation between the various standard compressed data blocks. When accessing data at a random address, the multiple continuous standard compressed data blocks where the data at the random address is located are input into multiple decompression engines for decompression, which can achieve double-speed decompression of the compressed data at the random address.

[0014] In a possible implementation, the first original data block is any one of the S original data blocks, wherein, when the length of the first original data block is equal to L1, the additional data in the canonical data block compressed from the first original data block is a legal codeword or the additional data includes an invalid flag and invalid data. When the length of the first original data block is less than L1, the additional data includes an invalid flag and invalid data. The invalid flag is used to indicate that the additional data where the invalid flag is located needs to be discarded; the invalid data may include data greater than or equal to 0 bits after the invalid flag. By setting invalid data or invalid flags in the above-mentioned various ways, the decompression engine can be stopped from decompressing, thereby reducing decompression operations while obtaining correct data, thereby reducing decompression power consumption.

[0015] Optionally, when a processing cycle includes multiple clock cycles, the additional data in the canonical compressed data block obtained by compressing the first original data block includes an invalid flag and invalid data. The invalid flag is used to indicate that the additional data where the invalid flag is located needs to be discarded; the invalid data may include data greater than or equal to 0 bits after the invalid flag.

[0016] In another possible implementation, a compression device independently compresses the original data block through a compression algorithm to obtain a compressed data block corresponding to the original data block, which may include the following steps: S1, obtaining a data group to be compressed from the original data, the data group including one or more bits of data; S2, judging whether it is necessary to compress the data group to be compressed, if it is necessary to compress, executing S3, otherwise executing S4; S3, compressing the data block group to be compressed, and executing S1 after compression; S4, using the currently compressed data as a compressed data block; wherein, an implementation of judging whether it is necessary to compress the data group to be compressed may include: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than L2, it is determined that compression is required; if the sum of x and Δx is not greater than L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than L2, it is determined that compression is not required; if the sum of x and Δx is greater than L1, it is determined that compression is not required.

[0017] The above method can realize compression of data blocks, which has the advantages of simple implementation and high efficiency.

[0018] In another possible implementation, the compression device independently compresses the original data block through a compression algorithm to obtain a compressed data block corresponding to the original data block. Another implementation method may be: the compression device compresses the data block with a length of L1 in the original data with a data group as a step length to obtain a temporary compressed data block; when the length of the temporary compressed data block is greater than L2, the data block with a length of L1 is removed from the tail i data groups to obtain a data block as the original data block, and the data obtained by compressing the original data block is used as a compressed data block, wherein the length of the compressed data block is not greater than L2, i is a positive integer, and i<L1; when the length of the temporary compressed data block is not greater than L2, the data block with a length of L1 is the original data block, and the temporary compressed data block is the compressed data block. The above method provides another specific implementation, and has the advantages of simple implementation and high compression efficiency.

[0019] Optionally, i is a minimum positive integer that makes the compressed data block no larger than L2. The smaller i is, the more compressed data is, thereby increasing the compression efficiency.

[0020] In another possible implementation, L1 is the exit bit width of a single decompression engine, and L2 is the entry bit width of a single decompression engine. In this way, existing hardware resources can be used as much as possible to increase the amount of data compressed each time, thereby increasing compression efficiency.

[0021] In the second aspect, an embodiment of the present application also provides a compression device, comprising: a device for compressing original data into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device comprises multiple decompression engines, each of which is capable of decompressing one standard compressed data block within one processing cycle, and L2 is an integer greater than 1, comprising: a processing module, for processing the original data block based on a compression algorithm to obtain the standard compressed data block corresponding to the original data block, wherein the original data block is the data included in the original data.

[0022] In a possible implementation, the processing module includes:

[0023] A compression unit, configured to independently compress the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block, wherein the length d of the compressed data block is not greater than L2;

[0024] The standardization unit is used to add additional data of a length of L2-d after the compressed data block, so as to generate the standard compressed data block of the length of L2, where d is an integer greater than 0.

[0025] Optionally, the original data block includes a plurality of continuous data in the original data, and the length of the original data block is not greater than L1, where L1 is an exit bit width of the decompression engine.

[0026] In a possible implementation, the processing module may include:

[0027] A compression unit, used for independently compressing each of S consecutive original data blocks in the original data by a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of consecutive data in the original data, a length of the plurality of consecutive data is not greater than L1, a length d of each compressed data block in the S compressed data blocks is not greater than L2, and S is a positive integer;

[0028] A standardization unit is used to add additional data of a length of L2-d after each of the S compressed data blocks to generate S standard compressed data blocks of a length of L2, wherein the standard data block of length L2 is a data block that can be decompressed by a single decompression engine in the decompression device within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2.

[0029] In a possible implementation, when the length of the original data block is equal to L1, the additional data in the canonical data block obtained by compressing the original data block is a legal codeword; or the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded; or,

[0030] When the length of the original data block is less than L1, the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded.

[0031] In another possible implementation, the compression unit specifically includes a step of performing the following steps:

[0032] S1. Obtain a data group to be compressed from the original data, wherein the data group includes one or more bits of data;

[0033] S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4;

[0034] S3, compressing the data block group to be compressed, and executing S1 after the compression is completed;

[0035] S4, using the currently compressed data as the compressed data block;

[0036] Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than L2, it is determined that compression is required; if the sum of the x and the Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than L2, it is determined that compression is not required; if the sum of the x and the Δx is greater than the L1, it is determined that compression is not required.

[0037] In another possible implementation, the compression unit specifically includes a step of performing the following steps:

[0038] Compress the data block with a length of L1 in the original data with the data group as the step length to obtain a temporary compressed data block;

[0039] When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having a length of L1 is used as the original data block, and data obtained by compressing the original data block is used as the compressed data block, wherein the length of the compressed data block is not greater than L2, i is a positive integer, and i<L1;

[0040] When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the original data block, and the temporary compressed data block is used as the compressed data block.

[0041] Optionally, i is a minimum positive integer that ensures that the compressed data block is no larger than L2.

[0042] In yet another possible implementation, the L1 is an egress bit width of a single decompression engine; and / or the L2 is an ingress bit width of a single decompression engine.

[0043] In the third aspect, an embodiment of the present application also provides a compression device, comprising: a processor and a memory, wherein the memory is used to store data and program instructions, and when the processor calls the memory to store data and program instructions, the compression device implements any one of the data compression methods described in the first aspect.

[0044] In a fourth aspect, an embodiment of the present application further provides an electronic device with a compression function, specifically a first electronic device in an embodiment of the present application, the electronic device comprising a compression device and a processing device, the compression device may be any one of the compression devices described in the second aspect or the third aspect; the processing device is used to send the original data described above to the compression device.

[0045] In a fifth aspect, an embodiment of the present application further provides a computer storage medium, which is used to store computer software instructions, and when the computer software instructions are executed by a computer, the computer implements any one of the data compression methods described in the first aspect.

[0046] In a sixth aspect, an embodiment of the present application further provides a computer program, wherein the computer program includes computer software instructions, and when the computer software instructions are executed by a computer, the computer implements any one of the data compression methods described in the first aspect.

[0047] In the seventh aspect, an embodiment of the present application further provides a decompression device, comprising: a first memory, M decompression engines and a second memory, wherein the first memory is used to store multiple standard compressed data blocks (for example, N standard compressed data blocks), wherein one of the standard compressed data blocks is a data block with a length of L2 generated after independently compressing the original data block, and one of the standard compressed data blocks can be decompressed by a single decompression engine within one processing cycle, M is an integer greater than 1, M and N are positive integers, M≥N; the M decompression engines are used to perform parallel decompression on the standard compressed data blocks stored in the first memory, and one of the decompression engines is used to decompress one of the standard compressed data blocks, for example, the M decompression engines include N decompression engines, and each of the N decompression engines is used to obtain one of the N standard compressed data blocks, and decompress the obtained standard compressed data block to obtain N original data blocks corresponding one to one to the N standard compressed data blocks.

[0048] The second memory is used to store original data blocks obtained by decompressing the standard compressed data blocks, for example, N original data blocks, and the standard compressed data blocks correspond to the original data blocks one by one.

[0049] Optionally, the length of the original data block is not greater than L1, and the exit bit width of the decompression engine is L1.

[0050] In the decompression device provided by the above embodiment, multiple standard compressed data blocks are decompressed in parallel by multiple decompression engines respectively, so that each decompression engine can decompress the multiple standard compressed data blocks in parallel when decompressing the multiple standard compressed data blocks, thereby improving the decompression efficiency of the data. Moreover, compared with compressed data blocks compressed into variable lengths, the present application sets a standard compressed data block of a fixed length L2, and the compression end does not need to record the length of each compressed data block, and the decompression end does not need to record the length of the acquired data, making the design of the decompression end simpler.

[0051] Furthermore, in the prior art, the number of original data blocks into which the original data is divided is equal to the number of decompression engines, that is, the number of decompression engines limits the granularity at which the original data can be divided, so that the length of the multiple compressed data blocks formed by compressing the original data is relatively large. When accessing data at a random address, it is necessary to decompress the compressed data block where the data at the random address is located. During the decompression process, due to the correlation of the data in the compressed data block, the compressed data block can only be decompressed in one decompression engine, and it is impossible to achieve double-speed decompression of the compressed data. Through the above-mentioned decompression device, the granularity at which the original data can be divided is independent of the number of decompression engines, and the original data can be compressed into a large number of smaller standard compressed data blocks. When accessing data at a random address, the multiple continuous standard compressed data blocks where the data at the random address is located are input into multiple decompression engines for decompression, which can achieve double-speed decompression of the compressed data at the random address.

[0052] In a possible implementation, the decompression device further includes a splicing circuit, which is used to splice the original data blocks stored in the second storage to obtain the original data. In this way, the original data can be decompressed and processed by the subsequent processing circuit, ensuring that the data is not lost.

[0053] In one embodiment, the decompression device may further include a length counter for recording the length of the decompressed data, so that the splicing circuit can obtain data of a specific length for splicing according to the value in the length counter.

[0054] In another possible implementation, each of the M decompression engines is further configured to stop the decompression operation when the length of the original data block obtained by decompression is equal to the L1 or when an invalid flag is identified. By identifying the invalid flag to stop decompression, the decompression operation can be saved and the power consumption can be reduced while ensuring that correct data is obtained.

[0055] In another possible implementation, the decompression device further includes a control circuit, which is used to: when the length of the original data block decompressed by the first decompression engine is equal to the L1, stop the decompression operation of the first decompression engine, and the first decompression engine is any one of the M decompression engines. By stopping the decompression after determining that the data of the required length (L1) is obtained, the decompression operation can be saved and the power consumption can be reduced while ensuring that the correct data is obtained.

[0056] In another possible implementation, the second memory includes M storage sub-units corresponding one-to-one to the M decompression engines, respectively, and the first storage sub-unit is used to store the original data blocks decompressed by the decompression engine corresponding to the first storage sub-unit, and the first storage sub-unit is any one of the M storage sub-units.

[0057] Among them, the decompression engine and various circuits (such as control circuit, splicing circuit) can be implemented based on hardware circuits (such as FPGA, ASIC), so that the processing rate can be increased, thereby improving the decompression speed.

[0058] In an eighth aspect, an embodiment of the present application further provides an integrated circuit, comprising: any one of the decompression devices described in the seventh aspect.

[0059] Optionally, the integrated circuit may further include a computing device coupled to the decompression device, wherein the computing device is used to perform data processing according to original data decompressed by the decompression device.

[0060] In the ninth aspect, an embodiment of the present application also provides a data decompression method, which is applied to a decompression device, wherein the decompression device includes M decompression engines, where M is a positive integer, and the method includes: the decompression device obtains N standard compressed data blocks, wherein any one of the N standard compressed data blocks is a data block with a length of L2 generated after independently compressing the original data block, and the standard compressed data block with a length of L2 is a data block that can be decompressed by a single decompression engine within one processing cycle, where N is a positive integer, and M≥N; the N standard compressed data blocks are decompressed by N decompression engines respectively to obtain N original data blocks corresponding one to one to the N standard compressed data blocks; the N original data blocks are spliced ​​to obtain original data, wherein L1 and L2 are integers greater than 1, and L1 is greater than L2.

[0061] By executing the above method, N standard compressed data blocks of length L2 are synchronously decompressed by N decompression engines respectively, so that each decompression engine can synchronously decompress the multiple standard compressed data blocks, thereby improving data decompression efficiency.

[0062] Moreover, compared with compressing into variable-length compressed data blocks, the present application sets a standard compressed data block of fixed length L2. The compression end does not need to record the length of each compressed data block, and the decompression end does not need to record the length of the acquired data, making the design of the decompression end simpler.

[0063] Furthermore, in the prior art, the number of original data blocks into which the original data is divided is equal to the number of decompression engines, that is, the number of decompression engines limits the granularity at which the original data can be divided, so that the length of the multiple compressed data blocks compressed by the original data is relatively large. When accessing data at a random address, it is necessary to decompress the compressed data block where the data at the random address is located. During the decompression process, due to the correlation of the data in the compressed data block, the compressed data block can only be decompressed in one decompression engine, and it is impossible to achieve double-speed decompression of the compressed data. By executing the technical solution described in the above method, the granularity at which the original data can be divided is independent of the number of decompression engines, and the original data can be compressed into a large number of smaller standard compressed data blocks. When accessing data at a random address, the multiple continuous standard compressed data blocks where the data at the random address is located are input into multiple decompression engines for decompression, which can achieve double-speed decompression of the compressed data at the random address.

[0064] In a possible implementation, the N decompression engines include the decompression engine k, the N compressed data blocks include the compressed data block k, and the N original data blocks include the original data block k obtained by decompressing the compressed data block k, where k is a positive integer and is not greater than N; and the N decompression engines are used to decompress the N standard compressed data blocks respectively to obtain the N compressed data blocks corresponding to the N original data blocks one by one, specifically including:

[0065] Decompressing the compressed data block k by the decompression engine k;

[0066] When the length of the original data block k decompressed by the decompression engine k is equal to L1, the decompression operation of the decompression engine k on the compressed data block k is stopped, and the original data block k is output; or, when the decompression engine k recognizes an invalid flag, the decompression operation of the decompression engine k on the compressed data block k is stopped, and the decompressed original data block k is output.

[0067] In the tenth aspect, the embodiment of the present application also provides an electronic device with a decompression function, which may specifically be the second electronic device in the embodiment of the present application, and may include a processor, a memory coupled to the processor, and a decompression device, wherein the decompression device is any one of the decompression devices described in the seventh aspect, and the processor is used to call the data and instructions stored in the memory to implement the function of the second electronic device.

[0068] In the eleventh aspect, an embodiment of the present application further provides a compression and decompression system, the system comprising a compression device which is any one of the compression devices described in the second aspect or the third aspect, and the decompression device which is any one of the decompression devices described in the seventh aspect.

[0069] In the twelfth aspect, an embodiment of the present application further provides a compression and decompression system, including a first electronic device and a second electronic device, the first electronic device being the electronic device with a compression function as described in the fourth aspect, and the second electronic device being the electronic device with a decompression function as described in the tenth aspect.

[0070] In a thirteenth aspect, the embodiment of the present application further provides a data compression method, including:

[0071] Each of S consecutive original data blocks in the original data is independently compressed by a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, the length d of each compressed data block in the S compressed data blocks is not greater than L2, S is a positive integer, a compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2;

[0072] The length of each compressed data block in the S compressed data blocks is recorded.

[0073] The above data compression method can be applied to a compression device or a third electronic device with a compression function. Here, the compression device is taken as an example for description.

[0074] The difference from the data compression method described in the first aspect is that the data compression method described in the first aspect finally obtains a standard compressed data block, whose length is fixed to L2, which is achieved by adding additional data after the compressed data block obtained by compressing the original data block. In the above embodiment, the compressed data block obtained by compressing the original data block is the compressed data block finally obtained, and its length is variable. The compression device needs to record the length of each compressed data block so that the decompression device can distinguish compressed data blocks of different lengths.

[0075] Similar to the first aspect, by executing the above method, when compressing the original data, the length of the S compressed data blocks generated is less than L2, so that each decompression engine can complete the decompression of the first compressed data block within one processing cycle, thereby realizing synchronous decompression of each decompression engine and improving data decompression efficiency.

[0076] Furthermore, in the prior art, the number of original data blocks into which the original data is divided is equal to the number of decompression engines, that is, the number of decompression engines limits the granularity at which the original data can be divided, so that the length of the multiple compressed data blocks compressed by the original data is relatively large. When accessing data at a random address, it is necessary to decompress the compressed data block where the data at the random address is located. During the decompression process, due to the correlation of the data in the compressed data block, the compressed data block can only be decompressed in one decompression engine, and it is impossible to achieve double-speed decompression of the compressed data. By executing the technical solution described in the above method, the granularity at which the original data can be divided is independent of the number of decompression engines, and the original data can be compressed into a large number of smaller compressed data blocks. When accessing data at a random address, the multiple continuous standard compressed data blocks where the data at the random address is located are input into multiple decompression engines for decompression, which can achieve double-speed decompression of the compressed data at the random address.

[0077] In a possible implementation manner, the first original data block is any one of the S original data blocks, the first compressed data block is a compressed data block corresponding to the first original data block among the S compressed data blocks, and the compression device independently compresses the first original data block through a compression algorithm, and an implementation manner of obtaining the first compressed data block may include:

[0078] S1. Obtain a data group to be compressed;

[0079] S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4;

[0080] S3, compressing the data block group to be compressed, and executing S1 after the compression is completed;

[0081] S4, using the currently compressed data as the first compressed data block;

[0082] Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than the L2, it is determined that compression is required; if the sum of x and Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than the L2, it is determined that compression is not required; if the sum of x and Δx is greater than the L1, it is determined that compression is not required.

[0083] In another possible implementation manner, the first original data block is any one of the S original data blocks, the first compressed data block is a compressed data block corresponding to the first original data block among the S compressed data blocks, and the compression device independently compresses the first original data block in the original data by using a compression algorithm to obtain another implementation manner of the first compressed data block may be:

[0084] The compression device compresses the data block with a length of L1 in the original data with the data group as a step length to obtain a temporary compressed data block;

[0085] When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having the length L1 is used as the first original data block, and data obtained by compressing the first original data block is used as the first compressed data block, wherein the length of the first compressed data block is not greater than L2, i is a positive integer, and i<L1;

[0086] When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the first original data block, and the temporary compressed data block is used as the first compressed data block.

[0087] Optionally, i is a minimum positive integer that ensures that the first compressed data block is no larger than L2.

[0088] In a possible implementation, an implementation of the third electronic device or the compression device recording the length of the first compressed data block may be:

[0089] The compression device records the length d of the first compressed data block in a start field of the first compressed data block, where the start field is a field starting from the start bit of the first compressed data block and having a length equal to a third length threshold; or,

[0090] The length d of the first compressed data block is written into the first file; the first file includes the lengths corresponding to the S compressed data blocks obtained by sequentially compressing the original data by the second compression device.

[0091] In a fourteenth aspect, the present application also provides a compression device, including:

[0092] A compression unit, used for independently compressing each of S consecutive original data blocks in the original data by a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, the length d of each compressed data block in the S compressed data blocks is not greater than L2, S is a positive integer, a compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2;

[0093] The recording unit is used to record the length of each compressed data block in the S compressed data blocks.

[0094] In a possible implementation, the first original data block is any one of the S original data blocks, the first compressed data block is a compressed data block corresponding to the first original data block among the S compressed data blocks, and the compression unit includes a step for executing:

[0095] S1. Obtain a data group to be compressed;

[0096] S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4;

[0097] S3, compressing the data block group to be compressed, and executing S1 after the compression is completed;

[0098] S4, using the currently compressed data as the first compressed data block;

[0099] Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than the L2, it is determined that compression is required; if the sum of x and Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than the L2, it is determined that compression is not required; if the sum of x and Δx is greater than the L1, it is determined that compression is not required.

[0100] In another possible implementation, the first original data block is any one of the S original data blocks, the first compressed data block is a compressed data block corresponding to the first original data block among the S compressed data blocks, and the compression unit includes a device specifically configured to:

[0101] Compress the data block with a length of L1 in the original data with the data group as the step length to obtain a temporary compressed data block;

[0102] When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having the length L1 is used as the first original data block, and data obtained by compressing the first original data block is used as the first compressed data block, wherein the length of the first compressed data block is not greater than L2, i is a positive integer, and i<L1;

[0103] When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the first original data block, and the temporary compressed data block is used as the first compressed data block.

[0104] Optionally, i is a minimum positive integer that ensures that the first compressed data block is no larger than L2.

[0105] In a possible implementation, the recording unit is specifically used for:

[0106] Record the length d of the first compressed data block in the start field of the first compressed data block, where the start field is a field starting from the start bit of the first compressed data block and having a length equal to a third length threshold; or,

[0107] The length d of the first compressed data block is recorded and written into the first file; the first file includes the lengths corresponding to the plurality of first compressed data blocks obtained by sequentially compressing the original data by the second compression device.

[0108] In the fifteenth aspect, an embodiment of the present application further provides a compression device, comprising: a processor and a memory, wherein the memory is used to store data and program instructions, and the processor calls the memory to store data and program instructions to execute the data compression method as described in the thirteenth aspect.

[0109] In the sixteenth aspect, the embodiments of the present application further provide an electronic device with a compression function, which may specifically be the third electronic device in the embodiments of the present application, comprising: any one of the compression devices described in the fourteenth aspect or the fifteenth aspect.

[0110] In the seventeenth aspect, an embodiment of the present application further provides a computer storage medium, which is used to store computer software instructions, and when the computer software instructions are executed by a computer, the computer implements any one of the data compression methods described in the thirteenth aspect.

[0111] In the eighteenth aspect, an embodiment of the present application further provides a computer program, wherein the computer program includes computer software instructions, and when the computer software instructions are executed by a computer, the computer implements any one of the data compression methods described in the thirteenth aspect.

[0112] In a nineteenth aspect, an embodiment of the present application further provides a data decompression method, which is applied to a decompression device, the decompression device including M decompression engines, and the method includes:

[0113] Obtain compressed data, wherein the compressed data includes N compressed data blocks and the length of each compressed data block in the N compressed data blocks, where N is a positive integer; wherein any one of the N compressed data blocks is an independently compressed data block with a length not greater than L2, where N is a positive integer, and M≥N;

[0114] Splitting the compressed data into N compressed data blocks according to the length of each of the N compressed data blocks;

[0115] Decompressing the N standard compressed data blocks respectively by using N decompression engines to obtain N original data blocks corresponding to the N compressed data blocks one by one;

[0116] The N original data blocks are concatenated to obtain original data.

[0117] In a twentieth aspect, an embodiment of the present application further provides a decompression device, including a controller and M decompression engines, wherein the controller includes an acquisition unit, a splitting unit, a data distribution unit, and a splicing unit, wherein:

[0118] The acquisition unit is used to: acquire compressed data, the compressed data including N compressed data blocks and the length of each compressed data block in the N compressed data blocks, N is a positive integer; wherein any one of the N compressed data blocks is an independently compressed data block with a length not greater than L2, N is a positive integer, and the compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle;

[0119] The splitting unit is used to split the compressed data into N compressed data blocks according to the length of each compressed data block in the N compressed data blocks,

[0120] The data distribution unit is used to distribute the N compressed data blocks to N decompression engines among the M decompression engines, so that the N decompression engines decompress the N compressed data blocks respectively to obtain N original data blocks corresponding to the N compressed data blocks, wherein the length of any original data block among the N original data blocks is not greater than L1, and L1 and L2 are integers greater than 1, and L1 is greater than L2;

[0121] The splicing unit is used to splice the N original data blocks to obtain original data.

[0122] In a twenty-first aspect, an embodiment of the present application further provides a decompression device, the decompression device comprising: a controller, a first memory, M decompression engines, and a second memory, wherein:

[0123] The controller is used to obtain compressed data, the compressed data including N compressed data blocks and the length of each first compressed data block in the N compressed data blocks, and split the compressed data into N compressed data blocks according to the length of each first compressed data block in the N compressed data blocks;

[0124] The first memory is used to store N compressed data blocks, wherein any one of the N compressed data blocks is an independently compressed data block with a length not greater than L2, N is a positive integer, M ≥ N, and the compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle;

[0125] Each of the N decompression engines is used to obtain a compressed data block among the N compressed data blocks, and decompress the obtained compressed data blocks respectively, so as to obtain N compressed data blocks corresponding to N original data blocks one by one, wherein the length of any original data block among the N original data blocks is not greater than L1, and L1 and L2 are integers greater than 1, and L1 is greater than L2;

[0126] The second memory is used to store the N compressed data blocks.

[0127] In a possible implementation, the decompression device further includes: a splicing circuit,

[0128] The splicing circuit is used to splice the N original data blocks to obtain original data.

[0129] In aspect 22, an embodiment of the present application further provides an integrated circuit, comprising: any one of the decompression devices described in aspect 21. Optionally, the integrated circuit may further comprise a computing device coupled to the decompression device, the computing device being used to perform data processing according to the original data decompressed by the decompression device.

[0130] In the twenty-third aspect, the embodiment of the present application also provides an electronic device with a decompression function, which may specifically be the fourth electronic device in the embodiment of the present application, comprising: any one of the decompression devices described in the first twenty aspects or the twenty-first aspect, a processor coupled to the decompression device, and a memory coupled to the processor.

[0131] In the twenty-fourth aspect, an embodiment of the present application further provides a computer storage medium, which is used to store computer software instructions, and when the computer software instructions are executed by a computer, the computer implements any one of the data decompression methods described in the nineteenth aspect.

[0132] In the twenty-fifth aspect, an embodiment of the present application further provides a computer program, which includes computer software instructions, and when the computer software instructions are executed by a computer, the computer executes any one of the data decompression methods described in the nineteenth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0134] Figure 1 It is a schematic diagram of the principle of decompression by multiple decompression engines in the prior art;

[0135] Figure 2 is a schematic diagram of the architecture of an electronic device provided by an embodiment of the present invention;

[0136] Figure 3 is a structural schematic diagram of a decompression device provided by an embodiment of the present invention;

[0137] Figure 4 is a flow chart of a data compression method provided by an embodiment of the present invention;

[0138] Figure 5 is a schematic diagram illustrating a data compression method provided by an embodiment of the present invention;

[0139] Fig. 6A is a schematic diagram illustrating original data and compressed data provided by an embodiment of the present invention;

[0140] Figure 6B This is a schematic diagram of the structure of the standard compressed data block provided by the embodiment of the present invention.

[0141] Figure 7 is a flowchart of a method for implementing step S41 provided in an embodiment of the present invention;

[0142] Figure 8 is a flowchart of another method for implementing step S41 provided in an embodiment of the present invention;

[0143] Fig. 9 is a structural schematic diagram of a compression device provided by an embodiment of the present invention;

[0144] Fig.10 is a schematic structural diagram of another compression device provided by an embodiment of the present invention;

[0145] Fig.11 It is a flowchart of a data decompression method provided by an embodiment of the present invention;

[0146] Fig.12 is a schematic structural diagram of another decompression device provided by an embodiment of the present invention;

[0147] Fig.13 is a structural diagram of a second electronic device provided by an embodiment of the present invention

[0148] Fig.14 is a flow chart of another data compression method provided by an embodiment of the present invention;

[0149] Fig.15 is a structural schematic diagram of another compression device provided by an embodiment of the present invention;

[0150] Fig.16 is a structural schematic diagram of another compression device provided by an embodiment of the present invention;

[0151] Fig.17 is a structural schematic diagram of another decompression device provided by an embodiment of the present invention;

[0152] Fig.18 is a flow chart of another data decompression method provided by an embodiment of the present invention;

[0153] Fig.19 is a structural schematic diagram of another decompression device provided by an embodiment of the present invention;

[0154] Fig. 20 It is a structural schematic diagram of a fourth electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0155] The following briefly introduces the relevant concepts involved in each embodiment of the present invention:

[0156] (1) System on Chip (SoC), SoC is called system-on-chip, also known as system on chip, which means it is a product, an integrated circuit with a dedicated purpose, which contains all the contents of a complete system and embedded software. The so-called complete system generally includes a central processing unit (CPU), memory, and peripheral circuits. At the same time, it is a technology that is used to realize the entire process from determining the system function to software / hardware division and completing the design. In the embodiment of the present application, the decompression device can also be integrated on the SOC.

[0157] (2) Random Access Memory (RAM), also known as "random access memory", is the internal memory that directly exchanges data with the CPU, also called main memory (memory). It can be read and written at any time and at a very high speed. It is usually used as a temporary data storage medium for the operating system or other running programs.

[0158] (3) Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM), abbreviated as DDR, is a general term for a series of memory interface protocols and memories that use this interface protocol, including DDR1, DDR2, DDR3, DDR4, etc. DDR memory can transmit data twice in one clock cycle, that is, once during the rising period and once during the falling period of the clock. Therefore, DDR memory can achieve a higher data transmission rate at the same bus frequency as SDRAM.

[0159] (4) AXI (Advanced eXtensible Interface) is a bus protocol. This protocol is the most important part of the AMBA (Advanced Microcontroller Bus Architecture) 3.0 protocol. It is an on-chip bus for high performance, high bandwidth, and low latency. The AXI bus sends addresses, read data, write data, and handshake signals in different channels. The order of different accesses can be disrupted, and BUSID is used to indicate the ownership of each access. The master device can issue multiple read and write operations without receiving return data. The order of the read-back data can be disrupted, and non-aligned data access is also supported.

[0160] (5) Compression algorithm refers to the algorithm for data compression, which is also often referred to as signal coding in the field of electronics and communications, and includes two steps: compression and restoration (or encoding and decoding). The compression algorithm involved in this application is characterized in that the electronic device needs to sequentially scan the characters to be compressed and encode them sequentially, so that the electronic device can monitor the length of the compressed data while compressing.

[0161] Optionally, the compression algorithm may be a general compression algorithm such as entropy encoding, run length coding (RLC), dictionary coding, and their combination methods or variants, which are not limited in the embodiments of the present application. Among them, entropy coding may include but is not limited to Huffman coding, arithmetic coding, etc.; dictionary coding includes but is not limited to the LZ (Lempel-Ziv) coding series, and the LZ coding series includes deflate, LZ77, LZ78, LZW, LZO and other codes, which are not limited in the embodiments of the present application.

[0162] (6) Huffman Coding, also known as Huffman coding, is a type of coding. Huffman coding is a type of variable length coding (VLC). Huffman coding constructs codewords with the shortest average length of different prefixes based on the probability of character occurrence. The codewords (i.e., the codes of each symbol) encoded by Huffman are different prefix codewords, that is, any codeword will not be the front part of another codeword. This allows the codewords to be transmitted together without adding additional isolation symbols in the middle. As long as there are no errors during transmission, the receiving end can still separate the codewords without confusion.

[0163] (7) Run Length Coding (RLC), also known as "run length coding" or "run length coding", is a statistical coding method that belongs to lossless compression coding and is an important coding method for raster data compression. Its basic principle is to replace consecutive symbols with the same value (consecutive symbols constitute a continuous "run") with a symbol value or string length, so that the symbol length is less than the length of the original data. Only when the code of each row or column of data changes, the code and the number of repetitions of the same code are recorded once, thereby achieving data compression.

[0164] (8) Data group: When the compression algorithm compresses a character string, it uses data groups as units. Usually a data group is one character or multiple consecutive characters, which is related to the encoding format of the original data. In this application, a character is the smallest meaningful unit in the encoding format of the original data. For example, a text in English format is 1 byte (8 bits), and an integer in INT32 format is 4 bytes. The length of the data group (i.e., the compression step) is related to the compressed data content, the compression algorithm, etc. In this application, one compression refers to compressing a data group.

[0165] For example, for Huffman coding, when compressing a character string, if the compression is performed in units of single characters, one data group is one character.

[0166] For another example, for run-length encoding, assuming that the data to be compressed is "AAAABBB", during the first compression, the data group is "AAAA", and the length of the data group corresponding to the first compression is 4 characters; during the second compression, the data group is "BBB", and the length of the data group corresponding to the second compression is 3 characters.

[0167] (9) Decompression engine: hardware that implements data decompression, such as an integrated circuit or a field programmable gate array (FPGA). The decompression engine corresponds to the compression algorithm and can only decompress data compressed by a specific compression algorithm.

[0168] (10) A standard compressed data block has a fixed length, namely, L2, which is a data block that can be processed (decompressed) by a single decompression engine in a decompression device within one processing cycle, and the size of the original data block obtained after decompression is no larger than L1. Wherein, L1 and L2 are integers greater than 1, and L1 is greater than L2.

[0169] (11) Processing cycle, which can be one clock cycle or multiple clock cycles. A single decompression engine can complete the processing of a compressed data block with a maximum length of L2 within one processing cycle.

[0170] (12) Independent compression: S original data blocks are compressed independently, and S compressed data blocks are obtained that correspond to the S original data blocks one by one. There is no correlation between the S compressed data blocks (that is, when compressing a certain original data block, it is not necessary to compress it based on other original data blocks), and they can be decompressed independently (that is, when decompressing, it is not necessary to decompress it based on other compressed data blocks).

[0171] For example, the original data is composed of an original data block A and an original data block B. When compressing the original data, when compressing the original data block A through a compression algorithm, compression is started from the first data group of the original data block A. After the compression of the original data block A is completed, a compressed data block A' is obtained; when compressing the original data block B, compression is started again from the first data group of the original data block B. After the compression of the original data block B is completed, a compressed data block B' is obtained. When decompressing the compressed data block A' and the compressed data block B', the compressed data block A' and the compressed data block B' can be decompressed separately, the compressed data block A' and the compressed data block B' can be decompressed in any order, or the compressed data block A' and the compressed data block B' can be decompressed at the same time.

[0172] See also Figure 2 , Figure 2 2 is a schematic diagram of the architecture of an electronic device provided by an embodiment of the present invention, and the electronic device may include: a processor 21, a memory 22, and a decompression device 23. The processor 21 may be connected to the memory 22 and the decompression device 23 via a bus 24.

[0173] The memory 22 may be a read-only memory (ROM), a random access memory (RAM) or other memory. In the embodiment of the present application, the memory 22 is used to store raw data and various software programs, such as the data compression program in the present application. In an embodiment of the present application, the memory 22 may be a DDR to increase the data transmission rate between the memory and the processor or between the memory and the decompression device.

[0174] The processor 21 is the control center of the computer device. In this application, the processor refers to a circuit or a collection of multiple circuits with processing capabilities. For example, it can be a processor based on an instruction set such as a central processing unit (CPU) (also called a "software processor"), or it can also be a processing circuit (also called a "hardware processor") based on hardware implementations such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA). For a processor based on an instruction set, the processor 21 can perform various functions of the electronic device by running or executing a software program stored in a memory in the computer device, and calling data stored in the memory. For a processor based on FPGA, the FPGA circuit can be configured by reading a configuration file to complete the corresponding function. For a circuit based on ASIC implementation, the ASIC can be designed to have corresponding circuit functions to perform these functions after power-on. In this application, the processor 21 can be packaged into a chip alone or together with other circuits, or the processor can be packaged into several chips (such as packaging multiple chips for different functional circuits), which is not limited in this application.

[0175] In the embodiment of the present application, the processor 21 can call the original data stored in the memory 22 and the data compression program to implement the data compression method described in the embodiment of the present application.

[0176] The bus 24 may be an AXI bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, and the like.

[0177] See also Figure 3 The structural diagram of the decompression device shown in FIG. 2 shows that the decompression device 23 may include a first memory 231, M decompression engines 232, a second memory 233, a splicing circuit 234, etc. The processor 21 may also control the startup of the decompression device 24 through the bus 24. The decompression device 23 may obtain the compressed data block from the memory 22 through the bus 24, and decompress the compressed data block in real time through the decompression engine. In order to make the decompression speed faster, the decompression device may be specifically based on a hardware processor such as an ASIC or FPGA. Of course, the present application is not limited to the decompression based on a software processor such as a CPU.

[0178] The first memory 231 is the entry storage of the decompression device, and is used to store the received compressed data blocks. Optionally, the first memory 231 may include M storage subunits corresponding to the M decompression engines one by one, and one storage subunit may store a compressed data block, such as a standard compressed data block. In an embodiment of the present application, the compressed data block may also be a compressed data block obtained by compressing the original data block whose length is not greater than L1. In other words, L2 can be determined as the data length that can be stored by any storage subunit in the first memory.

[0179] The M decompression engines are used to decompress the standard compressed data blocks stored in the first memory in parallel, and one decompression engine is used to decompress one standard compressed data block.

[0180] In a specific implementation of the embodiment of the present application, the M decompression engines include N decompression engines, and the N standard compressed data blocks can be decompressed in parallel by the N decompression engines to obtain N original data blocks. Each decompression engine 232 in the N decompression engines is used to obtain a standard compressed data block in the N standard compressed data blocks, and decompress the obtained standard compressed data block to obtain N original data blocks corresponding to the N standard compressed data blocks. Among them, the length of any original data block in the N original data blocks may not be greater than L1.

[0181] The second memory is an exit storage of the decompression device, and is used to store N original data blocks. Optionally, the second memory includes M storage subunits corresponding to the M decompression engines, respectively, and the storage subunits are used to store the original data blocks decompressed by the decompression engines corresponding to the storage subunits, that is, L1 can be determined as the data length that can be stored in any storage subunit in the second memory.

[0182] It can be understood that the inlet storage size of the decompression device is the sum of the inlet storage sizes of the M decompression engines included in the decompression device, and the outlet storage size of the decompression device is the sum of the outlet storage sizes of the M decompression engines included in the decompression device.

[0183] The splicing circuit is used to splice the original data blocks (for example, N original data blocks) stored in the second storage to obtain original data. The splicing circuit is not a necessary component of the decompression device, and the decompression device may not include the splicing circuit, and the data splicing is realized by software, which is not limited in the embodiments of the present application.

[0184] In one embodiment of the present application, N decompression engines include a decompression engine i, N compressed data blocks include a compressed data block i, and N original data blocks include an original data block i obtained by decompressing the compressed data block i, i is a positive integer, and i is not greater than N; the decompression engine i is specifically used to: decompress the compressed data block i through the decompression engine i. Furthermore, when the length of the original data block i decompressed by the decompression engine i is equal to L1, the decompression operation on the compressed data block i is stopped, and the original data block i is output; or, when the decompression engine i recognizes an invalid flag, the decompression operation on the compressed data block i is stopped, and the decompressed original data block i is output.

[0185] It can also be said that the first decompression engine is any one of the N decompression engines, and the first standard compressed data block is any one of the N standard compressed data blocks; taking the first decompression engine for the first standard compressed data block as an example to illustrate the decompression process of any decompression engine, specifically: the decompression device can decompress the first standard compressed data block through the first decompression engine; when the length of the original data block decompressed by the first decompression engine is equal to L1, stop the first decompression engine's decompression operation on the first standard compressed data block, and output the original data block; or, when the first decompression engine recognizes an invalid flag, stop the first decompression engine's decompression operation on the first standard compressed data block, and output the original data block currently decompressed.

[0186] In one embodiment of the present application, the decompression device also includes a control circuit for stopping the decompression operation of the first decompression engine on the obtained compressed data block when the length of the original data block decompressed by the first decompression engine among the N decompression engines is equal to L1.

[0187] Optionally, the decompression device may further include a data distribution circuit, which is used to distribute the N compressed data blocks stored in the first memory to the N decompression engines in sequence.

[0188] It should be noted that a storage subunit in the first memory and / or a storage subunit in the second memory can be set in the corresponding decompression engine, which is not limited in the embodiment of the present application.

[0189] Optionally, the electronic device may further include a computing device, which is coupled to the decompression device and the processor and is used to perform data processing according to the original data decompressed by the decompression device.

[0190] In a specific implementation, the computing device may be a specific integrated circuit, such as hardware running a machine learning algorithm, hardware running a video or image processing algorithm, etc. The processor may also control the start-up of the decompression device and the computing device through the bus; the decompression device may obtain a compressed data block from the memory through the bus, and decompress the multiple compressed data blocks in real time through multiple decompression engines to obtain the original data corresponding to the multiple compressed data blocks, and input the original data into the computing device; the computing device uses the original data to complete the calculation, for example, the computing device uses the original data to train or reason the machine learning model.

[0191] The computing device may be one or more microprocessors, or one or more field programmable gate arrays (FPGAs), etc., for realizing specific computing functions of the computing module.

[0192] In one embodiment of the present invention, the computing device and the decompression device can form an integrated circuit, which can be integrated with a processor, a memory, etc. on the same chip. The integrated circuit can also be used as an independent chip in an electronic device to achieve acceleration of calculations through hardware.

[0193] Although, Figure 2 The electronic device can simultaneously run the data compression method in the present application and the data decompression method corresponding to the data compression method. However, in one embodiment of the present application, the electronic device that executes the data compression method (also referred to as the first electronic device in the present application) may not include the computing device and the decompression device. Similarly, the processor in the electronic device that executes the data decompression method (such as the second electronic device) may not run the data compression method.

[0194] It should be noted that the specific implementation of the data compression function of the electronic device and the first electronic device can refer to the relevant description in the embodiment of the following data compression method; the specific implementation of the data decompression function implemented by the second electronic device can refer to the relevant description in the embodiment of the following data decompression method, and the embodiments of this application will not be repeated.

[0195] The data compression method according to the embodiment of the present invention is described in detail below. Figure 4 Please refer to the flowchart of the data compression method shown in Figure 5 The data compression method is a schematic diagram of a data compression method shown in FIG. 1 , wherein the data compression method is performed by a first electronic device or a compression device, and the first electronic device may be as shown in FIG. Figure 2The electronic device shown may also be other first electronic devices described in the embodiments of the present application. The following description takes a compression device as an example. The data compression method is used to compress metadata into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device includes multiple decompression engines, each of which can complete the decompression of a standard compressed data block within one processing cycle, L2 is an integer greater than 1, and in the process of compressing the original data, the compression device can perform one or more original data block compressions, and the original data block compression may include: the compression device processes the original data block based on a compression algorithm to obtain a standard compressed data block corresponding to the original data block, wherein the original data block is the data included in the original data.

[0196] The embodiment of the present application takes the compression process of an original data block as an example to illustrate, and the specific implementation of the compression may be: independently compressing the original data block through a compression algorithm to obtain a compressed data block corresponding to the original data block, and the length d of the compressed data block is not greater than L2; further, adding additional data of length L2-d after the compressed data block, thereby generating a standard compressed data block of length L2, d is an integer greater than 0. It should be understood that d is only in accordance with the length of the compressed data block, and the actual value of d may be different for different compressed data blocks.

[0197] In another implementation of the embodiment of the present application, the method for compressing original data may include all or part of the following steps.

[0198] Step S41: Each of the S continuous original data blocks in the original data is independently compressed by a compression algorithm to obtain the S compressed data blocks, and the original data blocks correspond to the compressed data blocks one by one. Wherein, each of the S original data blocks is a plurality of continuous data in the original data, and the length of the plurality of continuous data is not greater than L1, and the length d of each compressed data block in the S compressed data blocks is not greater than L2, and S is a positive integer.

[0199] Original data can be data in the form of a file or in a non-file form (for example, some data stored in memory). Original data includes multiple original data blocks, which are sequentially connected to form original data, where "data block" refers to a collection of multiple data. In the field of computer applications, one or more bits can be used to represent a data, that is, a data block can include multiple bits. The length of each original data block in the multiple original data blocks is no greater than L1. The length of the compressed data block corresponding to each original data block is no greater than L2. It should be understood that the length of the original data block may be different in different original data blocks. Please refer to Fig. 6ASchematic illustration of original data and compressed data shown.

[0200] Optionally, before step S41, the method may further include:

[0201] Step S40: Determine that L1 is the exit bit width of a single decompression engine and that L2 is the entry bit width of a single decompression engine.

[0202] The "determination" in this step can be determined by configuring L1 and L2 in advance (for example, configuring the specific values ​​of L1 and L2 in the program code or circuit implementation), or it can be determined by reading the relevant configuration values ​​when the program or hardware circuit is running (for example, determining the values ​​of L1 and L2 by reading a configuration file that records the values ​​of L1 and L2 and is stored locally or remotely).

[0203] It can be understood that in the present application, the values ​​of L1 and L2 can be determined according to the compression support of the compression system's hardware, that is, the determination of these thresholds needs to be supported by hardware, and it is best to maximize the use of hardware resources. For example, for L1, since it involves the original data block, it is necessary to ensure that the size of the decompression engine's export storage (i.e., the memory used to store the decompressed data) during the final parallel decompression can accommodate the decompressed data, that is, not less than the export bandwidth of the decompression engine (i.e., the maximum number of data bits that can be stored after decompressing the standard compressed data block in one processing cycle). Therefore, it is necessary to make L1 less than or equal to the export bit width of a single decompression engine. In order to maximize the utilization rate, L1 can be made equal to the export bit width of a single decompression engine. Similarly, it is necessary to ensure that the entry storage of a single decompression engine can accommodate the compressed data, that is, not less than the entry bandwidth of the decompression engine (i.e., the width of the data that a decompression engine can read in one processing cycle, for example, 8 bytes are read each time, which is 64 bits wide). Therefore, from the perspective of maximizing utilization, L2 can be set equal to the entry bit width of a single decompression engine.

[0204] It should be understood that the entry bit width of a single decompression engine may be the storage size that can be allocated to the single decompression engine by the first memory or the size of the storage subunits in the first memory corresponding to the single decompression engines.

[0205] Among them, the egress bit width, L1, ingress bit width and L2 can be expressed in bits.

[0206] Those skilled in the art can understand that the exit bit width and the entry bit width can be determined by the performance indicators to be met by the compression and decompression system in combination with specific software and hardware. For example, it can be determined according to one or more factors including the compression algorithm adopted by the decompression engine, the circuit design of the decompression engine, the length of data that can be input into the decompression device in each clock cycle, and the number, area, and power consumption of the decompression engines in the decompression device. For example, if the main frequency of the hardware is f, and the bandwidth allocated to the entry of the decompression device is b, then the data entering the compression device in each clock cycle is b / f. If it is known that the decompression capacity of a single decompression engine is c bytes / clock cycle (byte / cycle), then at least b / (f*c) decompression engines are required. Similarly, if it is known that the decompression capacity of a single decompression engine is c bytes / clock cycle, and the decompression device includes M decompression engines, then the data entering the compression device in each clock cycle is c*M bytes. Therefore, the storage size of the first memory in the decompression device must be able to store the data entering the compression device in each clock cycle, and the entry storage of a single decompression engine must be able to store the amount of data entering the compression engine in each clock cycle, that is, c bytes.

[0207] For example, the main frequency of the hardware is 1GHz, and the bandwidth allocated to the entrance of the decompression device is 32GBps. Then, the data entering the compression device in each clock cycle is 32 bytes. If it is known that the decompression capacity of a single decompression engine is 8 bytes / clock cycle (byte / cycle), the decompression device requires at least 4 decompression engines, and the storage resource size corresponding to each decompression engine is not less than 8 bytes.

[0208] The outlet bit width of a single decompression engine is greater than the inlet bit width of a single engine. The outlet bit width of a single engine may be equal to the storage size allocated to the single decompression engine by the second memory in the decompression device, or equal to the size of the storage subunits in the second storage unit corresponding to the single decompression engine.

[0209] Optionally, the inlet bit width of a single decompression engine can be set to the maximum length of a compressed data block that can be decompressed by the single decompression engine in one clock cycle; similarly, the outlet bit width of a single decompression engine can be set to the maximum length of original data that can be decompressed by the single decompression engine in one clock cycle.

[0210] Optionally, the compression device can obtain specification parameters of the decompression device, including the outlet bit width, inlet bit width, number of decompression engines, etc. Then, L1 is determined to be less than or equal to the outlet bit width of a single decompression engine, and L2 is less than or equal to the inlet bit width of a single decompression engine.

[0211] It should be noted that L1 may also be smaller than the egress bit width of a single decompression engine, and L2 may also be smaller than the ingress bit width of a single decompression engine.

[0212] Step S42: Additional data of length L2-d is added after each of the S compressed data blocks to generate S standard compressed data blocks of length L2. A standard data block of length L2 can be decompressed by a single decompression engine in the decompression device within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2.

[0213] Among them, the first original data block is any one of the S original data blocks, the first compressed data block is the data obtained by compressing the first original data block among the S compressed data blocks, and when the length of the first compressed data block is equal to L2, d is L2, that is, the compression device does not need to add additional data after the first compressed data block, and the first compressed data block is the required standard compressed data block with a length of L2.

[0214] It can be understood that the length of the additional data in the canonical compressed data blocks obtained by compressing different original data blocks may be different.

[0215] There are three ways to add additional data:

[0216] The first case: the length y of the first original data block is equal to L1. The additional data in the canonical compressed data block obtained by compressing the first original data block can be filled arbitrarily. When decompressing, the decompressor only retains the data with a length of L1 in the front, and the subsequent data can be discarded. For example, the additional data (1) is a legal codeword or (2) includes an invalid flag and invalid data, where the invalid flag is used to indicate that the additional data needs to be discarded, and the invalid data is located after the invalid flag. The number of bits of wireless data after the invalid flag can be 0 or not less than 1. The number of bits of invalid data depends on the length of the data compressed from the first original data block, and the sum of the number of bits of invalid data and the number of bits of invalid flag is L2, so as to ensure that a canonical compressed data block with a length of L2 can be obtained. Please refer to Figure 6B The schematic diagram of the structure of the canonical compressed data block shown in the figure, wherein the legal codeword is a codeword that the decompression engine can recognize and decompress. When the additional data is the legal codeword of the first type, although the decompression engine can recognize and decompress the additional data, since the length of the first original data block is L1, the decompression engine can only take data of L1 length at most, and the data decompressed from the additional part can be discarded. For the second type, the invalid flag can be composed of multiple bits, and the specific number of bits depends on the decompression algorithm. For different decompression algorithms, the number of bits of the invalid flag can be different. Regardless of the algorithm, it only needs to ensure that the decompression engine can recognize the invalid flag and know that all data starting from the invalid flag are invalid data and can be discarded. Since the decompression engine does not decompress the data after the invalid flag, the value of the invalid data filled after the invalid flag is not limited, and any data can be filled.

[0217] The second case: the length y of the first original data block is less than L1, and the additional data includes the "invalid flag" and "invalid data" mentioned in the first case. In this case, since y is less than L1, if the method of filling with legal code words is adopted, the decompression end will not be able to output the decompression result (because unlike the first case, the data of L1 can be uniformly taken. In this case, the length of the valid data decompressed is unknown). Therefore, it is necessary to adopt the "invalid flag + invalid data" method to stop decompression. At this time, the decompressed data before the invalid flag is the original data block. It should be noted that in another embodiment, if the length of the original data block can be known in some way, it can also be achieved by adding legal code words. It can be understood that by adding the method of invalid flag, the second case can share a set of software codes with the first case, which is more convenient to handle and reduces design costs.

[0218] The above two situations are applicable to the case where the processing cycle is one or more clock cycles.

[0219] The third case: For the case where the processing cycle is multiple clock cycles, the additional data in the standard data block obtained by compressing the first original data block can also be (3) including an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded, the invalid data is located after the invalid flag, and the number of bits of the wireless data after the invalid flag can be 0 or not less than 1.

[0220] Code length (bit) Encoding (binary) 2 00 3 001 3 011 3 100 3 101 3 110 5 11100 5 11101 5 11110 6 111110 6 111111

[0221] Table 1

[0222] The compression device can determine that the legal codeword is the code corresponding to the code length of L2-d in Table 1 according to the value of L2-d, or determine that the legal codeword is a combination of multiple codes with a total length of L2-d in Table 1.

[0223] It should be noted that the above Table 1 is only used to illustrate the principle of selecting legal codewords. The embodiments of the present application may adopt other legal codes or combinations of legal codes according to the specific compression algorithm.

[0224] In the above embodiment, when compressing the original data, the length of the multiple standard compressed data blocks generated is a fixed value L2, and the length of the original data block used to generate the standard compressed data block is no more than L1, which standardizes the length of the compressed data block generated by the compression of the original data. In addition, each standard compressed data block is independently compressed and can be independently decompressed, so that each decompression engine can decompress the multiple standard compressed data blocks synchronously, thereby improving the data decompression efficiency.

[0225] Moreover, compared with compressing into variable-length compressed data blocks, the present application sets a standard compressed data block of fixed length L2. The compression end does not need to record the length of each compressed data block, and the decompression end does not need to record the length of the acquired data, making the design of the decompression end simpler.

[0226] Furthermore, in the prior art, the number of original data blocks into which the original data is divided is equal to the number of decompression engines, that is, the number of decompression engines limits the granularity at which the original data can be divided, so that the length of the multiple compressed data blocks compressed by the original data is relatively large. When accessing data at a random address, it is necessary to decompress the compressed data block where the data at the random address is located. During the decompression process, due to the relevance of the data in the compressed data block, the compressed data block can only be decompressed in one decompression engine, and it is impossible to achieve double-speed decompression of the compressed data. In the embodiment of the present application, the granularity at which the original data can be divided is independent of the number of decompression engines, and the original data can be compressed into a large number of smaller standard compressed data blocks. When accessing data at a random address, the multiple continuous standard compressed data blocks where the data at the random address is located are input into multiple decompression engines for decompression, which can achieve double-speed decompression of the compressed data at the random address.

[0227] The following takes the example of a compression device independently compressing the first original data block through a compression algorithm to obtain a first compressed data block to introduce the implementation method in step S41. The first original data block is an original data block in the original data. It can be understood that step S41 can also include other implementation methods, which are not limited in the embodiments of the present application.

[0228] The first implementation method:

[0229] Typically, data compression is performed with data groups as steps. A data group is one or more continuous data that can be compressed, and is a compressible unit under the compression algorithm. The length of the data group is determined by the compression algorithm and the data content of the data group, and is less than or equal to L1; moreover, the length of a data block generated by compression of a single data group is less than or equal to L2. Its specific implementation is the prior art and is not elaborated in this application.

[0230] The first implementation method of step S41 includes the following process:

[0231] S1: The compression device obtains a data group to be compressed from the original data.

[0232] The data group to be compressed may be the first data group in the first original data block or the next data group to be compressed after one or more compressions are completed on the data group in the first original data block. The data group includes one or more bits of data.

[0233] S2: Determine whether the data group to be compressed needs to be compressed. If compression is required, execute step S3; otherwise, execute step S4.

[0234] Wherein, judging whether it is necessary to compress the data group to be compressed includes: judging whether the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1, and judging whether the length y of the compressed data block generated after the data group to be compressed is compressed is not greater than L2. Those skilled in the art can understand that the length of the compressed data block generated after the data group is compressed refers to the length of the original compressed data block plus the length increased after the current compression, that is, the length of the compressed data block will be updated after each data group is compressed.

[0235] 1) If x+Δx≤L1 and y≤L2, compression is required.

[0236] If the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than L2, it means that the compressed data block obtained after another compression still meets the condition that the length is not greater than L2, and the data group to be compressed can be compressed and it is judged that compression is needed.

[0237] 2) If x+Δx≤L1 and y>L2, it is determined that compression is not required.

[0238] If the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than L2, it means that although the length still meets the condition of being less than L1 after another compression, the length of the compressed data block obtained after compression does not meet the condition of being no greater than L2. Therefore, it is judged that compression is not needed. At this time, the compression of the data group to be compressed should be stopped and step S4 is executed.

[0239] 3) If x+Δx>L1, it is determined that compression is not required.

[0240] If the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is greater than L1, it means that after another compression, the length will exceed L1. Therefore, it is determined that compression is not needed. At this time, the compression of the data group to be compressed should be stopped and step S5 should be executed.

[0241] S3, compress the data group to be compressed, and execute S1 after the compression is completed.

[0242] S4, taking the currently compressed data as the first compressed data block. At this time, the currently compressed data group is the first original data group.

[0243] It should be noted that when the data group to be compressed does not need to be compressed, the data group to be compressed needs to be put back into the original data to serve as the first data group to be compressed for the next original data block.

[0244] It should be understood that the present application compresses the original data into S standard compressed data blocks, and the data group in the present application is the first data group refers to the first data group in the first original data block.

[0245] It should be understood that the compression device may first determine the size of x+Δx and L1, and then determine the size of y and L2.

[0246] For example, see Figure 7 The flowchart of a method for independently compressing the first original data block by a compression algorithm to obtain a first compressed data block is shown. The method includes the following process:

[0247] S411: The compression device obtains a first data group and compresses the first data group.

[0248] Since the length Δx of the data group is not greater than L1, and the data block formed by compressing a single data group is not greater than L2, the compression device can perform compression by default when obtaining the first data group. The compression device can also determine whether to compress the first data group when obtaining the first data group. In this case, the first data group must meet the conditions for being able to be compressed. Further, let k=2 to execute step S412, k is a positive integer, k≤M.

[0249] S412: The compression device obtains the kth data group, the length of which is Δx k .

[0250] Optionally, the compression device may preferentially determine the length x of the compressed one or more numbers and the length Δx of the kth data group. k Is the sum not greater than L1?

[0251] S413: The compression device determines the total length x of the compressed one or more data groups and the length Δx of the kth data group. k It can be understood that the total length x of one or more data groups refers to the length obtained by accumulating the length of each data group in the one or more data groups.

[0252] The one or more compressed data groups are the 1st to k-1th data groups.

[0253] When x+Δx k > L1, it means that after compressing the kth data group, the length of the 1st to kth data groups will exceed L1. In this case, the kth data group does not need to be compressed, and the compression of the Kth data group should be stopped, and step S414 is executed; when x+Δx k ≤L1, it means that if the lengths of the 1st to kth data groups are less than or equal to L1 after the kth data group is compressed, step S415 is executed.

[0254] S414: The compressed data block obtained by compressing the compressed one or more data groups (i.e., the 1st to k-1st data groups) is used as the first compressed data block. At this time, the compressed one or more data groups (i.e., the 1st to k-1st data groups) are the first original data block in step S41.

[0255] S415: Determine whether the length y of the compressed data block generated after compressing the kth data group is greater than L2.

[0256] When y>L2, it means that the length y of the compressed data block generated after compressing the kth data group is greater than L2. At this time, it is determined that the kth data group does not need to be compressed, and S414 can be executed. When y≤L2, it means that the length y of the compressed data block generated after compressing the kth data group is not greater than L2, and the kth data group needs to be compressed. At this time, let k=k+1 and repeat step S412.

[0257] It should be noted that, for the first or more data groups to be compressed, if the corresponding parameters can be agreed in advance to meet certain conditions to ensure that the conditions for one or more compressions can be met (for example, L1 is much larger than the data group length, and L2 is also relatively large. Even after one or more compressions, the compressed data will not exceed L1, and the length of the compressed data block will not exceed L2), then, it is possible to directly perform one or more compressions on the first or more data without performing a judgment.

[0258] The second implementation method:

[0259] Please also read Figure 8 The embodiment of the present invention shown in the figure provides another flow chart of independently compressing the first original data through a compression algorithm to obtain a first compressed data block. The implementation method may include the following steps:

[0260] Step S416: compressing the data block with a length of L1 in the original data with the data group as the step length to obtain a temporary compressed data block.

[0261] Step S417: Determine whether the length y′ of the temporary compressed data block is greater than L2.

[0262] When y′>L2, it means that the length of the temporary compressed data block obtained by compressing the data block with a length of L1 is greater than L2, that is, greater than the length of the standard compressed data block. At this time, execute step S418; otherwise, when y′≤L2, execute step S419.

[0263] Step S418: The data block obtained by removing the i data groups at the end of the data block of length L1 is used as the first original data block, and the data obtained by compressing the first original data block is used as the first compressed data block, wherein the length of the first compressed data block is not greater than L2.

[0264] Optionally, i may be a minimum positive integer such that the length of the first compressed data block is not greater than . The method for determining i may be: the compression device removes data groups one by one from the end of the data block of length L1 until the length of the compressed data block obtained by compressing the data block of length L1 after removing the i-th data group is not greater than L2.

[0265] It should be noted that the removed i data groups need to be put back into the original data to serve as the first data group for compressing the next original data block.

[0266] Step S419: The data block with the length L1 is used as the first original data block, and the temporary compressed data block is used as the first compressed data block. The compression device may execute step S42.

[0267] exist Figure 8 In the illustrated embodiment, the compression device compresses a data block of length L1 in the data to be compressed with a data group as a step length to obtain a temporary compressed data block; compares the length y′ of the temporary compressed data block with the size of L2, and determines the first original data block and the first compressed data block generated by compressing the first original data block according to the comparison result, thereby obtaining a first compressed data block that meets the requirements, and further obtaining a standard compressed data block of a fixed length L2.

[0268] See also Fig. 9 , Fig. 9 is a schematic diagram of the structure of a compression device provided by an embodiment of the present invention, the compression device 90 comprises: the compression device 90 is used to compress original data into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device comprises a plurality of decompression engines, each of which can decompress a standard compressed data block in one processing cycle, L2 is an integer greater than 1, and the compression device 90 comprises:

[0269] The processing module is used to process the original data block based on a compression algorithm to obtain the standard compressed data block corresponding to the original data block, wherein the original data block is data included in the original data.

[0270] In one implementation of the embodiment of the present application, the processing module may include:

[0271] A compression unit 91 is used to independently compress the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block, wherein the length d of the compressed data block is not greater than L2;

[0272] The standardization unit 92 is used to add additional data of a length of L2-d after the compressed data block, so as to generate the standard compressed data block of the length of L2, where d is an integer greater than 0.

[0273] In other words, the compression unit 91 is used to independently compress each of the S continuous original data blocks in the original data by using a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, and the length d of each compressed data block in the S compressed data blocks is not greater than L2, and S is a positive integer;

[0274] The standardization unit 92 adds additional data of a length of L2-d after each of the S compressed data blocks to generate S standard compressed data blocks of a length of L2, wherein the standard data block of length L2 is a data block that can be processed by a single decompression engine in the decompression device within one processing cycle, the L1 and the L2 are integers greater than 1, the d is an integer greater than 0, and the L1 is greater than the L2.

[0275] The specific implementation of each functional unit included in the compression device 90 can refer to the relevant description in the above data compression method, which will not be repeated here. Each unit in the compression device 90 can implement the above data compression method based on hardware or software. For example, when it is based on hardware, each unit can be completed based on circuits such as FPGA and ASIC, or when it is implemented based on software, the corresponding function can be executed by allowing the CPU to read the code that can realize the function of the above unit stored in the memory.

[0276] See also Fig.10 , Fig.10 1 is a schematic diagram of the structure of another compression device provided by an embodiment of the present invention. The compression device 100 can implement the above data compression method. The compression device 100 includes: a processor 101 and a memory 102. The processor 101 is connected to the memory 102 via a bus 103.

[0277] The memory 102 may be a read-only memory (ROM), a random access memory (RAM) or other memory. In the embodiment of the present application, the memory 102 is used to store data such as raw data and various software programs, such as the data compression program in the present application. Optionally, the memory 102 may be DDR to increase the data transmission rate between the memory and the processor or between the memory and the decompression device.

[0278] The processor 101 is the control center of the first computer device, and can be a processor or a general term for multiple processing elements. In a specific implementation, as an optional embodiment, the processor 101 may include one or more CPUs. The processor 101 may also include a specific integrated circuit, such as one or more microprocessors, one or more field programmable gate arrays (FPGAs), etc., that implement the data compression method in the embodiment of the present invention.

[0279] Optionally, the compression device further includes a communication interface 104, and the communication interface 104 is used to implement data exchange with other devices, such as a decompression device.

[0280] The processor 101 is used to call the data and program code stored in the memory 102 to perform: one or more compressions of the original data block, wherein the original data block compression includes:

[0281] The original data block is processed based on a compression algorithm to obtain the standard compressed data block corresponding to the original data block, wherein the original data block is data included in the original data.

[0282] In one implementation of an embodiment of the present application, taking the compression of an original data block as an example, the processor 101 may execute: independently compressing the original data block through a compression algorithm to obtain a compressed data block corresponding to the original data block, wherein the length d of the compressed data block is not greater than L2; adding additional data of a length of L2-d after the compressed data block, thereby generating the standard compressed data block of a length of L2, wherein d is an integer greater than 0.

[0283] In other words, the processor 101 can execute: independently compress each of the S consecutive original data blocks in the original data through a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, and the length d of each of the S compressed data blocks is not greater than L2; add additional data of length L2-d after each of the S compressed data blocks to generate S standard compressed data blocks of length L2, wherein the standard data block of length L2 is a data block that can be processed by a single decompression engine in the decompression device within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2.

[0284] Optionally, the original data block may include a plurality of continuous data in the original data, and the length of the original data block is not greater than L1, where L1 is an exit bit width of the decompression engine.

[0285] In an implementation of the embodiment of the present application, the first original data block is any original data block among the S original data blocks, wherein:

[0286] When the length of the first original data block is equal to L1, the first additional data in the standard data block compressed from the first original data block is a legal codeword or the first additional data includes an invalid flag for indicating to the decompression engine that the first additional data needs to be discarded and greater than or equal to 0 invalid data following the invalid flag.

[0287] When the length of the first original data block is less than L1, the first additional data includes an invalid flag for indicating to the decompression engine that the first additional data in the canonical data block compressed by the first original data block needs to be discarded and greater than or equal to 0 invalid data following the invalid flag.

[0288] In one implementation of the present application, the processor 101 performs the independent compression of the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block, specifically including executing part or all of the following processes:

[0289] S1. Obtain a data group to be compressed from the original data, wherein the data group includes one or more bits of data;

[0290] S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4;

[0291] S3, compressing the data block group to be compressed, and executing S1 after the compression is completed;

[0292] S4, using the currently compressed data as the compressed data block;

[0293] Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than the L2, it is determined that compression is required; if the sum of x and Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than the L2, it is determined that compression is not required; if the sum of x and Δx is greater than the L1, it is determined that compression is not required.

[0294] In another implementation of the present application, the processor 101 performs the independent compression of the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block, specifically including executing part or all of the following processes:

[0295] Compressing a data block with a length of L1 in the original data with a data group as a step length to obtain a temporary compressed data block;

[0296] When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having the length L1 is used as the original data block, and data obtained by compressing the original data block is used as the compressed data block, wherein the length of the compressed data block is not greater than L2, and i may be a minimum positive integer that makes the compressed data block not greater than L2;

[0297] When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the original data block, and the temporary compressed data block is used as the compressed data block.

[0298] In one implementation of the present application, L1 is the egress bit width of a single decompression engine; and / or, L2 is the ingress bit width of a single decompression engine.

[0299] It should be understood that the functional implementation of each device in the compression device in the embodiment of the present application can refer to the relevant description in the embodiment of the above-mentioned data compression method, and the embodiment of the present application will not be repeated.

[0300] The present application also provides a first electronic device including a data compression function. The first electronic device may include the above-mentioned Fig. 9 or Fig.10 Any compression device described in.

[0301] Combine the following Figure 2 The architecture of the electronic device shown, Figure 3 The decompression device shown and Fig.11 The flowchart of the data decompression method shown in the figure describes in detail the data decompression method in the embodiment of the present invention.

[0302] The data decompression method is executed by a second electronic device having a data decompression function or by the above Figure 3 The decompression device shown is executed. The second electronic device may be Figure 2 The electronic device shown in the figure or other electronic devices including a decompression device, the decompression device may also be as follows Figure 3 The decompression device shown in the figure may also be other decompression devices, which are not limited in the present embodiment. Figure 2 The electronic device shown, the decompression device is Figure 3 The decompression device shown is used as an example to illustrate the data decompression method.

[0303] The memory in the electronic device where the decompression device is located can store the first data and the compressed data blocks after the first data is compressed. The computing device can request the electronic device through the bus to obtain the second data with the starting address being the random address add1 and the length being Q, wherein the first data includes the second data; the electronic device can determine the P compressed data blocks including the second data according to the request address, and then send the P compressed data blocks to the decompression device; the decompression device inputs the original data obtained after the P compressed data blocks are decompressed and spliced ​​into the computing device, wherein the original data includes the second data; the computing device can obtain the second data from the original data based on the request address.

[0304] It should be noted that the electronic device can also determine the P compressed data blocks including the second data according to the address of the second data it needs, and then read the P compressed data blocks stored in the memory to the decompression device through the bus. This embodiment of the present application is not limited to this.

[0305] In the embodiment of the present application, the decompression device is set to include M decompression engines, each of which is a decompression engine corresponding to the compression algorithm in the above-mentioned data decompression method embodiment, and is used to decompress the compressed data generated by the compression algorithm. A single decompression engine can decompress one standard compressed data block in one clock cycle. That is, the decompression device can decompress a maximum of M standard compressed data blocks in one clock cycle.

[0306] The embodiment of the present application assumes that P=N, N≤M, that is, the second data required by the computing device can be decompressed in one clock cycle. It is understandable that P can also be greater than M. In this case, the second data needs to be decompressed through multiple clock cycles. The specific decompression method is similar to decompressing N compressed data blocks, and the embodiment of the present application will not be repeated.

[0307] See also Fig.11 The flowchart of the data decompression method shown is that the data decompression method is executed by a second electronic device or a decompression device. The embodiment of the present application is described using a decompression device as an example. The method includes all or part of the following steps.

[0308] Step S111: The decompression device obtains N standard compressed data blocks, wherein any one of the N standard compressed data blocks is a data block with a length of L2 generated after independently compressing the original data block, and one standard compressed data block can be decompressed by a single decompression engine within one processing cycle, N is a positive integer, and M≥N.

[0309] The decompression device obtains N standard compressed data blocks through the bus, and the length of the original data blocks after decompression may not be greater than L1. For the compression process of the standard compressed data blocks, please refer to the above Figure 4 , Figure 7 and Figure 8 The relevant description of the data compression method is not repeated in the embodiment of the present application. Wherein, L1 and L2 are integers greater than 1, and L1 is greater than L2.

[0310] Optionally, the decompression device may include a distribution circuit for distributing the N canonical compressed data blocks to N decompression engines.

[0311] At this time, the specific implementation method of step S111 may be: the decompression device receives N standard compressed data blocks sent by the processor through the bus; and distributes the N standard compressed data blocks to N decompression engines in sequence through the distribution circuit.

[0312] Step S112: N decompression engines are used to decompress N standard compressed data blocks respectively to obtain N original data blocks. The standard compressed data blocks correspond to the original data blocks one by one, wherein the length of any one of the N original data blocks may not be greater than L1.

[0313] In one implementation of the embodiment of the present application, the first decompression engine is any one of N decompression engines, and the first standard compressed data block is any one of the N standard compressed data blocks. Taking the first decompression engine for the first standard compressed data block as an example, the decompression process of any decompression engine is specifically: the decompression device can decompress the first standard compressed data block through the first decompression engine; when the length of the original data block decompressed by the first decompression engine is equal to L1, the first decompression engine stops the decompression operation of the first compressed data block and outputs the original data block; or, when the first decompression engine recognizes an invalid flag, the first decompression engine stops the decompression operation of the first compressed data block and outputs the original data block currently decompressed.

[0314] In one implementation of an embodiment of the present application, N decompression engines include a decompression engine i, N standard compressed data blocks include a standard compressed data block i, and N original data blocks include an original data block i obtained by decompressing the standard compressed data block i, where i is a positive integer and is not greater than N. The first decompression engine is used to decompress the received standard compressed data block i to obtain and output the original data block i.

[0315] One implementation of decompression by decompression engine i may be:

[0316] Decompression engine i decompresses standard compressed data block i;

[0317] Decompression engine i may include a counter to record the length of the original data block i decompressed by decompression engine i and compare the length of the original data block i with the size of L1 through a comparator. When the length of the original data block i decompressed by the first decompression engine is equal to L1, the first decompression engine stops the decompression operation on the standard compressed data block i and outputs the original data block i.

[0318] When the first decompression engine recognizes the invalid flag, the first decompression engine stops the decompression operation on the canonical compressed data block i and outputs the original data block i obtained by decompression.

[0319] Another implementation of decompression by decompression engine i may be:

[0320] The decompression engine i completely decompresses the standard compressed data block i to obtain the original data block. When the length of the original data block is greater than L1, the data after the data with a length of L1 in the original data block is discarded to obtain the original data block i with a length equal to L1.

[0321] When the decompression engine i recognizes the invalid flag, the decompression engine i stops the decompression operation on the standard compressed data block i and outputs the original data block i obtained by decompression.

[0322] Optionally, the decompression device may include a control circuit. When the length of the original data block i obtained by decompressing the standard compressed data block i by the decompression engine i is equal to the said L1, the decompression device may stop the decompression operation of the decompression engine i on the standard compressed data block i through the control circuit.

[0323] Step S113: splicing the N original data blocks to obtain original data.

[0324] After the N decompression engines decompress the N original data blocks, the decompression device may splice the N original data blocks to obtain the original data.

[0325] Optionally, the decompression device may also realize splicing N original data blocks into original data through a built-in splicing circuit.

[0326] After step S113, the decompression device can input the original data into the computing device.

[0327] exist Fig.11 In the embodiment shown, the decompression device obtains N canonical compressed data blocks, and decompresses the N canonical compressed data blocks through N decompression engines respectively, obtains the N canonical compressed data blocks corresponding to N original data blocks one by one, and splices the obtained N original data blocks to obtain original data. Among them, any canonical compressed data block among the N canonical compressed data blocks is a data block with a length of L2 generated after independently compressing the original data block, so that each decompression engine can decompress the multiple canonical compressed data blocks synchronously, thereby improving the decompression efficiency of the data.

[0328] Furthermore, the granularity into which the original data can be divided is independent of the number of decompression engines. The original data can be compressed into a large number of smaller standard compressed data blocks. When accessing data at a random address, the multiple consecutive standard compressed data blocks containing the data at the random address are input into multiple decompression engines for decompression, thereby achieving double-speed decompression of the compressed data at the random address.

[0329] See also Fig.12 , Fig.12 1 is a schematic diagram of the structure of a decompression device provided by an embodiment of the present invention. The decompression device 12 includes a controller 120 and M decompression engines 121, wherein the controller includes an acquisition unit 1201, a data distribution unit 1202 and a splicing unit 1203, wherein:

[0330] The acquisition unit 1201 is used to acquire N standard compressed data blocks, wherein any one of the N standard compressed data blocks is a data block with a length of L2 generated by independently compressing the original data block, and one standard compressed data block can be decompressed by a single decompression engine within one processing cycle, N is a positive integer, and M≥N;

[0331] The data distribution unit 1202 decompresses the N standard compressed data blocks in parallel through N decompression engines to obtain N original data blocks, where the standard compressed data blocks correspond to the original data blocks one by one;

[0332] The splicing unit 1203 is used to splice the N original data blocks to obtain original data.

[0333] In one implementation of the embodiment of the present application, the data distribution unit 1202 can be specifically used to distribute N standard compressed data blocks to N decompression engines among M decompression engines, so that the N standard compressed data blocks are respectively decompressed by the N decompression engines to obtain N original data blocks corresponding to the N standard compressed data blocks, wherein the length of any original data block among the N original data blocks is not greater than L1. It should be noted that the specific implementation of each unit in the decompression device 12 in the embodiment of the present application can refer to the above Fig.11 The relevant description in the data decompression method shown is not repeated in the embodiment of the present application.

[0334] Each unit in the controller 120 can implement the above data compression method based on hardware or software. For example, when it is based on hardware, each unit can be completed based on circuits such as FPGA and ASIC, or when it is based on software, the corresponding function can be executed by allowing the CPU to read the code that can realize the function of the above unit stored in the memory. In order to achieve faster decompression speed, hardware is used in this application to realize the functions of related units.

[0335] See also Fig.13 , Fig.13 It is a structural diagram of a second electronic device with a data decompression function provided by an embodiment of the present invention. The second electronic device 13 includes: a processor 131, a memory 132 and a decompression device 133, wherein the processor 131 is connected to the memory 132 and the decompression device 133 via a bus 134.

[0336] The processor 131 is the control center of the decompression device, and may be one or more CPUs, one or more microprocessors, or one or more FPGAs, etc., which is not limited in the embodiment of the present invention.

[0337] The memory 132 is used to store data and instructions, and may be a read-only memory, a random access memory or other memory, etc. The processor 131 is used to call the instructions of the memory 132 to implement the functions of the second electronic device.

[0338] The decompression device 133 is used to decompress the received standard compressed data block to restore the original data block. The decompression device can be as follows: Figure 3 or Fig. 9 The decompression device shown. It should be noted that Fig.13 The specific implementation of each function in the decompression device shown can refer to Fig.11 The relevant description in the method embodiment shown will not be repeated in the embodiments of the present application.

[0339] The embodiment of the present application provides a compression and decompression system, which may include a compression device and a decompression device, wherein the compression device may be Fig. 9 or Fig.10 Any of the compression devices shown, the decompression device can be Figure 3 or Fig.12 It can be understood that the compression and decompression system can be set in the same electronic device, such as Figure 2 The electronic devices shown may also be distributed in different electronic devices.

[0340] An embodiment of the present application further provides a compression and decompression system, which may include a first electronic device and a second electronic device. For details, please refer to the relevant descriptions of the first electronic device and the second electronic device in the above embodiments, which will not be repeated here.

[0341] It should be understood that the electronic device, the first electronic device, and the second electronic device in the present application can be a personal computer (PC), a mobile phone, a laptop, a tablet computer, a server, a gateway, a router, etc. Among them, the first electronic device and the second electronic device can be set in the same electronic device, such as Figure 2 The electronic device shown may also be different devices, which is not limited in the embodiments of the present application.

[0342] Another data compression method provided in an embodiment of the present application is introduced below. The data compression method is applied to a compression device or a third electronic device that can implement data compression. The following description is made using a compression device as an example.

[0343] The compression device independently compresses each of the S continuous original data blocks in the original data through a compression algorithm, obtains S compressed data blocks corresponding to the S original data blocks, and records the length of each compressed data block in the S compressed data blocks. Each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, the length d of each compressed data block in the S compressed data blocks is not greater than L2, S is a positive integer, and the compressed data block with a length of L2 is a data block that a single decompression engine can process in one processing cycle. The compressed data blocks obtained by limiting the compression can be processed in one processing cycle, so that each decompression engine of the decompression device can decompress synchronously during decompression.

[0344] like Fig.14 The flowchart of a data compression method provided by an embodiment of the present application is shown, and the data compression method may include the following steps:

[0345] S141: The compression device independently compresses each of the S consecutive original data blocks in the original data by using a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks.

[0346] Among them, each of the S original data blocks is a plurality of continuous data in the original data, and the length of the plurality of continuous data is not greater than L1. The length d of each compressed data block in the S compressed data blocks is not greater than L2. S is a positive integer. The compressed data block with a length of L2 is a data block that can be processed by a single decompression engine in the decompression device within one processing cycle. L1 and L2 are integers greater than 1. The d is an integer greater than 0. L1 is greater than L2.

[0347] S142: The compression device records the length of each compressed data block in the S compressed data blocks.

[0348] For the description of “original data”, “data block”, “original data block”, “compressed data block”, “processing cycle”, etc., please refer to the above Figure 5 The data compression method or related descriptions in other embodiments are not repeated in the embodiments of the present application.

[0349] With the above Figure 5 The data compression method shown is different in that Figure 5The data compression method shown finally obtains a standard compressed data block, whose length is fixed to L2, which is achieved by adding additional data to the compressed data block. In the embodiment of the present application, the compressed data block is the compressed data block finally obtained, and its length is not fixed. The compression device needs to record the length of each compressed data block so that the decompression device can distinguish the compressed data blocks of different lengths in the obtained compressed data.

[0350] Among them, an implementation of step S141 can refer to the two implementations of S41 above, and the embodiments of this application will not be repeated. Through the two implementations of S41, the length of the compressed data block is made as close to L2 as possible, so that each decompression engine in the fourth electronic device or decompression device can maximize the use of the decompression engine when decompressing the first compressed data block, thereby improving the decompression efficiency.

[0351] The first original data block is any one of the S original data blocks, and the first compressed data block is a compressed data block corresponding to the first original data block in the S compressed data blocks. The implementation manner in which the compression device records the length of the compressed data block is described below by taking the first compressed data block as an example. It can be understood that the compression device may record the length of the first compressed data block including but not limited to the following two implementation manners:

[0352] The first implementation method:

[0353] The compression device records the length d of the first compressed data block in the start field of the first compressed data block, where the start field is a field starting from the start bit of the first compressed data block and having a length equal to the third length threshold. The third length threshold may be 4 bits or 8 bits, which is related to the size of L2. For example, if L2 is 8 bytes, that is, the maximum length of the first compressed data block is 64 bits, then the recorded length value does not exceed 64, and the third length threshold may be 8 bits.

[0354] After receiving the compressed data, the decompression device reads the length stored in the start field to obtain the first compressed data block. Similarly, the next first compressed data block is obtained to obtain all the first compressed data blocks. Furthermore, the decompression device or equipment sequentially inputs the multiple first compressed data blocks into the decompression engine for decompression.

[0355] The second implementation method:

[0356] The compression device writes the length d of the first compressed data block into a first file, where the first file includes the lengths of S compressed data blocks obtained by sequential compression by the compression device.

[0357] The compression device records the lengths of the S compressed data blocks in the first file in sequence according to the compression order of the data. When the compression device transmits the compressed data to the fourth electronic device or the decompression device, the first file is sent to the fourth electronic device or the decompression device at the same time. The decompression device or device divides the received compressed data according to the multiple lengths contained in the first file to obtain S compressed data blocks. Further, the decompression device or device sequentially inputs the S compressed data blocks into the decompression engine for decompression.

[0358] The following describes a compression device provided by an embodiment of the present application. Fig.15 As shown, the compression device 150 includes: a compression unit 1501 and a recording unit 1502, wherein:

[0359] A compression unit 1501 is used to independently compress each of S consecutive original data blocks in the original data by using a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, the length d of each compressed data block in the S compressed data blocks is not greater than L2, S is a positive integer, a compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2;

[0360] The recording unit 1502 is used to record the length of each compressed data block in the S compressed data blocks.

[0361] The specific implementation of each functional unit included in the compression device 150 can refer to the above Fig.13 The relevant description of the data compression method is not repeated here. Each unit in the compression device 150 can implement the above data compression method based on hardware or software. For example, when it is based on hardware, each unit can be completed based on circuits such as FPGA and ASIC, or when it is based on software, the CPU can read the code that can realize the function of the above unit stored in the memory to execute the corresponding function.

[0362] Another compression device provided by the embodiment of the present application is described below. Fig.16 As shown, the compression device 160 includes: a processor 1601 and a memory 1602. The processor 1601 is connected to the memory 1602 via a bus 1603. The memory 1602 is used to store data and program instructions.

[0363] The memory 1602 may be a read-only memory (ROM), a random access memory (RAM) or other memory. In the embodiment of the present application, the memory 1602 is used to store data such as raw data and various software programs. Fig.14 The program of the data compression method, etc. Optionally, the memory 1602 may be DDR to improve the data transmission rate between the memory 1602 and the processor 1601 or between the memory 1601 and the decompression device.

[0364] The processor 1601 is the control center of the first computer device, which may be a processor or a general term for multiple processing elements. In a specific implementation, as an optional embodiment, the processor 1601 may include one or more CPUs. The processor 1601 may also include a specific integrated circuit, such as one or more microprocessors, one or more field programmable gate arrays (FPGAs), etc., that implement the data compression method in the embodiment of the present invention.

[0365] Optionally, the compression device further includes a communication interface 1604, and the communication interface 1604 is used to implement data exchange with other devices, such as a second decompression device.

[0366] The processor 1601 calls the data and program instructions stored in the memory 1602 to execute the following method:

[0367] Each of S consecutive original data blocks in the original data is independently compressed by a compression algorithm to obtain S compressed data blocks corresponding to the S original data blocks, each of the S original data blocks is a plurality of continuous data in the original data, the length of the plurality of continuous data is not greater than L1, the length d of each compressed data block in the S compressed data blocks is not greater than L2, S is a positive integer, a compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle, L1 and L2 are integers greater than 1, d is an integer greater than 0, and L1 is greater than L2;

[0368] The length of each compressed data block in the S compressed data blocks is recorded.

[0369] Optionally, the compression device may further include a communication interface 1604 to implement data transmission with a communication counterpart such as a decompression device.

[0370] The specific implementation of the compression device 160 when executing the steps can refer to the above figure Fig.13The relevant description of the data compression method will not be repeated here.

[0371] Another decompression device involved in the embodiment of the present application is described below. Fig.17 , the decompression device 170 may include a controller 1700, a first memory 1701, M decompression engines 1702 and a second memory 1703, wherein:

[0372] The controller 1700 is used to obtain compressed data, the compressed data including N compressed data blocks and the length of each of the N compressed data blocks, and split the compressed data into N compressed data blocks according to the length of each of the N compressed data blocks;

[0373] The first memory 1701 is used to store N compressed data blocks, wherein any one of the N compressed data blocks is an independently compressed data block with a length not greater than L2, N is a positive integer, M≥N, the compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle, L1 and L2 are integers greater than 1, and L1 is greater than L2;

[0374] Each of the N decompression engines is used to obtain a compressed data block from the N compressed data blocks, and decompress the obtained compressed data blocks respectively to obtain N original data blocks corresponding to the N compressed data blocks, wherein the length of any one of the N original data blocks is not greater than L1, and the M decompression engines include the N decompression engines;

[0375] The second memory 1703 is used to store the N original data blocks.

[0376] In a possible implementation, the decompression device 170 further includes: a splicing circuit 1704, wherein the splicing circuit 1704 is configured to splice the N original data blocks to obtain original data.

[0377] Optionally, the first memory 1701 may include M storage subunits corresponding to the M decompression engines one by one, and one storage subunit may store one compressed data block. Similarly, the second memory 1703 may include M storage subunits corresponding to the M decompression engines one by one, and one storage subunit may store one original data block.

[0378] It should be noted that a storage subunit in the first memory and / or a storage subunit in the second memory can be set in the corresponding decompression engine, which is not limited in the embodiment of the present application.

[0379] It should be noted that the controller 1700 is not a necessary device for the decompression device 170. The decompression device 170 may not include the controller 1700, but the controller 1700 is disposed in the fourth electronic device including the decompression device 170. Each process in the controller 1700 may implement the above data compression method based on hardware or software. For example, when it is based on hardware, each unit may be completed based on circuits such as FPGA and ASIC, or when it is implemented based on software, the corresponding function may be executed by allowing the CPU to read the code that can implement the above process stored in the memory.

[0380] The embodiment of the present application further provides a fourth electronic device capable of realizing a data decompression function. The fourth electronic device may include: Fig.17 Any decompression device shown.

[0381] The following is an introduction to the Fig.14 A data decompression method of the data compression method shown in FIG. The method is applied to a fourth electronic device or a decompression device. The fourth electronic device may be as follows: Figure 2 The electronic device shown in the figure or other electronic devices including a decompression device, the decompression device can be as follows Fig.17 The decompression device shown may also be other decompression devices.

[0382] like Fig.18 A flow chart of a decompression method is shown, and the method includes but is not limited to the following steps:

[0383] S181: Obtain compressed data, wherein the compressed data includes N compressed data blocks and the length of each of the N compressed data blocks, where N is a positive integer; wherein any one of the N compressed data blocks is an independently compressed data block with a length not greater than L2, where N is a positive integer, M ≥ N, and the compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle.

[0384] S182: Split the compressed data into N compressed data blocks according to the length of each compressed data block in the N compressed data blocks.

[0385] S183: Decompress the N compressed data blocks respectively through N decompression engines to obtain N original data blocks corresponding to the N compressed data blocks one by one, wherein the length of any original data block among the N original data blocks is not greater than L1, L1 and L2 are integers greater than 1, and L1 is greater than L2.

[0386] S184: splicing the N original data blocks to obtain original data.

[0387] Among them, the compressed data block is obtained by Fig.14The compressed data block obtained by the data compression method is shown in the figure. Fig.14 The relevant description of the data compression method shown in the figure will not be repeated in the embodiment of the present application.

[0388] In one embodiment of the present application, after receiving the compressed data, the decompression device reads the length stored in the start field to obtain a compressed data block. Similarly, the next compressed data block is obtained to obtain N compressed data blocks. Further, the decompression device sequentially inputs the N compressed data blocks into the decompression engine for decompression.

[0389] In one embodiment of the present application, the compression device may record the length of each compressed data block in the first file in sequence according to the compression order of the data. After the compression device transmits the compressed data including N compressed data blocks and the first file to the decompression device, the decompression device may divide the received compressed data according to the multiple lengths contained in the first file to obtain N compressed data blocks. Further, the decompression device sequentially inputs the N compressed data blocks into N decompression engines for decompression.

[0390] See also Fig.19 , Fig.19 1 is a schematic diagram of the structure of another decompression device provided by an embodiment of the present invention. The decompression device 190 may include a controller 1900 and M decompression engines 1901, wherein the controller 1900 includes the following units:

[0391] The acquisition unit 1902 is used to acquire compressed data, wherein the compressed data includes N compressed data blocks and the length of each of the N compressed data blocks, where N is a positive integer; wherein any one of the N compressed data blocks is an independently compressed data block with a length not greater than L2, where N is a positive integer, M ≥ N, and the compressed data block with a length of L2 is a data block that can be processed by a single decompression engine within one processing cycle;

[0392] A splitting unit 1903 is used to split the compressed data into N compressed data blocks according to the length of each compressed data block in the N compressed data blocks;

[0393] The data distribution unit 1904 is used to distribute the N compressed data blocks to the N decompression engines, so that the N decompression engines decompress the N compressed data blocks respectively to obtain N original data blocks corresponding to the N compressed data blocks one by one, and the length of each of the N original data blocks is not greater than L1, L1 and L2 are integers greater than 1, and L1 is greater than L2;

[0394] The splicing unit 1905 is used to splice the N original data blocks to obtain original data.

[0395] The specific implementation of each functional unit included in the decompression device 190 can refer to the above Fig.18 The relevant description of the data decompression method is not repeated here. Each unit in the controller 1900 can implement the above data compression method based on hardware or software. For example, when it is based on hardware, each unit can be completed based on circuits such as FPGA and ASIC, or when it is based on software, the corresponding function can be executed by allowing the CPU to read the code that can realize the function of the above unit stored in the memory. In order to achieve a faster decompression speed, hardware is used in this application to implement the functions of the relevant units.

[0396] The embodiment of the present application further provides a fourth electronic device capable of realizing a data decompression function. The fourth electronic device may include: Fig.17 or Fig.19 For example, the fourth electronic device 200 includes: a processor 2001 , a memory 2002 , and a decompression device 2003 , wherein the processor 2001 is connected to the memory 2002 and the decompression device 2003 via a bus 2004 .

[0397] The processor 2001 is the control center of the decompression device, and may be one or more CPUs, one or more microprocessors, or one or more FPGAs, etc., which is not limited in the embodiment of the present invention.

[0398] The memory 2002 is used to store data and instructions, and may be a read-only memory, a random access memory or other memory, etc. The processor 2001 is used to call the instructions of the memory 2002 to implement the functions of the second electronic device.

[0399] The decompression device 2003 is used to decompress the received compressed data block to restore the original data block. The decompression device can be as follows: Fig.17 or Fig.19 The decompression device shown. It should be noted that Fig. 20 The specific implementation of each function in the decompression device shown can refer to Fig.18 The relevant description in the method embodiment shown will not be repeated in the embodiments of the present application.

[0400] Optionally, the fourth electronic device may further include a computing device, which is coupled to the decompression device and the processor, and is used to perform data processing according to the original data decompressed by the decompression device. Figure 2 The relevant description of the computing device in the described electronic device will not be repeated here.

[0401] The embodiment of the present application also provides a compression and decompression system, which may include a compression device and a decompression device, wherein the compression device may be Fig.15 or Fig.16 Any of the compression devices shown, the decompression device can be Fig.17 or Fig.19 Any decompression device shown. It can be understood that the compression and decompression system can be set in the same electronic device or distributed in different electronic devices.

[0402] An embodiment of the present application further provides a compression and decompression system, which may include a third electronic device and a fourth electronic device. For details, please refer to the relevant descriptions of the third electronic device and the fourth electronic device in the above embodiments, which will not be repeated here.

[0403] It should be understood that the third electronic device and the fourth electronic device in the present application can be personal computers (PCs), mobile phones, laptops, tablet computers, servers, gateways, routers, etc. The third electronic device and the fourth electronic device can be set in the same electronic device or different devices, which is not limited in the embodiments of the present application.

[0404] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

Claims

1. A data compression method, characterized in that: The method is used for compressing original data into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device comprises a plurality of decompression engines, each of which can decompress one standard compressed data block in one processing cycle, and L2 is an integer greater than 1, and the method comprises: Performing one or more original data block compressions, the original data block compressions comprising: Processing the original data block based on a compression algorithm to obtain the canonical compressed data block corresponding to the original data block, wherein the original data block is data included in the original data; The processing of the original data block based on the compression algorithm to obtain the standard compressed data block corresponding to the original data block comprises: The original data block is independently compressed by a compression algorithm to obtain a compressed data block corresponding to the original data block, wherein the length d of the compressed data block is not greater than L2; the original data includes a plurality of data groups; the data group includes one or more bits of data; Adding additional data of length L2-d after the compressed data block, thereby generating the standard compressed data block of length L2, wherein d is an integer greater than 0; the multiple compressed data blocks obtained by independently compressing the multiple original data blocks are independent of each other; The sum of the lengths of the original data block and the first data group in the original data that follows the original data block is greater than L1, or the length of the compressed data block obtained by compressing the original data block and the first data group in the original data that follows the original data block is greater than L2, and L1 is greater than L2.

2. The method according to claim 1, characterized in that The original data block includes a plurality of continuous data in the original data, and the length of the original data block is no greater than L1, where L1 is the exit bit width of the decompression engine.

3. The method according to claim 2, characterized in that When the length of the original data block is equal to the L1, the additional data in the canonical compressed data block corresponding to the original data block is a legal codeword; or the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded; or, When the length of the original data block is smaller than the L1, the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded.

4. The method according to claim 2 or 3, characterized in that The independently compressing the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block comprises: S1. Obtaining a data group to be compressed from the original data; S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4; S3, compress the data group to be compressed, and execute S1 after the compression is completed; S4, using the currently compressed data as the compressed data block; Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than the L2, it is determined that compression is required; if the sum of x and Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than the L2, it is determined that compression is not required; if the sum of x and Δx is greater than the L1, it is determined that compression is not required.

5. The method according to claim 2 or 3, characterized in that: The independently compressing the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block comprises: Compressing a data block with a length of L1 in the original data with a data group as a step length to obtain a temporary compressed data block; When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having the length L1 is used as the original data block, and data obtained by compressing the original data block is used as the compressed data block, wherein the length of the compressed data block is not greater than L2, and i is a minimum positive integer that makes the compressed data block not greater than L2; When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the original data block, and the temporary compressed data block is used as the compressed data block.

6. The method according to any one of claims 1 to 3, characterized in that: The L2 is the entry bit width of a single decompression engine.

7. A compression device, characterized in that: Used to compress the original data into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device includes a plurality of decompression engines, each of which can decompress one standard compressed data block in one processing cycle, and L2 is an integer greater than 1, including: A processing module, configured to process the original data block based on a compression algorithm to obtain the canonical compressed data block corresponding to the original data block, wherein the original data block is data included in the original data; The processing module comprises: A compression unit, configured to independently compress the original data block by a compression algorithm to obtain a compressed data block corresponding to the original data block, wherein the length d of the compressed data block is not greater than L2; the original data includes a plurality of data groups; the data group includes one or more bits of data; a standardization unit, configured to add additional data of a length of L2-d after the compressed data block, thereby generating the standard compressed data block of the length of L2, wherein d is an integer greater than 0; and the multiple compressed data blocks obtained by independently compressing the multiple original data blocks are independent of each other; The sum of the lengths of the original data block and the first data group in the original data that follows the original data block is greater than L1, or the length of the compressed data block obtained by compressing the original data block and the first data group in the original data that follows the original data block is greater than L2, and L1 is greater than L2.

8. The compression device according to claim 7, characterized in that The original data block includes a plurality of continuous data in the original data, and the length of the original data block is no greater than L1, where L1 is the exit bit width of the decompression engine.

9. The compression device according to claim 8, characterized in that When the length of the original data block is equal to the L1, the additional data in the canonical compressed data block corresponding to the original data block is a legal codeword; or the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded; or, When the length of the original data block is smaller than the L1, the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded.

10. The compression device according to claim 7 or 8, characterized in that The compression unit is specifically used to perform the following steps: S1, obtaining a data group to be compressed from the original data, the data group including one or more bits of data; S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4; S3, compress the data group to be compressed, and execute S1 after the compression is completed; S4, using the currently compressed data as the compressed data block; Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than the L2, it is determined that compression is required; if the sum of x and Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than the L2, it is determined that compression is not required; if the sum of x and Δx is greater than the L1, it is determined that compression is not required.

11. The compression device according to claim 7 or 8, characterized in that The compression unit is specifically used for: Compressing a data block with a length of L1 in the original data with a data group as a step length to obtain a temporary compressed data block; When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having the length L1 is used as the original data block, and data obtained by compressing the original data block is used as the compressed data block, wherein the length of the compressed data block is not greater than L2, and i is a minimum positive integer that makes the compressed data block not greater than L2; When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the original data block, and the temporary compressed data block is used as the compressed data block.

12. The compression device according to any one of claims 7 to 9, characterized in that: The L2 is the entry bit width of a single decompression engine.

13. A compression device, characterized in that: The invention is used for compressing original data into one or more standard compressed data blocks of length L2 for parallel decompression by a decompression device, wherein the decompression device comprises a plurality of decompression engines, each of which can decompress one standard compressed data block in one processing cycle, and L2 is an integer greater than 1. The compression device comprises: a processor and a memory, wherein the memory is used to store data and program instructions, and the processor calls the data and program instructions stored in the memory to perform: one or more original data block compressions, wherein the original data block compressions include: Processing the original data block based on a compression algorithm to obtain the canonical compressed data block corresponding to the original data block, wherein the original data block is data included in the original data; The processor executes the processing of the original data block based on the compression algorithm to obtain the standard compressed data block corresponding to the original data block, specifically including executing: The original data block is independently compressed by a compression algorithm to obtain a compressed data block corresponding to the original data block, wherein the length d of the compressed data block is not greater than L2; the original data includes a plurality of data groups; the data group includes one or more bits of data; Adding additional data of length L2-d after the compressed data block, thereby generating the standard compressed data block of length L2, wherein d is an integer greater than 0; the multiple compressed data blocks obtained by independently compressing the multiple original data blocks are independent of each other; The sum of the lengths of the original data block and the first data group in the original data that follows the original data block is greater than L1, or the length of the compressed data block obtained by compressing the original data block and the first data group in the original data that follows the original data block is greater than L2, and L1 is greater than L2.

14. The compression device according to claim 13, characterized in that The original data block includes a plurality of continuous data in the original data, and the length of the original data block is no greater than L1, where L1 is the exit bit width of the decompression engine.

15. The compression device according to claim 14, characterized in that When the length of the original data block is equal to the L1, the additional data in the canonical compressed data block corresponding to the original data block is a legal codeword; or the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded; or, When the length of the original data block is smaller than the L1, the additional data includes an invalid flag and invalid data, wherein the invalid flag is used to indicate that the additional data needs to be discarded.

16. The compression device according to claim 14 or 15, characterized in that The processor executes the independent compression of the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block, specifically including executing the following steps: S1. Obtain a data group to be compressed from the original data, wherein the data group includes one or more bits of data; S2, determining whether the data group to be compressed needs to be compressed, if compression is required, executing S3, otherwise executing S4; S3, compress the data group to be compressed, and execute S1 after the compression is completed; S4, using the currently compressed data as the compressed data block; Among them, the judgment of whether it is necessary to compress the data group to be compressed includes: if the sum of the total length x of one or more compressed data groups and the length Δx of the data group to be compressed is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is not greater than the L2, it is determined that compression is required; if the sum of x and Δx is not greater than the L1 and the length of the compressed data block generated after the data group to be compressed is compressed is greater than the L2, it is determined that compression is not required; if the sum of x and Δx is greater than the L1, it is determined that compression is not required.

17. The compression device according to claim 14 or 15, characterized in that The processor executes the independent compression of the original data block by using a compression algorithm to obtain a compressed data block corresponding to the original data block, specifically including executing: Compressing a data block with a length of L1 in the original data with a data group as a step length to obtain a temporary compressed data block; When the length of the temporary compressed data block is greater than L2, a data block obtained by removing i data groups at the tail of the data block having the length L1 is used as the original data block, and data obtained by compressing the original data block is used as the compressed data block, wherein the length of the compressed data block is not greater than L2, and i is a minimum positive integer that makes the compressed data block not greater than L2; When the length of the temporary compressed data block is not greater than L2, the data block with the length of L1 is used as the original data block, and the temporary compressed data block is used as the compressed data block.

18. The compression device according to any one of claims 13 to 15, characterized in that: The L2 is the entry bit width of a single decompression engine.

19. An electronic device with a compression function, the electronic device comprising a compression device and a processing device, the compression device is a compression device as described in any one of claims 7 to 12 or a compression device as described in any one of claims 13 to 18; the processing device is used to send the original data to the compression device.

20. A decompression device, characterized in that: include: A first memory, M decompression engines and a second memory, wherein: The first memory is used to store a plurality of standard compressed data blocks, wherein one standard compressed data block is a data block with a length of L2 generated by independently compressing an original data block, one standard compressed data block can be decompressed by a single decompression engine within one processing cycle, and M is an integer greater than 1; The M decompression engines are used to decompress the standard compressed data blocks stored in the first memory in parallel, and one decompression engine is used to decompress one standard compressed data block; The second memory is used to store the original data blocks obtained by decompressing the standard compressed data blocks, and the standard compressed data blocks correspond to the original data blocks one by one; The original data includes multiple data groups; the data groups include one or more bits of data; and the multiple compressed data blocks obtained by independently compressing the multiple original data blocks are independent of each other.

21. The decompression device according to claim 20, characterized in that: The decompression device also includes a splicing circuit, The splicing circuit is used to splice the original data blocks stored in the second memory to obtain original data.

22. The decompression device according to claim 20 or 21, characterized in that: Each of the M decompression engines is further used for: When the length of the original data block obtained by decompression is equal to L1 or when an invalid flag in the standard compressed data block is identified, the decompression operation is stopped, the export bit width of the decompression engine is L1, and the length of the original data block is not greater than L1.

23. The decompression device according to claim 20 or 21, characterized in that: The decompression device also includes a control circuit, which is used for: When the length of the original data block decompressed by the first decompression engine is equal to L1, the decompression operation of the first decompression engine is stopped, the first decompression engine is any one of the M decompression engines, and the L1 is greater than the L2.

24. The decompression device according to claim 23, characterized in that: The exit bit width of the decompression engine is L1, and the length of the original data block is not greater than L1.

25. The decompression device according to any one of claims 20, 21 and 24, characterized in that: The second memory includes M storage sub-units corresponding one-to-one to the M decompression engines respectively. The first storage sub-unit is used to store the original data block decompressed by the decompression engine corresponding to the first storage sub-unit. The first storage sub-unit is any one of the M storage sub-units.

26. An integrated circuit, characterized in that: The integrated circuit comprises a decompression device as claimed in any one of claims 20 to 25.

27. A data decompression method, characterized in that: Applied to a decompression device, the decompression device includes M decompression engines, M is an integer greater than 1, and the method includes: Obtain N standard compressed data blocks, wherein any one of the N standard compressed data blocks is a data block with a length of L2 generated by independently compressing an original data block, and one standard compressed data block can be decompressed by a single decompression engine within one processing cycle, N is a positive integer, and M≥N; Decompressing the N standard compressed data blocks in parallel respectively by N decompression engines to obtain N original data blocks, wherein the standard compressed data blocks correspond to the original data blocks one by one; Splicing the N original data blocks to obtain original data; The original data includes multiple data groups; the data groups include one or more bits of data; and the multiple compressed data blocks obtained by independently compressing the multiple original data blocks are independent of each other.

28. The data decompression method according to claim 27, characterized in that: The length of any one of the N original data blocks is no longer than L1, where L1 is the exit bit width of the decompression engine.

29. The data decompression method according to claim 28, characterized in that: The first decompression engine is any one of the N decompression engines, and the first standard compressed data block is any one of the N standard compressed data blocks; Decompressing the N standard compressed data blocks in parallel by using N decompression engines to obtain N original data blocks specifically includes: Decompressing the first canonical compressed data block by the first decompression engine; When the length of the original data block decompressed by the first decompression engine is equal to the L1, the first decompression engine stops decompressing the first standard compressed data block and outputs the original data block; or, when the first decompression engine recognizes an invalid flag, the first decompression engine stops decompressing the first standard compressed data block and outputs the original data block currently decompressed.

30. An electronic device with a decompression function, characterized in that: It comprises a processor, a memory and a decompression device, wherein the processor is coupled to the memory and the decompression device, wherein the decompression device is the decompression device according to any one of claims 20-25.

31. An electronic device with a decompression function, characterized in that: It includes a processor, a memory and M decompression engines, the processor is coupled to the memory and the M decompression engines, the memory is used to store data and program instructions, and the processor calls the data and program instructions stored in the memory to implement the data decompression method as described in any one of claims 27-29.

32. A compression and decompression system, characterized in that: It comprises a compression device and a decompression device, wherein the compression device is the compression device according to any one of claims 7 to 18, and the decompression device is the decompression device according to any one of claims 20 to 25.

33. A compression and decompression system, characterized in that: It comprises a first electronic device and a second electronic device, wherein the first electronic device is the electronic device with a compression function as claimed in claim 19, and the second electronic device is the electronic device with a decompression function as claimed in claim 30 or 31.

Images