LZ process hardware compression method and LZ process hardware compression system based on parallel and pipeline design

Abstract

The invention belongs to the field of computer storage data compression, and discloses a LZ process hardware compression method and a LZ process hardware compression system based on parallel and pipeline design. The LZ process is divided into six modules, which are respectively a calculation hash module, a search hash table module, an update hash table module, a matching module, a last beat resultmerge module, and an output module. A parallel design is adopted inside the modules, that is, multiple bytes are processed in the module at the same time; and the six modules form a six-stage pipeline through the design completed by each module in one clock cycle. The invention avoids the merge operation in the modules of the same pipeline, and in the merge module with the upper beat result, thematching result of the previous pipeline and the matching result of the pipeline are designed to be combined to produce a longer matching result.

Description

technical field [0001] The invention belongs to the field of computer storage data compression, in particular to a LZ process hardware compression method and system based on parallel and pipeline design. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: [0003] With the development of modern science and technology, especially the explosion of cloud computing and big data, the transmission and storage of massive data has become an issue of increasing concern in the computer field. Therefore, compression algorithms provide technical possibilities for reducing transmission bandwidth and increasing storage efficiency. Generally speaking, compression algorithms are divided into lossless compression and lossy compression. For data-sensitive applications, lossless compression is generally used to reduce the amount of data, and when needed, the corresponding decompression algorithm is used to restore the original data...

AI Technical Summary

Problems solved by technology

[0007](1) The existing technology is generally implemented by software, and the performance of the LZ compression process fluctuates greatly due to the different compression ratios of the compressed files;

[0008](2) The performance of the fastest LZ-based compression algorithm can only reach 100 megabits per second (such as LZ4), but for memory, ultra-high-speed transmission and other applications Scenarios where the compression performance cannot meet the requirements (for example, above GB / s);

[0009](3) The existing technology does not formulate hardware compression based on parallel and pipeline design for the LZ process, resulting in the inability to greatly improve the performance of the LZ series compression

Images