A data compression method and device

Abstract

The invention discloses a data compression method, which comprises the following steps: receiving block data to be compressed input according to a preset first length; compressing the block data to obtain compressed block data; compressing the block data Concatenating and arranging according to the compression order to obtain concatenated compressed block data; dividing the concatenated compressed block data into a plurality of data blocks whose length is a preset second length, so that the system releases space in units of data blocks. Applying the technical solution provided by the embodiment of the present invention, through the block data compression method of fixed-length input and variable-length output, and the compressed block data of variable-length output are connected and arranged according to the compression order, and the connected and arranged The block data is divided into multiple blocks whose length is the preset second length, which can reduce the waste of space, improve the compression efficiency of block data, and simplify the indexing of metadata. The invention also discloses a data compression device, which has corresponding technical effects.

Description

technical field [0001] The invention relates to the technical field of data compression, in particular to a data compression method and device. Background technique [0002] With the development of data compression technology, the amount of data in the storage system has been greatly reduced, thereby reducing the storage space and improving the efficiency of system transmission, storage and processing. [0003] Existing data compression methods mainly include two types. The first method is to compress fixed-length block data to fixed-length block data, such as figure 1 As shown, for example, the block data of 64KB is compressed into an integer multiple of 8KB, or the block data of 8KB is compressed into an integer multiple of 1KB. Divide the block data delivered by the host into 8KB-aligned block data. If it is not aligned, it will be completed and read, so that the block data to be compressed will be rounded into 8KB, and then the 8KB block data will be compressed. The com...

AI Technical Summary

Problems solved by technology

[0004] Both of the above two data compression methods have certain shortcomings. Among them, the compressed block data of the first method is fixed-length, and must be rounded with a certain granularity, which has a large waste of space and affects the compression efficiency of the data.

Secondly, because the length of each block of data is different in method 2, it is necessary to record the metadata information of each block of data including the address of each block of data input to the compression processor, and the index of metadata is complicated

Images