Method, system and device for compressing data into disk and readable storage medium

Abstract

The invention discloses a method for compressing data into a disk, which comprises the steps of obtaining compressed data, and determining the length of the compressed data; storing the compressed data into a linked list, and adding the length of the compressed data into the length of the linked list; wherein the length of the linked list is the total length of all compressed data in the linked list; judging whether the linked list length exceeds a threshold value or not; and if yes, allocating idle threads to merge all the compressed data in the linked list, and performing disk falling operation on the merged compressed data. According to the method, the merging process of the compressed data is decoupled from the IO path, compressed data merging does not need to be carried out in the IOprocess of compression disk falling, and the idle threads are specially allocated to carry out the merging work of the compressed data, so that the influence on IO performance is avoided, CPU resources are fully utilized, and the storage performance is improved. Meanwhile, the invention further provides a system and equipment for compressing the data into the disk and a readable storage medium, and the system and equipment have the above beneficial effects.

Description

technical field [0001] The present application relates to the field of compressing data to disk, and in particular to a method, system, device and readable storage medium for compressing data to disk. Background technique [0002] Data compression refers to reducing the amount of data to reduce storage space without losing useful information, improving its transmission, storage and processing efficiency, or reorganizing data according to a certain algorithm to reduce data redundancy and storage space a technical method. With the development of data compression technology, the amount of data in the storage system has been greatly reduced, thereby reducing the storage space and improving the efficiency of system transmission, storage and processing. [0003] However, with the increasing expansion of data services, the requirements for high load and low latency of storage devices are getting higher and higher, which requires storage devices to have a sufficiently high read and...

AI Technical Summary

Problems solved by technology

[0003] However, with the increasing expansion of data services, the requirements for high load and low latency of storage devices are getting higher and higher, which requires storage devices to have a sufficiently high read and write speed. In the existing data compression process, it is necessary to Merge the compressed data together and then perform the disk operation. This process involves the copying of byte-level data in the memory, and the merging action of compressed data is embedded in the IO path, which affects the IO performance.

Images