Compressed Data Linked List Merging for Disk Write Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression processes negatively impact Input/Output (IO) performance due to the need to merge compressed data before writing it to disk, affecting read and write speeds in storage devices.
Innovation Solution
A method that stores compressed data in a linked list and allocates an idle thread to merge and write the data when the list exceeds a threshold, decoupling the merging process from the IO path, allowing continuous data storage without merging during IO operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If compressed data is merged before write-to-disk operation, then data can be written to disk, but IO performance is affected
Solution Approach 1:
The patent extracts the data merging operation from the IO path by using a separate worker thread to perform merging in the background. This allows the main IO thread to continue writing compressed data to disk without waiting for merging to complete, thereby resolving the contradiction between completing write operations and maintaining IO performance.
Solution Approach 2:
The patent implements preliminary action by pre-allocating a buffer pool and using a linked list to temporarily store compressed data chunks before merging. This preparation structure allows data to be accumulated and merged efficiently without blocking the IO path, enabling continuous writing while maintaining performance.
2Productivity
If compressed data is stored and merged in memory, then data can be written to disk, but CPU resources are not fully utilized
Solution Approach 1:
The patent implements self-service by having worker threads automatically merge compressed data in the background without requiring CPU intervention during the IO path. The buffer pool and linked list structures enable data to be managed and merged autonomously, allowing CPU resources to be fully utilized for other tasks while storage operations proceed efficiently.
Data Source
AI summary
A method, system, and device for writing compressed data to a disk, and a readable storage medium. The method includes: acquiring compressed data, and determining a length of the compressed data; storing the compressed data in a linked list, and adding the length of the compressed data to a linked list length, wherein the linked list length is a total length of all compressed data in the linked list; determining whether the linked list length exceeds a threshold value; and if YES, allocating an idle thread to merge all the compressed data in the linked list, and carrying out a write-to-disk operation on the merged compressed data.