Output Buffer Compaction for Mixed Compressed Data Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for compressing and compacting data blocks result in significant wasted space and CPU inefficiencies due to the use of fixed grain units and variable compression ratios, leading to inefficient storage and processing of compressed and uncompressed data blocks.
Innovation Solution
A device and method that sequentially compact compressed and uncompressed data blocks into an output buffer, starting from predefined regions, using actual block sizes and eliminating waste by aligning blocks based on their sizes, and utilizing separate buffers for small blocks, thereby reducing wasted space and improving storage efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If fixed grain units (1KB) are used to pack compressed data blocks, then data blocks can be written to storage media using standardized units, but significant wasted space occurs when compressed blocks do not align with grain unit boundaries
Solution Approach 1:
The patent segments the packing process into two independent phases: first compressing data blocks to variable sizes, then separately packing them into grain units. This allows the compression algorithm to optimize for size without being constrained by grain unit boundaries, reducing wasted space while maintaining standardized storage units.
Solution Approach 2:
The patent introduces dynamic adjustment of compression parameters based on data characteristics, allowing the compression ratio to vary rather than being fixed. This dynamic approach enables better space utilization by adapting compression intensity to the specific compressibility of each data block, thereby reducing overall wasted space.
2Quantity of substance
If compression algorithms are applied to all data blocks, then storage capacity is increased and SSD wear is reduced, but CPU cycles are significantly consumed for compression and decompression operations
Solution Approach 1:
The patent changes the parameter of compression intensity by implementing adaptive compression that adjusts the compression ratio based on data characteristics and storage conditions. This allows the system to use higher compression for compressible data (increasing storage capacity) while using lower or no compression for already-compressed or incompressible data (reducing CPU consumption).
Solution Approach 2:
The patent implements feedback mechanisms that monitor compression ratios and data patterns, using this information to dynamically adjust compression settings. When compression yields poor results or consumes excessive CPU cycles, the system adapts by reducing compression intensity or switching to alternative strategies, thereby optimizing the balance between storage capacity and processing overhead.
3Loss of substance
If compressed data blocks of variable sizes are written to storage, then storage efficiency is improved, but additional data structures for address indirection are required for random read operations
Solution Approach 1:
The patent performs preliminary organization of compressed data blocks into standardized grain units before writing to storage. By pre-packaging variable-sized compressed blocks into fixed-size grain units with embedded metadata, the system eliminates the need for complex external address translation structures, as all necessary information is already organized in the stored format.
4Extent of automation
If non-compressible data blocks are compressed, then uniform compression processing is applied, but the output block becomes bigger than the input block resulting in discarded compression results
Solution Approach 1:
The patent uses feedback from the compression process itself to determine whether to keep or discard compression results. By monitoring the output size relative to input size, the system automatically detects when compression increases data size and discards those results, keeping the original data instead. This maintains automatic processing while avoiding counterproductive compression of non-compressible data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to the field of data compression and data compaction. In particular, the invention provides a device tor an improved compacting of compressed and uncompressed data blocks into an output buffer. Thereby, wasted space is reduced or eliminated. The device is configured to obtain a set of input data blocks comprising at least one of a compressed data block and an uncompressed data block; compact the compressed data blocks into the output buffer, starting from a first predefined region in the output buffer, such that the compressed data blocks are sequentially compacted; and compact the uncompressed data blocks into the output buffer, starting from a second predefined region in the output buffer, such that the uncompressed data blocks are sequentially compacted.