Pre-Compression Data Reordering for Disordered File Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data compression techniques fail to effectively reduce storage space for highly disordered or naturally originated data, such as images and videos, due to inefficient prediction models and file storage blocking issues, leading to wasteful compression and decompression processes without significant storage size reduction.
Innovation Solution
The approach involves identifying and grouping sequentially efficiently compressible data sequences, transforming data to increase orderliness, and packing similar data streams together to optimize compression efficiency, using adaptive predictors and entropy encoding techniques to maintain a consistent prediction model and reduce disruptive sequences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If conventional compression algorithms are applied to highly disordered data, then compression process is performed, but storage size reduction is not achieved and file size may increase
Solution Approach 1:
The patent applies pre-compression transformations to reorder and transform the input data stream before it enters the compression engine. This preliminary action modifies the data structure to create patterns that are more amenable to compression, allowing the compression algorithm to effectively reduce storage size even for previously incompressible highly disordered data
Solution Approach 2:
The patent transforms the input data stream by changing its parameters and structure through pre-compression processing. This includes reordering operations that modify the sequence and arrangement of data elements, converting highly disordered data into a form with exploitable patterns that enable effective compression and achieve storage size reduction
2Quantity of substance
If compression is performed on small files, then compression process is executed, but storage space is not actually reduced due to file system blocking
Solution Approach 1:
The patent merges multiple small files or data streams into a single consolidated compression unit before applying compression. This combining approach overcomes file system blocking issues by creating a larger effective data unit that can achieve actual storage reduction, while the merged structure enables efficient processing that reduces wasted compression time
3Productivity
If conventional compression methods are used, then compression is achieved, but decompression speed is limited by balanced throughput design
Solution Approach 1:
The patent segments the decompression process into multiple parallel stages including pre-processing, main decompression, and post-processing operations. This segmentation allows different parts of the decompression pipeline to operate concurrently, significantly increasing decompression speed while the compression phase maintains optimized throughput through dedicated processing paths
Solution Approach 2:
The patent implements dynamic processing where the compression and decompression operations can be independently optimized and scaled. The system allows compression to operate at one throughput level while decompression operates at a higher speed level, breaking the conventional balance constraint and enabling asymmetric performance optimization for each direction
Data Source
AI summary
Prediction-based compression engines are spoon-fed with sequentially efficiently compressible (SEC) streams of input data that make it possible for the compression engines to more efficiently compress or otherwise compact the incoming data than would be possible with streams of input data accepted on a TV-raster scan basis. Various techniques are disclosed for intentionally forming SEC input data streams. Among these are the tight packing of alike files or fragments into concatenation suitcases and the decomposition of files into substantially predictably consistent (SPC) fragments or segments that are routed to different suitcases according to their type. In a graphics-directed embodiment, image frames are partitioned into segment areas that are internally SPC and multidirectional walks (i.e., U-turning walks) are defined in the segment areas where these defined walks are traced during compression and also during decompression. A variety of pre-compression data transformation methods are disclosed for causing apparently random data sequences to appear more compressibly alike to each other. The methods are usable in systems that permit substantially longer times for data compaction operations than for data decompaction operations.