Data Compression Model Training With Generated Redundancy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional compression algorithms, including those using AI and ML, are limited by their focus on finding existing redundancy in fixed or small data blocks, leading to a bottleneck in compression performance.
Innovation Solution
A method and apparatus that analyze the possibility of adding redundancy in data blocks using a redundancy generator algorithm (RGA) to generate additional redundant data, exceeding traditional compression ratios by employing advanced mathematical functions and AI algorithms to identify and create more redundancy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional compression algorithms focus on finding existing redundancy in fixed or small data blocks, then the algorithms are simple and easy to implement, but the compression performance hits a bottleneck and cannot discover new ways of generating redundancy
Solution Approach 1:
The patent transforms fixed-size data blocks into dynamic variable-length data blocks that adapt to the redundancy characteristics of the input data. The data block size is no longer constrained to traditional exponential values (4, 8, 16, 32, 63, 128, 256 bytes) but can vary dynamically to optimize compression ratios, allowing the algorithm to discover and exploit redundancy patterns that fixed blocks would miss
Solution Approach 2:
The patent changes the fundamental parameter of data block size from fixed to variable, enabling the compression algorithm to adapt to different data characteristics. By allowing data blocks to expand beyond traditional size constraints, the algorithm can capture larger redundancy patterns and achieve superior compression performance while maintaining implementation feasibility
2Use of energy by moving object
If compression algorithms only check redundancy in small data blocks with sizes that are exponents of 2, then the algorithms are computationally efficient, but they fail to discover redundancy in larger data structures and hit a compression bottleneck
Solution Approach 1:
The patent implements dynamic data block sizing where the block size is determined by the actual redundancy patterns in the data rather than being constrained to fixed exponential sizes. This allows the algorithm to efficiently process data by adapting block sizes to match the scale of redundancy patterns, capturing both small and large redundancy structures without excessive computational overhead
Solution Approach 2:
The patent extends the compression search space by allowing data blocks to transcend traditional size dimensions. Instead of being confined to small fixed blocks, the algorithm can explore larger data structures and multi-dimensional redundancy patterns, effectively adding a dimension to the compression search space and discovering redundancy that would be invisible to traditional algorithms
3Device complexity
If traditional algorithms focus on removing existing redundancy rather than creating new redundancy, then the approach is straightforward and computationally simple, but it limits the achievable compression ratio
Solution Approach 1:
The patent applies preliminary action by generating synthetic redundant data before the actual compression process. Instead of only removing existing redundancy, the algorithm first creates additional redundant patterns in the data, then compresses both the original and generated redundancy. This preliminary generation of redundancy opens new compression opportunities and achieves higher compression ratios
Solution Approach 2:
The patent introduces an intermediary step of redundancy generation between data input and compression. This intermediary process creates synthetic redundancy patterns that serve as a bridge, allowing the compression algorithm to work with both original and generated redundancy. This additional layer enables the system to achieve superior compression by exploiting redundancy that wouldn't naturally exist in the original data
Data Source
AI summary
A method and an apparatus for training a data compression model, and a storage medium are provided. The method includes: reading a data block with a predefined size; analyzing a possibility of adding redundancy in the data block; determining an index of a function for generating redundant data in the data block; and generating, with the function corresponding to the index, redundant data in the data block.