Context-Based Data Compression for Non-Adjacent Redundancy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods only eliminate redundancy between adjacent data, resulting in low compression flexibility and inefficiency, particularly for numeric data with high entropy.
Innovation Solution
Classify data into types based on context information, using predictors and hash tables to identify same and similar data, and apply specific compression algorithms to each type, enhancing redundancy elimination across adjacent and non-adjacent data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is compressed based only on adjacent data relationships, then the compression method is simple, but the compression flexibility is low and compression rate is limited
Solution Approach 1:
The patent segments data into multiple types (first data, second data, third data) based on their contextual characteristics. Different segmentation strategies are applied: first data is segmented by exact repetition patterns, second data by similarity thresholds, and third data by contextual relationships. This segmentation enables targeted compression approaches for each data type, improving overall compression flexibility without requiring a completely complex system redesign.
Solution Approach 2:
The patent applies different compression qualities and methods to different data segments. First data (exact repetitions) receives the most aggressive compression using repetition codes, second data (similar values) uses intermediate compression with similarity-based encoding, and third data uses contextual compression. This local quality approach allows the system to optimize compression flexibility for specific data patterns while maintaining manageable complexity through differentiated handling.
2Productivity
If data classification is performed to eliminate redundancy between non-adjacent data, then compression rate improves, but processing complexity increases
Solution Approach 1:
The patent performs preliminary classification of data into different types before applying compression algorithms. By pre-identifying first data (exact repetitions), second data (similar values), and third data (contextual patterns) before compression, the system enables more effective redundancy elimination across non-adjacent data. This preliminary action improves compression rate by preparing data structures that facilitate broader pattern matching, while managing complexity through a structured two-stage process.
Solution Approach 2:
The patent extends compression analysis from the traditional adjacent-data dimension to multiple dimensions by introducing context information and similarity thresholds. Second data classification based on similarity thresholds enables detection of patterns across non-adjacent positions, adding a temporal/spatial dimension to redundancy detection. This dimensional expansion improves compression rate by capturing broader patterns while organizing complexity through defined classification criteria.
3Adaptability or versatility
If multiple data types are classified and compressed differently, then compression flexibility improves, but the compression system becomes more complex
Solution Approach 1:
The patent changes key parameters to enable flexible compression: introducing similarity thresholds as adjustable parameters, defining context information windows, and setting repetition detection parameters. By making these parameters configurable and data-type-specific, the system achieves high compression flexibility for different data patterns while managing complexity through parameterized approaches rather than hard-coded complex logic for each data type.
Data Source
AI summary
Embodiments of this application disclose a data compression method and apparatus, a computer device, and a storage medium, and belong to the field of data storage technologies. The method includes: obtaining a to-be-compressed data stream, where the data stream includes a plurality of pieces of data; determining a plurality of types of data from the plurality of pieces of data based on context information of each of the plurality of pieces of data, where each of the plurality of types of data includes at least one of the plurality of pieces of data, or the context information of each piece of data indicates data before and/or after corresponding data in the data stream; and compressing at least one of the plurality of types of data.