Content-Based Data Compression for Mixed Telemetry Blocks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques often fail to optimize compression by applying a single method to an entire data set, missing opportunities for more effective reduction in size by using different techniques for varying data portions.
Innovation Solution
A method and system for content-based dynamic data compression that involves a training stage to extract statistical features from data sets, an installation stage to set up the compression parameters, a working stage where individual data blocks are compressed using optimally selected techniques based on those features, and a consumption stage for decompression, allowing for variable compression techniques across a data set.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single compression technique is applied to the entire data set, then the compression process is simple and fast, but the compression ratio is suboptimal because it cannot adapt to varying data portions
Solution Approach 1:
The patent divides the data set into multiple data blocks and applies different compression techniques to different blocks based on their characteristics. This segmentation allows the system to optimize compression ratio for each block while maintaining manageable complexity through automated selection.
Solution Approach 2:
The patent implements dynamic compression by selecting compression techniques based on the actual content of each data block rather than using a static approach for the entire data set. This dynamic adaptation enables optimal compression ratios while the automation keeps complexity manageable.
2Productivity
If different compression techniques are used for different data blocks, then the compression ratio is optimized, but the complexity of selecting and managing multiple techniques increases
Solution Approach 1:
The patent enables the compression system to automatically analyze each data block and select the most appropriate compression technique without requiring manual intervention. This self-service approach optimizes compression ratios while maintaining ease of operation through automation.
Solution Approach 2:
The patent changes the parameter of compression technique selection based on the characteristics of each data block. By dynamically adjusting which compression technique is applied to each block, the system achieves optimal compression ratios while the automated parameter selection keeps the process simple to operate.
3Quantity of substance
If compression techniques are dynamically selected for each data block, then the smallest compressed data set size is achieved, but the processing time and computational resources increase
Solution Approach 1:
The patent applies partial analysis to each data block, focusing on key characteristics that determine the best compression technique rather than performing exhaustive analysis. This approach achieves minimal compressed data set size while limiting the time and computational resources required by applying only necessary analysis to each block.
Data Source
AI summary
The present disclosure describes methods and systems that provide for content based dynamic data compression. During a training stage a plurality of training sets are compressed using multiple data compression techniques, and each training data set is associated with a most effective compression technique. Statistical features can be extracted from each of the training data sets with consideration of the most effective compression techniques and are saved as a compression parameters file. During a working stage, individual data blocks within a telemetry data set are compressed using a data compression technique selected for the data block in light of the statistical feature presented in the compression parameters file, and the compressed data blocks are written to a compressed data set, along with encoding tags that identify the compression technique used on each data block. During a consumption stage, the compressed data blocks are decompressed using techniques identified by the encoding tags.