Multidimensional Data Compression Using Clustered Sparse Dictionaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dictionary-learning approaches for multidimensional data compression require long computation times and are inefficient in terms of memory usage and compression speed, while lacking high-quality reconstruction capabilities.
Innovation Solution
A method involving super pixel generation, clustering, and dictionary training using sparse coding techniques, including super pixel segmentation, clustering, and batch orthogonal matching pursuit (OMP) to achieve efficient compression and high-quality reconstruction of multidimensional data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If dictionary learning based on sparse representation is applied for data compression, then compression effectiveness and storage efficiency are improved, but computation time increases significantly
Solution Approach 1:
The patent segments the training process into multiple iterations where only a subset of training samples is processed in each iteration. The training data is divided into batches, and the dictionary is updated incrementally rather than processing all samples at once, significantly reducing computation time per iteration while maintaining compression effectiveness.
Solution Approach 2:
The patent performs preliminary actions by pre-processing the training data to identify and select representative samples before actual dictionary training. This preliminary selection step reduces the overall dataset size that needs to be processed during compression, thereby reducing computation time while preserving the essential features needed for effective compression.
2Quantity of substance
If existing dictionary-learning approaches are used, then data compression is achieved, but compression speed remains slow
Solution Approach 1:
The patent implements periodic action by using iterative refinement where the compression process is divided into multiple passes. In each pass, the dictionary is trained on a subset of data, then applied to compress the full dataset, with periodic updates to the dictionary based on reconstruction error analysis. This periodic approach accelerates compression speed while maintaining high compression ratios.
Solution Approach 2:
The patent applies partial action by processing only the most significant components of the data during compression rather than attempting to compress every detail. By focusing computational resources on capturing the dominant features and patterns, the system achieves fast compression speed while maintaining acceptable compression ratios for the most important data characteristics.
3Quantity of substance
If multidimensional data is compressed using traditional methods, then storage space is reduced, but reconstruction quality deteriorates
Solution Approach 1:
The patent changes parameters by dynamically adjusting the sparsity constraint and dictionary size based on the specific characteristics of the multidimensional data being compressed. By adapting these parameters to match the data's inherent structure and dimensionality, the system achieves high compression ratios that reduce memory usage while preserving reconstruction quality appropriate for each data type.
Solution Approach 2:
The patent applies local quality by training different dictionaries for different regions or types of multidimensional data. Instead of using a single global dictionary, the system identifies local patterns and characteristics within the data and creates specialized dictionaries for each region, ensuring that reconstruction quality is optimized for local features while achieving overall compression efficiency.
Data Source
AI summary
A method is provided for compressing multidimensional data having a plurality of data point into a compressed file. The method comprises: generating a plurality of super pixels from the multidimensional data; clustering the plurality of super pixels into a plurality of super pixel clusters; using the plurality of super pixel clusters to train a plurality of dictionaries respectively; splitting the multidimensional data into a plurality of data sub-blocks; vectorizing the plurality of data sub-blocks to obtain a plurality of vectorized data sub-blocks; selecting a dictionary for each of the vectorized data sub-blocks; sparse coding each of vectorized data sub-blocks with a corresponding selected dictionary to obtain a compressed representation; and storing the obtained compressed representations and indexes of corresponding selected dictionaries in the compressed file.