Data Compression Using Input Blocks as Delta Dictionaries
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The selection of a similarity candidate for delta-compression is computationally expensive, particularly in applications requiring near-real-time latencies, leading to inefficiencies in data storage and transmission.
Innovation Solution
An alternated compression dictionary method is introduced, allowing the input data block to be used as a compression dictionary, with decision-making based on reference information such as compression ratio, recursion depth, and CPU cycles to optimize computational efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional delta-compression dictionary selection is used, then data compression is achieved, but computational cost increases significantly
Solution Approach 1:
Instead of selecting a dictionary from previously processed data blocks, the patent inverts the approach by using the current input data block itself as the compression dictionary. This eliminates the computationally expensive dictionary selection process while maintaining compression functionality, as the input block is directly used to compress subsequent blocks without needing to search for similar candidates.
Solution Approach 2:
The input data block serves dual purposes: it is both the data to be processed and the compression dictionary for subsequent blocks. This self-service approach allows the system to use its own input data as the reference for compression, eliminating the need for external dictionary selection and reducing computational overhead.
2Loss of time
If dictionary selection is performed for near-real-time applications, then compression is achieved, but latency increases
Solution Approach 1:
The patent performs preliminary action by designating the current input block as the dictionary for the next compression operation before any dictionary selection would be needed. This proactive approach eliminates the time-consuming search process, as the dictionary is already determined (the input block itself) rather than requiring real-time selection from historical data.
Solution Approach 2:
The patent inverts the traditional sequence where a past block is selected as dictionary for current compression. Instead, the current input block becomes the dictionary for future compression, reversing the temporal dependency and eliminating the need to look back through historical blocks, thus reducing latency.
3Loss of energy
If processed data blocks are used as compression dictionaries, then compression ratio is improved, but device complexity increases
Solution Approach 1:
The patent extracts and eliminates the complex dictionary selection mechanism from the compression system. By removing the need to search through and select from multiple processed data blocks, the system simplifies its structure while maintaining compression functionality through the direct use of input blocks as dictionaries.
Solution Approach 2:
The input data block serves multiple functions simultaneously: it is the data being processed, the output of the compression operation, and the compression dictionary for subsequent blocks. This multi-functionality reduces the need for separate dictionary management structures, thereby reducing device complexity.
Data Source
AI summary
This application provides a method that includes: obtaining an input data block; and processing the input data block by selecting one of alternatives: delta-compressing one or more target data blocks with the input data block used as a compression dictionary, where each target data block is one of processed data blocks; delta-compressing or deduplicating the input data block by using one of processed blocks as a dictionary or as a reference block for deduplication; independently compressing the input data block; or passing the input data block uncompressed for further processing.