Dictionary-Index Compression Using Selected Symbol Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing lossless data compression algorithms require substantial computational resources and storage capacity due to the need to calculate and store hash functions for every dictionary entry and new data, which hampers efficiency and storage requirements.
Innovation Solution
A method that selects a subset of input data symbols and generates a dictionary-index value, matching ranges are identified and stored as pointers and lengths, reducing the number of updates and comparisons needed, using a hash function only for every Nth byte and updating the dictionary-index for selected symbols, thereby reducing computational and storage demands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hash functions are calculated and stored for every dictionary entry and new data, then compression accuracy is improved, but computational resources and storage capacity are substantially increased
Solution Approach 1:
The patent divides the data stream into blocks and processes only selected symbols within each block (e.g., every Nth symbol) rather than every symbol. This segmentation approach maintains compression accuracy for representative symbols while substantially reducing the computational burden of hash function calculations and dictionary updates.
Solution Approach 2:
The patent applies partial action by performing hash function calculations and dictionary updates only for selected symbols rather than all symbols in the data stream. This partial processing maintains sufficient compression accuracy while significantly reducing computational resource consumption and storage requirements.
2Measurement precision
If hash functions are calculated and stored for every dictionary entry and new data, then compression accuracy is improved, but storage capacity is substantially increased
Solution Approach 1:
The patent segments the dictionary update process to only include selected symbols from the data stream. By updating the dictionary-index only for representative symbols (e.g., every Nth symbol) rather than all symbols, the storage capacity required for the dictionary is substantially reduced while maintaining compression accuracy for the segmented data blocks.
Solution Approach 2:
The patent applies partial action by storing hash functions and dictionary entries only for selected symbols rather than all symbols. This partial storage approach reduces the quantity of data that must be stored in the dictionary-index while maintaining sufficient compression accuracy through representative sampling.
3Productivity
If all input data symbols are processed for compression, then compression ratio is improved, but the number of comparisons and processing time are increased
Solution Approach 1:
The patent segments the input data stream into blocks and processes only selected symbols within each block for compression operations. This segmentation reduces the number of comparisons required while maintaining compression effectiveness for the segmented data, thereby reducing processing time without completely sacrificing compression ratio.
Solution Approach 2:
The patent applies partial action by processing selected symbols rather than all symbols in the input data stream. This partial processing approach reduces the number of comparisons and processing time while maintaining sufficient compression ratio through representative symbol selection and matching.
Data Source
AI summary
A compression method applies a selection rule to input symbols and generates a reduced partial set of symbols. The partial set is checked against a dictionary-index for a match. A match identifies a range of matching symbols in a dictionary. The length of the matching range is iteratively increased by checking previous and next symbols in the input data and the dictionary until a matching range length meets a threshold limit or the length of the matching range cannot be increased further. Compressed data corresponding to the input symbols is provided where input symbols are copied over and symbols in a matched range of data are replaced with a representation of their corresponding start location and length in the dictionary.