Delimiter-Based Data Compression for Distant Repeated Strings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques, such as the sliding dictionary method, face efficiency issues when compressing data with repeated character strings occurring at distant positions, leading to increased compressed data size and reduced compression efficiency.
Innovation Solution
A data compression device that extracts delimiter characters and character strings from the data, rearranges them to identify occurrence orders, and performs compression using the LZMA algorithm on intermediate data where repeated character strings are likely to occur near each other, improving compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the same character string repeatedly occurs at relatively distant positions in compression target data, then the original data structure is preserved, but the size of compressed data increases and compression efficiency deteriorates
Solution Approach 1:
The patent segments the compression target data into multiple lines based on newline characters, and further segments each line into fields using delimiter characters. This hierarchical segmentation allows the compression algorithm to process smaller, more manageable units independently, enabling better compression ratios when repeated strings appear at distant positions across different lines or fields.
Solution Approach 2:
The patent transforms the one-dimensional compression problem into a two-dimensional structure by organizing data into lines and fields with hierarchical delimiters. This dimensional transformation allows the LZMA algorithm to exploit repetition patterns more effectively by searching within and across the structured hierarchy, improving compression efficiency for distant repetitions.
2Device complexity
If conventional compression algorithms are used on raw data, then the processing is simple, but the compression efficiency deteriorates when repeated character strings occur at distant positions
Solution Approach 1:
The patent performs preliminary segmentation of the compression target data into lines and fields before applying the LZMA compression algorithm. By pre-organizing the data structure with delimiter characters and hierarchical segmentation, the compression algorithm can operate more efficiently on the structured intermediate data, achieving better compression ratios without significantly increasing overall processing complexity.
Data Source
AI summary
A data compression device includes a processor configured to extract delimiter characters and character strings from compression target data. The character strings each are sandwiched between two of the delimiter characters in the compression target data. The processor is configured to generate first data by arranging the delimiter characters to allow an order in which the delimiter characters occur in the compression target data to be identified. The processor is configured to generate second data by arranging the character strings for each type of combination of two of the delimiter characters, which sandwich each of the character strings in the compression target data, to allow an order in which the character strings occur in the compression target data to be identified. The processor is configured to perform compression on the first data and the second data. The processor is configured to output a result of the compression.