Symbol-Pair Compression With Non-Adjacent Distance Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression techniques are inefficient for large datasets ('Big Data') and do not effectively track the frequency of symbol pairs, limiting their scalability and analytical capabilities.
Innovation Solution
A computerized method that identifies and counts unique symbol pairs in documents, replacing them with corresponding symbols and indicating their distance, allowing for improved compression and analysis by creating a symbol dictionary that prioritizes frequent pairs and uses recursive cycles to achieve higher compression ratios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional data compression techniques are used, then the compression process is simple, but the compression efficiency is low and scalability is limited
Solution Approach 1:
The patent segments the data processing into distinct phases: identifying symbol pairs, counting their frequencies, selecting top pairs based on frequency thresholds, and replacing them with compressed representations. This segmentation allows the system to handle large datasets efficiently by breaking down the complex compression task into manageable steps, thereby improving compression efficiency without overwhelming system complexity.
Solution Approach 2:
The patent performs preliminary actions by first identifying and counting all symbol pair frequencies before actual compression occurs. This preliminary frequency analysis creates a roadmap for compression, allowing the system to prioritize which symbol pairs to compress based on their frequency of occurrence. This approach ensures that the most beneficial compressions are performed first, maximizing compression efficiency while maintaining manageable process complexity.
2Adaptability or versatility
If only adjacent symbol pairs are compressed, then the compression method is simple, but the analytical capabilities are limited
Solution Approach 1:
The patent extends the compression method from one dimension (adjacent symbol pairs only) to multiple dimensions by incorporating non-adjacent symbol pairs with varying distance parameters. This dimensional expansion allows the system to capture long-range dependencies and patterns in the data that adjacent-pair-only methods miss, significantly enhancing analytical capabilities while managing complexity through systematic distance-based categorization.
3Adaptability or versatility
If frequency tracking is not implemented, then the processing is faster, but the analytical capabilities are reduced
Solution Approach 1:
The patent performs frequency counting as a preliminary action before compression, organizing all symbol pair frequencies in advance. This preliminary organization allows the system to quickly reference and select the most frequent pairs during compression without re-counting frequencies repeatedly. The upfront investment in frequency analysis pays dividends during the compression phase, maintaining analytical capabilities while minimizing processing time loss.
4Productivity
If all symbol pairs are replaced, then the compression ratio is maximized, but the loss of information increases
Solution Approach 1:
The patent introduces frequency thresholds and distance caps as controllable parameters to regulate compression. By adjusting these parameters, the system can optimize the balance between compression ratio and information preservation. Frequent symbol pairs within acceptable distance ranges are compressed, while rare or distant pairs are preserved in their original form, ensuring that compression does not exceed the point of significant information loss.
Data Source
AI summary
A computerized method of compressing symbolic information organized into a plurality of documents, each document having a plurality of symbols, includes: (i) automatically identifying a plurality of sequential and non-sequential symbol pairs in an input document; (ii) counting the number of appearances of each unique symbol pair; and (iii) producing a compressed document that includes a replacement symbol at each position associated with one of the plurality of symbol pairs, at least one of which corresponds to a non-sequential symbol pair. For each non-sequential pair the compressed document includes corresponding indicia indicating a distance between locations of the non-sequential symbols of the pair in the input document. In some instances the plurality of symbol pairs includes only those pairs of non-sequential symbols for which the distance between locations of the non-sequential symbols of the pair in the input document is less than a numeric distance cap.