Pattern-Based String Compression Using Radix Tree Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in efficiently compressing large volumes of data while maintaining high read and write speeds, particularly in cloud-based systems where storage costs are high and existing compression techniques are not suitable for high-speed operations.
Innovation Solution
The method involves generating patterns in strings to compress data by identifying common prefix, expression, and suffix portions, using radix trees to represent strings and their reversals, and merging nodes to reduce storage size, allowing for efficient data compression and decompression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If advanced data storage systems and cloud storage services are used to store massive quantities of data, then data storage capacity is improved, but storage costs and computational power requirements increase
Solution Approach 1:
The patent extracts and stores only the unique portions of strings rather than complete strings. By identifying common patterns and storing only the distinctive parts, the system reduces storage requirements while maintaining data integrity and accessibility.
Solution Approach 2:
The patent changes the representation parameter of stored data from complete strings to pattern-based unique portions. This parameter transformation enables significant space reduction while preserving the essential information needed for data retrieval and processing.
2Quantity of substance
If compression techniques are applied to reduce data size, then storage requirements are reduced, but read and write speeds are impacted
Solution Approach 1:
The patent performs preliminary pattern identification and string analysis during the data ingestion phase. By pre-processing data to identify common patterns and extract unique portions upfront, the system minimizes computational overhead during read and write operations, thereby maintaining high speeds.
Solution Approach 2:
The patent creates a compressed representation (copy) of the original data that preserves essential information. The pattern-based unique portions serve as efficient copies that can be stored and retrieved quickly, reducing the need to handle full-size strings during operations.
3Quantity of substance
If pattern matching and radix trees are used to identify common portions of strings, then compression efficiency is improved, but computational complexity increases
Solution Approach 1:
The patent merges multiple pattern matching operations into a unified radix tree structure. By combining common prefix identification, common suffix identification, and unique portion extraction into a single data structure, the system reduces computational complexity while maintaining compression efficiency.
Solution Approach 2:
The radix tree structure serves multiple functions simultaneously: it identifies common patterns, extracts unique portions, and enables efficient data retrieval. This multi-functional approach reduces the need for separate processing steps, thereby lowering overall computational complexity.
Data Source
AI summary
The disclosure relates to compressing strings by reducing the number of string characters that are stored. For example, a system may generate a first radix tree for a set of strings and a second radix tree for a reverse of each of the set of strings. The system may merge nodes of the first radix tree and/or second radix tree based on a tuning parameter. The system may identify, based on the first radix tree, beginning portions of at least two strings that match and identify, based on the second radix tree, ending portions of at least two strings that match. The system may use the matching beginning portions, the unique portions, and/or the matching ending portions to generate a pattern that matches the two or more strings. The system may store the two or more strings in association with the generated pattern without their matching beginning and/or ending portions.