Suffix Tree Memory Optimization via Block Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Suffix tree structures require significant memory for storing and processing large binary data strings, leading to performance degradation due to high space complexity, and existing solutions either do not adequately address this issue or result in loss of functionality through compression techniques.
Innovation Solution
A method of storing data strings in a structured form by dividing them into subsets of predetermined length, arranging these subsets into a tree structure with prefix tables and storing the subsets, tree structure, and postfix, allowing for efficient pattern matching by dividing search strings into sub-searches and comparing them against the stored data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a suffix tree is constructed to support pattern matching on large binary data strings, then search functionality is improved, but memory space requirements increase significantly
Solution Approach 1:
The patent divides binary data into fixed-size blocks (e.g., 8-bit bytes) and processes them sequentially. Instead of building a traditional suffix tree on individual bits, the method segments the binary string into manageable units, creating a compressed representation that maintains pattern matching capability while reducing memory requirements. The segmented approach allows the data to be processed in chunks rather than requiring the entire dataset to be held in memory simultaneously.
Solution Approach 2:
The patent changes the fundamental parameter of data representation from bit-level operations to byte-level or larger unit operations. By transforming the data processing paradigm from operating on individual bits to operating on fixed-size blocks (bytes, words, or custom-sized units), the memory requirements are reduced while maintaining the ability to perform pattern matching. This parameter change allows the use of more efficient data structures that operate at a higher level of abstraction.
2Quantity of substance
If compression techniques are applied to reduce space requirements for storing the tree structure, then memory usage is reduced, but functionality of the suffix tree is lost
Solution Approach 1:
The patent performs preliminary segmentation and organization of the binary data into fixed-size blocks before pattern matching operations. By pre-processing the data into a compressed block-based representation, the system eliminates the need for complex tree structures while preserving pattern matching functionality. The preliminary organization of data into manageable units allows for efficient searching without requiring the full functionality of a traditional suffix tree.
Solution Approach 2:
The patent extracts only the essential elements needed for pattern matching from the traditional suffix tree structure. Instead of maintaining the complete tree with all its nodes, edges, and auxiliary structures, the method extracts the core functionality by working directly on segmented blocks of the original data. This extraction approach removes unnecessary structural overhead while preserving the ability to perform pattern matching operations.
3Measurement precision
If binary data is processed at the bit level to support pattern matching, then search accuracy is maintained, but processing time and space consumption increase
Solution Approach 1:
The patent merges multiple bit-level operations into single block-level operations. By combining adjacent bits into fixed-size blocks (such as bytes or larger units), the system performs pattern matching on consolidated data units rather than processing each bit individually. This merging approach maintains matching accuracy because the block boundaries are chosen to preserve the integrity of pattern boundaries, while significantly reducing the number of operations required.
Solution Approach 2:
The patent creates a compressed copy of the binary data in block-based format that can be used for pattern matching without requiring the original bit-level representation to be actively processed. By working with this compressed block-based copy, the system maintains the ability to perform accurate pattern matching while avoiding the time and space costs of manipulating the full bit-level data structure during search operations.
Data Source
AI summary
This invention relates to methods for storing and searching data. Embodiments of the invention make use of suffix trees to support binary pattern matching. Embodiments of the invention can be shown to have comparable search speeds to searches of known suffix trees, but are advantageous in that they have lower memory usage requirements which is important in large data environments.