Suffix Tree Binary Data Segmentation for Memory Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Suffix tree structures require significant memory for large binary data strings, leading to performance degradation, and existing solutions for reducing memory usage often compromise functionality or are inefficient.
Innovation Solution
The method involves dividing binary data strings into subsets of predetermined length, creating a suffix tree structure with prefix tables and storing subsets, prefix tables, and postfix information, which allows for efficient pattern matching by comparing sub-searches to the stored data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a suffix tree structure is used to store binary data strings, then pattern matching capability is improved, but memory consumption increases significantly
Solution Approach 1:
The binary data string is divided into multiple segments of fixed length (e.g., 8 bits each). Each segment is processed and stored separately in the suffix tree structure, rather than treating the entire binary string as a single unit. This segmentation reduces the memory footprint while preserving pattern matching capabilities.
Solution Approach 2:
The invention changes the parameter representation by converting binary data into a different format suitable for tree storage. Specifically, binary segments are transformed into numerical or symbolic representations that can be efficiently stored and searched in the suffix tree, reducing memory requirements while maintaining search functionality.
2Quantity of substance
If the data string length increases, then more data can be stored, but construction time and memory requirements increase
Solution Approach 1:
The data string is segmented into fixed-length chunks before being inserted into the suffix tree. This pre-segmentation allows for more efficient construction by processing smaller units independently, reducing the overall construction time while enabling storage of longer data strings through batched insertion of segments.
Solution Approach 2:
The binary data is pre-processed and segmented into fixed-length units before being fed into the suffix tree construction algorithm. This preliminary action of segmentation and formatting prepares the data in an optimal structure for efficient tree construction, reducing the computational burden during the indexing phase.
3Measurement precision
If binary data is processed at the bit level, then precise pattern matching is achieved, but processing complexity and space requirements increase
Solution Approach 1:
The invention changes the processing parameter from individual bits to fixed-length binary segments (e.g., bytes). This parameter change maintains pattern matching precision by preserving the complete segment information in the suffix tree, while reducing processing complexity by working with larger, more manageable units that can be handled more efficiently by standard algorithms.
Data Source
Figure 1
Figure 2
AI summary
This invention relates to methods for storing and searching data. Embodiments of the invention make use of suffix trees to support binary pattern matching. Embodiments of the invention can be shown to have comparable search speeds to searches of known suffix trees, but are advantageous in that they have lower memory usage requirements which is important in large data environments.