Pattern Matching via Field Concatenation and Boundary Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
As computer memory capacities increase, existing pattern matching techniques become inefficient in searching large amounts of data, leading to increased computational overhead and slower search times.
Innovation Solution
The approach involves treating a concatenation of discrete field values as a single value, using boundary information to determine if a matching pattern occurs within a single discrete value, and employing pattern-matching algorithms like the Boyer-Moore-Horspool or Knuth-Morris-Pratt to accelerate the detection of matching patterns by reducing the need for multiple searches across separate field values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional pattern matching techniques are used to search large amounts of data, then search coverage is comprehensive, but search time increases and computational overhead increases
Solution Approach 1:
The patent segments the search process into two phases: first, a rapid filtering phase using a Bloom filter to identify candidate records that may contain the pattern, and second, a precise verification phase using traditional pattern matching algorithms. This segmentation allows the system to quickly eliminate non-matching records without performing expensive pattern matching operations on them, thereby significantly reducing overall search time while maintaining comprehensive search coverage.
Solution Approach 2:
The patent performs preliminary actions by pre-processing the data into a Bloom filter structure before the actual pattern matching search. The Bloom filter is constructed in advance to encode presence information of patterns, enabling rapid candidate identification. This preliminary setup allows subsequent searches to benefit from optimized candidate selection, reducing the time required for the main search operation.
2Measurement precision
If traditional pattern matching is performed on each field value separately, then accuracy is maintained, but computational overhead increases
Solution Approach 1:
The patent merges multiple field value searches into a single consolidated search operation. Instead of performing separate pattern matching operations on each field value, the system combines the field values and performs a unified search using the Bloom filter approach. This merging reduces the total number of search operations required while maintaining accuracy through the verification phase, thereby reducing computational overhead without sacrificing pattern matching precision.
Solution Approach 2:
The patent introduces a Bloom filter as an intermediary structure between the query pattern and the field values. This intermediary enables efficient candidate identification by encoding pattern presence information, allowing the system to quickly filter out non-matching records before applying the computationally expensive pattern matching algorithms. The Bloom filter acts as a mediator that reduces the workload on the subsequent verification stage.
3Adaptability or versatility
If data is stored in distributed format across multiple fields, then data organization is flexible, but search efficiency decreases
Solution Approach 1:
The patent creates a universal search approach that works effectively regardless of how data is distributed across fields. The Bloom filter construction and search methodology is field-agnostic, meaning it can handle data organized in any distributed format across multiple fields without requiring structural changes to the underlying data organization. This universal approach maintains data organization flexibility while significantly improving search efficiency through the rapid filtering capability.
Data Source
AI summary
Techniques related to accelerated detection of matching patterns are disclosed. In an embodiment, a method may involve, for a single field of a body of records, contiguously storing field values, thereby storing the field values as a concatenation. The method may also involve identifying a match in the concatenation that matches a value pattern specified by a value pattern expression. The value pattern may include a first partial pattern and a second partial pattern. Further, the method may involve determining that the match is contained in a single field value of the body of records. Still further, the method may involve establishing the match as a matching value. The match may be determined to be contained in the single field value based on searching a data structure comprising length information or offset information for a subset of the field values stored as the concatenation.