Parallel Bloom Filter Circuit for Approximate Pattern Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing approximate pattern matching techniques are inefficient and resource-intensive when dealing with large sets of patterns, particularly in applications like computational biology and computer communications, due to the need for extensive computer resources and complex filtering processes.
Innovation Solution
A scalable filtering circuit and reduction stage utilizing parallel Bloom filters or Bloom filter arrays with a single hash key generator to detect potential pattern matches, reducing the number of patterns for verification, and employing a verification stage with an approximate match engine to confirm matches, allowing for efficient processing of large data streams.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional filtering techniques are used for approximate pattern matching, then the matching accuracy is maintained, but the processing speed and resource efficiency deteriorate when dealing with large sets of patterns
Solution Approach 1:
The patent divides the pattern matching process into distinct stages: a filtering stage that quickly eliminates non-matching patterns, and a verification stage that performs detailed approximate matching only on candidate patterns. This segmentation allows the system to maintain accuracy while improving processing speed by avoiding exhaustive verification of all patterns.
Solution Approach 2:
The patent introduces an intermediary filtering mechanism that sits between the input data stream and the verification stage. This filter acts as a mediator that pre-processes the data and identifies candidate matches before they reach the computationally intensive verification stage, thereby reducing the overall complexity and resource requirements.
2Reliability
If exhaustive verification of all patterns is performed, then matching accuracy is ensured, but processing time and computational resources increase significantly
Solution Approach 1:
The patent performs preliminary filtering actions before the main verification process. By pre-processing the data stream and identifying candidate patterns that are likely to match, the system prepares the input for the verification stage in advance, ensuring that only relevant patterns undergo time-consuming verification.
Solution Approach 2:
The patent applies partial verification by performing exhaustive checking only on a subset of patterns that pass the filtering stage. Instead of verifying all patterns completely, the system performs partial verification on candidates and full verification only when necessary, reducing overall verification time while maintaining reliability for critical matches.
3Productivity
If multiple parallel filter mechanisms are used to detect patterns, then processing capacity increases, but the number of false positive matches increases
Solution Approach 1:
The patent implements feedback mechanisms where the results from parallel filter mechanisms are analyzed and used to adjust subsequent filtering operations. The system learns from false positives generated by parallel filters and refines its filtering criteria, thereby maintaining high processing capacity while progressively improving match detection accuracy.
Data Source
AI summary
A system and method for inspecting a data stream for data segments matching one or more patterns each having a predetermined allowable error, which includes filtering a data stream for a plurality of patterns of symbol combinations with a plurality of parallel filter mechanisms, detecting a plurality of potential pattern piece matches, identifying a plurality of potentially matching patterns, reducing the identified plurality of potentially matching patterns to a set of potentially matching patterns with a reduction stage, providing associated data and the reduced set of potentially matching patterns, each having an associated allowable error, to a verification stage, and verifying presence of a pattern match in the data stream from the plurality of patterns of symbol combinations and associated allowable errors with the verification stage.