VBMI Teddy Algorithm 128-Bit Boundary False Positives
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Hyperscan's Teddy algorithm suffers from front-end accuracy loss, leading to false positives at 128-bit boundaries due to padding zeros introduced by VPSLLDQ instructions during shifting.
Innovation Solution
The VBMI Teddy algorithm employs a 512-bit SIMD instruction to perform a shift operation based on a control vector, using the VPERMB instruction to prevent false positives at 128-bit boundaries and improve front-end accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If VPSLLDQ instructions are used for shifting operations in the front-end, then processing speed is improved, but false positives occur at 128-bit boundaries due to padding zeros
Solution Approach 1:
The patent extracts and eliminates the harmful padding zeros introduced by VPSLLDQ instructions at 128-bit boundaries. By identifying and removing these spurious zero values that cause false positives, the solution maintains the high-speed processing benefit while correcting the accuracy degradation caused by the shifting operation's side effect.
Solution Approach 2:
The patent converts the harmful effect of padding zeros (which cause false positives) into a beneficial filtering process. The solution deliberately identifies these padding-induced false positives and applies correction logic to eliminate them, thereby transforming the originally harmful artifact into an identifiable and correctable condition that improves overall matching accuracy.
2Device complexity
If conventional Teddy algorithm is used, then implementation simplicity is maintained, but throughput is limited compared to VBMI Teddy
Solution Approach 1:
The patent changes the instruction set parameter from conventional SIMD instructions to VBMI (Vector Byte Manipulation Instructions) specifically VPERMB for permutation and VPSLLDQ for shifting. This parameter change in the instruction level enables more efficient byte-level operations and parallel processing, achieving 5.15× throughput improvement while managing complexity through systematic application of these specialized instructions.
Data Source
AI summary
Methods, software, and apparatus for implementing a high-performance multi-literal matching algorithm. Under aspects of a method, multi-literal matching is used during front-end processing to identify match candidates based on suffix patterns. An extended SHIFT-OR algorithm is performed using the chunk of data as an input to identify match candidates for the suffix patterns, wherein the extended SHIFT-OR algorithm shifts match indicia in the plurality of rows across predetermined bit boundaries and aligns the match indicia corresponding to target suffixes into single columns. Match candidates are indicated when OR'ed column values yield a ‘0’. Match verification for one or more search strings containing the suffix patterns is then performed by a back-end. The scheme eliminates false positives at the predetermined bit boundaries, providing substantial improvement in front-end accuracy and performance.