Fuzzy Substring Search Using Character and Hop Index Values
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing fuzzy substring search methods often return false positives and fail to capture expected results due to limitations in handling misspelled inputs and longer strings, particularly in NVARCHAR columns, and lack flexibility in adjusting search parameters for optimal performance.
Innovation Solution
The method determines tokens from an input string, calculates similarity scores based on character and hop index values, and identifies optimal paths within reference strings to reduce false positives and support longer strings up to 5000 characters, with configurable search parameters for balancing result exhaustiveness and response time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If n-gram matching is used for fuzzy search, then search speed is improved and results are returned quickly, but false positives increase and expected results are not captured
Solution Approach 1:
The patent segments the search process into two distinct phases: (1) n-gram matching for rapid candidate generation, and (2) dynamic programming-based path finding for precise similarity verification. This segmentation allows the system to leverage the speed of n-gram matching while eliminating its false positives through the more accurate second phase verification.
Solution Approach 2:
The patent introduces an intermediary verification mechanism using dynamic programming to bridge the gap between fast but inaccurate n-gram matching and precise but slow exact matching. This intermediary phase filters false positives and validates true matches, combining the advantages of both approaches.
2Measurement precision
If traditional fuzzy search methods are used, then short strings are searched effectively, but longer strings up to 5000 characters are not supported
Solution Approach 1:
The patent creates a universal search algorithm that handles both short and long strings (up to 5000 characters) using the same dynamic programming approach. The system maintains consistent search precision across different string lengths by using character index values and hop index values that scale appropriately with input size.
Solution Approach 2:
The patent extends the search space by introducing a new dimension for handling long strings through configurable parameters such as maximum token length, maximum hops, and character error thresholds. This allows the system to adapt to varying string lengths while maintaining search effectiveness.
3Device complexity
If search parameters are fixed, then algorithm simplicity is maintained, but flexibility in adjusting parameters for optimal performance is lost
Solution Approach 1:
The patent implements dynamic search parameters that can be configured based on specific search requirements. Parameters such as maximum token length, maximum hops, character error thresholds, and token score thresholds are made adjustable, allowing the system to adapt to different performance needs while maintaining a relatively simple core algorithm structure.
Data Source
AI summary
The present disclosure describes methods, systems, and computer program products for performing fuzzy substring search. One computer-implemented method includes determining, by operation of a hardware processor, one or more tokens of an input string received from a client application, identifying, from a database, a reference string based on the one or more tokens of the input string, determining a plurality of character index values of the reference string, each character index value indicating a position of a character in the reference string, determining one or more hop index values of the input string, each hop index value indicating a position of a separator in the input string, and calculating a similarity score of the reference string relative to the input string based on the plurality of character index values of the reference string and the one or more hop index values of the input string.