Adaptive Entity Recognition for Fuzzy String Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current database systems face challenges in accurately matching and updating records due to stringent matching thresholds and the inability to recognize fuzzy matches in strings, particularly in parsing street addresses and person names, which leads to missed matches or false positives.
Innovation Solution
The implementation of an adaptive entity recognition system that uses a combination of regular expressions and statistical methods to parse strings into tokens, employing probabilistic matching and heuristic decision-making to recognize entities, allowing for flexible pattern recognition and fuzzy matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a database system uses a matcher such as Edit Distance to match strings with a stringent matching threshold, then false matches are reduced, but good matches are missed
Solution Approach 1:
The patent segments strings into parsed entities (tokens) with different importance weights rather than treating them as uniform sequences. This allows the system to focus matching on critical entities while being more flexible with less important ones, resolving the contradiction between strict matching and capturing variations.
Solution Approach 2:
The patent changes the matching parameters by applying different algorithms and thresholds to different parsed entities based on their importance. Critical entities use stricter matching while less important entities allow more flexibility, enabling both high accuracy and completeness simultaneously.
2Device complexity
If a database system uses regular string comparison algorithms like Edit Distance or Jaro-Winkler distance to directly match strings, then the matching process is simple, but the matching accuracy decreases for strings representing the same entity
Solution Approach 1:
The patent performs preliminary parsing of strings into structured entities before matching. This preprocessing step transforms unstructured strings into weighted components, enabling more accurate comparison while maintaining computational efficiency through organized data structures.
Solution Approach 2:
The patent transforms the matching approach by changing from uniform string distance metrics to entity-weighted comparison. Different entities contribute differently to the overall match score based on their importance, significantly improving accuracy for entities like person names while keeping the process manageable.
3Reliability
If a database system uses grammar-based parsing methods like regular expressions, then the parsing is strict and deterministic, but approximate matches are rejected
Solution Approach 1:
The patent introduces dynamic weighting to parsed entities, allowing the system to adapt its strictness based on entity importance. Critical entities maintain strict matching while less important entities allow approximate matches, combining consistency with adaptability.
Solution Approach 2:
The patent changes the rigidity parameter of parsing by assigning different importance weights to different entities. This allows the system to be deterministic for critical entities while flexible for others, resolving the contradiction between strict parsing and fuzzy matching.
4Adaptability or versatility
If a database system uses statistical methods to parse strings, then fuzziness in data is handled, but a significant amount of training data and parameters are required
Solution Approach 1:
The patent applies statistical methods selectively only to entities where fuzziness is expected and important, rather than to all strings. This partial application reduces the training data requirement while still handling fuzzy cases effectively.
Solution Approach 2:
The patent applies different parsing strategies to different parts of the string based on entity type and importance. Statistical methods are used locally where needed for fuzzy matching, while deterministic methods handle clear cases, reducing overall computational and data requirements.
Data Source
AI summary
A system receives a record which includes a string and separates the string into a number of tokens, including a token and another token. The system identifies a pattern that includes an entity, another entity, and a number of entities that equals the number of tokens, and another pattern that includes the same number of entities as the number of tokens. The system determines a combined probability that combines a probability based on the number of entries in the entity's dictionary which stores the token, and another probability based on a number of character types in the other entity that match characters in the other token. If the combined probability associated with the pattern is greater than another combined probability associated with the other pattern, the system matches the record to a system record based on recognizing the token as the entity and the other token as the other entity.