Data Normalization System Using Phonetic Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Integrating data files listing user names can lead to issues due to varying formats, errors, and inconsistencies, resulting in duplicate records or missed data, as the distinction between given names and surnames may be incorrectly swapped or not discernible.
Innovation Solution
A data normalization system utilizes metaphone strings and algorithms to convert user names into phonetic spellings, comparing these strings against a name index to determine confidence scores for correct ordering, and uses character patterns to identify surnames, ensuring accurate formatting and minimizing errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data files are integrated without normalization, then integration speed is maintained, but data quality and accuracy deteriorate due to varying formats and duplicate records
Solution Approach 1:
The patent replaces manual data normalization processes with automated phonetic algorithms (metaphone strings) that convert names into phonetic representations. This substitution of mechanical/manual operations with algorithmic processing resolves the contradiction by maintaining data quality through systematic phonetic matching while reducing the complexity of manual data review and verification.
Solution Approach 2:
The patent transforms names from their original spelling form into phonetic parameter representations (metaphone strings). By changing the parameter from literal spelling to phonetic encoding, the system can reliably match variant spellings and identify duplicates without being affected by spelling inconsistencies, thus improving data quality while automating the process.
2Measurement precision
If phonetic algorithms are used to normalize names, then name matching accuracy is improved, but processing time increases
Solution Approach 1:
The patent performs phonetic conversion of all names into metaphone strings during the initial data import and normalization phase. By completing this preprocessing action beforehand, the system establishes accurate phonetic representations that enable rapid matching during subsequent operations, thus improving name matching accuracy while minimizing time loss during actual data processing and querying.
3Quantity of substance
If confidence thresholds are lowered to include more names, then data completeness is improved, but accuracy deteriorates due to potential false matches
Solution Approach 1:
The patent implements a confidence scoring mechanism that provides feedback on the reliability of each name match. The system calculates confidence scores based on phonetic string matching and allows users to set thresholds that balance completeness and accuracy. This feedback loop enables users to adjust the trade-off between including more names (completeness) and ensuring high match quality (accuracy) based on their specific needs.
Data Source
AI summary
A data normalization system receives a first string and a second string that are ordered according to an initial string ordering. The data normalization system analyzes, the first string and the second string based on a list of known character sets included in surnames, yielding an analysis, and determines, based on the analysis, that a set of characters in the second string matches a known character set included in the list of known character sets included in surnames. In response to determining that the set of characters in the second string matches a known character set included in the list of known character sets included in surname, the data normalization system orders the first string and the second string according to an updated string ordering.