Modified Levenshtein Distance Algorithm for OCR Code Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The accuracy of mapping optical character recognition (OCR) text strings to categories in a coding dictionary is compromised by errors in handwritten text recognition, leading to inaccurate Levenshtein distances and improper code assignments in large surveys.
Innovation Solution
The implementation of a modified Levenshtein distance algorithm that adjusts for common OCR errors using penalty matrices and considers multiple alternative character choices, dynamically varying penalties for substitutions, insertions, and deletions, and weighting based on character characteristics and confidence levels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the standard Levenshtein distance algorithm is used to compare OCR text strings with coding dictionary entries, then the comparison process is simple and fast, but the accuracy of code assignment deteriorates due to OCR errors
Solution Approach 1:
The patent applies different penalty weights to different types of character errors based on their likelihood in OCR. Common OCR errors like substituting '1' for 'l' or 'O' for '0' receive lower penalties, while rare substitutions receive higher penalties. This local differentiation of error treatment improves accuracy without uniformly increasing complexity.
Solution Approach 2:
The patent changes the parameters of the Levenshtein distance algorithm by introducing weighted penalties for substitutions, insertions, and deletions. Instead of uniform cost, each operation type has dynamically assigned weights based on error probability models, transforming the standard algorithm into a context-aware matching system.
2Reliability
If multiple alternative character choices from OCR are considered to improve matching accuracy, then the reliability of code assignment increases, but the computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary filtering by comparing the primary OCR recognition result against the coding dictionary first. Only when the weighted Levenshtein distance exceeds a threshold does the system then evaluate alternative character choices. This preliminary action avoids unnecessary computation with alternatives when the primary match is sufficient.
Solution Approach 2:
The patent evaluates alternatives partially by considering only the top N most likely alternative characters from OCR rather than all possible alternatives. This partial evaluation provides sufficient reliability improvement while constraining processing time by limiting the search space of alternative combinations.
Data Source
AI summary
Methods and systems of mapping of an optical character recognition (OCR) text string to a code included in a coding dictionary by supplementing the Levenshtein Distance Algorithm (LDA) with additional information in the form of adjustments based on particular character substitutions, insertions and deletions together with weighting based on multiple alternatives for the OCR text string. In one embodiment, an OCR text string mapping method (100) includes receiving (110) an OCR text string, comparing (120) it with selected text strings from a coding dictionary, computing (130) modified Levenshtein distances associated with the comparisons by determining (140) substitution penalties, determining (150) insertion penalties, determining (160) deletion penalties and combining (170) the penalties, selecting (180) the best matching text string from the coding dictionary based on the modified Levenshtein distances, determining (190) whether a maximum threshold distance is met, and assigning (200) a code associated with the best matching text string to the OCR text string when met, and assigning (210) a null or no code when not met.