Spatial Label Correction for Document Data Classification Errors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated data classification techniques often result in erroneous classifications, particularly for text strings in documents like spreadsheets, due to reliance on text analysis alone, leading to inaccuracies such as misclassifying '35759' as a zip code when it refers to a monetary amount.
Innovation Solution
A machine learning model that utilizes spatial context information, including preceding text strings and their classifications, to correct automated classifications by analyzing spatial context through a spatial correction model trained on historical data, using character-level embeddings and bi-directional LSTM layers to improve classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated classification techniques are used based solely on text content, then classification speed is improved, but classification accuracy deteriorates
Solution Approach 1:
The patent transitions from one-dimensional text content analysis to multi-dimensional classification by incorporating spatial context information (surrounding cells, rows, columns) and semantic relationships. This allows the system to maintain fast automated processing while improving accuracy through additional contextual dimensions.
Solution Approach 2:
The patent introduces an intermediary correction model that acts between the initial automated classification and the final result. This correction model uses spatial and semantic analysis to identify and correct misclassifications without requiring complete manual review, thus maintaining efficiency while improving accuracy.
2Measurement precision
If spatial context information is incorporated into classification, then classification accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent segments the classification process into two distinct stages: an initial automated classification stage and a correction stage. This segmentation allows the system to handle complex spatial analysis only when necessary (for correction), rather than applying it to all data, thus reducing overall processing complexity while maintaining accuracy improvements.
Solution Approach 2:
The patent applies complex spatial and semantic analysis locally only to cases where misclassification is detected or suspected, rather than uniformly to all data. This localized application of advanced analysis reduces processing complexity while maintaining the accuracy benefits where they are most needed.
Data Source
AI summary
Aspects of the present disclosure provide techniques for automated data classification error correction through machine learning. Embodiments include receiving a set of predicted labels corresponding to a set of consecutive text strings that appear in a particular order in a document, including: a first text string corresponding to a first predicted label; a second text string that follows the first text string in the particular order and corresponds to a second predicted label; and a third text string that follows the second text string in the particular order and corresponds to a third predicted label. Embodiments include providing inputs to a machine learning model based on: the third text string; the second text string; the second predicted label; and the first predicted label. Embodiments include determining a corrected third label for the third text string based on an output provided by the machine learning model in response to the inputs.