Incremental Learning Data Construction via Confidence Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The construction of learning data for statistical methodologies in fields like information retrieval, information extraction, and natural language processing is time-consuming and costly, with existing methods either lacking efficient data enhancement or suffering from performance deterioration due to errors and batch learning limitations.
Innovation Solution
A semi-automated method and apparatus that uses machine learning to generate learning models, automatically tag raw corpus data, calculate confidence scores, select learning data candidates, and allow user correction, enabling incremental learning to reduce construction time and cost while maintaining high performance with a small dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If batch learning is used to repeatedly perform machine learning, then learning models can be trained with existing learning data, but learning time increases considerably with each repetition as the amount of learning data increases
Solution Approach 1:
The patent segments the learning process into two distinct modes: offline batch learning for initial model training with existing data, and online incremental learning for continuous model updates with new data. This segmentation allows the system to avoid repeatedly processing entire datasets, thereby reducing learning time while maintaining model accuracy through progressive refinement.
Solution Approach 2:
The patent performs preliminary batch learning to establish initial learning models before deploying them for automatic tagging. These pre-trained models serve as a foundation that can be quickly refined through incremental learning, eliminating the need to reprocess all historical data with each new tagging task and significantly reducing subsequent learning time.
2Productivity
If automatic tagging is performed without error correction, then processing speed is maintained, but performance deteriorates due to inclusion of errors in auto tagging results
Solution Approach 1:
The patent implements a feedback mechanism where automatic tagging results are evaluated against confidence scores, and low-confidence results are flagged for manual review or correction. This feedback loop allows the system to maintain high productivity by automatically processing high-confidence cases while selectively applying human review only where needed, thus preserving accuracy without sacrificing overall speed.
Solution Approach 2:
The patent introduces confidence score thresholds as a parameter to differentiate between automatic acceptance and manual review of tagging results. By adjusting this parameter, the system can dynamically balance between productivity and reliability, accepting higher automation rates when thresholds are lenient and improving accuracy when thresholds are stringent, thereby optimizing the trade-off between speed and accuracy.
3Reliability
If more learning data is constructed to improve statistical methodology performance, then higher performance is achieved, but construction time and effort increase greatly
Solution Approach 1:
The patent enables the system to automatically generate new learning data through automatic tagging of raw corpora using existing learning models. This self-service capability allows the system to expand its own training data without external human intervention, continuously improving performance while minimizing additional construction time by leveraging its own computational resources.
Solution Approach 2:
The patent recovers and reuses previously tagged data that may have been discarded or set aside, incorporating it into expanded learning datasets. This approach maximizes the utility of existing data assets, allowing the system to construct larger learning datasets without proportionally increasing construction time, as it reprocesses and repurposes previously collected information.
Data Source
AI summary
An apparatus and method for efficiently constructing learning data required in statistical methodology used in information retrieval, information extraction, translation, natural language processing, etc. are provided. The method includes the steps of: generating learning models by performing machine learning with respect to learning data; attaching tags to a raw corpus automatically by using the generated learning models to thereby generate learning data candidates; calculating confidence scores of the generated learning data candidates, and then selecting a learning data candidate using the confidence scores; and allowing a user to correct an error in the selected learning data candidate through an interface and adding the error-corrected learning data candidate to the learning data, thereby adding new learning models incrementally.