Training Semantic Matching Models with Noisy Annotation Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training data for semantic matching models often contains noise and vague annotations, making it difficult to achieve accurate and efficient model training.
Innovation Solution
A method involving the acquisition of a first annotation dataset, training an initial classification model, performing predictions, generating a second annotation dataset based on prediction results, and further training the model using this dataset to improve accuracy and utilization of high-noise data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If annotation data is acquired for training semantic matching models, then model training can be performed, but the annotation data contains noise and vague annotations which reduces training accuracy
Solution Approach 1:
The training process is divided into multiple stages: first training an initial classification model on the noisy annotation data, then using this model to identify and filter high-quality samples, and finally training the semantic matching model on this filtered subset. This segmentation allows the system to handle noisy data systematically while extracting the most valuable training signals.
Solution Approach 2:
An intermediate classification model is introduced as a mediator between the noisy annotation data and the semantic matching model training. This intermediate model filters and selects high-quality training samples from the noisy data, acting as a bridge that prevents direct propagation of annotation errors to the final model.
2Productivity
If all annotation data is used for training, then data utilization is maximized, but noise in the data reduces model accuracy
Solution Approach 1:
Instead of using all available annotation data, the method selectively uses only the high-quality samples identified by the intermediate classification model. This partial action approach prioritizes quality over quantity, using a subset of data that provides the most reliable training signals while discarding noisy samples that would harm model accuracy.
Solution Approach 2:
The intermediate classification model automatically identifies and selects high-quality training samples from the noisy annotation data without requiring manual intervention. The system serves itself by using the initial model to curate its own training data, efficiently separating signal from noise in the annotation set.
Data Source
AI summary
The present disclosure discloses a method, apparatus, device, and storage medium for training a model, relates to the technical fields of knowledge graph, natural language processing, and deep learning. The method may include: acquiring a first annotation data set, the first annotation data set including sample data and a annotation classification result corresponding to the sample data; training a preset initial classification model based on the first annotation data set to obtain an intermediate model; performing prediction on the sample data in the first annotation data set using the intermediate model to obtain a prediction classification result corresponding to the sample data; generating a second annotation data set based on the sample data, the corresponding annotation classification result, and the corresponding prediction classification result; and training the intermediate model based on the second annotation data set to obtain a classification model.