Retrieval Model Training Data Quality Assessment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The requirement for high-quality training data is excessive and costly in the process of training retrieval models, which hampers the efficiency of information retrieval systems.
Innovation Solution
The method involves acquiring initial training data, updating negative text examples using an initial retrieval model, and expanding positive and negative text examples through subsequent training operations to obtain improved retrieval models, thereby reducing the need for high-quality training data while maintaining retrieval effectiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If high-quality manually annotated training data is used to train retrieval models, then model performance is improved, but training cost and time consumption increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-processing unannotated data through automated quality assessment before full manual annotation. The system evaluates data quality metrics (diversity, difficulty, representativeness) and prioritizes which unannotated samples need manual labeling, thereby reducing the overall annotation workload while maintaining model performance.
Solution Approach 2:
The patent implements self-service through automated data quality assessment mechanisms that evaluate and select training samples without human intervention. The system autonomously identifies high-value training data through computational metrics, reducing dependency on manual annotation for every sample while preserving the ability to achieve high model performance.
2Reliability
If high-quality manually annotated training data is used to train retrieval models, then model performance is improved, but annotation cost increases
Solution Approach 1:
The patent applies preliminary action by pre-processing unannotated data through automated quality assessment before full manual annotation. The system evaluates data quality metrics (diversity, difficulty, representativeness) and prioritizes which unannotated samples need manual labeling, thereby reducing the overall annotation workload and cost while maintaining model performance.
Solution Approach 2:
The patent implements self-service through automated data quality assessment mechanisms that evaluate and select training samples without human intervention. The system autonomously identifies high-value training data through computational metrics, reducing dependency on manual annotation while preserving the ability to achieve high model performance.
3Measurement precision
If more training data is collected to improve model performance, then retrieval accuracy increases, but data processing complexity increases
Solution Approach 1:
The patent applies the extraction principle by isolating and evaluating specific quality dimensions (diversity, difficulty, representativeness) of training data separately. This allows the system to selectively process and prioritize only the most valuable data characteristics, simplifying the overall data processing pipeline while improving retrieval accuracy through targeted data selection.
Solution Approach 2:
The patent applies parameter changes by transforming raw training data into evaluated data with associated quality scores. The system changes the state of training data from unannotated to assessed, introducing quality metrics as new parameters that enable automated selection and prioritization, thereby managing processing complexity while enhancing retrieval accuracy.
Data Source
AI summary
The present application discloses a method and apparatus for training a retrieval model, device and computer storage medium that relate to intelligent search and natural language processing technologies. An implementation includes: acquiring initial training data; performing a training operation using the initial training data to obtain an initial retrieval model; selecting texts with the correlation degrees with a query in the training data meeting a preset first requirement from candidate texts using the initial retrieval model; performing a training operation using the updated training data to obtain a first retrieval model; and selecting texts with the correlation degrees with the query in the training data meeting a preset second requirement from the candidate texts using the first retrieval model; and/or selecting texts with the correlation degrees with the query meeting a preset third requirement; and performing a training operation using the expanded training data to obtain a second retrieval model.