Neural Network Training with Label Weighting for Annotation Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge in developing autonomous vehicle systems lies in the need for large datasets to accurately classify objects in traffic scenes, which is labor-intensive and prone to variability, especially when human annotators work under pressure and with varying quality, leading to inconsistencies and reduced accuracy.
Innovation Solution
A method that combines unsupervised and supervised learning of convolutional neural networks (CNNs) using a weighting criterion to assess the reliability of labels, allowing all learning data sets to be considered during training, regardless of label quality, and iteratively refining the weighting based on a filtering instance's performance to improve annotation accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If human annotators work under pressure to annotate large datasets quickly, then productivity increases, but annotation quality and consistency deteriorate
Solution Approach 1:
The patent replaces the mechanical human annotation process with an automated machine learning system. A pre-trained model automatically generates annotations for training datasets, eliminating the need for human annotators to manually label data under pressure. This substitution maintains high productivity while ensuring consistent annotation quality, as the automated system does not suffer from human fatigue or variability.
Solution Approach 2:
The system enables self-service annotation through automated model generation. The training data generation unit automatically creates annotated training datasets by leveraging the pre-trained model's capabilities, allowing the system to annotate its own training data without external human intervention. This self-service mechanism ensures both high productivity and reliable annotation quality.
2Ease of operation
If human annotators work with varying quality standards, then ease of operation increases, but measurement precision of annotations deteriorates
Solution Approach 1:
The patent changes the operational parameters of the annotation process by transitioning from human-based subjective judgment to machine-based objective classification. The pre-trained model uses consistent algorithmic parameters to annotate data, eliminating variability in annotation standards while maintaining ease of operation through automated processing. This parameter change ensures measurement precision without sacrificing operational simplicity.
3Manufacturing precision
If only high-quality labeled datasets are used for supervised training, then manufacturing precision of model accuracy improves, but quantity of training data deteriorates
Solution Approach 1:
The patent applies local quality by differentiating the quality requirements for different portions of the training process. High-quality pre-trained models are used to generate annotations, and then quality filtering is applied selectively to retain only the highest-quality training samples. This approach maintains model accuracy by ensuring that only premium training data is used, while still leveraging a substantial volume of data through the automated generation process.
Solution Approach 2:
The pre-trained model serves as an intermediary that bridges the gap between raw data and high-quality training datasets. It automatically generates accurate annotations that would otherwise require extensive human effort, enabling the creation of large volumes of high-quality training data. This intermediary mechanism allows the system to maintain both high model accuracy and substantial training data volume.
4Quantity of substance
If all learning datasets including low-quality labels are used, then quantity of training data improves, but manufacturing precision of model training deteriorates
Solution Approach 1:
The patent implements a dynamic training approach where the quality threshold for training data is adjusted based on the specific training phase and model requirements. The quality determination unit dynamically evaluates each training sample and assigns it an appropriate weight or inclusion status. This dynamic mechanism allows the system to incorporate a large volume of training data while maintaining training accuracy by selectively filtering or down-weighting low-quality samples based on real-time assessment.
Data Source
Figure 1
Figure 2~3
Figure 4a~4c
AI summary
A method (100) for training an artificial neural network, ANN (1), with learning data records (2) that each comprise a sample (11a) and an associated label (12a), wherein each sample (11a) represents values of one or more input variables (11) of the ANN (1) and the associated label (12a) represents values of one or more output variables (12) of the ANN (1), having the steps of: - the ANN (1) is trained (110) in unmonitored fashion on the basis of the samples (11a) contained in the learning data records (2); - at least one quality criterion (3) for the reliability and/or quality of the association of labels (12a) with samples (11a) in the learning data records (2) is taken as a basis for determining (120) a weighting (2a) of the learning data records (2); - this weighting (2a) is taken into consideration for training (130) the ANN (1) in monitored fashion on the basis of the samples (11a) and associated labels (12a) contained in the learning data records (2). Trained ANN (1*) and system (50) having this ANN (1*). Associated computer program.