Auto Labeling Device Using Uncertainty Scores for Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional auto labeling methods are labor-intensive and costly due to low throughput and the need for skilled inspectors to correct auto-labeled training images, making it difficult to maintain high accuracy and efficiency.
Innovation Solution
A method using uncertainty scores to automatically verify and correct labels by employing a combination of object detection networks, convolution networks, and classifiers to generate and refine bounding boxes and feature maps, allowing unskilled inspectors to focus on verifying uncertain labels, thereby increasing throughput and reducing the number of inspectors needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional auto labeling methods are used with inspectors to correct labels, then labeling accuracy is maintained, but throughput is low and costs are high
Solution Approach 1:
The system uses uncertainty scores generated by the neural network to automatically identify and flag only those labels that require human inspection. The auto labeling device serves itself by prioritizing which labels need verification, reducing reliance on manual inspection while maintaining accuracy.
Solution Approach 2:
The patent introduces uncertainty scores as a new parameter to quantify label reliability. By changing from uniform inspection of all labels to selective inspection based on uncertainty thresholds, the system achieves both high throughput and maintained accuracy.
2Productivity
If the number of inspectors is increased to improve throughput, then labeling capacity increases, but costs increase
Solution Approach 1:
The patent extracts only the uncertain or potentially incorrect labels from the full set of auto-labeled data for human inspection. This selective approach allows fewer inspectors to handle a smaller subset of problematic labels while the majority of confident predictions are accepted automatically.
Solution Approach 2:
Instead of having inspectors verify all labels (excessive action), the system applies partial verification only to labels exceeding uncertainty thresholds. This partial action approach maintains accuracy where needed while avoiding unnecessary inspection costs.
3Reliability
If skilled inspectors are used to maintain accuracy, then labeling quality is high, but it is difficult to acquire enough inspectors
Solution Approach 1:
The uncertainty score acts as an intermediary that guides inspector attention. Instead of relying on inspectors to identify problematic labels through expertise, the system uses the uncertainty metric to automatically flag cases needing review, making the inspection process accessible to less experienced personnel.
4Reliability
If all training images are manually inspected to ensure accuracy, then labeling quality is maintained, but time consumption increases
Solution Approach 1:
The system performs preliminary filtering by calculating uncertainty scores for all labels before human inspection. This preliminary action identifies and prioritizes only the uncertain cases for manual review, eliminating the need for time-consuming inspection of all labels while maintaining accuracy.
Data Source
AI summary
A method for training an auto labeling device capable of performing automatic verification by using uncertainty scores of labels is provided. The method includes steps of: a learning device (a) inputting unlabeled training images into a trained object detection network and a trained convolution network to generate bounding boxes for training and feature maps for training; and (b) (i) instructing an ROI pooling layer to generate pooled feature maps for training, (ii) at least one of (ii-1) inputting the pooled feature maps for training into a first classifier to generate first class scores for training and first box uncertainty scores for training, and (ii-2) inputting the pooled feature maps for training into a second classifier to generate second class scores for training and second box uncertainty scores for training, and (iii) training one of the first classifier using first class losses and the second classifier using second class losses.