Detecting Label Errors in Image Recognition Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In machine learning, manually labeling correct answer data for image recognition is prone to errors, leading to incorrect learning of detection targets, which affects model performance, and existing methods struggle to estimate errors in partial labeling of detection objects.
Innovation Solution
An information processing apparatus that includes a storage unit for correct answer data and detection data, a target determination unit to extract mismatching data, an investigation unit to analyze property information, and an error determination unit to identify error candidate data, allowing for accurate estimation of error portions in correct answer data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is performed for correct answer data, then detection targets can be recognized in images, but errors readily occur in the labeling process
Solution Approach 1:
The system applies the detection model to correct answer data to generate detection results, then compares these results with the original correct answer data to identify mismatches. This feedback loop enables automatic error detection in manually labeled data, allowing users to confirm and correct erroneous labels, thereby improving the reliability of correct answer data without sacrificing labeling capability.
Solution Approach 2:
The detection model performs self-validation by automatically comparing its own detection results against the correct answer data it was trained on. This self-service mechanism identifies potential errors in the correct answer data without requiring external verification, reducing the burden on users while maintaining high accuracy standards.
2Reliability
If all mismatching data is confirmed by user, then detection performance is improved, but burden on user increases
Solution Approach 1:
The system extracts only the mismatching portions between detection results and correct answer data, presenting these specific portions to users for confirmation rather than requiring review of entire images or all data. This extraction approach focuses user attention on only the problematic areas, significantly reducing user burden while maintaining detection performance improvement.
Solution Approach 2:
Instead of requiring users to confirm all possible mismatching data, the system applies a threshold to identify only those mismatches that exceed a certain significance level. This partial action approach confirms only the most critical errors, reducing user workload while still achieving sufficient detection performance improvement.
3Ease of manufacture
If one label is added to one entire detection object, then labeling is simplified, but error estimation for partial labeling cannot be performed
Solution Approach 1:
The system segments the detection object into multiple regions and generates detection results for each region separately. By comparing these segmented detection results with the corresponding portions of correct answer data, the system can precisely identify which specific regions contain errors, enabling accurate error estimation even when one label covers the entire detection object.
Solution Approach 2:
The system applies different evaluation criteria to different regions of the detection object based on local characteristics. By analyzing mismatches at the regional level rather than treating the entire object uniformly, the system can provide precise error estimation for specific portions while maintaining the simplified one-label-per-object structure.
Data Source
AI summary
An information processing apparatus comprises a storage unit configured to store correct answer data used to detect at least one portion of a detection object from an image and detection data detected as the at least one portion of the detection object from the image; a target determination unit configured to extract mismatching data between the correct answer data and the detection data, which exists within a predetermined range from a region in which the correct answer data and the detection data match, and determine the mismatching data as evaluation target data; an investigation unit configured to investigate property information of the evaluation target data; and an error determination unit configured to determine, based on the property information, whether the evaluation target data is error candidate data of the correct answer data.