Aggregated Image Annotation via Weighted Confidence Channels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for training machine learning models to recognize objects require large amounts of manually annotated data, which can be inconsistent across annotators, making it difficult to combine and utilize effectively, especially since simple averaging techniques do not always result in accurate bounding boxes.
Innovation Solution
A system that aggregates bounding box annotations from multiple contributors by generating an additional image channel based on weighted averages of confidence measures, then applies an object detection model to this channel to produce a more accurate aggregated annotation dataset for training purposes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple annotators manually annotate objects in images, then the quantity of training data increases, but the consistency and accuracy of annotations deteriorate
Solution Approach 1:
The patent combines multiple annotators' bounding box annotations into a single aggregated annotation. It processes multiple annotation results by calculating intersection over union (IoU) metrics and merging overlapping bounding boxes to produce a consolidated annotation that represents the consensus of multiple annotators, thereby maintaining data quantity while improving consistency.
Solution Approach 2:
The system uses feedback from multiple annotators to iteratively improve annotation quality. By comparing overlapping annotations and calculating IoU scores, the system identifies discrepancies and uses this feedback to refine the final aggregated annotation, ensuring higher accuracy through collective input.
2Productivity
If simple averaging techniques are used to combine bounding boxes, then the processing speed increases, but the accuracy of the resulting bounding box deteriorates
Solution Approach 1:
The patent changes the parameter combination method from simple arithmetic averaging to a more sophisticated approach based on IoU calculations and overlap analysis. Instead of uniformly averaging boundary coordinates, the system weights annotations based on their overlap程度 and consistency, producing more accurate bounding boxes that reflect the true object boundaries.
Solution Approach 2:
The system segments the annotation combination process into distinct stages: first identifying overlapping regions between bounding boxes, then calculating IoU metrics for each overlap, and finally merging boxes based on these calculated metrics. This segmented approach ensures accuracy by treating each annotation relationship individually rather than applying a blanket averaging formula.
3Reliability
If a large number of manual annotations are collected, then the quality of training data improves, but the time and computational resources required increase
Solution Approach 1:
The system performs preliminary filtering and preprocessing of annotations before full aggregation. It first identifies obvious overlaps and consensus regions, pre-processes bounding box coordinates, and prepares annotation pairs for efficient merging. This preliminary action reduces the computational burden of the subsequent aggregation process while maintaining data quality.
Data Source
AI summary
Image annotation includes: accessing an image and a plurality of annotation data sets for the image, wherein the plurality of annotation data sets are made by a plurality of contributors, and the image has a plurality of original image channels; aggregating the plurality of annotation data sets to obtain an aggregated annotation data set for the image; and outputting the aggregated annotation data set. Aggregating the plurality of annotation data sets to obtain an aggregated annotation data set for the image includes: generating an additional image channel based at least in part on weight averages of confidence measures of the plurality of contributors; and applying an object detection model to at least a part of the plurality of original image channels and at least a part of the additional image channel to generate the aggregated annotation data set.