Medical Image Annotation Consensus via Commonality Data Derivation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning model creation processes face challenges in obtaining accurate and consistent annotation information for medical images, as different annotators may assign varying labels to the same image, leading to inconsistencies and difficulties in generating reliable correct answer data.
Innovation Solution
A machine learning model creation support apparatus and method that acquires annotation information from multiple annotators, derives commonality data indicating the consistency of label assignments, and generates confirmed annotation information based on this data and preset confirmation conditions, ensuring accurate and consistent correct answer data for training and evaluation phases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple annotators assign labels to the same medical image, then the quantity of annotation information increases, but the consistency and reliability of the annotation information deteriorates due to varying label assignments
Solution Approach 1:
The patent combines multiple annotation information pieces from different annotators into a single confirmed annotation information by deriving commonality data and applying confirmation conditions. This merging process integrates the quantity of annotations while filtering for consistency, resolving the contradiction between having more annotations and maintaining reliability.
Solution Approach 2:
The patent introduces commonality data as an intermediary between raw annotation information and confirmed annotation information. This intermediary represents the consistency level among annotators and serves as a basis for filtering and confirming annotations, enabling the system to maintain reliability while utilizing multiple annotators.
2Measurement precision
If annotation information is manually generated by multiple annotators, then the accuracy of correct answer data can be improved through consensus, but the time required for data generation increases
Solution Approach 1:
The patent implements self-service through automated derivation of commonality data and automatic confirmation of annotation information based on preset conditions. The system automatically processes multiple annotators' inputs without requiring manual review or intervention, maintaining high accuracy while significantly reducing the time required compared to manual verification processes.
Solution Approach 2:
The patent changes the processing parameters from manual review to automated computational derivation. By transforming the confirmation process into a computational task that calculates commonality data and applies confirmation conditions automatically, the system maintains measurement precision while reducing time loss through parameter transformation.
3Reliability
If the system processes annotation information from multiple annotators, then the reliability of training data improves, but the device complexity increases due to additional processing steps
Solution Approach 1:
The patent segments the annotation processing into distinct functional modules: acquiring annotation information, deriving commonality data, and generating confirmed annotation information. This segmentation organizes the complexity into manageable steps, each handled by specific processing units, thereby maintaining reliability while making the system architecture more manageable and less complex.
Data Source
AI summary
A machine learning model creation support apparatus including a processor, in which the processor acquires a plurality of pieces of annotation information generated by a plurality of annotators assigning a plurality of labels according to a plurality of classes to a region of the same medical image, derives, for each of the classes, commonality data indicating commonality in how the labels are assigned by the plurality of annotators for the plurality of pieces of annotation information, and generates confirmed annotation information used as correct answer data of a machine learning model based on the commonality data and a preset confirmation condition.