Medical Data Annotation Selection via AI Inference Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The annotation of medical data for training artificial intelligence models is costly and time-consuming, and randomly selecting data can lead to suboptimal training performance due to the need for skilled experts and the inefficiency of data selection.
Innovation Solution
A method and apparatus for selecting medical data for annotation based on the current training performance of an AI model, using a selection policy to identify target data for annotation, which includes determining a selection policy for medical data to be used in next training and selecting data based on inference results to improve training performance while minimizing annotation costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If random data extraction is used for annotation, then annotation cost is reduced, but training performance deteriorates
Solution Approach 1:
The system uses the AI model's inference results as feedback to identify which data samples are most beneficial for annotation. By analyzing prediction confidence levels and classification outcomes, the system determines which samples will most improve training performance, creating a closed-loop selection process that adapts to model needs
Solution Approach 2:
The system changes the selection parameter from random sampling to inference-based sampling. By using the AI model's inference confidence levels and prediction outcomes as selection criteria, the system transforms the data selection process from stochastic to deterministic based on model performance needs
2Reliability
If all medical data is annotated, then training performance is improved, but annotation time increases
Solution Approach 1:
The system extracts only the most valuable subset of data for annotation by using inference results to identify samples that will most benefit training. Instead of annotating all data, it extracts a targeted portion based on model confidence levels and performance gaps
Solution Approach 2:
The system performs partial annotation on carefully selected data samples rather than complete annotation of all data. By annotating only the most informative samples identified through inference analysis, it achieves sufficient training performance with reduced annotation effort
3Measurement precision
If expert annotation is performed on all data, then data quality is improved, but annotation cost increases
Solution Approach 1:
The system applies expert annotation only to specific local regions of the data space - namely, the samples identified through inference results as most beneficial for training. High-quality expert annotation is concentrated on critical samples rather than uniformly applied to all data
4Productivity
If data selection is based on current training performance, then training efficiency is improved, but system complexity increases
Solution Approach 1:
The system performs self-service by using its own inference results to guide data selection for annotation. The AI model evaluates its own performance on candidate samples and automatically identifies which data would be most beneficial for retraining, eliminating the need for external performance evaluation systems
Data Source
AI summary
An operating method of a medical data selecting apparatus operated by at least one processor includes generating training data including partial medical data sampled from mass medical data and annotated data of the partial medical data, extracting candidate data for annotation from the mass medical data, the candidate data being at least a portion of the mass medical data, acquiring inference results that are inferred from the candidate data by an artificial intelligence (AI) model trained based on the training data and selecting target data for annotation to be used in next training of the AI model, from among the candidate data based on the inference results.