X-Ray In-Product Learning Through Uncertainty-Based Image Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems for medical imaging face challenges in handling high uncertainty cases, particularly in x-ray data, which can hinder the improvement of pathology identification, and require time-consuming manual labeling for regulatory compliance.
Innovation Solution
A system that analyzes uncertainty values and outlier data to identify images useful for retraining, allowing adaptive learning without requiring recertification, by using a criterion-based approach to determine which images are beneficial for improving model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning systems incorporate all high uncertainty cases for retraining, then model performance may improve, but computational resources and time are wasted on cases that are not appropriate for improving pathology identification
Solution Approach 1:
The system applies different selection criteria to different classes of medical images based on their specific characteristics. For each class, it evaluates whether high uncertainty cases are appropriate for improving pathology identification or if they represent inappropriate cases (such as poor quality images or artifacts). This localized quality assessment ensures that retraining resources are allocated efficiently to only those cases that will actually improve model performance for each specific pathology class.
2Manufacturing precision
If manual labeling is performed for regulatory compliance, then labeled training data can be obtained, but the process is time-consuming and reduces productivity
Solution Approach 1:
The system performs self-labeling by automatically determining the appropriateness of high uncertainty cases for retraining. The selection module autonomously identifies which cases should be included in the retraining dataset based on uncertainty analysis and class-specific criteria, eliminating the need for time-consuming manual labeling by radiologists or annotators while maintaining data quality standards required for regulatory compliance.
Solution Approach 2:
The system performs preliminary filtering and selection of training cases before the actual retraining process. By pre-identifying appropriate high uncertainty cases that meet quality criteria, the system prepares the training dataset in advance, reducing the need for subsequent manual intervention and accelerating the overall workflow while ensuring regulatory compliance requirements are met.
3Adaptability or versatility
If adaptive learning is implemented to improve model performance over time, then the system can learn from new data, but regulatory recertification may be required for each model update
Solution Approach 1:
The system implements dynamic adaptive learning where the selection criteria and retraining process can evolve over time. The class-specific criteria are adaptable based on accumulated deployment experience, allowing the system to learn from new data and improve performance continuously. The framework supports dynamic adjustment of what constitutes appropriate training cases as the system gains more experience with different pathology classes and uncertainty patterns.
4Measurement precision
If comprehensive uncertainty analysis is performed for all input images, then appropriate training cases can be identified, but the analysis complexity and processing time increase
Solution Approach 1:
The system segments the uncertainty analysis process into distinct modular components: an uncertainty determination module that calculates uncertainty values, a selection module that evaluates appropriateness for retraining, and a class-specific criterion application module. This segmentation allows comprehensive uncertainty analysis to be performed systematically while managing complexity through modular design, where each component has a specific function and can be independently optimized or adjusted.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A system (SYS)and related method for providing training data. The system is configured to receive a classification result for a class from plural pre-defined classes (Cj). The classification result is produced by a trained machine learning model (M) in response to processing an input image. A decision logic (DL) of the system is configured to analyze input data (pi,qi) comprising the received classification result value (pi) and an uncertainty value (qi) associated with the classification result value. The system outputs, per class, an associated indication whether the input image is or is not useful for re-training the model (M) in respect of the said class.