Cross-Modal Medical ML Training via Mutual Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models for medical image analysis and natural language processing in medical reports is hindered by the need for manual labeling, which is time-consuming, expensive, and prone to errors, especially when large labeled datasets are difficult to obtain.
Innovation Solution
Concurrently training visual machine learning models and natural language processing models using a dataset of medical images and corresponding text reports, where the output of one model serves as ground truth for the other, allowing for iterative improvement with minimal initial labeling, and utilizing a combined visual and NLP consensus loss function.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is used to train ML models, then model accuracy can be improved, but the process becomes slow and expensive
Solution Approach 1:
The system enables self-service training by having the visual ML model generate labels for NLP training and the NLP ML model generate labels for visual training. Each model serves itself and the other, eliminating the need for manual human labeling while maintaining training quality through iterative mutual refinement.
Solution Approach 2:
The patent introduces an intermediary labeling system where the visual ML model and NLP ML model act as mediators for each other's training. Instead of direct manual labeling, each model's outputs serve as intermediate training data for the other, creating a self-sustaining training loop that reduces human involvement.
2Measurement precision
If manual labeling is performed by trained professionals, then labeling accuracy improves, but costs and time requirements increase
Solution Approach 1:
The system replaces expensive human professionals with self-service automated labeling. The visual ML model and NLP ML model mutually generate training labels for each other, eliminating the need to hire and train human annotators while maintaining high labeling accuracy through the cross-modal verification process.
Solution Approach 2:
The patent creates copies of expert knowledge by training ML models to replicate the labeling capabilities of trained professionals. Once trained on initial expert-labeled data, the models can generate high-quality labels independently, effectively copying and distributing expert-level labeling capability across multiple training iterations without recurring expert involvement.
3Reliability
If large labeled datasets are obtained through manual processes, then model performance improves, but the process becomes prohibitively expensive and time-consuming
Solution Approach 1:
The system implements continuous useful action through iterative concurrent training where the visual and NLP models continuously generate and refine labels for each other. This creates an unending stream of training data generation without the bottlenecks of manual processes, allowing the models to improve progressively through continuous mutual refinement rather than discrete manual labeling batches.
Solution Approach 2:
The patent applies preliminary action by having the models generate preliminary labels automatically before human review or final training. The visual ML model generates preliminary visual labels and the NLP ML model generates preliminary text labels, which then serve as the foundation for subsequent training iterations, eliminating the need to start from scratch with manual labeling for each training cycle.
Data Source
AI summary
There is provided a method, comprising: providing a training dataset including, medical images and corresponding text based reports, and concurrently training a natural language processing (NLP) machine learning (ML) model for generating a NLP category for a target text based report and a visual ML model for generating a visual finding for a target image, by: training the NLP ML model using the text based reports of the training dataset and a ground truth comprising the visual finding generated by the visual ML model in response to an input of the images corresponding to the text based reports of the training dataset, and training the visual ML model using the images of the training dataset and a ground truth comprising the NLP category generated by the NLP ML model in response to an input of the text based reports corresponding to the images of the training dataset.