Medical Image Segmentation Training With Dynamic External Data Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training deep neural networks for medical image segmentation face challenges in effectively utilizing external data due to variations in image quality and relevance, leading to computational inefficiencies and inaccurate learning.
Innovation Solution
A data-efficient training algorithm that assigns importance weights to 'good' external data and suppresses 'bad' data by employing a block coordinate descent optimization scheme, optimizing the weighted loss function to enhance network learning with external data while minimizing computational cost.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If external data from alternative sources is used to supplement training data, then the quantity of training data increases, but the reliability of training results deteriorates due to variation in image quality and relevance
Solution Approach 1:
The patent applies local quality by assigning different importance weights to different external data samples based on their individual quality and relevance characteristics. Instead of treating all external data uniformly, the system evaluates each sample's contribution and assigns weights accordingly, allowing high-quality external data to positively influence training while limiting the impact of low-quality samples. This resolves the contradiction by enabling quantity increase without proportional reliability degradation.
Solution Approach 2:
The patent changes the parameter of data importance through dynamic weight assignment during training. The importance weights are adjusted based on loss values and discrepancy measurements, transforming the static treatment of training data into a dynamic process where external data samples are selectively emphasized or de-emphasized. This parameter change allows the system to utilize large quantities of external data while maintaining reliability by down-weighting problematic samples.
2Reliability
If external data with large loss values is dropped during training, then the reliability of training improves by avoiding harmful patterns, but the productivity of training deteriorates due to loss of informative data
Solution Approach 1:
The patent applies discarding and recovering by initially assigning low weights to external data samples with large loss values, effectively discarding their harmful influence, while preserving the option to recover informative patterns through iterative weight adjustment. The system does not permanently discard these samples but rather dynamically adjusts their importance weights based on their actual contribution to learning, allowing recovery of useful information while maintaining reliability.
Solution Approach 2:
The patent converts the harmful effect of high-loss external data into a benefit by using the loss values themselves as signals for weight adjustment. Samples with initially high loss values are not simply discarded but are used to guide the optimization process, where their discrepancy from ground truth informs the weight assignment mechanism. This transforms potentially harmful data into useful training signals that improve the model's ability to distinguish between harmful and beneficial patterns.
3Reliability
If gradient descent detection of mini-batch data is used to learn external data weights, then the reliability of weight estimation improves, but the computational cost increases due to requirement of second order gradients
Solution Approach 1:
The patent substitutes the complex mechanical system of second-order gradient computation with a simpler alternative approach. Instead of using gradient descent that requires calculating Hessian matrices and second-order derivatives, the system employs a discrepancy-based weight assignment mechanism that relies on first-order loss values and simple comparisons. This substitution maintains reliable weight estimation while dramatically reducing computational complexity and energy consumption.
Solution Approach 2:
The patent changes the parameters used for weight estimation from second-order gradient information to first-order loss values and discrepancy metrics. By transitioning from a parameter regime requiring computationally expensive second derivatives to one based on simpler first-order statistics, the system achieves reliable weight estimation with significantly reduced computational cost, making the training process more scalable and efficient.
Data Source
AI summary
A method of training a learning network to perform image segmentation includes receiving training data from a trusted source and an alternative source; assigning weights to data in the second corpus; dividing the data into batches with samples from the first and second corpuses; and completing training epochs by, for each batch: generating an output for each sample; determining a loss by comparing the generated output with a ground truth for each sample; defining a discrepancy for each second corpus sample as a difference between the sample loss and the average loss for the first corpus, approximated as the average loss for the first corpus from the preceding epoch modified by a change in the average loss from the preceding to the current epoch; identifying weights for the next batch which minimise the weighted sum of discrepancies and network parameters which minimise the average weighted loss.