Deep Neural Network Hard Negative Mining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep neural networks and convolutional neural networks face challenges in accurately classifying medical images due to the prevalence of false negatives, where abnormalities are often not depicted, leading to inefficiencies in processing vast sample sizes and reduced accuracy in medical diagnostics.
Innovation Solution
A focus-learning function is implemented to improve classification by extracting samples from medical images, comparing initial classifications to annotations, and updating the network to correct erroneous classifications, particularly focusing on 'hard negative' samples to adjust weights and revive dead neurons, thereby enhancing the network's predictive power and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If deep neural networks are trained with large volumes of medical images, then the network can process more samples, but false negatives increase and classification accuracy decreases
Solution Approach 1:
The patent segments the training process into two distinct phases: pre-training with large volumes of data to build general features, and fine-tuning with a curated subset of hard samples to improve accuracy. This segmentation allows the network to benefit from both large-scale processing and precision-oriented learning, resolving the contradiction between processing capacity and classification accuracy.
Solution Approach 2:
The patent applies local quality by identifying and focusing computational resources on specific difficult-to-classify samples (hard positives and hard negatives) rather than treating all samples uniformly. This targeted approach improves classification accuracy for challenging cases without requiring reprocessing of the entire large dataset, thus maintaining productivity while enhancing precision.
2Ease of operation
If the deep neural network processes all samples uniformly, then processing is straightforward, but computational resources are wasted on easy samples and accuracy on hard samples remains low
Solution Approach 1:
The patent performs preliminary classification of all samples to identify hard positives and hard negatives before the fine-tuning phase. This preliminary action separates easy samples (which can be processed quickly) from hard samples (which require focused attention), allowing the system to maintain operational simplicity while improving reliability through targeted resource allocation.
Solution Approach 2:
The system uses the initial classification results as feedback to identify which samples require further attention. By feeding back the classification confidence scores and comparing against ground truth annotations, the system automatically identifies hard samples that need refined processing, thus improving diagnostic accuracy without complicating the overall workflow.
3Ease of manufacture
If the network uses standard training approaches, then implementation is simple, but it fails to correctly identify abnormalities leading to false negatives
Solution Approach 1:
The patent changes the training parameters by introducing a two-stage process with different learning rates, batch sizes, and loss function weights for hard samples versus easy samples. These parameter changes allow the network to maintain implementation simplicity (using standard deep learning frameworks) while significantly improving abnormality detection accuracy through optimized training dynamics.
Solution Approach 2:
The patent introduces dynamic adjustment of training focus by switching between pre-training and fine-tuning phases, and dynamically adjusting which samples receive higher weight during gradient updates. This dynamic approach enables the system to adapt to different stages of learning, maintaining simplicity in implementation while achieving high precision in abnormality detection.
Data Source
AI summary
Methods, systems, and storage components for utilizing a deep neural network(s) for classifying at least one medical image. A deep neural network (DNN) can be configured by an image processing component to contain at least one revived neuron, where the revived neuron has an adjusted value based on a focus-learning function that is transferred to the DNN by the image processing component, where the focus-learning function provides the adjustment by updating the DNN with data that contains a corrected classification with respect to at least one normal image sample derived from a medical image, and where the correction is based on the focus-learning function comparing an annotation associated with an abnormal image sample derived from the medical image to another annotation associated with the at least one normal image.