Tissue Image MIL Classification Using Certainty-Weighted Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning techniques for image classification in digital pathology, such as those used for classifying tissue images, often act as a 'black box' and fail to accurately classify whole-slide images due to their reliance on richly annotated training data and instability in deep learning models, leading to false positives and negatives.
Innovation Solution
A Multiple-Instance-Learning (MIL) program with a certainty-value-based pooling function is used to classify tissue images, incorporating model uncertainty and requiring only weakly annotated training data, which computes certainty values using a dropout technique at test time to improve classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing machine-learning techniques are used for image classification, then classification speed is improved, but classification accuracy deteriorates due to false positives and negatives
Solution Approach 1:
The patent introduces an attention mechanism as an intermediary component between the deep learning model and the classification output. This attention mechanism computes attention weights for different image tiles and uses them to aggregate predictions, thereby mediating the classification process to reduce false positives and negatives while maintaining computational efficiency
Solution Approach 2:
The patent segments the whole-slide image into multiple smaller image tiles and processes them independently through the deep learning model. Each tile's prediction is then aggregated using attention-weighted pooling, allowing the system to maintain high processing speed while improving overall classification accuracy through collective decision-making
2Extent of automation
If deep learning models are used for tissue image classification, then automation extent is improved, but model stability deteriorates leading to inconsistent results
Solution Approach 1:
The patent implements a feedback mechanism through the attention weights, which provide information about the reliability and importance of each image tile's prediction. This feedback is used to adjust the aggregation of predictions, stabilizing the overall model output and reducing inconsistencies while maintaining full automation
Solution Approach 2:
The patent combines multiple independent tile predictions into a unified classification decision through attention-weighted aggregation. This merging process integrates diverse local predictions into a stable global decision, reducing the impact of individual model instabilities
3Measurement precision
If richly annotated training data is used for training, then measurement precision is improved, but device complexity deteriorates due to annotation requirements
Solution Approach 1:
The patent applies partial annotation by training the deep learning model on only a subset of image tiles rather than requiring full annotation of entire whole-slide images. The attention mechanism then learns to identify and weight the most relevant tiles, achieving high precision with reduced annotation complexity
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for classifying tissue images. The method comprises: receiving (102) a plurality of digital tissue images; splitting (104) each received image into a set of image tiles; for each of the tiles, extracting (106) a feature vector from the tile; providing (108) a Multiple-Instance-Learning (MIL) program configured to use a model for classifying any input image as a member of one out of at least two different classes based on feature vectors extracted from all tiles of said image; for each of the tiles, computing (110) a certainty value indicating the certainty of the model regarding the contribution of the tile's feature vector on the classification of the image; for each of the images, using (114), by the MIL-program, a certainty-value- based pooling function for aggregating the feature vectors of the image or predictive values computed from the feature vectors of the image into an aggregated predictive value as a function of the certainty values of the tiles of the image; and classifying (116) each of the images as a member of one of the classes based on the aggregated predictive value.