Weakly Supervised Image Classification via Tile Feature Pooling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Histopathological image analysis in oncology faces challenges due to the high cost and limited availability of localized annotations, making it difficult to train effective machine learning models for disease detection and classification without pixel-level labels, which are necessary for accurate diagnosis and prognosis estimation.
Innovation Solution
A method using weakly supervised learning with pre-trained convolutional neural networks, applying tiling techniques to extract feature vectors from histopathology images, and employing multi-instance learning to classify images without localized annotations, allowing for disease localization and diagnosis prediction using whole-slide labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If localized annotation masks are used as training data, then classification accuracy is improved, but annotation cost and time consumption increase significantly
Solution Approach 1:
The image is divided into multiple tiles or patches, and the classification problem is decomposed into multiple instance learning where each tile is an instance. This allows the model to learn from image-level labels without requiring pixel-level annotation masks, resolving the contradiction between accuracy and annotation cost.
Solution Approach 2:
A pooling operation (e.g., max pooling or mean pooling) is introduced as an intermediary between the feature extraction stage and the classification stage. This pooling operation aggregates features from multiple tiles to represent the entire image, enabling the model to achieve accurate classification using only global labels without localized annotations.
2Manufacturing precision
If pixel-by-pixel segmentation maps are created, then disease localization precision is improved, but pathologist workload and cost increase
Solution Approach 1:
Instead of requiring complete pixel-level segmentation of all diseased regions, the model uses image-level labels to guide the classification process. This partial action approach achieves sufficient diagnostic accuracy without the excessive effort of creating comprehensive segmentation maps.
Solution Approach 2:
The model performs self-localization by identifying which tiles contribute most to the classification decision through attention mechanisms or gradient-based methods. This allows the system to automatically identify suspicious regions without requiring manual segmentation annotations.
3Reliability
If deep learning models are trained with localized annotations, then diagnostic accuracy is improved, but data availability is limited
Solution Approach 1:
The model is designed to work with image-level labels which are universally available in histopathology datasets, rather than requiring the specialized localized annotations that limit data availability. This multi-functionality allows the same model architecture to be trained on various datasets with different annotation types.
Solution Approach 2:
The model extracts features from image tiles and aggregates them through pooling operations before classification. This preliminary feature extraction and aggregation creates a robust representation that can be trained effectively with limited labeled data, improving reliability even when dataset size is constrained.
Data Source
AI summary
A method and apparatus of a device that classifies an image is described. In an exemplary embodiment, the method includes tiling at least one region of interest of the input image into a set of tiles. For each tile, the method includes extracting a feature vector of the tile by applying a convolutional neural network, wherein a feature is a local descriptor of the tile; and computing a score of the tile from the extracted feature vector, said tile score being representative of a contribution of the tile into a classification of the input image. The method also includes sorting a set of the tile scores and selecting a subset of the tile scores based on their value and/or their rank in the sorted set. The method also includes applying a classifier to the selected tile scores in order to classify the input image.