Regional Embeddings in Multiple-Instance Learning for Weakly Labeled Pathology
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning models for digital pathology face challenges in training due to the high spatial resolution and morphological variance of whole-slide images, often relying on weakly annotated data, which limits the effectiveness of patch-level labeling and instance-level classification.
Innovation Solution
A multiple-instance learning approach that considers both selected patches and their regions, using a pre-trained model to generate patch and regional embeddings, followed by a trainable model to create a global embedding for classification, incorporating regional features with local patch features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If patch-level labeling is used for training deep learning models, then instance-level classification accuracy can be improved, but the labeling process becomes extremely time-consuming and often impossible for expert pathologists
Solution Approach 1:
The whole-slide image is segmented into multiple patches, and the labeling task is segmented from instance-level to bag-level. Each bag contains multiple patches, and only bag-level labels are required, eliminating the need for time-consuming instance-level labeling while still enabling effective training through multiple-instance learning
Solution Approach 2:
Multiple-instance learning acts as an intermediary framework between patch-level and whole-slide-level analysis. It introduces the concept of bags as intermediate units that contain multiple patches, allowing the model to learn from bag-level labels without requiring instance-level annotations
2Loss of information
If the entire whole-slide image is used as input, then complete information is available for analysis, but the high spatial resolution and morphological variance make training difficult and computationally intensive
Solution Approach 1:
The whole-slide image is divided into multiple patches that are then grouped into bags. This segmentation allows the model to process manageable units while maintaining the ability to capture global contextual information through the bag structure and multiple-instance learning framework
Solution Approach 2:
The problem is transformed from processing entire whole-slide images in one dimension to processing patches and bags in multiple dimensions. The hierarchical structure (patches → bags → whole slide) creates additional dimensional layers for information processing
3Productivity
If only selected patches are used for classification, then computational efficiency is improved, but regional contextual information is lost reducing detection accuracy
Solution Approach 1:
The image is segmented into patches that are organized into regional bags. This segmentation enables efficient processing of individual patches while the bag structure preserves regional contextual information, resolving the trade-off between efficiency and accuracy
Solution Approach 2:
Multiple patches within each bag are merged to form a regional unit. The model processes patches efficiently individually but merges their information at the bag level to capture regional context, achieving both computational efficiency and detection accuracy
Data Source
Figure 1
Figure 2
Figure 3(a)
AI summary
Systems, methods, and computer programs disclosed herein relate to training a machine learning model and using the trained machine learning model to classify images, preferably medical images, using multiple-instance learning techniques. The machine learning model can be trained, and the trained machine learning model can be used for various purposes, in particular for the detection, identification and/or characterization of tumor types and/or gene mutations in tissues.