Masked Image Pretraining for 3D Medical Image Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high cost and time-consuming nature of obtaining expert annotations for 3D medical volumetric images limits the effectiveness of deep learning approaches in medical image analysis, and the sheer data volume associated with 3D medical images increases processing complexity, making it difficult to integrate radiomics endpoint information with other bio-marker data for clinical studies.
Innovation Solution
A self-supervised masked image modeling (MIM) training process is employed to pre-train an image encoder on unannotated multi-dimensional medical images using Transformer-based architectures, followed by a supervised training process to fine-tune the encoder for specific vision tasks, leveraging unlabeled data to reduce annotation requirements and processing complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If expert annotations are obtained for 3D medical volumetric images to train deep learning models, then model performance is improved, but cost and time consumption increase significantly
Solution Approach 1:
The patent applies self-supervised pre-training on unlabeled 3D medical images before fine-tuning with annotated data. The image encoder learns useful representations from abundant unlabeled data in advance, so that when annotated data is later used for fine-tuning, the model converges faster and achieves better performance with less annotation time.
Solution Approach 2:
The patent uses self-supervised learning where the model learns from unlabeled data without requiring expert annotations. The masking and reconstruction process enables the model to self-train on abundant unlabeled 3D medical images, reducing dependency on costly and time-consuming expert annotations while still learning effective features.
2Loss of information
If the sheer data volume of 3D medical images is processed using traditional methods, then comprehensive analysis is achieved, but processing complexity increases
Solution Approach 1:
The patent divides 3D medical images into multiple 2D slices or patches for processing. The image encoder processes these segmented parts independently and then reconstructs the comprehensive 3D representation, reducing processing complexity while maintaining analysis completeness through the hierarchical feature extraction capability of Transformers.
Solution Approach 2:
The patent introduces an image encoder as an intermediary component that processes 3D medical images through self-supervised pre-training. This intermediary learns to extract meaningful features from the complex 3D data, simplifying subsequent analysis tasks by transforming raw high-complexity 3D images into processed feature representations.
3Loss of energy
If traditional supervised training is used on limited annotated data, then training cost is reduced, but training convergence speed decreases
Solution Approach 1:
The patent performs self-supervised pre-training on unlabeled data before supervised fine-tuning on annotated data. This preliminary action allows the model to learn general features from abundant unlabeled data at low cost, so that subsequent supervised training on limited annotated data converges much faster, achieving both cost efficiency and fast convergence.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method (1400) includes obtaining a first training data set (201) including unannotated multi-dimensional medical images (202) and executing a self-supervised masked image modeling (MIM) training process (200) to pre-train an image encoder (150) on the first training data set. The method also includes obtaining a second training data set (203) that includes annotated multi-dimensional medical images (204). Here, each annotated multi-dimensional medical image includes a plurality of image voxels (206) each paired with a corresponding ground-truth label (208) indicating a class the corresponding image voxel belongs to. The method also includes executing a supervised training process (160) to train an image analysis model (170) on the second training data set to teach the image analysis model to learn how to predict the corresponding ground-truth labels for the plurality of image voxels of each annotated multi-dimensional medical image. The image analysis model incorporates the pre-trained image encoder.