3D Medical Image Segmentation With Masked Modeling for Sparse Labels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing semi-supervised learning (SSL) methods for 3D medical image segmentation face challenges due to scarce unlabeled data and tight coupling between student and teacher networks, leading to poor robustness, generalization, and accuracy, especially when anatomical targets vary slightly.
Innovation Solution
A method employing masked modeling with random masking operations on input images to create diverse tasks for student and teacher networks, incorporating HybridFormer modules, auxiliary decoders, and specific training loss functions to enhance feature learning and decoupled inter-student learning, ensuring effective unsupervised guidance and error correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional SSL methods use slight perturbations for consistency constraint, then the training process is simple, but the task diversity is insufficient and student model quickly agrees with teacher model
Solution Approach 1:
The patent applies preliminary action by performing random masking operations on input images before feeding them to student and teacher networks. This preprocessing step creates diverse tasks in advance, ensuring that the networks encounter varied segmentation challenges throughout training, preventing premature convergence while maintaining manageable training complexity through standardized masking procedures.
Solution Approach 2:
The patent implements dynamics by introducing dynamic masking strategies where different random masks are applied to student and teacher networks during training. This creates dynamically varying tasks that adapt throughout training, preventing the student model from quickly agreeing with the teacher model while keeping the training process structured through consistent masking application.
2Reliability
If weight synchronization is used to merge student weights into teacher, then the teacher predictions are stable, but the two models become tightly coupled and teacher cannot sense student errors
Solution Approach 1:
The patent introduces an intermediary mechanism by applying different random masks to student and teacher networks as intermediate processing steps. This intermediary masking layer decouples the direct weight synchronization relationship, allowing the teacher to maintain stability while independently processing masked inputs, thereby enabling the teacher to detect errors that would otherwise be masked by tight coupling.
Solution Approach 2:
The patent applies segmentation by dividing the input processing into separate masked streams for student and teacher networks. This segmentation creates independent processing paths that reduce direct coupling between models, allowing each network to learn complementary features from differently masked versions of the same input while maintaining their distinct identity and error-detection capability.
3Measurement precision
If numerous label data are used for fully-supervised training, then the segmentation accuracy is high, but the manual annotation cost is expensive and labor-intensive
Solution Approach 1:
The patent implements self-service by enabling the teacher network to generate pseudo-labels from unlabeled data through its stable predictions on masked inputs. This self-service mechanism allows the system to automatically create training labels without manual annotation, reducing the quantity of labeled data needed while maintaining high segmentation accuracy through the teacher's reliable predictions that serve as self-generated supervision signals.
Solution Approach 2:
The patent applies parameter changes by transforming the training paradigm from fully-supervised to semi-supervised learning. This parameter change in the learning setup allows the system to achieve high segmentation accuracy with fewer labeled data by leveraging unlabeled data through consistency regularization and the teacher-student framework, thereby reducing the quantity of labeled data required.
Data Source
AI summary
Disclosed is a method for establishing a 3D medical image segmentation model based on masked modeling and application thereof includes: establishing a semi-supervised learning network, wherein a student network includes an encoding module for extracting latent features and a segmentation decoder that predicts segmentation results, a teacher network includes an encoding module and a segmentation decoder that are structurally consistent with the student network; training the semi-supervised learning network, wherein during training, two random masking operations are performed on each image, and the image is input to the two networks respectively; optimizing and updating the weight of the student network, and transferring the updated weight to the teacher network, wherein the training loss function includes prototype representation loss, which is used to characterize the difference between the prototypes extracted and generated by the two networks; the student network may further include a reconstruction decoder and an auxiliary segmentation decoder.