Multi-view Neural Network for 3D Medical Image Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional deep learning methods face challenges in accurately segmenting 3D medical images due to the high cost of human labeling and the complexity of training 3D deep networks, especially in semi-supervised settings where labeled data is scarce.
Innovation Solution
The proposed solution involves a multi-view volumetric co-training strategy that utilizes asymmetric 2D-initialized 3D branches to leverage the strengths of both 2D and 3D networks, incorporating multi-view constraint losses and uncertainty-aware attention mechanisms to improve segmentation accuracy and robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional deep learning methods are used for 3D medical image segmentation, then the task can be performed with standard approaches, but accuracy is insufficient and human labeling cost is high
Solution Approach 1:
The patent segments the 3D segmentation task into multiple 2D view processing sub-tasks. Different 2D views (axial, coronal, sagittal) are processed separately by dedicated networks, and their predictions are integrated to achieve accurate 3D segmentation. This reduces the need for expensive 3D labeled data while maintaining segmentation accuracy.
Solution Approach 2:
The patent introduces 2D views as intermediary representations between the original 3D medical images and the final segmentation results. These 2D projections serve as intermediate steps that are easier to label and process, facilitating semi-supervised learning where limited 3D labeled data can be augmented with abundant 2D labeled data.
2Reliability
If 3D deep networks are trained directly, then volumetric information can be captured, but training complexity and computational cost increase significantly
Solution Approach 1:
The training process is segmented into two stages: first training multiple 2D networks on relatively simple 2D data, then training a 3D network that integrates these 2D predictions. This progressive training approach reduces overall training complexity compared to directly training a full 3D network from scratch.
Solution Approach 2:
The patent performs preliminary training of 2D networks before training the 3D network. The 2D networks are pre-trained on 2D projections of the 3D data, extracting useful features that are then transferred to guide the 3D network training. This preliminary action simplifies the subsequent 3D training process.
3Quantity of substance
If limited labeled data is used for training, then labeling cost is reduced, but segmentation performance deteriorates
Solution Approach 1:
The patent transforms the 3D volumetric data into multiple 2D projection views (dimensionality reduction). This allows the system to leverage 2D labeled data, which is cheaper and more abundant, to train view-specific networks that collectively provide accurate 3D segmentation performance even with limited 3D labeled data.
Solution Approach 2:
The patent creates a multi-functional system where 2D networks process different views (axial, coronal, sagittal) and their predictions are integrated by a 3D network. This multi-view approach allows the system to utilize diverse data sources and training strategies, improving performance with limited labeled data through semi-supervised learning.
Data Source
AI summary
Volumetric quantification can be performed for various parameters of an object represented in volumetric data. Multiple views of the object can be generated, and those views provided to a set of neural networks that can generate inferences in parallel. The inferences from the different networks can be used to generate pseudo-labels for the data, for comparison purposes, which enables a co-training loss to be determined for the unlabeled data. The co-training loss can then be used to update the relevant network parameters for the overall data analysis network. If supervised data is also available then the network parameters can further be updated using the supervised loss.