3D Anisotropic Hybrid Network for Feature Transfer in Uneven Volumes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional 3D convolutional neural networks face challenges in effectively processing anisotropic 3D volumes due to unequal image resolutions across dimensions, limited GPU memory, and the need for large datasets, making it difficult to learn features and generalize to unseen data.
Innovation Solution
A 3D anisotropic hybrid network (AHNet) transfers convolutional features from 2D images by using a 2D encoder network initialized with pre-trained weights, extending 2D kernels to 3D, adding a feature decoder with anisotropic convolutional blocks, and employing focal loss for end-to-end learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If 3D CNN is applied to anisotropic volumes, then 3D context information can be utilized, but the network performance deteriorates due to unequal resolutions across dimensions and limited GPU memory
Solution Approach 1:
The 3D anisotropic volume is segmented into multiple 2D slices along the z-axis. Each slice is processed independently by 2D CNNs to extract features, avoiding the performance degradation of direct 3D CNN application while preserving 3D context through sequential processing of slices with different resolutions
Solution Approach 2:
The approach transforms the 3D processing problem into a sequence of 2D problems by processing slices in the z-dimension. This dimensionality change allows leveraging pre-trained 2D networks while maintaining the ability to capture 3D contextual relationships through the sequence of 2D feature maps
2Reliability
If 3D networks with more features are used, then feature representation capability is improved, but GPU memory constraints limit the network width and depth
Solution Approach 1:
The network is segmented into multiple 2D CNNs that process each slice independently. This avoids the need for a single large 3D network with excessive parameters, distributing the computational load across multiple smaller 2D networks that fit within GPU memory constraints
Solution Approach 2:
Multiple copies of 2D CNNs are used to process different slices. Instead of one large 3D network, several smaller 2D network copies operate in parallel on different slices, achieving comparable or better feature representation with reduced memory requirements
3Reliability
If 3D networks are trained from scratch, then the network can be optimized for 3D data, but training time increases and large datasets are required
Solution Approach 1:
2D CNNs are pre-trained on large 2D image datasets before being applied to 3D medical volumes. This preliminary training on abundant 2D data provides good initial feature extractors, avoiding the need to train from scratch and significantly reducing the time and data requirements for 3D task optimization
Solution Approach 2:
The approach changes the parameter space from 3D convolutional kernels to 2D convolutional kernels. This parameter change allows leveraging pre-trained 2D network weights, transferring knowledge from 2D to 3D domain and reducing training time while maintaining effectiveness for 3D medical image analysis
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A computer-implemented method for identifying features in 3D image volumes includes dividing a 3D volume into a plurality of 2D slices and applying a pre-trained 2D multi-channel global convolutional network (MC-GCN) to the plurality of 2D slices until convergence. Following convergence of the 2D MC-GCN, a plurality of parameters are extracted from a first feature encoder network in the 2D MC-GCN. The plurality of parameters are transferred to a second feature encoder network in a 3D Anisotropic Hybrid Network (AH-Net). The 3D AH-Net is applied to the 3D volume to yield a probability map;. Then, using the probability map, one or more of (a) coordinates of the objects with non-maximum suppression or (b) a label map of objects of interest in the 3D volume are generated.