Two-Stage 3D Image Segmentation Under Memory Constraints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for segmenting 3D images, particularly brain MRI images, face challenges such as long processing times, poor performance due to large deviations from atlases, sub-optimal repeatability across rescan, and memory constraints when using convolutional neural networks (CNNs), leading to sub-optimal results and misalignment issues.
Innovation Solution
A two-stage neural network approach comprising a low-resolution module trained on low-resolution images to generate feature maps and a high-resolution module trained on high-resolution image patches, leveraging global context to produce complete high-resolution segmentations while managing memory constraints.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a multi-atlas registration algorithm is used for segmentation, then segmentation can be performed on high resolution scans, but the running time becomes very long (a few minutes to a few hours)
Solution Approach 1:
The patent replaces the traditional multi-atlas registration algorithm (mechanical/image processing system) with a deep learning-based neural network system. This substitution enables the system to achieve comparable or superior segmentation accuracy while dramatically reducing processing time from minutes/hours to seconds, as the neural network has learned optimal segmentation patterns during training.
Solution Approach 2:
The patent employs a two-stage processing approach where a low-resolution module first performs preliminary segmentation to generate initial feature maps and segmentations. These preliminary results are then refined by a high-resolution module. This preliminary action allows the system to capture global context quickly and then focus computational resources on refining local details, thereby reducing overall processing time while maintaining accuracy.
2Measurement precision
If convolutional neural networks are used on full 3D volumes to maintain global context, then segmentation quality improves, but memory requirements become prohibitively high
Solution Approach 1:
The patent divides the neural network into two distinct modules: a low-resolution module that processes downsampled volumes to capture global context, and a high-resolution module that processes original-resolution volumes for detailed segmentation. This segmentation of the network architecture allows each module to operate within feasible memory constraints while collectively achieving both global context understanding and high-quality segmentation.
Solution Approach 2:
The patent introduces low-resolution feature maps as an intermediary representation between the input volume and the final segmentation output. The low-resolution module extracts global contextual features from downsampled data, and these features serve as guidance for the high-resolution module. This intermediary approach enables the system to leverage global context without requiring the high-resolution module to process the entire high-resolution volume alone, thus managing memory requirements effectively.
3Ease of manufacture
If 3D images are separated into individual 2D images or smaller 3D patches for CNN processing, then memory constraints are satisfied, but global context is lost leading to sub-optimal results and misalignment issues
Solution Approach 1:
The patent merges the outputs of two separate processing streams: the low-resolution module that captures global context and the high-resolution module that provides local detail. By combining these complementary results, the system achieves both computational feasibility (through downsampling) and segmentation accuracy (through multi-scale feature integration), avoiding the pitfalls of processing methods that use only single resolution levels.
Data Source
AI summary
Systems and methods for segmenting 3D images are provided. In an embodiment, the system includes a neural network having a low-resolution module trained to infer a complete low-resolution segmentation from an input low-resolution 3D image and to generate corresponding low-resolution feature maps; and a high-resolution module trained to infer a complete high-resolution segmentation from an input high-resolution 3D image and the feature maps from the low-resolution module. Methods for training the neural network and measuring a volume of an object using the 3D segmentations are also described.