3D Medical Image Segmentation Using 2D Bounding Box Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic segmentation methods for 3D medical images, particularly brain MR images, face challenges in resource efficiency and reliability, especially for smaller organs with low contrast and varying shapes, leading to high computational demands and inaccuracies.
Innovation Solution
A two-stage method using 2D convolutional neural networks for initial localization followed by 3D segmentation, where 2D U-Nets determine a bounding box in three orthogonal orientations, and a 3D U-Net refines the segmentation within this box, reducing computational load and improving accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If fully automatic segmentation methods are used, then productivity is improved, but reliability deteriorates due to insufficient reliability when target anatomy differs from the norm
Solution Approach 1:
The patent applies segmentation by dividing the 3D medical image into multiple 2D slices along different orientations (axial, coronal, sagittal). Each slice is processed independently by 2D CNNs to generate probability maps, which are then integrated to form the final 3D segmentation. This approach improves reliability by breaking down the complex 3D problem into manageable 2D tasks while maintaining productivity through automated processing.
Solution Approach 2:
The patent transforms the 3D segmentation problem into a series of 2D classification problems by processing images in three orthogonal orientations. The 2D probability maps from each orientation are then integrated to reconstruct the 3D segmentation. This dimensionality change allows the use of computationally efficient 2D CNNs while achieving accurate 3D segmentation, resolving the contradiction between speed and accuracy.
2Manufacturing precision
If image registration-based methods are used, then manufacturing precision is improved, but use of energy worsens due to high computational demands
Solution Approach 1:
The patent segments the 3D volume into multiple 2D slices and processes each slice independently using 2D CNNs. This segmentation approach reduces computational complexity compared to processing the entire 3D volume at once, thereby reducing energy consumption while maintaining segmentation precision through the integration of multiple 2D probability maps.
Solution Approach 2:
The patent replaces traditional image registration-based methods with a machine learning approach using 2D and 3D CNNs. This substitution eliminates the computationally intensive registration process while achieving comparable or superior segmentation precision, thereby reducing energy consumption and computational resource requirements.
3Reliability
If manual outlining is used, then reliability is improved, but productivity deteriorates due to time-consuming labor
Solution Approach 1:
The patent implements self-service by training the 2D and 3D CNN models to perform segmentation automatically without requiring manual outlining. The system learns from training data and autonomously segments new images, eliminating the need for time-consuming manual work while maintaining high accuracy through the learned features from training.
Solution Approach 2:
The patent performs preliminary action by training the 2D and 3D CNN models on annotated training data before deployment. This preliminary training phase enables the model to automatically perform segmentation with high accuracy, replacing the need for manual outlining during actual use and significantly improving productivity while maintaining reliability.
4Manufacturing precision
If 3D segmentation is performed directly, then manufacturing precision is improved, but use of energy worsens due to computational complexity
Solution Approach 1:
The patent segments the 3D volume into multiple 2D slices and processes each slice independently using computationally efficient 2D CNNs. This segmentation approach reduces the computational burden compared to direct 3D segmentation while maintaining accuracy through the integration of probability maps from multiple orientations. The method then applies a 3D CNN only to the localized bounding box region, further optimizing resource usage.
Solution Approach 2:
The patent transforms the computationally intensive 3D segmentation problem into a series of 2D classification problems that can be solved more efficiently. By processing images in three orthogonal orientations and integrating the results, the method achieves accurate 3D segmentation with reduced computational energy compared to direct 3D processing.
5Use of energy by moving object
If 2D slice processing is used, then use of energy is reduced, but measurement precision deteriorates due to loss of 3D context
Solution Approach 1:
The patent processes images in three orthogonal orientations (axial, coronal, sagittal) and integrates the probability maps from each orientation to reconstruct the 3D segmentation. This multi-dimensional approach preserves 3D context information that would be lost in single-orientation 2D processing, thereby maintaining measurement precision while benefiting from the computational efficiency of 2D CNNs.
Solution Approach 2:
The patent merges the probability maps generated from three different orthogonal orientations to form the final 3D segmentation. By combining information from multiple views, the method preserves 3D context and improves measurement precision while maintaining the computational efficiency of 2D processing approaches.
6Use of energy by moving object
If bounding box localization is performed, then use of energy is reduced by focusing computation, but device complexity increases
Solution Approach 1:
The patent performs preliminary localization to identify a bounding box containing the target anatomy before performing detailed segmentation. This preliminary action focuses subsequent computational resources on the relevant region, reducing overall energy consumption. The localization step uses 2D CNNs on orthogonal slices to efficiently determine the bounding box coordinates.
Solution Approach 2:
The patent segments the processing task into two stages: (1) bounding box localization using 2D CNNs on orthogonal slices, and (2) detailed segmentation using 3D CNNs only within the localized bounding box. This segmentation of the processing pipeline reduces overall computational energy by avoiding processing of entire 3D volumes, while the added complexity is managed through modular implementation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, a system and a computer readable medium for automatic segmentation of a 3D medical image, the 3D medical image comprising an object to be segmented, the method characterized by comprising: carrying out, by using a machine learning model, in at least two of a first, a second and a third orthogonal orientation, 2D segmentations for the object in slices of the 3D medical image to derive 2D segmentation data; determining a location of a bounding box (10) within the 3D medical image based on the 2D segmentation data, the bounding box (10) having predetermined dimensions; and carrying out a 3D segmentation for the object in the part of the 3D medical image corresponding to the bounding box (10).