3D Medical Image Domain Conversion With 2D Slice Discriminators
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to generate high-resolution three-dimensional medical images across different imaging modalities due to the challenge of obtaining data with consistent resolution conditions, particularly when using low-resolution three-dimensional data for training, which limits the effectiveness of domain conversion in medical image processing.
Innovation Solution
A method utilizing a generative adversarial network with a three-dimensional convolutional neural network generator and two-dimensional discriminators to train a model that converts three-dimensional images across different domains, even when the input data has varying resolutions, by using isotropic processing and slice plane direction-specific discriminators to enhance image resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If low-resolution three-dimensional data is used for training, then the training process becomes simpler and faster, but the generated images have low resolution and poor quality
Solution Approach 1:
The three-dimensional data is divided into multiple two-dimensional slice images along different slice plane directions. By processing and training on these segmented 2D slices rather than the complete 3D volume, the system achieves faster training convergence while maintaining the ability to generate high-resolution three-dimensional generated images through the generator's volumetric synthesis capability.
Solution Approach 2:
The patent transitions from processing three-dimensional data directly to processing two-dimensional slice images extracted from different directions. This dimensional reduction allows the use of computationally efficient 2D convolutional neural networks for training while still enabling the generation of high-resolution 3D images through the generator's ability to synthesize volumetric data from multiple 2D perspectives.
2Manufacturing precision
If high-resolution three-dimensional data is obtained for all cross sections, then the image quality is high, but the data acquisition time and storage requirements increase significantly
Solution Approach 1:
Instead of requiring uniformly high resolution across all three-dimensional data and all cross-sectional views, the patent applies resolution enhancement selectively to two-dimensional slice images extracted from specific slice plane directions. The generator reconstructs high-resolution three-dimensional generated images by synthesizing information from these selectively processed 2D slices, achieving high quality output without the time and storage cost of acquiring and processing all possible high-resolution cross sections.
3Adaptability or versatility
If a three-dimensional convolutional neural network is used for domain conversion, then the model can process volumetric data effectively, but the computational complexity and training time increase
Solution Approach 1:
The patent segments the three-dimensional data into two-dimensional slice images and uses 2D convolutional neural networks for training instead of 3D CNNs. This segmentation reduces computational complexity and training time while the generator maintains the capability to process and generate volumetric data by synthesizing three-dimensional generated images from the processed 2D slices.
Solution Approach 2:
The patent substitutes the direct use of computationally intensive three-dimensional convolutional operations with a two-dimensional convolutional approach. By extracting 2D slice images from 3D data and processing them with 2D CNNs, the system achieves comparable volumetric processing capability with reduced computational burden, replacing the mechanical 3D convolution operations with 2D operations on sliced data.
Data Source
AI summary
A method of generating a trained model uses a first generator configured using a three-dimensional convolutional neural network that receives an input of a three-dimensional image of a first domain and that outputs a three-dimensional generated image of a second domain different from the first domain, and a first discriminator configured using a two-dimensional convolutional neural network that receives an input of a two-dimensional image indicating a cross section image in a first slice plane direction cut out from the three-dimensional generated image of the second domain and that discriminates authenticity of the input two-dimensional image. The method includes performing, by a computer, training the first generator and the first discriminator in an adversarial manner based on training data including a three-dimensional image captured under a first imaging condition and a three-dimensional image captured under a second imaging condition different from the first imaging condition.