Medical Image Domain Conversion Using Body Coordinate Channels
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for converting medical images between different modalities, such as CT and MR, are susceptible to misregistration issues during training, leading to ineffective image conversion when datasets with significant misalignment are used.
Innovation Solution
Introduce coordinate information of the human body coordinate system into the training process, using three-dimensional convolutional neural networks to enhance the robustness of image conversion by incorporating x, y, and z coordinate data with image data, and perform adversarial training to improve the accuracy of image generation across different medical imaging modalities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing methods for converting medical images between different modalities are used, then image conversion can be performed, but the conversion is susceptible to misregistration issues during training, leading to ineffective image conversion when datasets with significant misalignment are used
Solution Approach 1:
The patent introduces coordinate information as an additional dimension to the image data. Specifically, x-coordinate information and z-coordinate information are added as new channels to the input image, transforming the data from 3D (height, width, channels) to 5D (height, width, channels, x-coordinate, z-coordinate). This allows the neural network to explicitly perceive spatial position and alignment, resolving the misregistration problem by incorporating positional information as an extra dimension rather than relying solely on image content matching.
2Adaptability or versatility
If datasets with significant misalignment are used for training, then more diverse training data is available, but the image conversion becomes ineffective due to misregistration
Solution Approach 1:
The patent introduces coordinate information as an intermediary that mediates between the image content and the alignment relationship. Instead of directly comparing images that may be misaligned, the coordinate channels serve as an intermediate representation that explicitly encodes spatial position. This intermediary allows the network to understand the relationship between images from different modalities even when they are misaligned, enabling effective training on diverse datasets with significant misalignment.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Provided are a method of generating a trained model, a machine learning system, a program, and a medical image processing apparatus capable of implementing training of domain conversion robust against a misregistration of data used for training. By using a learning model having a structure of a generative adversarial network including a first generator configured using a first convolutional neural network that receives an input of a medical image of a first domain and that outputs a first generated image of a second domain, and a first discriminator configured using a second convolutional neural network that receives an input of data including first image data, which is the first generated image or a medical image of the second domain included in a training dataset and coordinate information of a human body coordinate system corresponding to each position of a plurality of unit elements configuring the first image data, and that discriminates authenticity of the input image, a computer acquires a plurality of pieces of training data including the medical image of the first domain and the medical image of the second domain; and performs training processing.