Adversarial Network for Medical Image Modality Simulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current medical imaging technologies face challenges in obtaining complementary information from different imaging modalities or sequences, as there is no direct method to convert data from one modality or sequence to another, limiting the availability of specific types of information.
Innovation Solution
An apparatus and method using a deep learning-based adversarial network to simulate medical image data from one modality or sequence to another, by training an image synthesizer and discriminator to produce realistic images that mimic the characteristics of the target modality or sequence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If multiple imaging modalities are acquired to obtain complementary information, then information completeness is improved, but device complexity and acquisition time increase
Solution Approach 1:
The patent creates a simulated copy of the target modality image from the source modality image using deep learning. The synthesizer generates a simulated T2-weighted image from a T1-weighted image, providing complementary information without requiring actual T2 acquisition, thus reducing device complexity and scan time while maintaining information completeness
Solution Approach 2:
The system performs preliminary simulation of the target modality image before actual acquisition. By generating a simulated T2 image from T1 data in advance, the system can perform registration and analysis without requiring the actual T2 scan, thereby reducing overall acquisition time and complexity
2Loss of information
If multiple imaging modalities are acquired to obtain complementary information, then information completeness is improved, but acquisition time increases
Solution Approach 1:
The synthesizer creates a simulated copy of the target modality image from the source modality image. This allows the system to obtain complementary information that would normally require a separate T2-weighted scan, thereby reducing acquisition time while maintaining information completeness
Solution Approach 2:
The deep learning synthesizer serves multiple functions: it generates simulated images for registration, provides complementary information, and reduces acquisition time. This multi-functional approach eliminates the need for separate acquisition sequences, improving efficiency while maintaining diagnostic information quality
3Ease of operation
If direct registration between different modalities is performed, then workflow simplicity is improved, but registration accuracy deteriorates
Solution Approach 1:
The patent introduces a simulated target modality image as an intermediary between the source and actual target images. The synthesizer creates a simulated T2 image from T1 data, which then serves as the intermediary for registration with the actual T2 image, improving accuracy while maintaining workflow simplicity
Solution Approach 2:
The registration process is segmented into two stages: first, register the simulated image to the actual target image (which has similar characteristics and achieves high accuracy); second, transfer the transformation parameters to register the source image to the target image. This segmentation improves overall registration accuracy while keeping the workflow manageable
Data Source
AI summary
There is provided an apparatus comprising processing circuitry configured to: receive first medical image data obtained using a first type of imaging procedure, wherein the first medical image data is representative of an anatomical region of a subject; and apply a simulator to perform a simulation process on the first medical image data to obtain simulated second medical image data, the simulated second medical image data having properties so as to simulate image data that is obtained using a second type of imaging procedure. The simulator comprises an image synthesizer that is trained in combination with a discriminator in an adversarial fashion by repeatedly alternating an image synthesizer training process in which the image synthesizer is trained to produce simulated medical image data, and a discriminator training process in which the discriminator is trained to distinguish between real medical image data and simulated medical image data.