Method and device for generating pet-MRI fusion image by using artificial intelligence
The AI-based method generates accurate PET-MRI fusion images from MRI, addressing the challenges of separate PET and MRI scans by using a conditional GAN model, reducing costs and errors, and enhancing accessibility by utilizing MRI alone.
Patent Information
- Application Number
- PCT/KR2024/005932
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-22
- Filing Date
- 2024-05-02
- Publication Date
- 2024-11-28
AI Technical Summary
The existing methods of combining PET and MRI for imaging require separate tests, leading to errors due to time differences and posture changes, and the all-in-one PET-MRI device is costly and not universally accessible.
A method and device using AI, specifically a conditional generative adversarial network (GAN)-based generative model, to generate PET-MRI fusion images from MRI images, allowing for precise fusion without the need for separate PET and MRI scans, thereby reducing costs and errors.
Enables the generation of accurate PET-MRI fusion images using only MRI, reducing examination costs, radiation exposure, and increasing accessibility by leveraging the advantages of MRI over PET, while minimizing the disadvantages of the all-in-one PET-MRI device.
Smart Images

Figure KR2024005932_28112024_PF_FP_ABST
Abstract
Description
Method and device for generating PET-MRI fusion images using artificial intelligence
[0001] The present invention relates to a method and device for generating a fusion image of position emission tomography (PET) and magnetic resonance imaging (MRI) based on artificial intelligence (AI).
[0002] Attempts to combine molecular and high-resolution imaging have been ongoing for a long time, and PET and MRI are the most widely used imaging modalities in both clinical and neuroscience research. In particular, PET and MRI are known to have advantages in identifying the metabolic status and anatomic structure of lesions, respectively.
[0003] While MRI can indirectly predict metabolic status through image signal intensity, PET significantly outperforms MRI in cancer, infectious lesions, and other conditions. Previously, when both metabolic status and anatomical structure needed to be identified, separate PET and MRI scans were typically performed for comparison. However, when the two scans were performed while moving between the two examination equipment at different times, the images obtained from each scan often exhibit significant errors in metabolic status and anatomical location due to the time difference and changes in posture between the scans.
[0004] Meanwhile, the recently emerging integrated PET-MRI device is a type of medical imaging diagnostic equipment that simultaneously acquires PET and MRI images from a single lying position, then processes the images to create a fusion PET-MRI image. Integrated PET-MRI devices minimize errors that occur when performing PET and MRI separately, enabling them to produce precise fusion PET-MRI images.
[0005] However, integrated PET-MRI systems have the disadvantage of being more expensive and time-consuming than PET or MRI systems. Furthermore, these integrated PET-MRI systems are currently only available at select tertiary medical institutions, making them difficult to use widely.
[0006] [Prior Art Literature]
[0007] [Patent Document]
[0008] Republic of Korea Patent Publication No. 10-1836322
[0009] The present invention has been devised to solve the above problems, and the purpose of the present invention is to provide a method and device for generating a PET-MRI fusion image based on MRI using AI.
[0010] In order to achieve the above object, a method for generating a PET-MRI fusion image using artificial intelligence according to an embodiment of the present invention includes the steps of: when a patient identifier (ID) is input, loading MRI images corresponding to the patient ID and selecting one MRI image among them; and applying a conditional generative adversarial network (GAN)-based generation model trained to derive a PET-MRI fusion image of the same anatomical location as the MRI image from the MRI image, thereby generating a PET-MRI fusion image of the same anatomical location as the selected MRI image.
[0011] In order to achieve the above object, according to one embodiment of the present invention, a device for generating a PET-MRI fusion image using artificial intelligence comprises: an input / output unit for, when a patient identifier (ID) is input, loading MRI images corresponding to the patient ID and selecting one MRI image from among them; a storage unit for storing a conditional generative adversarial network (GAN)-based generation model trained to derive a PET-MRI fusion image of the same anatomical location as the MRI image from the MRI image; and a processor for applying the conditional GAN-based generation model to generate a PET-MRI fusion image of the same anatomical location as the selected MRI image.
[0012] According to one aspect of the present invention described above, the present invention provides a method and device for generating a PET-MRI fusion image based on MRI using AI, thereby enabling the advantages of PET to be utilized with only MRI without performing expensive PET, and overcoming the disadvantages of contrast agent use, long shooting time, high examination cost, and radiation exposure during PET-MRI examination.
[0013] In addition, it produces PET-MRI fusion images based on MRI, which is relatively inexpensive and accessible compared to PET, which is expensive and has low accessibility, thereby helping medical staff make decisions and having a positive effect on reducing overall medical costs.
[0014] FIG. 1 is a diagram showing an example of a generative model that generates a PET-MRI fusion image based on a conditional GAN according to an embodiment of the present invention.
[0015] FIG. 2 is a diagram showing an example of a platform applying a conditional GAN-based generative model according to an embodiment of the present invention to generate a PET-MRI fusion image.
[0016] FIG. 3 is a flowchart showing the PET-MRI fusion image generation operation of a platform to which a conditional GAN-based generative model according to an embodiment of the present invention is applied.
[0017] FIG. 4 is a flowchart showing the PET-MRI fusion image generation operation of a platform to which a conditional GAN-based generative model is applied according to another embodiment of the present invention.
[0018] FIG. 5 is a diagram showing an example of extracting a sagittal plane including a spinal column from an MRI image according to another embodiment of the present invention;
[0019] And, FIG. 6 is a block diagram showing a device for generating a PET-MRI fusion image according to an embodiment of the present invention.
[0020] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. It should be understood that the various embodiments of the present invention, while different from each other, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention. Furthermore, it should be understood that the positions or arrangements of individual components within each disclosed embodiment may be modified without departing from the spirit and scope of the present invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the present invention is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled, if properly described. Like reference numerals in the drawings designate the same or similar functionality throughout the several aspects.
[0021] The components according to the present invention are defined by functional distinctions rather than physical distinctions, and can be defined by the functions each component performs. Each component may be implemented as hardware or program code and processing units that perform each function, and the functions of two or more components may be implemented by including them in a single component. Therefore, the names given to the components in the following embodiments are not intended to physically distinguish each component, but rather to suggest the representative functions performed by each component, and it should be noted that the technical spirit of the present invention is not limited by the names of the components.
[0022] Before describing an embodiment of the present invention, the AI algorithm applied in the present invention will be described.
[0023] Synthetic medical data is defined as "nonexistent but plausible fake data" created based on actual medical data, such as clinical information, genomes, and life logs. It is created to be applied in specific environments where direct acquisition is not possible or to protect the privacy and confidentiality of data. A representative AI algorithm applied to the creation of synthetic medical data is the generative adversarial network (GAN) algorithm. Unlike existing deep learning algorithms, it consists of a generator that creates fake data and a discriminator that distinguishes between real and fake data. In this way, the two opposing systems can compete to create new medical image data.
[0024] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the drawings.
[0025] FIG. 1 is a diagram illustrating an example of a generative model that generates a PET-MRI fusion image based on a conditional GAN according to an embodiment of the present invention.
[0026] Referring to FIG. 1, a generation model according to an embodiment of the present invention is an artificial intelligence model trained to derive a PET-MRI fusion image of the same anatomical location as an MRI image from an MRI image, and the generation model is operated by a generator (106) and a discriminator (108).
[0027] In more detail, the generator (106) learns MRI images (102) and PET-MRI fusion images (104) for the same anatomical location obtained from an integrated PET-MRI device. That is, from hundreds of PET-MRI examinations, MRI images and PET-MRI fusion images for the same anatomical location for each PET-MRI examination are extracted and used as learning and verification data. The learned MRI images and PET-MRI fusion images are paired, and various sequence images can be used for the MRI images.
[0028] The generator (106) that has learned the MRI image (102) and PET-MRI fusion image (104) for the same anatomical location generates a PET-MRI fusion image for the same anatomical location as the MRI image (102).
[0029] The discriminator (108) performs learning to determine authenticity by comparing the generated PET-MRI fusion image (110) with the actual PET-MRI fusion image (112), and repeats the learning process until the similarity between the generated PET-MRI fusion image (110) and the actual PET-MRI fusion image (112) reaches the maximum. Here, the maximum refers to a threshold value used to measure the similarity between the generated PET-MRI fusion image (110) and the actual PET-MRI fusion image (112), and is a value indicating that the similarity between the two images is very high.
[0030] In addition, the discriminator (108) measures the error value (loss) between the generated PET-MRI fusion image (110) and the actual PET-MRI fusion image (112) and feeds it back to the generator (106). That is, the discriminator (108) updates the generation model (116) by feeding back the error value between the two images, and through this, the generator (106) can generate a PET-MRI fusion image that is more similar to the actual PET-MRI fusion image (112).
[0031] FIG. 2 is a drawing showing an example of a platform to which a generative model according to an embodiment of the present invention is applied to generate a PET-MRI fusion image.
[0032] Referring to FIG. 2, it is assumed that the platform (204) operates a conditional GAN-based generative model according to an embodiment of the present invention and that various software programs can be run on this platform.
[0033] When the driving program starts, the platform (204) receives an identifier (ID) of a patient for whom a PET-MRI fusion image is to be generated from the user, retrieves the MRI image of the patient from the medical image storage of each medical institution, and displays it on the screen. The MRI image displayed on the screen may be organized in the form of a list of MRI images by sequence.
[0034] The user selects an MRI image (202) of a desired sequence from a list of MRI images displayed on the screen and designates a region of interest among the selected MRI images (202). Here, the configuration of designating a region of interest among the MRI images (202) is not a mandatory configuration and may be omitted in some cases.
[0035] When an MRI image (202) selected by a user is input, the platform (204) applies a conditional GAN-based generation model according to an embodiment of the present invention to generate a PET-MRI fusion image (206) of the same anatomical location as the MRI image (202), and displays the generated PET-MRI fusion image (206) on the screen.
[0036] FIG. 3 is a flowchart illustrating a PET-MRI fusion image generation operation of a platform to which a conditional GAN-based generative model according to an embodiment of the present invention is applied.
[0037] Referring to FIG. 3, when the ID of a patient for whom a PET-MRI fusion image is to be created is input through a program being operated, the platform (S302) retrieves the MRI image of the patient from the medical image storage of each medical institution and displays a list of MRI images on the screen (S304). Here, the list of MRI images includes MRI images for various sequences.
[0038] When an MRI image is selected from the list of MRI images displayed in S304, (S306) a conditional GAN-based generation model according to an embodiment of the present invention is applied to the selected MRI image to generate a PET-MRI fusion image of the same anatomical location as the selected MRI image. (S308)
[0039] Afterwards, the platform displays the PET-MRI fusion image generated by S308 on the screen through the driving program (S310).
[0040] In Fig. 3, the operation of generating a PET-MRI fusion image of the same anatomical location as the entire area of the selected MRI image is described when an MRI image is selected. However, to further improve the performance of the conditional GAN-based generative model according to an embodiment of the present invention, it is possible to extract MRI images for a portion of the region of interest from the selected MRI image, and of course, it is also possible to generate a PET-MRI fusion image based on the extracted MRI image.
[0041] FIG. 4 is a flowchart illustrating a PET-MRI fusion image generation operation of a platform to which a conditional GAN-based generative model is applied according to another embodiment of the present invention.
[0042] Referring to FIG. 4, when the ID of a patient for whom a PET-MRI fusion image is to be created is input through a program being operated, the platform retrieves the MRI image of the patient from the medical image storage of each medical institution (S402) and displays a list of MRI images on the screen (S404). Here, the list of MRI images includes MRI images for various sequences.
[0043] When an MRI image is selected from the list of MRI images displayed in S404, (S406) a sagittal view including a portion of an area of interest, for example, the spinal column, is extracted from the selected MRI image. (S408) Here, the sagittal view refers to a virtual plane dividing the body into left and right sides, and it is assumed that the MRI image selected in S406 is a T2FS (T2 Fat Saturation) sequence MRI image. Here, the T2FS sequence is an MRI sequence designed for fat attenuation, which is a sequence that suppresses high signals from fat tissue to better view signals from other tissues.
[0044] In addition, the extraction operation of S408 can be performed by the user directly specifying on the screen to extract the sagittal plane including the spinal column, and as another example, the sagittal plane including the spinal column can be automatically extracted through repetitive machine learning of an AI model trained to extract the sagittal plane including the spinal column from an MRI image. The reason why only the sagittal plane including the spinal column is extracted from the MRI image and used to generate a PET-MRI fusion image is that the sagittal plane including various organs other than the spine on the left and right outer sides of the spinal column can deteriorate the performance of the generation model according to an embodiment of the present invention.
[0045] Thereafter, the platform applies a conditional GAN-based generation model according to an embodiment of the present invention to the MRI image extracted in S408 to generate a PET-MRI fusion image of the same anatomical location as the extracted MRI image (S410), and displays the generated PET-MRI fusion image on the screen through a driving program (S412).
[0046] FIG. 4 illustrates another embodiment of the present invention, in which the PET-MRI fusion image generation technology proposed in the present invention is applied to the spinal column as an example. However, the PET-MRI fusion image generation technology proposed in the present invention can be applied to any area of the body according to the process described in FIG. 3. In addition, although the present specification describes that the PET-MRI fusion image is generated using a T2FS sequence MRI image, it is of course possible to use other sequence images that are more suitable depending on the area of the body to which the PET-MRI fusion image generation technology is applied.
[0047] FIG. 5 is a drawing showing an example of extracting a sagittal plane including the spinal column from an MRI image according to another embodiment of the present invention.
[0048] Referring to FIG. 5, by extracting only the sagittal image (506) including the vertebral body (502) from among the sagittal images obtained in the horizontal plane of the spine in the MRI image (508) and excluding the sagittal image (504) located on the left and right outer sides of the vertebral body (502), an extracted MRI image used in an embodiment of the present invention can be obtained.
[0049] FIG. 6 is a block diagram illustrating a device for generating a PET-MRI fusion image according to an embodiment of the present invention.
[0050] Referring to FIG. 6, a PET-MRI fusion image generation device (600) according to an embodiment of the present invention includes an input / output unit (610), a storage unit (620), and a processor (630). In addition, although not shown, the PET-MRI fusion image generation device (600) according to an embodiment of the present invention may further include a communication unit for transmitting and receiving input / output data.
[0051] The PET-MRI fusion image generation device (600) receives a patient ID, an MRI to be used for PET-MRI fusion image generation, etc. from a user through an input / output unit (610), and generates a PET-MRI fusion image by applying the conditional GAN-based generation model proposed in the present invention to the input MRI through a processor (630). In addition, it outputs a list of MRI images of the corresponding patient and the generated PET-MRI fusion image, etc. through the input / output unit (610).
[0052] The PET-MRI fusion image generation device (600) stores a conditional GAN-based generation model applied when generating a PET-MRI fusion image in a storage unit (620), and temporarily or permanently stores data processed by other processors (630). The storage unit (620) may include a volatile storage medium or a non-volatile storage medium, but the scope of the present invention is not limited thereto.
[0053] In addition, when the PET-MRI fusion image generation device (600) generates a PET-MRI fusion image for a sagittal plane including a portion of an MRI image of interest, for example, the spinal column, as described in FIGS. 4 and 5, the storage unit (620) may store an AI model, etc. applied when extracting the sagittal plane including the spinal column.
[0054] The PET-MRI fusion image generation method proposed in the present invention can be implemented in the form of program commands that can be executed by various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium can include program commands, data files, data structures, etc., either singly or in combination.
[0055] The program commands recorded on the above computer-readable recording medium may be specially designed and configured for the present invention or may be known and available to those skilled in the art of computer software.
[0056] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, and flash memory.
[0057] Examples of program instructions include not only machine language codes, such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.
[0058] Although various embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.
[0059] [Explanation of symbols]
[0060] 502: Vertebrae of the spine
[0061] 504: Sagittal image of the left and right lateral surfaces of the spinal column
[0062] 506: Sagittal image including the vertebral body
[0063] 508: MRI image
[0064] 600: PET-MRI fusion image generation device
Claims
1. A method for generating a fusion image of positron emission tomography (PET) and magnetic resonance imaging (MRI) using artificial intelligence, When a patient identifier (ID) is entered, a process of loading MRI images corresponding to the patient ID and selecting one of the MRI images; A PET-MRI fusion image generation method comprising a process of generating a PET-MRI fusion image at the same anatomical location as the selected MRI image by applying a conditional generative adversarial network (GAN)-based generation model trained to derive a PET-MRI fusion image at the same anatomical location as the MRI image from the MRI image.
2. In paragraph 1, A process of extracting a sagittal view including the spinal column from the selected MRI image, A PET-MRI fusion image generation method further comprising a process of generating a PET-MRI fusion image of the same anatomical location as the extracted MRI image by applying the conditional GAN-based generative model.
3. In paragraph 2, A PET-MRI fusion image generation method characterized in that the above extraction process extracts a sagittal plane including the spinal column bones directly designated by the user, or automatically extracts a sagittal plane including the spinal column bones through repeated machine learning of an artificial intelligence model trained to extract a sagittal plane including the spinal column bones from an MRI image.
4. In paragraph 1, A PET-MRI fusion image generation method characterized in that the above-mentioned selected MRI image is a T2FS (T2 Fat Saturation) sequence MRI image designed for fat attenuation.
5. In paragraph 1, The above conditional GAN-based generative model operates by a generator and a discriminator. A PET-MRI fusion image generation method characterized in that the generator learns MRI images and PET-MRI fusion images of the same anatomical location obtained from an integrated PET-MRI device, thereby generating a PET-MRI fusion image of the same anatomical location as the selected MRI image, the discriminator compares the generated PET-MRI fusion image with an actual PET-MRI fusion image obtained from the integrated PET-MRI device, and feeds back an error value between the compared images to the generator, and the generator reflects the error value to generate a PET-MRI fusion image of the same anatomical location as the selected MRI image.
6. In a device that generates a fusion image of positron emission tomography (PET) and magnetic resonance imaging (MRI) using artificial intelligence, When a patient identifier (ID) is input, an input / output unit that retrieves MRI images corresponding to the patient ID and selects one MRI image among them; A storage unit that stores a conditional generative adversarial network (GAN)-based generative model that is trained to derive a PET-MRI fusion image of the same anatomical location as the MRI image from the MRI image, A PET-MRI fusion image generation device including a processor that generates a PET-MRI fusion image of the same anatomical location as the selected MRI image by applying the conditional GAN-based generative model.
7. In paragraph 6, A PET-MRI fusion image generation device characterized in that the processor extracts a sagittal view including the spinal column from the selected MRI image and applies the conditional GAN-based generation model to generate a PET-MRI fusion image of the same anatomical location as the extracted MRI image.
8. In paragraph 7, A PET-MRI fusion image generation device characterized in that the processor extracts a sagittal plane including the spinal column bone directly designated by the user, or automatically extracts a sagittal plane including the spinal column bone through repeated machine learning of an artificial intelligence model trained to extract a sagittal plane including the spinal column bone from an MRI image.
9. In paragraph 6, A PET-MRI fusion image generation device characterized in that the above-mentioned selected MRI image is a T2FS (T2 Fat Saturation) sequence MRI image designed for fat attenuation.
10. In paragraph 6, The above conditional GAN-based generative model operates by a generator and a discriminator. A PET-MRI fusion image generation device characterized in that the generator learns MRI images and PET-MRI fusion images of the same anatomical location obtained from an integrated PET-MRI device, thereby generating a PET-MRI fusion image of the same anatomical location as the selected MRI image, the discriminator compares the generated PET-MRI fusion image with an actual PET-MRI fusion image obtained from the integrated PET-MRI device, and feeds back an error value between the compared images to the generator, and the generator reflects the error value to generate a PET-MRI fusion image of the same anatomical location as the selected MRI image.
Citation Information
Patent Citations
Apparatus for transferring petㆍMRI fusion equipment
KR101836322B1
Ultra-low dose PET image reconstruction system and method based on deep learning
CN113808106A
System, apparatus and method for providing patient-specific diagnostic assistant information
KR101929953B1
Multimodal three-dimensional medical image fusion method and system, and electronic device
WO2021022752A1
KR20210110236A
