Method and apparatus for converting medical images with improved clarity

Abstract

The present invention relates to a generative medical image conversion method, and more particularly, to a medical image conversion method and apparatus therefor, in which a learning model that uses a generative adversarial network (GAN) performs learning at the level of a feature map generated in the process of encoding a medical image, thereby outputting a medical image with improved clarity. According to one embodiment of the present invention, a medical image conversion device is provided that compares a feature map encoded in an input image with a feature map encoded in a target image and learns to reduce the difference between the images, thereby reducing the difference between the images in the feature map dimension, thereby further improving the clarity of the output result.

Description

[Technical Field]

[0001] One aspect of the present invention relates to a method and apparatus for converting medical images using artificial intelligence, and more particularly, to a method and apparatus for converting medical images using artificial intelligence, in which a learning model that uses a generative adversarial network (GAN) performs learning at the level of a feature map generated during the encoding of the medical image, thereby outputting a medical image with improved clarity. [Background technology]

[0002] The material described in this section merely provides background information for embodiments of the present invention and may not constitute prior art.

[0003] For the diagnosis and treatment of patients with cerebral hemorrhage and tumors, medical staff can use non-enhanced CT images, which do not use contrast enhancers, and enhanced CT images, which do use contrast enhancers.

[0004] Contrast-enhanced CT images are a type of medical image taken to search for blood vessels, various organs, cancer, etc., using a contrast agent to increase tissue contrast.

[0005] However, contrast agents often cause side effects, and in severe cases can lead to death due to cardiac arrest, shock, and other causes.

[0006] Since side effects caused by contrast agents cannot be predicted in advance, if image conversion can be used to convert non-contrast-enhanced CT or MR images into contrast-enhanced CT or MR images, such side effects can be prevented in advance and medical costs associated with contrast agents can be reduced.

[0007] In addition, in cases where MR images are available but X-ray irradiation is not possible and CT images are not available, such as pregnant women, or in cases where CT images are available but MR images cannot be taken due to time, cost, or implants, image conversion between imaging modalities still allows medical staff to determine tissue boundaries or the location of tumors without the use of contrast agents.

[0008] In addition, it can help medical staff better understand information about lesions through conversion between MR images, for example, conversion from T1 image to T2 image, or conversion from T1 image to diffusion image.

[0009] Meanwhile, a typical learning method for machine learning models is to extract feature maps from input data, synthesize output data based on these, and then compare the synthesized output data with the target data to reduce the difference. The feature maps extracted in this process provide important information for generating output data, so if the appropriate information is not extracted from the input data, the clarity of the output results will be reduced.

[0010] The above-mentioned background art is technical information that the inventor possessed for the purpose of deriving the embodiments of the present invention or that he acquired in the process of deriving the embodiments of the present invention, and cannot necessarily be said to be publicly known art that was disclosed to the general public before the filing of the embodiments of the present invention. Summary of the Invention [Problem to be solved by the invention]

[0011] In order to solve the above-mentioned problems, one aspect of the present invention is to provide a medical image conversion device that can improve the clarity of output results by comparing a feature map encoded from an input image with a feature map encoded from a target image and learning to reduce the difference between the two, thereby reducing the difference between the images in the feature map dimension.

[0012] Another object of the present invention is to provide a medical image conversion method that enables medical staff to perform medical examinations, diagnoses, and treatments with accurate information through conversion between medical images, thereby enabling patients to receive appropriate medical services.

[0013] The technical problems that the present invention aims to achieve are not limited to the technical problems mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art to which the present invention pertains from the following description. [Means for solving the problem]

[0014] In order to achieve the above-mentioned object, one aspect of the present invention is an artificial intelligence learning method for medical image conversion using a generative adversarial network (GAN), comprising:

[0015] The learning method includes: synthesizing a third image with a first image; comparing the third image with the second image; and reflecting the results of the comparison in learning.

[0016] a first step of encoding the first image to generate a first feature map;

[0017] a second step of encoding the second image to generate a second feature map; and

[0018] a third step of comparing the first feature map with the second feature map and performing learning in a direction to reduce a difference between the first feature map and the second feature map.

[0019] According to an embodiment, the second feature map may be generated in a process of encoding the second image and synthesizing the second image.

[0020] According to an embodiment, the first feature map may be generated in a process of encoding the first image and synthesizing the third image.

[0021] According to an embodiment, the difference derived in the third step may be fed back to the process of generating the first feature map in the first step.

[0022] In some embodiments, the first image may be input data input as learning data to an artificial intelligence learning model, the third image may be an image in a different domain from the first image, and the second image may constitute a paired dataset with the first image.

[0023] Another aspect of the present invention is a medical image conversion device that converts medical images using a generative adversarial network (GAN), comprising:

[0024] a first learning model that extracts first image features from the first image and generates a third image;

[0025] a second learning model for extracting second image features from the second image; and

[0026] a difference extraction unit that extracts differences between the second image features and the first image features to improve the clarity of the third image and feeds the differences back to the first learning model;

[0027] It is possible to provide a medical image conversion device including the above.

[0028] According to an embodiment, the first learning model may be characterized by receiving an input of a first image and generating a third image based on the first image.

[0029] According to an embodiment, the second learning model may be characterized by receiving an input of a second image and generating the second image based on the second image.

[0030] According to an embodiment, the first image feature may be a feature map obtained by encoding the first image, and the second image feature may be a feature map obtained by encoding the second image.

[0031] According to an embodiment, the first image may be input data input as learning data to an artificial intelligence learning model, the third image may be an image in a different domain from the first image, and the second image may constitute a paired data set with the first image. [Effects of the Invention]

[0032] As described above, according to one embodiment of the present invention, a medical image conversion device is provided that can improve the clarity of output results by comparing a feature map encoded from an input image with a feature map encoded from a target image and learning to reduce the difference between them, thereby reducing the difference between the images in terms of the feature map dimension.

[0033] The present invention has an effect of providing a medical image conversion method that enables medical staff to perform medical examinations, diagnoses, and treatments with accurate information through conversion between medical images, thereby enabling patients to receive appropriate medical services.

[0034] In addition, the present invention has various effects such as excellent versatility depending on the embodiment, and such effects can be clearly confirmed in the description of the embodiments below. [Brief explanation of the drawings]

[0035] The following drawings attached to this specification illustrate one embodiment of the present invention and, together with the detailed description of the invention described above, serve to further understand the technical concept of the present invention, and therefore the present invention is not limited to the details shown in such drawings.

[0036] [Figure 1] FIG. 1 illustrates a medical image conversion method according to one embodiment of the present invention. [Figure 2] FIG. 2 illustrates an artificial intelligence learning model according to one embodiment of the present invention. [Figure 3] FIG. 3 shows an embodiment of calculating the match rate between the output image and the target image. [Figure 4] FIG. 4 illustrates an artificial intelligence learning method according to another embodiment of the present invention. [Figure 5] FIG. 5 shows a medical image conversion device according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0037] The advantages and features of the present invention, and methods for achieving them, will become clearer with reference to the following detailed description of the embodiments accompanied by the accompanying drawings. However, the present invention is not limited to the following embodiments, but may be embodied in various different forms, and should be understood to include all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. The following embodiments are provided to ensure a complete disclosure of the present invention and to fully convey the scope of the invention to those skilled in the art. In describing the present invention, if a detailed description of related prior art is considered to obscure the gist of the present invention, such a detailed description will be omitted.

[0038] The terms used in this application are merely used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

[0039] In this application, the terms "comprise" or "have" are intended to specify the presence of a specified feature, number, step, operation, component, part, or combination thereof, and should be understood as not precluding the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof. Terms such as "first," "second," etc. may be used to describe various components, but the components should not be limited by these terms. These terms are used only to distinguish one component from another.

[0040] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, identical or corresponding components will be given the same drawing numbers, and duplicate descriptions thereof will be omitted.

[0041] FIG. 1 illustrates a medical image conversion method according to an embodiment of the present invention, and FIG. 2 illustrates an artificial intelligence learning model according to an embodiment of the present invention.

[0042] According to one embodiment of the present invention, there is provided a medical image conversion method for converting medical images using a generative adversarial network (GAN).

[0043] The medical image conversion method according to the present embodiment includes an input step (S100) of receiving a first image 110 selected from a paired data set including a first image 110 and a second image 120; a generation step (S110) of generating a third image 130 corresponding to the first image 110 and having a domain different from that of the first image 110 based on the first image 110;

[0044] The method may further include a learning step (S120) of comparing the third image 130 with the second image 120 and learning a learning model in consideration of the comparison result.

[0045] The first image 110 and the second image 120 may represent medical images. Here, the medical images may include magnetic resonance (MR) images, such as 2D MR, 3D MR, 2D streaming MR, 4D MR, 4D volumetric MR, and 4D cine MR; functional MR images, such as fMR, DCE-MR, and diffusion MR; computed tomography (CT) images, such as 2D CT, conebeam CT, 3D CT, and 4D CT; ultrasound images, such as 2D ultrasound, 3D ultrasound, and 4D ultrasound; positron emission tomography (PET) images; X-ray images; fluoroscopic images; radiotherapy portal images; single-photon emission computed tomography (SPECT) images; and computer-generated synthetic images, such as pseudo-CT.

[0046] In addition, the medical image may include medical image data, such as a training image, a ground truth image, a contoured image, a dose image, and the like.

[0047] Depending on the embodiment, the medical image may include an image acquired from an image acquisition device, a computer-generated image, etc. The image acquisition device may include, for example, an MR imaging device, a CT imaging device, a PET imaging device, an ultrasound imaging device, a fluoroscopic device, a SPECT imaging device, an integrated linear accelerator and MR imaging device, etc. In addition, the image acquisition device is not limited to the above examples and may include various medical imaging devices within the scope of the technical concept for acquiring a medical image of a patient.

[0048] The paired data set may include a set of training data including images of an object, for example, a specific part of a patient.

[0049] A CT image and an MR image can be taken of a specific part of a patient, for example, the head, using a CT imaging device and an MR imaging device, respectively, and in this case, the CT image and the MR image can form a paired data set.

[0050] When non-enhanced CT images and MR images are taken of a specific area of ​​a patient, the non-enhanced CT images and MR images can form a paired data set, and when contrast-enhanced CT images and MR images are taken of a specific area of ​​a patient, the contrast-enhanced CT images and MR images can form a paired data set.

[0051] When T1 and T2 images are taken of a specific part of a patient, the T1 and T2 images can form a paired data set.

[0052] Here, the specific part of a patient may refer to a specific part of the same patient, or may refer to a specific part of different patients as long as the parts being imaged are the same.

[0053] For example, CT images and MR images of the same specific area of ​​a patient can form a paired data set, and CT images and MR images of the same area of ​​different patients can also form a paired data set.

[0054] What does it mean for two images to have different domains? For example, if the contrast of the patterns contained in the two corresponding images differs between the two images, or if the intensity distribution or pattern differs between the two images, then the domains are different.

[0055] In addition, images taken with different imaging devices have different domains. For example, CT images and MR images are taken with CT imaging devices and MR imaging devices, respectively, so they have different domains. Similarly, there are different domains between ultrasound images and CT images, between ultrasound images and MR images, and between MR images and PET images.

[0056] Furthermore, non-contrast-enhanced images and contrast-enhanced images have different intensity distributions for the same region, and therefore have different domains.

[0057] In the case of MR images, T1 images and T2 images are the same MR images, but they have different intensity distributions, so they have different domains. In other words, if there is a difference in the signal, sequence, or image creation process that is the basis for image creation between two images created through reconstruction based on signals provided by an individual, the two image information with the corresponding difference will have different domains. In addition, the domains of the above-mentioned examples of medical images are different.

[0058] To explain a medical image conversion method according to an embodiment of the present invention, an example will be described in which an image acquisition device acquires a first image 110 and a second image 120 of the same part of the same patient and obtains paired data sets. In this embodiment, the first image 110 is an original CT image, the second image 120 is an original MR image, and the third image 130 is a combined MR image.

[0059] The input step (S100) may include a step of receiving, as input to a learning model, a first image 110 selected from a paired dataset including a first image 110 and a second image 120. Here, the first image 110 may include input data obtained by processing an original CT image using learning data.

[0060] The manufacturer of the software medical device that performs the medical image conversion method according to this embodiment can determine which of the first image 110 and the second image 120 to select as input data in the paired data set including the first image 110 and the second image 120. In this case, the image to be selected can be automatically selected by a selection algorithm or manually selected by an operator.

[0061] A manufacturer can create a paired dataset by acquiring original CT images using a CT imaging device and original MR images using an MR imaging device, or can select the original CT images as input data for the paired dataset using an algorithm or manually.

[0062] The generation step (S110) may include a step in which a learning model that receives an input of a first image 110 generates a third image 130 based on the first image 110, corresponding to the first image 110 and having a different domain from the first image 110.

[0063] The learning model can receive original CT images as input data and generate synthetic MR images based on the original CT images.

[0064] The process in which the learning model generates the third image 130 from the first image 110, which has a domain different from that of the first image 110, may be performed through the following two processes according to an embodiment. However, the process in which the learning model generates the third image 130 is not limited to the following two processes.

[0065] The first process is composed of one or more layers that receive a first image 110 acquired by an image acquisition device and perform convolution operations to extract high-dimensional features smaller than the first image 110, and this can be expressed as an artificial neural network that extracts features. Such an artificial neural network that extracts features receives input of 2D or 3D actual image data acquired by an image acquisition device and performs convolution operations to extract image features from the original image.

[0066] To exemplarily explain this process, a convolution layer extracts image features through a filter and a pooling layer strengthens the features and reduces the size of the image. Image features can be extracted by repeating convolution and pooling.

[0067] According to an embodiment, in a convolution layer configured with multiple layers, the first convolution layer may extract features of input data provided as an input and output a feature map as a result, the pooling layer may receive the input of this feature map, and output a result in which the features are enhanced and the image size is reduced, and the output of the pooling layer may again be input to the second convolution layer.

[0068] The second step is composed of one or more layers that perform deconvolution operations to generate image data of different modalities from the result of the first step, and can be expressed as an artificial neural network that generates images of different modalities. Such an artificial neural network that generates images of different modalities can generate two-dimensional or three-dimensional image data of different modalities by performing deconvolution operations from the results of the convolution operation performed by the artificial neural network that extracts features in the first step.

[0069] The learning step (S120) may include a step of comparing the third image 130 with the second image 120 and training a learning model based on the comparison results. If the original CT image is determined as input data from among the original CT image and the original MR image included in the paired dataset, the original MR image can be determined as the target image. Here, the second image 120 may correspond to ground truth. After generating the third image 130, the learning model may compare the generated third image 130 with the second image 120 and perform learning to reduce the difference between the two. That is, the learning model may continuously compare the synthetic MR image with the original MR image to determine whether the synthetic MR image has been generated to be close to the original MR image, thereby training the synthetic MR image to be the same as the original MR image. According to an embodiment, the method may include a step of calculating a loss value between the third image 130 and the second image 120 by applying a loss function to the algorithm, and updating the parameters of the learning model based on the loss value.

[0070] According to an embodiment, in the step of calculating the loss value, the calculation of the loss value for updating the learning model may be performed for each iteration. That is, the learning model may calculate the loss value between the synthetic MR image and the original MR image for each iteration while performing learning.

[0071] When the number of repetitions includes a first repetition interval and a second repetition interval that are connected sequentially, depending on the embodiment, if the absolute value of the change between the first loss value calculated in the first repetition interval and the second loss value calculated in the second repetition interval is smaller than a reference value, the learning model may regard this as overfitting and terminate learning.

[0072] Once the learning model has sufficiently learned medical image conversion, it can complete the learning without further learning. The learning model that has completed the learning will now have the ability to convert medical images into third images 130 that have a different domain from the first images 110 for various first images 110 each having a different data distribution.

[0073] According to an embodiment of the present invention, the learning model 200 may be a GAN model and may include a generator and a classifier. The generator may perform image generation learning, and the classifier may perform image segmentation learning.

[0074] Referring to FIG. 2, in a learning model 200 to which a GAN model is applied, when an original CT image 110 is input as input data to a generator 210, the generator 210 performs learning to generate a synthetic MR image 130 through the above-mentioned convolution operation, and a classifier 220 can perform learning to distinguish between the original MR image 120, i.e., the actual medical image, and the synthetic MR image 130.

[0075] According to an embodiment, a cycle-GAN model (not shown) may be applied to the learning model 200 according to this embodiment. In this case, the learning model 200 may be configured to include a first generator 210, a first division molecule 220, a second generator, and a second division molecule. When the original CT image 110 is input as input data to the first generator 210, the first generator 210 performs learning to generate a synthetic MR image 130 through the convolution operation described above, and the first division molecule 220 may perform learning to distinguish between the original MR image 120, i.e., the actual medical image, and the synthetic MR image. Furthermore, when the synthetic MR image 130 generated by the first generator 210 is input to the second generator, the second generator may perform learning to generate a synthetic CT image based on the synthetic MR image 130, and the second division molecule may perform learning to distinguish the synthetic CT image from the original CT image.

[0076] Here, the original CT image 110 corresponds to the first image 110 in the above-described medical image conversion method, the synthesized MR image 130 corresponds to the third image 130, and the original MR image 120 corresponds to the second image 120.

[0077] According to an embodiment, the third step (S120) may include calculating a match rate between the third image 130 and the second image 120 and feeding the calculated value back to the learning model. Here, calculating the match rate is a method for determining how much difference there is between two images belonging to the same domain.

[0078] FIG. 3 shows an embodiment of calculating the match rate between the output image and the target image.

[0079] In some embodiments, if a lesion area is displayed in the same location in the composite MR image and the original MR image, an expert can calculate the match rate by comparing the size of the lesion area displayed in each image. For example, the positions of tumors and other lesions can be displayed in the composite MR image and the original MR image, and the match rate can be calculated by switching between binary images.

[0080] In this case, the match rate can be calculated using evaluation indices such as the Hausdorff distance or the Dice similarity coefficient, or can be quantitatively calculated based on the difference in the centers of mass of the two lesions. If the target and output images are CT images, the match rate between the two images can be determined by comparing the calculated radiation doses for the lesions. Depending on the embodiment, the difference between the synthesized and original MR images can be quantified using MAE, RMSE, SSIM, PSNR, etc. (hereinafter referred to as performance indices) for determination.

[0081] Referring to Figure 3, the process of aligning the lesion area between the synthesized MR image 130, which is the output image, and the original MR image 120, which is the target image, and then applying the Hausdorff distance measurement method to compare the contours of each lesion and calculate the matching rate is shown.

[0082] 4 shows an artificial intelligence learning method according to another embodiment of the present invention, and FIG. 5 shows a medical image conversion device according to another embodiment of the present invention.

[0083] An artificial intelligence learning method according to an embodiment of the present invention may include an artificial intelligence learning method for medical image conversion using a generative adversarial network (GAN).

[0084] The AI ​​learning method according to this embodiment may synthesize a third image 130 using a first image 110, compare the third image 130 with the second image 120, and reflect the results of comparing the third image 130 with the second image 120 in learning. For example, if the first image 110 is an original CT image, the second image 120 is an original MR image, and the third image 130 is a synthetic MR image, the AI ​​learning method according to this embodiment may receive the original CT image as input, generate a synthetic MR image based on the original CT image using a generative adversarial network (GAN), compare the synthetic MR image with the original MR image, and reflect the comparison results in learning. Here, reflecting in learning includes reflecting in the learning process of an AI learning model and upgrading the parameters of the learning model.

[0085] The AI ​​learning method according to this embodiment includes a first step (S200) of encoding the first image 110 to generate a first feature map;

[0086] A second step (S210) of encoding the second image 120 to generate a second feature map; and

[0087] and a third step (S220) of comparing the first feature map with the second feature map and performing learning in a direction to reduce a difference between the first feature map and the second feature map.

[0088] According to an embodiment, the second feature map may be generated in a process of encoding the second image 120 and synthesizing the second image 120.

[0089] According to an embodiment, the first feature map may be generated in a process of encoding the first image 110 and synthesizing the third image 130.

[0090] According to an embodiment, the difference generated in the third step (S220) may be fed back to the process of generating the first feature map in the first step (S200).

[0091] In some embodiments, the first image 110 may be input data input as learning data to an artificial intelligence learning model, the third image 130 may be an image in a different domain from the first image 110, and the second image 120 may constitute a paired dataset with the first image 110.

[0092] For example, if the first image 110 is an original CT image, the third image 130 is a synthetic MR image generated based on the original CT image, and the second image 120 is an original MR image that forms a paired data set together with the original CT image,

[0093] In this embodiment, when the learning model compresses information of the original CT image in the process of generating a synthetic MR image from the original CT image, the feature map may contain compressed information of the image features of the original CT image. For example, the feature map may be an array containing information such as lines and curves. When the learning model receives the original CT image and performs a convolution operation, the feature map is obtained as a result of the convolution operation.

[0094] In this embodiment, encoding may include extracting image feature information from the first image 110 and compressing it in the process of generating the third image 130 from the first image 110. In this embodiment, the learning model may encode the first image 110, e.g., the original CT image, to extract a first feature map, and encode the second image 120, e.g., the original MR image, to extract a second feature map. In this case, the first feature map may include image features of the original CT image, and the second feature map may include image features of the original MR image.

[0095] According to the present invention, the learning model compares a first feature map extracted in the process of generating a third image 130 from a first image 110 with a second feature map extracted from a second image 120 that constitutes a paired data set together with the first image 110, calculates the difference, and reflects the result in the learning of the learning model to upgrade the learning model at the feature map level. More specifically, the learning model performs learning in a direction to reduce the difference between the first feature map and the second feature map. According to this embodiment, the difference between the first and second feature maps calculated in the third step (S220) is fed back to the process of the learning model generating the first feature map in the first step (S200), thereby updating the learning model. As a result, the learning model performs learning in a direction to make the first feature map more similar to the second feature map.

[0096] In this embodiment, the second feature map may be extracted in the process of the learning model encoding the second image 120 to generate the second image 120. That is, the second feature map may include image features of the second image 120.

[0097] The extraction of the second feature map can be performed by encoding the second image 120 and then decoding it again to generate the second image 120, for example, by compressing the original MR image and then decoding it again to generate the original MR image. That is, it can be performed by restoring the original MR image. Since the second feature map extracted by this process reflects the image features of the original MR image, the second feature map is applied to the process of generating a synthetic MR image from the original CT image, so that the synthetic MR image accurately reflects the original MR image.

[0098] Here, the learning model for generating the second feature map may be the same as the learning model for generating the first feature map, or may be a different learning model, depending on the embodiment.

[0099] The learning model generates synthetic MR images based on compressed information of the image features of the original CT image. Therefore, if this compressed information does not contain information appropriate for converting to MR, the synthetic MR images generated based on this will not be able to properly represent the MR information.

[0100] If the image features compressed from the original CT image (the input data) at the feature map level are similar to the image features of the original MR image (the target image), a synthetic MR image is generated based on the image features extracted from the original CT image, so the resulting synthetic MR image can more accurately reflect the original MR image (the target image).

[0101] Therefore, a synthetic MR image with improved clarity and similar to the original can be output.

[0102] According to another embodiment of the present invention, a medical image conversion device that converts medical images using a generative adversarial network (GAN) can be provided.

[0103] The medical image conversion device according to this embodiment includes a first learning model 200 that extracts first image features from a first image 110 to generate a third image 130, a second learning model 300 that extracts second image features from a second image 120, and

[0104] In order to improve the clarity of the third image 130, the third image 130 may be configured to include a difference extraction unit 400 that extracts differences between the second image features and the first image features and feeds the differences back to the first learning model 200.

[0105] According to an embodiment, the first learning model 200 may receive the first image 110 as an input and generate the third image 130 based on the first image 110 .

[0106] According to an embodiment, the second learning model 300 may receive the second image 120 as an input and generate the second image 120 based on the second image 120 .

[0107] According to an embodiment, the first image feature may include a feature map encoding the first image 110, and the second image feature may include a feature map encoding the second image 120.

[0108] As described above, depending on the embodiment, the first image 110 may be input data input as learning data to an artificial intelligence learning model, the third image 130 may be an image in a different domain from the first image 110, and the second image 120 may constitute a paired dataset with the first image 110.

[0109] The medical image conversion device according to this embodiment may perform the medical image conversion method according to the above-described embodiment. Here, the first learning model 200 may receive, for example, an original CT image as input data, encode the original CT image to extract a feature map of the original CT image, and decode the original CT image to output a synthetic MR image. That is, the first learning model 200 may extract the first feature map during the process of encoding the original CT image.

[0110] The second learning model 300 may receive a target image, for example, an original MR image constituting a paired dataset with an original CT image as input data, encode the image to extract a feature map of the original MR image, and decode the image to output the original MR image. That is, the second learning model 300 may extract a second feature map in the process of encoding the original MR image.

[0111] In this embodiment, the difference extraction unit 400 compares the first feature map extracted by the first learning model 200 with the second feature map extracted by the second learning model 300, calculates the difference, and reflects the result back in the learning process of the first learning model 200. The first learning model 200 can receive the difference calculation result value fed back by the difference extraction unit 400 and update the parameters of the learning model in a direction that reduces the difference between the first feature map and the second feature map.

[0112] According to the present invention, the first learning model 200 is trained to reduce the difference between the first feature map and the second feature map, thereby generating a synthetic MR image that is close to the original MR image.

[0113] The above-described embodiments of the present invention may be implemented in the form of a computer program that can be executed by various components on a computer, and such a computer program may be recorded on a computer-readable medium, which may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROMs, RAMs, and flash memories.

[0114] Meanwhile, the computer program may be one specially designed and constructed for the present invention, or one that is known and available to those skilled in the art of computer software. Examples of the computer program include not only machine language code, such as that produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc.

[0115] In the present specification (particularly in the claims), the use of the term "said" and similar indicators may be applicable to both the singular and the plural. Furthermore, when a range is described in the present invention, it is considered to include inventions to which individual values ​​within the range are applied (unless otherwise specified), and is the same as if each individual value constituting the range were described in the detailed description of the invention.

[0116] Unless explicitly stated or stated to the contrary, steps constituting the method of the present invention may be performed in any suitable order. The present invention is not necessarily limited to the order of steps described above. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by the claims. Furthermore, those skilled in the art will recognize that various modifications, combinations, and variations can be made within the scope of the appended claims or their equivalents, depending on design conditions and factors.

[0117] Therefore, the concept of the present invention should not be limited to the above-described embodiments, and not only the scope of the claims described below, but also all scopes equivalent to or modified equivalently from the scope of the claims belong to the scope of the concept of the present invention. [Explanation of symbols]

[0118] 110 First Image 120 Second Image 130 Third Image 200 First Learning Model 300 Second Learning Model 400 Difference extraction part

Claims

1. An artificial intelligence learning method for medical image conversion using a generative adversarial network (GAN), The learning method includes: synthesizing a third image with a first image; comparing the third image with the second image; and reflecting the result of the comparison in learning; a first step of encoding the first image to generate a first feature map; a second step of encoding the second image to generate a second feature map; and a third step of comparing the first feature map with the second feature map and performing learning in a direction to reduce a difference between the first feature map and the second feature map. An artificial intelligence learning method for medical image conversion.

2. The second feature map is generated in a process of encoding the second image and synthesizing the second image. The artificial intelligence learning method for medical image conversion according to claim 1.

3. The first feature map is generated by encoding the first image and synthesizing the third image. The artificial intelligence learning method for medical image conversion according to claim 1.

4. The difference derived in the third step is fed back to the process of generating the first feature map in the first step. The artificial intelligence learning method for medical image conversion according to claim 1.

5. The first image is input data input as learning data to an artificial intelligence learning model, the third image is an image in a different domain from the first image, and the second image constitutes a paired dataset with the first image. The artificial intelligence learning method for medical image conversion according to claim 1.

6. A medical image conversion device that converts medical images using a generative adversarial network (GAN), a first learning model that extracts first image features from the first image and generates a third image; a second learning model for extracting second image features in the second image; and a difference extraction unit that extracts a difference between the second image feature and the first image feature to improve the clarity of the third image and feeds the difference back to the first learning model. A medical image conversion device characterized by:

7. The first learning model receives a first image as an input and generates a third image based on the first image.

7. The medical image conversion device according to claim 6.

8. The second learning model receives an input of a second image and generates the second image based on the second image.

7. The medical image conversion device according to claim 6.

9. The first image feature is a feature map encoding the first image, and the second image feature is a feature map encoding the second image.

7. The medical image conversion device according to claim 6.

10. The first image is input data input as learning data to an artificial intelligence learning model, the third image is an image in a different domain from the first image, and the second image constitutes a paired dataset with the first image.

7. The medical image conversion device according to claim 6.

Images