Apparatus and method for medical image conversion based on artificial intelligence

Abstract

An artificial intelligence-based medical image conversion device and method are disclosed, and an artificial intelligence-based medical image conversion method according to one embodiment of the present invention may include: a step of collecting training data including a first training image, which is a first type of medical image captured to include a liver portion of a subject, and a second training image, which is a second type of medical image captured to include a liver portion of the subject; a step of training an artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to the second type, based on the target image when a target image corresponding to the first type is input based on the training data; a step of receiving the target image; and a step of generating the converted image based on the artificial intelligence model.

Description

Technology Field

[0001] The present invention relates to an artificial intelligence-based medical image conversion device and method. For example, the present invention relates to an artificial intelligence-based image analysis technique for diagnosing liver lesions using a Generative Adversarial Network (GAN). Background Technology

[0002] The diagnosis of liver cancer using artificial intelligence is currently an actively researched field, with studies incorporating AI into various diagnostic techniques ranging from imaging diagnostics using ultrasound, CT, and MRI to clinical diagnostics such as protein structure analysis.

[0003] For example, an artificial intelligence study on the differential diagnosis of liver lesions in contrast-enhanced CT (liver dynamic CT, LD CT) using deep learning based on a Convolutional Neural Network (CNN) was published in Radiology in 2018.

[0004] Meanwhile, when diagnosing liver cancer early, it is very difficult to differentiate between high-grade dysplastic nodules (small, less than 1 cm in size) and early hepatocellular carcinoma (early HCC), and these lesions often have the characteristic of not showing distinct contrast enhancement on LD CT and MRI.

[0005] Furthermore, while Magnetic Resonance Imaging (MRI) offers higher sensitivity and specificity compared to CT, it requires a long examination time. It is also difficult to perform on patients with metal artifacts, respiratory difficulties, or claustrophobia, and its high cost and bulk make it difficult for small medical institutions to operate. In contrast, although CT has the disadvantage of radiation exposure, it offers the advantages of faster scanning times resulting in fewer motion artifacts, higher spatial resolution than MRI, and easier accessibility as it is operated by most medical institutions.

[0006] In addition, Computed Tomography during hepatic arteriography (CTHA) is known as the most sensitive test for the early diagnosis of hepatocellular carcinoma, as it has higher sensitivity than CT and MRI for small hepatocellular carcinomas smaller than 1 cm. However, it has limitations in that it cannot be used as a screening test because it is an invasive test that requires puncturing the patient's femoral artery to position a catheter into the visceral artery and then performing a CT scan.

[0007] The technology forming the background of this invention is disclosed in Korean Registered Patent Publication No. 10-2321487. The problem to be solved

[0008] The present invention aims to solve the problems of the aforementioned conventional technology by providing an artificial intelligence-based medical image conversion device and method capable of implementing a highly sensitive CT HA image using an LD CT image that can be captured relatively easily as a source.

[0009] However, the technical problems that the embodiments of the present invention aim to solve are not limited to the technical problems described above, and other technical problems may exist. means of solving the problem

[0010] As a technical means for achieving the above-mentioned technical task, an artificial intelligence-based medical image conversion method according to one embodiment of the present invention may include: a step of collecting training data comprising a first training image, which is a first type of medical image captured to include a liver area of ​​a subject, and a second training image, which is a second type of medical image captured to include a liver area of ​​the subject; a step of training an artificial intelligence model that, based on the training data, generates a converted image, which is a virtual medical image corresponding to the second type, based on the target image when a target image corresponding to the first type is input; a step of receiving the target image; and a step of generating the converted image based on the artificial intelligence model.

[0011] In addition, the first type may be a dynamic CT (Computed Tomography) image, and the second type may be a hepatic artery contrast CT (Computed Tomography during hepatic arteriography, CTHA) image.

[0012] In addition, an artificial intelligence-based medical image conversion method according to one embodiment of the present invention may include, after the step of collecting the training data, a step of performing preprocessing to segment local regions corresponding to the liver region from each of the first training image and the second training image.

[0013] In addition, the step of training the artificial intelligence model may apply slice matching to the first training image and the second training image.

[0014] In addition, the step of training the artificial intelligence model may train the artificial intelligence model based on a Generative Adversarial Network algorithm.

[0015] In addition, the above-mentioned generative adversarial network algorithm may be an algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation).

[0016] In addition, the step of training the artificial intelligence model may construct the artificial intelligence model based on a plurality of loss functions.

[0017] In addition, the plurality of loss functions may include at least one of a GAN loss function, a Cycle loss function, an Identity loss function, and a CAM loss function.

[0018] Additionally, the plurality of loss functions may include a pixel-location loss function that considers pixel features of each of the first training image and the second training image in order to extract a similar texture from the shape of the liver region reflected in the first training image and the second training image.

[0019] In addition, an artificial intelligence-based medical image conversion method according to one embodiment of the present invention may include a step of mutually masking the target image and the converted image to derive analysis information regarding an area suspected of contrast enhancement and / or wash-out.

[0020] Meanwhile, an artificial intelligence-based medical image conversion method according to one embodiment of the present invention may include the step of acquiring a target image, which is a first type of medical image captured to include a liver area of ​​a subject, and the step of inputting the target image into a previously established artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to a second type that is distinguished from the first type.

[0021] In addition, the artificial intelligence model may be constructed based on training data collected to include a first training image captured to include the liver region as a first type of medical image and a second training image captured to include the liver region as a second type of medical image.

[0022] Meanwhile, an artificial intelligence-based medical image conversion device according to one embodiment of the present invention may include: a collection unit that collects learning data including a first learning image, which is a first type of medical image captured to include a liver area of ​​a subject, and a second learning image, which is a second type of medical image captured to include a liver area of ​​the subject; a learning unit that trains an artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to the second type, based on the target image when a target image corresponding to the first type is input based on the learning data; an input unit that receives the target image; and a conversion unit that generates the converted image based on the artificial intelligence model.

[0023] In addition, the learning unit can train the artificial intelligence model based on a Generative Adversarial Network algorithm.

[0024] In addition, the above-mentioned generative adversarial network algorithm may include an algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation).

[0025] In addition, the learning unit can construct the artificial intelligence model based on a plurality of loss functions.

[0026] In addition, an artificial intelligence-based medical image conversion device according to one embodiment of the present invention may include a diagnostic assistance unit that mutually masks the target image and the converted image to derive analysis information regarding an area suspected of contrast enhancement and / or wash-out.

[0027] Meanwhile, an artificial intelligence-based medical image conversion device according to one embodiment of the present invention may include an input unit for acquiring a target image, which is a first type of medical image captured to include a liver area of ​​a subject, and a conversion unit for inputting the target image into a pre-established artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to a second type that is distinguished from the first type.

[0028] The means for solving the problem described above are merely exemplary and should not be interpreted as intended to limit the present invention. In addition to the exemplary embodiments described above, additional embodiments may exist in the drawings and the detailed description of the invention. Effects of the invention

[0029] According to the means for solving the problem of the present invention described above, an artificial intelligence-based medical image conversion device and method can be provided that can realize a highly sensitive CT HA image using an LD CT image that can be captured relatively easily as a source.

[0030] However, the effects obtainable from this invention are not limited to those described above, and other effects may exist. Brief explanation of the drawing

[0031] FIG. 1 is a schematic diagram of a medical image providing system including an artificial intelligence-based medical image conversion device according to one embodiment of the present invention. Figure 2 is a diagram showing a comparison between a Type 1 medical image and a Type 2 medical image. FIG. 3 is a conceptual diagram schematically illustrating the entire process of a heterogeneous conversion technique of a medical image performed by an artificial intelligence-based medical image conversion device according to one embodiment of the present invention. Figure 4 is a conceptual diagram illustrating the process of training an artificial intelligence model that converts an input target image into a converted image, which is a virtual heterogeneous type of medical image. FIGS. 5a and 5b are drawings showing a comparison of an actual Type 1 image, an actual Type 2 image, and a converted image derived by the artificial intelligence-based medical image conversion device disclosed herein. FIG. 6 is a schematic diagram of an artificial intelligence-based medical image conversion device according to one embodiment of the present invention. FIG. 7 is a flowchart of the operation of an artificial intelligence-based medical image conversion method according to the first embodiment of the present invention. FIG. 8 is a flowchart of the operation of an artificial intelligence-based medical image conversion method according to the second embodiment of the present invention. Specific details for implementing the invention

[0032] Embodiments of the present invention are described below with reference to the attached drawings to enable those skilled in the art to easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0033] Throughout this specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" or "indirectly connected" with other elements interposed between them.

[0034] Throughout the entire specification, when a component is described as being located "on," "on top," "on top," "under," "on bottom," or "on bottom" of another component, this includes not only cases where the component is in contact with the other component but also cases where another component exists between the two components.

[0035] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0036] The present invention relates to an artificial intelligence-based medical image conversion device and method. For example, the present invention relates to an artificial intelligence-based image analysis technique for diagnosing liver lesions using a Generative Adversarial Network (GAN).

[0037] FIG. 1 is a schematic diagram of a medical image providing system including an artificial intelligence-based medical image conversion device according to one embodiment of the present invention.

[0038] Referring to FIG. 1, a medical image providing system (10) according to one embodiment of the present invention may include an artificial intelligence-based medical image conversion device (100) (hereinafter referred to as 'conversion device (100)'), a learning DB (200), a medical image capturing device (300), and a user terminal (400).

[0039] Meanwhile, the medical image providing system (10) may refer to a Picture Archiving and Communication System (PACS) established in conjunction with a hospital, medical institution, etc., and the conversion device (100) may be implemented as a separate device that acquires a medical image captured by a medical image capturing device (300) included in the medical image providing system (10) as a target image from the medical image capturing device (300) and outputs a virtual converted image corresponding to the target image, or may be implemented in a form mounted on the medical image capturing device (300) (for example, in the form of software, program, etc. installed on the medical image capturing device (300).

[0040] The conversion device (100), the learning DB (200), the medical imaging device (300), and the user terminal (400) can communicate with each other through a network (20). The network (20) refers to a connection structure that enables information exchange between each node, such as terminals and servers. Examples of such a network (20) include, but are not limited to, a 3GPP (3rd Generation Partnership Project) network, an LTE (Long Term Evolution) network, a 5G network, a WIMAX (World Interoperability for Microwave Access) network, the Internet, a LAN (Local Area Network), a Wireless LAN (Wireless Local Area Network), a WAN (Wide Area Network), a PAN (Personal Area Network), a Wi-Fi network, a Bluetooth network, a satellite broadcasting network, an analog broadcasting network, and a DMB (Digital Multimedia Broadcasting) network.

[0041] The learning DB (200) may be a database, server, etc., that stores learning data used in the training process for generating an artificial intelligence model of the conversion device (100), which will be described in detail below. More specifically, the learning DB (200) may store learning data including a first learning image, which is a first type of medical image taken of a subject, and a second learning image, which is a second type of medical image taken of the subject (the same subject). Additionally, the learning DB (200) may possess a learning data set that includes a first learning image and a second learning image for each of a plurality of subjects.

[0042] The medical imaging device (300) may be a device that captures a target image to be converted into a virtual medical image (converted image) of a heterogeneous type by the conversion device (100). In other words, the target image captured by the medical imaging device (300) may be transmitted to the conversion device (100) and converted into a virtual heterogeneous medical image.

[0043] According to one embodiment of the present invention, the medical imaging device (300) may be a Computed Tomography (CT) scanner, but is not limited thereto. As another example, the medical imaging device (300) may be an X-ray imaging device, a Magnetic Resonance Imaging (MRI) scanner, an ultrasound imaging device, etc. Additionally, depending on the type of medical imaging device (300), the medical image provided to the conversion device (100) (in other words, the target image, etc. described below) may correspond to a Computed Tomography (CT) image (CT image), an X-ray image, a Magnetic Resonance Imaging (MRI) image, an ultrasound image, etc.

[0044] Specifically, according to one embodiment of the present invention, the target image obtained by the medical imaging device (300), the first training image and the second training image used as a training data set for building the artificial intelligence model described below, may be DICOM (Digital Imaging and Communications in Medicine) images.

[0045] For reference, FIG. 1 is illustrated as having a conversion device (100) that is independent of the medical imaging device (300), but it is not limited thereto. According to an embodiment of the present invention, the conversion device (100) may be embedded in the medical imaging device (300) as a sub-module of the medical imaging device (300) and may operate to output a virtual image that is simulated as a heterogeneous type of medical image as described below, which is an image captured by the medical imaging device (300).

[0046] The user terminal (400) may be any type of wireless communication device, such as a smartphone, smartpad, tablet PC, PCS (Personal Communication System), GSM (Global System for Mobile communication), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), or Wibro (Wireless Broadband Internet) terminal. For example, the user terminal (400) may be a device that outputs at least one of a target image and a converted image, which is a virtual heterogeneous medical image converted from the target image.

[0047] Figure 2 is a diagram showing a comparison between a Type 1 medical image and a Type 2 medical image.

[0048] Referring to FIG. 2, the conversion device (100) disclosed herein may acquire (receive) a target image, which is a first type of medical image captured to include the liver area of ​​a subject, and input the acquired target image into a pre-established artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to a second type that is distinguished from the first type.

[0049] In this regard, in the description of the embodiments of the present invention, the first type of medical image may be a dynamic CT (Computed Tomography) image (“LD CT”, see FIG. 2(a)), and the second type of medical image may be a hepatic arteriography CT (Computed Tomography during hepatic arteriography, CTHA) image (“CT HA”, see FIG. 2(b)).

[0050] Referring to FIG. 2, Computed Tomography during hepatic arteriography (CTHA) is known to be the most sensitive test for early diagnosis of hepatocellular carcinoma, as it has a higher sensitivity than CT and MRI for small hepatocellular carcinomas of less than 1 cm. However, considering the limitation that it cannot be used as a screening test because it is an invasive test that requires puncturing the patient's femoral artery and positioning a catheter to the internal artery before performing CT, the conversion device (100) disclosed in this invention can operate to produce a CT HA image (a medical image of the second type) with high sensitivity using an LD CT image (a medical image of the first type) as a source, which can be taken non-invasively and is relatively easy to take.

[0051] In this regard, FIG. 3 is a conceptual diagram schematically illustrating the entire process of a heterogeneous conversion technique of a medical image performed by an artificial intelligence-based medical image conversion device according to one embodiment of the present invention.

[0052] Referring to FIG. 3, the conversion device (100) can collect learning data including a first learning image, which is a first type of medical image taken to include the liver area of ​​the subject, and a second learning image, which is a second type of medical image taken to include the liver area of ​​the subject.

[0053] Specifically, referring to FIG. 3, the conversion device (100) can collect a pair of first training images and second training images for the same subject as training data. In addition, the conversion device (100) can collect a first training image, which is a first type of medical image, and a second training image, which is a second type of medical image, together for each of a plurality of subjects and use them as a data set for training an artificial intelligence model described later.

[0054] Additionally, the conversion device (100) can perform preprocessing to segment local regions corresponding to liver regions from each of the first training image and the second training image obtained as training data. Specifically, the conversion device (100) may be equipped with an artificial intelligence-based segmentation algorithm for selectively extracting and segmenting liver regions reflected in each of the first training image and the second training image (liver segmentation in CT).

[0055] Additionally, the conversion device (100) can train an artificial intelligence model that generates a converted image, which is a virtual medical image corresponding to a second type, based on the input target image when a target image corresponding to a first type is input based on the collected training data.

[0056] Figure 4 is a conceptual diagram illustrating the process of training an artificial intelligence model that converts an input target image into a converted image, which is a virtual heterogeneous type of medical image.

[0057] Referring to FIG. 4, the conversion device (100) can train an artificial intelligence model based on a Generative Adversarial Network algorithm.

[0058] Meanwhile, according to one embodiment of the present invention, the generative adversarial network algorithm may include an algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation).

[0059] In this regard, the U-GAT-IT GAN algorithm is a deep learning-based algorithm that outputs a synthetic image by performing image-to-image translation. It can perform learning by applying image windowing to the first and second training images of the extracted liver region to emphasize the liver region and the lesion within the liver region, and by applying various types of augmentation techniques. The U-GAT-IT GAN algorithm-based model improves the characteristic of conventional artificial intelligence models that learn shapes and textures reflected in heterogeneous images asymmetrically, thereby enabling symmetric learning between shapes and textures, and thus shows superior performance compared to conventional GAN ​​models.

[0060] As another example, the transformation device (100) may train an artificial intelligence model based on a Generative Adversarial Network algorithm that includes a forward cycle and a backward cycle. Here, the Generative Adversarial Network algorithm that includes a forward cycle and a backward cycle may be referred to as CycleGAN, etc.

[0061] In this regard, the conversion device (100) can perform forward learning through a generator ('Generator' in FIG. 4) that generates a second type converted image, which is a virtual medical image corresponding to the second type, based on a first learning image, which is a first type medical image, and a classifier ('Classifier' in FIG. 4) that determines whether the generated second type converted image is authentic.

[0062] In addition, correspondingly, the generator of the conversion device (100) can generate a first-type converted image, which is a virtual medical image corresponding to the first type, based on a second learning image, which is a second-type medical image, and likewise, the discriminator can perform backward learning to determine the authenticity of the generated first-type converted image.

[0063] Specifically, for forward learning, the discriminator may determine the authenticity of a Type 2 transformed image, which is a simulated image generated by the generator, based on at least a portion of a Type 2 medical image, which is a Type 2 learning image, that was actually captured; similarly, for backward learning, the discriminator may determine the authenticity of a Type 1 transformed image, which is a simulated image generated by the generator, based on at least a portion of a Type 1 medical image, which is a Type 1 learning image, that was actually captured.

[0064] In summary, the conversion device (100) can perform a forward learning (Forward Cycle) to perform the reconstruction (conversion) process for the target image more precisely through the competitive repetition of a process of repeatedly generating (reconstructing) a virtual converted image that is modeled to correspond to the second type based on the input first type of medical image and a process of repeatedly determining the authenticity of the generated converted image, and a backward learning (Backward Cycle) to improve the accuracy (precision) of the reconstruction process applied to the second type of medical image through the competitive repetition of a process of repeatedly generating (reconstructing) a virtual converted image that is modeled to correspond to the first type based on the input second type of medical image and a process of repeatedly determining the authenticity of the generated converted image.

[0065] As described above, the artificial intelligence model in this institution, which has been trained through the training of an artificial intelligence model based on a generative adversarial network (e.g., CycleGAN) in which two distinct learning cycles for image reconstruction in opposite directions are repeated, can perform the operation of converting a medical image of a subject of type 1 (e.g., LD CT image) into a virtual transformed image of a heterogeneous type (type 2) (e.g., CT HA image) that simulates a target part of the subject's body (e.g., liver area, etc.) as if it were captured based on type 2.

[0066] Thus, the artificial intelligence model built (learned) through the conversion device (100) can predict (infer) three-dimensional volume data by reflecting changes in morphological features and texture features associated with a specific part of the subject's body reflected in the target image, and can generate a heterogeneous type of converted image (simulated image) corresponding to the target image.

[0067] Additionally, referring to FIG. 4, the transformation device (100) can build an artificial intelligence model based on a plurality of loss functions including at least one of a GAN loss function, a Cycle loss function, an Identity loss function, and a CAM loss function.

[0068] Specifically, the conversion device (100) can identify the amount of change between the input training image and the output virtual simulation image (Weighted Activation Map) by utilizing an activation map derived based on the downsampling result of the encoder side, and by reflecting this in the loss function in the artificial intelligence model training, it can perform training that considers the change information between the training image and the simulation image.

[0069] According to one embodiment of the present invention, the activation map may refer to a Grad-CAM (gradient-class activation map). Additionally, the activation map may be utilized to derive a basis region, which is a primarily transformed area, in the process of generating a virtual second-type transformed image based on an input first-type target image.

[0070] In addition, according to one embodiment of the present invention, the conversion device (100) can build an artificial intelligence model based on a pixel-location loss function ('PLLoss') that considers pixel features of each of the first training image and the second training image in order to extract a similar texture from the shape of the liver region reflected in the first training image and the second training image.

[0071] Meanwhile, referring to FIG. 3, the conversion device (100) can perform training of an artificial intelligence model by assigning weights to the texture and shape characteristics of an object (e.g., liver part, etc.) reflected in the collected training data ('style transfer').

[0072] For example, the conversion device (100) may apply slice matching to the first training image and the second training image. In other words, the conversion device (100) may automatically segment a local region corresponding to a liver area within an image using a predetermined labeling tool for the training of an artificial intelligence model, and then proceed with the training of the artificial intelligence model through slice matching between images (first training image and second training image) obtained at the same location of the same patient using an image in which the corresponding local region is selectively separated.

[0073] Additionally, the conversion device (100) can acquire a target image corresponding to the first type. For example, the conversion device (100) can receive a target image from the medical imaging device (300) that is taken of a subject (person being photographed) for the purpose of diagnosing liver lesions, etc. by the medical imaging device (300).

[0074] Meanwhile, the conversion device (100) can perform preprocessing to divide local regions corresponding to liver areas on the input target image, similar to the preprocessing process performed on training data.

[0075] Additionally, referring to FIG. 3, the conversion device (100) can generate a converted image (synthetic image) corresponding to an input target image based on a learned artificial intelligence model. That is, the conversion device (100) can generate a converted image, which is a virtual medical image corresponding to a second type that is distinguished from a first type, by inputting the target image into a pre-established artificial intelligence model.

[0076] Additionally, referring to FIG. 3, the conversion device (100) can derive analysis information linked to the diagnosis of liver cancer within the abdomen of a subject by utilizing a generated virtual conversion image. For example, the conversion device (100) can mutually mask the input target image and the conversion image generated from the target image to derive analysis information regarding areas suspected of contrast enhancement and / or wash-out as analysis information linked to the diagnosis of liver cancer.

[0077] In other words, the conversion device (100) may have a diagnostic assistance function that masks the portal phase or delayed phase image of the input target image (liver dynamic CT image) and the composite image (converted image) using the image conversion algorithm described in detail above, searches for the location of a lesion area that shows characteristics (findiction) suspected of contrast enhancement and suspected of washout in the delayed phase image, and displays the location on the output image.

[0078] FIGS. 5a and 5b are drawings showing a comparison of an actual Type 1 image, an actual Type 2 image, and a converted image derived by the artificial intelligence-based medical image conversion device disclosed herein.

[0079] Specifically, (a) of FIGS. 5a and 5b shows an actual CT during hepatic arteriography image (CT HA, Type 2 image), (b) shows an actual Liver dynamic CT image (LD CT, Type 1 image), and (c) shows a transformed image (synthetic CT image) derived by the transformation device (100).

[0080] Referring to FIGS. 5a and 5b, it can be seen that a lesion (indicated by a circle) diagnosed in the CT during hepatic arteriography with the highest sensitivity shows subtle contrast enhancement in liver dynamic CT, making it difficult to perceive with the human eye, but in an image where a transformation using an artificial intelligence model is applied based on the image, it can be identified as a lesion strongly suspected of having contrast enhancement. Accordingly, it can be seen that the transformation device (100) disclosed herein not only helps to find lesions that are difficult to perceive with the human eye in liver dynamic CT, but also makes the contrast enhancement of lesions appear more prominent, thereby substantially assisting users such as medical staff in detecting lesions.

[0081] FIG. 6 is a schematic diagram of an artificial intelligence-based medical image conversion device according to one embodiment of the present invention.

[0082] Referring to FIG. 6, the conversion device (100) may include a collection unit (110), a preprocessing unit (120), a learning unit (130), an input unit (140), a conversion unit (150), and a diagnostic assistance unit (160).

[0083] The collection unit (110) can collect learning data including a first learning image, which is a first type of medical image taken to include the liver area of ​​the subject, and a second learning image, which is a second type of medical image taken to include the liver area of ​​the subject.

[0084] The preprocessing unit (120) can perform preprocessing to divide local regions corresponding to liver parts from each of the first training image and the second training image obtained as training data.

[0085] In addition, the preprocessing unit (120) can perform preprocessing to divide local regions corresponding to liver areas on the input target image, similar to the preprocessing process performed on training data.

[0086] The learning unit (130) can train an artificial intelligence model that generates a transformed image, which is a virtual medical image corresponding to the second type, based on the input target image when a target image corresponding to the first type is input based on the learning data.

[0087] Specifically, the learning unit (130) can apply slice matching to the first learning video and the second learning video.

[0088] Additionally, the learning unit (130) can train an artificial intelligence model based on a Generative Adversarial Network algorithm. Specifically, according to one embodiment of the present invention, the Generative Adversarial Network algorithm may include an algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation).

[0089] Additionally, the learning unit (130) can build an artificial intelligence model based on a plurality of loss functions including at least one of a GAN loss function, a Cycle loss function, an Identity loss function, and a CAM loss function.

[0090] In addition, according to one embodiment of the present invention, the learning unit (130) can build an artificial intelligence model based on a pixel-location loss function that considers pixel features of each of the first learning image and the second learning image in order to extract a similar texture from the shape of the liver region reflected in the first learning image and the second learning image.

[0091] The input unit (140) can acquire a target image corresponding to the first type. For example, the input unit (140) can receive a target image from the medical imaging device (300) that is taken of a subject (person being photographed) for the purpose of diagnosing liver lesions, etc. by the medical imaging device (300).

[0092] The conversion unit (150) can generate a converted image corresponding to an input target image based on a learned artificial intelligence model. That is, the conversion unit (150) can generate a converted image, which is a virtual medical image corresponding to a second type that is distinguished from a first type, by inputting the target image into a pre-established artificial intelligence model.

[0093] The diagnostic aid (160) can mutually mask the target image and the transformed image generated by the transformation unit (150) to derive analysis information for areas suspected of contrast enhancement and / or wash-out.

[0094] Below, based on the details described above, we will briefly examine the operation flow of the present invention.

[0095] FIG. 7 is a flowchart of the operation of an artificial intelligence-based medical image conversion method according to the first embodiment of the present invention.

[0096] The artificial intelligence-based medical image conversion method according to the first embodiment of the present invention illustrated in FIG. 7 can be performed by the conversion device (100) described above. Therefore, even if the content described below is omitted, the description of the conversion device (100) can be equally applied to the description of the artificial intelligence-based medical image conversion method according to the first embodiment of the present invention.

[0097] Referring to FIG. 7, in step S11, the collection unit (110) can collect learning data including a first learning image, which is a first type of medical image taken to include the liver area of ​​the subject, and a second learning image, which is a second type of medical image taken to include the liver area of ​​the subject.

[0098] In addition, according to one embodiment of the present invention, after step S11, the preprocessing unit (120) may perform preprocessing to divide local regions corresponding to liver parts from each of the first training image and the second training image obtained as training data.

[0099] Next, in step S12, the learning unit (130) can train an artificial intelligence model that generates a transformed image, which is a virtual medical image corresponding to type 2, based on the input target image when a target image corresponding to type 1 is input based on the learning data.

[0100] Specifically, in step S12, the learning unit (130) can apply slice matching to the first learning image and the second learning image.

[0101] Additionally, in step S12, the learning unit (130) can train an artificial intelligence model based on a Generative Adversarial Network algorithm. Specifically, according to one embodiment of the present invention, in step S12, the learning unit (130) can build an artificial intelligence model using a Generative Adversarial Network algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation).

[0102] In addition, according to one embodiment of the present invention, in step S12, the learning unit (130) can build an artificial intelligence model based on a plurality of loss functions including at least one of a GAN loss function, a Cycle loss function, an Identity loss function, and a CAM loss function.

[0103] In addition, according to one embodiment of the present invention, in step S12, the learning unit (130) can build an artificial intelligence model based on a pixel-location loss function that considers the pixel features of each of the first learning image and the second learning image in order to extract a similar texture from the shape of the liver region reflected in the first learning image and the second learning image.

[0104] Next, in step S13, the input unit (140) can acquire a target image corresponding to the first type. For example, in step S13, the input unit (140) can receive from the medical imaging device (300) a target image taken of a subject (person being photographed) for the purpose of diagnosing liver lesions, etc. by the medical imaging device (300).

[0105] Next, in step S14, the conversion unit (150) can generate a converted image corresponding to the input target image based on a learned artificial intelligence model.

[0106] Next, in step S15, the diagnostic aid (160) can mutually mask the target image and the transformed image generated through step S14 to derive analysis information for areas suspected of contrast enhancement and / or wash-out.

[0107] In the description above, steps S11 through S15 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order of the steps may be changed.

[0108] FIG. 8 is a flowchart of the operation of an artificial intelligence-based medical image conversion method according to the second embodiment of the present invention.

[0109] The artificial intelligence-based medical image conversion method according to the second embodiment of the present invention illustrated in FIG. 8 can be performed by the conversion device (100) described above. Therefore, even if the content described below is omitted, the description of the conversion device (100) can be equally applied to the description of the artificial intelligence-based medical image conversion method according to the second embodiment of the present invention.

[0110] Referring to FIG. 8, in step S21, the input unit (140) can acquire a target image, which is a first type of medical image taken to include the liver area of ​​the subject.

[0111] Next, in step S22, the conversion unit (150) can input a target image into a pre-established artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to a second type that is distinguished from a first type.

[0112] In the description above, steps S21 to S22 may be further divided into additional steps or combined into fewer steps according to an embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order between steps may be changed.

[0113] An artificial intelligence-based medical image conversion method according to one embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate as one or more software modules to perform the operation of the present invention, and vice versa.

[0114] In addition, the aforementioned artificial intelligence-based medical image conversion method can also be implemented in the form of a computer program or application executed by a computer stored on a recording medium.

[0115] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical concept or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0116] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and the concept of equivalents thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0117] 10: Medical Imaging Provision System 100: AI-based medical image conversion device 110: Collection Department 120: Preprocessing section 130: Learning Department 140: Input section 150: Conversion unit 160: Diagnostic aid 200: Learning DB 300: Medical imaging device 400: User terminal 20: Network

Claims

Claim 1 A method for converting medical images based on artificial intelligence comprises: a step of collecting training data including a first training image, which is a first type of medical image captured to include a liver area of ​​a subject, and a second training image, which is a second type of medical image captured to include a liver area of ​​the subject; a step of training an artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to the second type, based on the target image when a target image corresponding to the first type is input based on the training data; a step of receiving the target image; and a step of generating the converted image based on the artificial intelligence model, wherein the first type is a Computed Tomography (CT) image and the second type is a Computed Tomography during hepatic arteriography (CTHA) image. Claim 2 delete Claim 3 A transformation method according to claim 1, further comprising, after the step of collecting the training data, a step of performing preprocessing to segment local regions corresponding to the liver region from each of the first training image and the second training image. Claim 4 A transformation method according to claim 1, wherein the step of training the artificial intelligence model is characterized by applying slice matching to the first training image and the second training image. Claim 5 A transformation method according to claim 1, wherein the step of training the artificial intelligence model is to train the artificial intelligence model based on a Generative Adversarial Network algorithm. Claim 6 A transformation method according to claim 5, characterized in that the generative adversarial network algorithm is an algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation). Claim 7 A transformation method according to claim 5, wherein the step of training the artificial intelligence model is to construct the artificial intelligence model based on a plurality of loss functions, wherein the plurality of loss functions includes at least one of a GAN loss function, a Cycle loss function, an Identity loss function, and a CAM loss function. Claim 8 A transformation method according to claim 7, wherein the plurality of loss functions further include a pixel-location loss function that considers pixel features of each of the first training image and the second training image to extract a similar texture from the shape of the liver region reflected in the first training image and the second training image. Claim 9 A transformation method according to claim 1, further comprising the step of mutually masking the target image and the transformed image to derive analysis information regarding areas suspected of contrast enhancement and / or wash-out. Claim 10 A method for converting medical images based on artificial intelligence comprises: a step of acquiring a target image, which is a first type of medical image captured to include a liver region of a subject; and a step of inputting the target image into a previously constructed artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to a second type distinguished from the first type, wherein the artificial intelligence model is constructed based on training data collected to include a first training image captured to include the liver region as the first type of medical image and a second training image captured to include the liver region as the second type of medical image, wherein the first type is a Computed Tomography (CT) image and the second type is a Computed Tomography during hepatic arteriography (CTHA) image. Claim 11 An artificial intelligence-based medical image conversion device comprising: a collection unit for collecting learning data including a first learning image, which is a first type of medical image captured to include a liver area of ​​a subject, and a second learning image, which is a second type of medical image captured to include a liver area of ​​the subject; a learning unit for training an artificial intelligence model that, based on the learning data, when a target image corresponding to the first type is input, generates a converted image, which is a virtual medical image corresponding to the second type, based on the target image; an input unit for receiving the target image; and a conversion unit for generating the converted image based on the artificial intelligence model, wherein the first type is a dynamic CT (Computed Tomography) image and the second type is a hepatic arteriography CT (Computed Tomography during hepatic arteriography, CTHA) image. Claim 12 delete Claim 13 A transformation device according to claim 11, wherein the learning unit trains the artificial intelligence model based on a Generative Adversarial Network algorithm, and the Generative Adversarial Network algorithm includes an algorithm based on U-GAT-IT GAN (Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation). Claim 14 A transformation device according to claim 13, wherein the learning unit constructs the artificial intelligence model based on a plurality of loss functions, and the plurality of loss functions include at least one of a GAN loss function, a Cycle loss function, an Identity loss function, and a CAM loss function. Claim 15 A transformation device according to claim 14, wherein the plurality of loss functions further include a pixel-location loss function that considers pixel features of each of the first training image and the second training image to extract a similar texture from the shape of the liver region reflected in the first training image and the second training image. Claim 16 A conversion device according to claim 11, further comprising a diagnostic aid unit that mutually masks the target image and the conversion image to derive analysis information regarding an area suspected of contrast enhancement and / or wash-out. Claim 17 An artificial intelligence-based medical image conversion device comprising: an input unit for acquiring a target image, which is a first type of medical image captured to include a liver region of a subject; and a conversion unit for inputting the target image into a previously constructed artificial intelligence model to generate a converted image, which is a virtual medical image corresponding to a second type distinguished from the first type, wherein the artificial intelligence model is constructed based on collected training data including a first training image captured to include the liver region as the first type of medical image and a second training image captured to include the liver region as the second type of medical image, wherein the first type is a dynamic CT (Computed Tomography) image and the second type is a hepatic arteriography CT (Computed Tomography during hepatic arteriography, CTHA) image.

Images