Medical image conversion method, device, and computer program using style condition-based diffusion model

Abstract

A medical image conversion method using a style condition-based diffusion model is a method performed by a computing device and comprises the steps of: acquiring a first medical image; acquiring a style key indicating a style of a medical image that is to be generated on the basis of the acquired first medical image; and generating a second medical image reflecting the style corresponding to the acquired style key by converting the acquired first medical image via a pre-trained artificial intelligence model by using the acquired style key as a condition, wherein the pre-trained artificial intelligence model is a diffusion model in which a correlation between the first medical image and the second medical image and a correlation between the style of the second medical image and the style key are trained.

Description

Medical image transformation method, device and computer program using a style condition-based diffusion model

[0001] The present disclosure relates to a method, device and computer program for transforming medical images using a style condition-based diffusion model.

[0002] Image transformation technology has recently attracted significant attention in the medical field. This is because medical image analysis and processing play a crucial role in medical diagnosis and treatment planning. In particular, medical imaging techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) provide precise diagnostic information, but accurate analysis can be difficult if the acquired data is complex or contains noise. To address these issues, technologies that transform or correct medical images are essential.

[0003] Advances in image transformation technology have been made possible primarily by the introduction of deep learning.

[0004] Deep learning offers powerful tools for automatically performing various transformation tasks by learning image patterns. For example, deep learning-based generative models, such as generative adversarial networks (GANs) and diffusion models, are effective at transforming images from one domain to another. In medical imaging, these technologies are being utilized for tasks such as noise removal, resolution enhancement, and conversion between various image modalities. However, existing deep learning-based technologies often fail to adequately reflect the unique characteristics of medical imaging, limiting their application in real-world clinical settings.

[0005] This is particularly problematic when dealing with 3D medical images. To process 3D images, the image is typically divided into 2D slices, processed, and then reassembled. While this approach improves computational efficiency and reduces the complexity of deep learning models, it can also lead to stylistic or structural inconsistencies during the conversion process.

[0006] For example, because each 2D slice is processed independently, the final reconstructed 3D image may lose its original structural consistency or be visually distorted. Furthermore, given the critical importance of continuity between slices in medical imaging, these inconsistencies can reduce diagnostic accuracy and reliability.

[0007] The background technology described above is something that the inventor possessed or acquired in the process of deriving the disclosure of the present application, and cannot necessarily be said to be a publicly known technology disclosed to the general public prior to the present application.

[0008] One object of the present disclosure is to provide a method, device and computer program for converting medical images using a style condition-based diffusion model.

[0009] An object of the present disclosure is to provide a method, apparatus, and computer program for converting a medical image using a style condition-based diffusion model, which converts a first medical image to generate a second medical image of a different type, and generates a second medical image in which a style corresponding to the style key is reflected by converting the first medical image as a condition of a style key indicating the style of the medical image to be generated, thereby generating a medical image in which a desired style is reflected, and which can secure a result in the same style by adjusting / controlling the style of the result data based on the style key.

[0010] A method for converting a medical image using a style condition-based diffusion model according to one embodiment is a method performed by a computing device, comprising the steps of: obtaining a first medical image; obtaining a style key indicating a style of a medical image to be generated; and generating a second medical image in which a style corresponding to the acquired style key is reflected by converting the obtained first medical image with the obtained style key as a condition through a pre-learned artificial intelligence model, wherein the pre-learned artificial intelligence model may be a diffusion model in which a correlation between the first medical image and the second medical image and a correlation between the style of the second medical image and the style key are learned.

[0011] In one embodiment, the first medical image obtained is a CT image extracted from 3D CT data corresponding to one plane, and the second medical image obtained is generated by transforming the CT image and may be an MRI image corresponding to one plane.

[0012] In one embodiment, the acquired style key may include at least one of an intensity histogram representing a brightness distribution of a medical image to be generated using the first medical image, a cumulative histogram representing a cumulative distribution of brightness values, and a differential histogram representing a rate of change between brightness values.

[0013] In one embodiment, the diffusion model may be a Brownian Bridge Diffusion Model (BBDM).

[0014] In one embodiment, the diffusion model may be a model trained to minimize a preset loss function by using learning data that uses a specific first medical image and a specific style key as input data, and a specific second medical image generated by transforming the specific first medical image and reflecting a style corresponding to the specific style key as correct data.

[0015] In one embodiment, the preset loss function may be expressed as in the following mathematical expression 1.

[0016] <Mathematical Formula 1>

[0017]

[0018] Here, the above is the expected value, above is a latent variable at a specific point in time Intermediate state image in the above The first medical image is the initial image, is the style key, above is the actual noise and the above may be noise predicted by the diffusion model.

[0019] In one embodiment, the preset loss function may be expressed as in the following mathematical expression 2 when the diffusion model is a Brownian bridge diffusion model.

[0020] <Mathematical Formula 2>

[0021]

[0022] Here, the above is the expected value, above The second medical image is the target image, is a latent variable at a specific point in time Intermediate state image in the above is the weight, above is the interpolation coefficient, above is the time-step noise coefficient, The first medical image is the initial image, is the style key, above is the actual noise and the above may be noise predicted by the diffusion model.

[0023] In one embodiment, the step of generating the second medical image may include a step of generating a second medical image having a histogram corresponding to the acquired style key by transforming the acquired first medical image based on the acquired style key, if the acquired style key is a histogram of the medical image to be generated.

[0024] In one embodiment, the step of generating the second medical image may include, when there are a plurality of first medical images obtained, a step of generating a plurality of second medical images in which a style corresponding to the obtained style key is identically reflected by converting the plurality of first medical images based on the obtained style key.

[0025] A medical image conversion device using a style condition-based diffusion model according to one embodiment includes a processor, a network interface, a memory, and a computer program loaded into the memory and executed by the processor, wherein the computer program may include an instruction for obtaining a first medical image, an instruction for obtaining a style key indicating a style of a medical image to be generated, and an instruction for generating a second medical image in which a style corresponding to the obtained style key is reflected by converting the obtained first medical image based on the obtained style key through a pre-learned artificial intelligence model, wherein the pre-learned artificial intelligence model is a diffusion model in which a correlation between a first medical image and a second medical image and a correlation between a style of the second medical image and the style key are learned.

[0026] In one embodiment, a computer-readable non-transitory recording medium may be provided that records instructions for executing a method for converting a medical image using a style condition-based diffusion model, the method comprising: obtaining a first medical image; obtaining a style key indicating a style of a medical image to be generated; and generating a second medical image in which a style corresponding to the acquired style key is reflected by converting the acquired first medical image based on the acquired style key through a pre-trained artificial intelligence model, wherein the pre-trained artificial intelligence model is a diffusion model in which a correlation between the first medical image and the second medical image and a correlation between the style key and the style of the second medical image are learned.

[0027] A method, device, and computer program for converting a medical image using a style condition-based diffusion model according to one embodiment convert a first medical image to generate a second medical image of a different type, and generate a second medical image in which a style corresponding to the style key is reflected by converting the first medical image as a condition of a style key indicating a style of a medical image to be generated, thereby generating a medical image in which a desired style is reflected, and by adjusting / controlling the style of the result data based on the style key, it is possible to secure a result in the same style.

[0028] The effects of the medical image conversion method, device and computer program using a style condition-based diffusion model according to an embodiment are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0029] The following drawings attached to this specification illustrate preferred embodiments of the present disclosure and, together with the detailed description of the invention, serve to further understand the technical idea of ​​the present disclosure, and therefore, the present disclosure should not be interpreted as being limited to matters described in such drawings.

[0030] Figure 1 is a drawing showing an MRI image and a target MRI image converted according to a conventional technique.

[0031] FIG. 2 is a flowchart of a medical image transformation method using a style condition-based diffusion model according to one embodiment of the present disclosure.

[0032] FIG. 3 is a diagram illustrating a medical image conversion process using a style condition-based diffusion model according to one embodiment of the present disclosure.

[0033] FIG. 4 is a diagram illustrating a second medical image and a target MRI image generated according to a method for converting medical images using a style condition-based diffusion model according to one embodiment of the present disclosure.

[0034] FIG. 5 is a diagram illustrating a hardware configuration of a medical image conversion device using a style condition-based diffusion model according to one embodiment of the present disclosure.

[0035] The various embodiments described in this specification are exemplified for the purpose of clearly explaining the technical concept of the present disclosure and are not intended to be limited to specific embodiments. The technical concept of the present disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combined from all or part of the embodiments described herein. Furthermore, the scope of the technical concept of the present disclosure is not limited to the various embodiments presented below or the specific descriptions thereof.

[0036] Terms used herein, including technical or scientific terms, unless otherwise defined, may have the meaning commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0037] As used herein, expressions such as "includes," "may include," "comprises," "may have," "have," and "may have" indicate the presence of a target feature (e.g., a function, operation, or component), but do not exclude the presence of other additional features. In other words, such expressions should be understood as open-ended terms that imply the possibility of including a second embodiment.

[0038] In this specification, singular expressions include plural expressions unless the context clearly indicates otherwise. Furthermore, plural expressions include singular expressions unless the context clearly indicates otherwise. When a part of the specification is said to include a component, this does not exclude other components, but rather implies that other components may be included, unless otherwise specifically stated.

[0039] Also, the term 'module' or 'part' used in the specification means a software or hardware component, and the 'module' or 'part' performs certain roles. However, the 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside on an addressable storage medium and may be configured to execute one or more processors. Thus, as an example, the 'module' or 'part' may include at least one of components such as software components, object-oriented software components, class components, and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. The functionality provided within the components and 'modules' or 'parts' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0040] According to one embodiment of the present disclosure, a 'module' or 'unit' may be implemented as a processor and a memory. 'Processor' should be broadly construed to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, and the like. In some circumstances, a 'processor' may also refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), and the like. A 'processor' may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations. In addition, 'memory' should be broadly construed to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, etc. Memory is said to be in electronic communication with the processor if the processor can read information from, and / or write information to, the memory. Memory integrated in a processor is in electronic communication with the processor.

[0041] As used herein, the expressions “first,” “second,” or “first,” “second,” etc., unless the context indicates otherwise, are used to refer to multiple similar objects and to distinguish one object from another, and do not limit the order or importance among the objects.

[0042] As used herein, the expressions "A, B, and C," "A, B, or C," "A, B, and / or C," or "at least one of A, B, and C," "at least one of A, B, or C," "at least one of A, B, and / or C," "at least one selected from A, B, and C," "at least one selected from A, B, or C," "at least one selected from A, B, and / or C," and the like can mean each listed item or all possible combinations of the listed items. For example, "at least one selected from A and B" can refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) both A and B.

[0043] The expression "based on" as used herein is used to describe one or more factors that influence the decision, act of judgment, or action described in the phrase or sentence containing the expression, and this expression does not exclude additional factors that influence the decision, act of judgment, or action.

[0044] As used herein, the expression that a component (e.g., a first component) is “connected” or “connected” to another component (e.g., a second component) may mean that the component is directly connected or connected to the other component, as well as connected or connected via a new other component (e.g., a third component).

[0045] The expression "configured to" used herein may have the meanings of "set to", "having the ability to", "modified to", "made to", "capable of", etc., depending on the context. The expression is not limited to the meaning of "specifically designed in hardware", and for example, a processor configured to perform a specific operation may mean a generic-purpose processor that can perform the specific operation by executing software.

[0046] Hereinafter, various embodiments of the present disclosure will be described with reference to the attached drawings. In the attached drawings and the description of the drawings, identical or substantially equivalent components may be assigned the same reference numerals. Furthermore, in the description of various embodiments below, duplicate descriptions of identical or corresponding components may be omitted, but this does not mean that the corresponding components are not included in the embodiments.

[0047] Deep learning-based image transformation technology is a fundamental and widely applicable technology that can be applied to any problem that requires predicting another image from a specific one. In particular, in medical imaging, various problems such as image modality transformation (e.g., between MRI modalities, such as T1 MRI and T2FLAIR MRI, and between CT and MRI), denoising in PET / CT / MRI, artifact removal (e.g., motion artifacts, beam hardening in CT, magnetic susceptibility artifacts in MRI), super-resolution, and image restoration (inpainting) can all be interpreted as image transformation problems.

[0048] Many medical images, such as PET, CT, and MRI, often contain 3D data, which is volumetric and created by stacking 2D images. This data can be trained and transformed slice-by-slice using 2D encoder-decoder models (e.g., deep learning networks). For example, axial, sagittal, and coronal sections can be processed separately.

[0049] However, these 2D-based approaches (hereinafter referred to as "prior art") can lead to slice inconsistencies. This causes slice-level inferences to fail to maintain inter-slice connectivity. In particular, structural continuity is crucial for 3D data at the volume level, and such discontinuities can manifest as image quality degradation or artifacts.

[0050] Figure 1 is a drawing showing an MRI image and a target MRI image converted according to a conventional technique.

[0051] Referring to Fig. 1, when comparing an MRI image generated using a conventional technique with a target MRI image, it can be seen that style SI, such as brightness and contrast, appears in the MRI image converted using the conventional technique.

[0052] Meanwhile, 3D models have the advantage of resolving discontinuities between slices, but there are some technical limitations.

[0053] First, it requires high memory usage. When comparing 3D models with the same batch size and model capacity, memory usage is significantly higher than that of 2D models, necessitating a high-performance GPU for training 3D models.

[0054] Second, due to this high memory usage, 3D models are trained with much smaller batch sizes than 2D models, which leads to reduced learning speed and stability.

[0055] Third, 3D models require more data than 2D models. Because the data dimensionality encoded by 3D models is relatively much larger, they require more data to learn the high-dimensional spatial manifold.

[0056] Accordingly, according to the medical image conversion method using a style condition-based diffusion model according to various embodiments of the present disclosure, the problem of style mismatch among the slice mismatch problems that occur during image conversion, which is a problem of the prior art, can be resolved. This will be described in more detail below with reference to FIGS. 2 to 4.

[0057] Here, the methods described with reference to FIGS. 2 to 4 may be performed through a medical image conversion device (100) (hereinafter referred to as 'computing device (100)') using a style condition-based diffusion model illustrated in FIG. 5, but are not limited thereto.

[0058] FIG. 2 is a flowchart of a medical image conversion method using a style condition-based diffusion model according to one embodiment of the present disclosure, and FIG. 3 is a diagram illustrating a medical image conversion process using a style condition-based diffusion model according to one embodiment of the present disclosure.

[0059] Medical imaging such as CT, MRI, and PET scans can produce images of different styles, even for the same patient, depending on the type and parameters of the scanner. For example, MRI images of different patients acquired with different equipment can differ in the color differences (contrast) of the CSF (cerebrospinal fluid), gray matter, and white matter, as well as in overall brightness.

[0060] Typically, to enable a model to perform conversion between CT images and MRI images, it is trained based on a dataset of pairs of CT images and MRI images. Since the styles of MRIs paired with CTs are diverse, the AI ​​model is trained to be able to generate MRIs of various styles for a single CT.

[0061] When a model learned in this way is used to convert a number of CT images extracted from 3D CT data into MRI images, a number of MRI images of different styles are created, and when these are combined, 3D MRI data with inconsistent styles are created.

[0062] Considering these points, a computing device (100) according to one embodiment of the present disclosure can perform a medical image conversion method using a style condition-based diffusion model for the purpose of resolving a style mismatch problem between cross-sectional images.

[0063] Referring to FIGS. 2 and 3, at step S110, the computing device (100) can acquire a first medical image.

[0064] Here, the first medical image may be, but is not limited to, a plurality of first cross-sectional images extracted from the first 3D volume data, i.e., a plurality of CT cross-sectional images extracted from the 3D CT data.

[0065] At step S120, the computing device (100) can obtain a style key indicating the style of the medical image to be generated.

[0066] Here, the medical image to be generated based on the first medical image is the second medical image, and the second medical image is an MRI image generated by converting a CT image, so the style key may indicate the style of the MRI image to be generated based on the CT image.

[0067] Additionally, here, style may refer to image characteristics such as brightness, contrast, color variation, etc. of the image. For example, the style of a medical image may refer to characteristics such as overall brightness, contrast, and intensity distribution by tissue of the medical image, but is not limited thereto.

[0068] Additionally, the style key here may be an indicator that numerically expresses these image features. For example, the style key may be a histogram of the medical image to be generated, i.e., the target second medical image.

[0069] Even when different patients are photographed using different equipment, histograms of similar shapes can be derived if the resulting MRI images have similar image characteristics (brightness, contrast, color changes, etc.).

[0070] Additionally, even if the same patient is photographed using the same equipment, different histograms may be obtained if the resulting MRI images have different imaging characteristics.

[0071] That is, as described above, the histogram of an image is a tool that well represents the image characteristics of the image, and the computing device (100) can obtain the histogram of the medical image to be created as a style key corresponding to the medical image to be created using the first medical image.

[0072] Here, the histogram may be a one-dimensional histogram having B bins, and for example, the histogram may include at least one of an intensity histogram representing a distribution of brightness of a medical image, a cumulative histogram representing a cumulative distribution of brightness values, and a differential histogram representing a rate of change between brightness values, but is not limited thereto.

[0073] At step S130, the computing device (100) can generate a second medical image in which a style corresponding to the acquired style key is reflected by converting the first medical image based on the style key acquired through step S120.

[0074] In one embodiment, the computing device (100) can generate a second medical image reflecting a style corresponding to the style key by transforming the first medical image with the style key as a condition through a pre-learned artificial intelligence model.

[0075] Here, the pre-learned artificial intelligence model may be a model that learns the correlation between the first medical image and the second medical image and the correlation between the style and the style key of the second medical image.

[0076] An artificial intelligence model (e.g., a neural network) consists of one or more network functions, which may be comprised of a set of interconnected computational units, generally referred to as "nodes." These "nodes" may also be referred to as "neurons." One or more network functions comprise at least one node. The nodes (or neurons) comprising one or more network functions may be interconnected by one or more "links."

[0077] Within an AI model, one or more nodes connected via links can form a relationship between input nodes and output nodes. The concepts of input nodes and output nodes are relative, meaning that any node in an output node relationship with one node can also be in an input node relationship with another node, and vice versa. As described above, input node-to-output node relationships can be created around links. One input node can be connected to one or more output nodes via links, and vice versa.

[0078] In a relationship between input nodes and output nodes connected through a single link, the value of the output node can be determined based on the data input to the input node. Here, the node interconnecting the input nodes and output nodes can have a weight. The weight can be variable and can be varied by a user or an algorithm so that the artificial intelligence model can perform a desired function. For example, when one or more input nodes are interconnected to one output node through each link, the output node can determine the output node value based on the values ​​input to the input nodes connected to the output node and the weight set for the link corresponding to each input node.

[0079] As described above, an AI model consists of one or more nodes interconnected through one or more links, forming input and output node relationships within the AI ​​model. The characteristics of an AI model can be determined based on the number of nodes and links, the relationships between nodes and links, and the weights assigned to each link. For example, if two AI models exist with the same number of nodes and links but different weight values ​​between the links, the two AI models may be perceived as different from each other.

[0080] Some of the nodes that make up an AI model can form a layer based on their distances from the initial input node. For example, a set of nodes that are n distances from the initial input node can form n layers. The distance from the initial input node can be defined by the minimum number of links that must be passed to reach the node from the initial input node. However, this definition of a layer is arbitrary for illustrative purposes, and the order of layers within an AI model can be defined in a different way than described above. For example, the layer of nodes can also be defined by their distance from the final output node.

[0081] The initial input node may refer to one or more nodes in an AI model into which data is directly input without going through a link in its relationship with other nodes. Alternatively, in an AI model network, in terms of the relationship between nodes based on links, it may refer to nodes that do not have other input nodes connected by links. Similarly, the final output node may refer to one or more nodes in an AI model that do not have an output node in its relationship with other nodes. In addition, a hidden node may refer to nodes that constitute an AI model other than the initial input node and the final output node. An AI model according to one embodiment of the present disclosure may be an AI model in which the number of nodes in the input layer may be greater than that in the hidden layer closer to the output layer, and the number of nodes decreases as it progresses from the input layer to the hidden layer.

[0082] An AI model may include one or more hidden layers. Hidden nodes in a hidden layer can receive the output of the previous layer and the output of surrounding hidden nodes as input. The number of hidden nodes in each hidden layer may be the same or different. The number of nodes in the input layer may be determined based on the number of data fields in the input data and may be the same as or different from the number of hidden nodes. Input data entered into the input layer may be computed by the hidden nodes in the hidden layer and output by the output layer, a fully connected layer (FCL).

[0083] In various embodiments, the artificial intelligence model may be a deep learning model.

[0084] A deep learning model (e.g., a deep neural network (DNN)) can refer to an artificial intelligence model that includes multiple hidden layers in addition to input and output layers. Using a deep neural network, it is possible to identify latent structures in data. That is, it is possible to identify latent structures in photos, text, videos, voices, and music (e.g., what objects are in the photo, what the content and emotion of the text are, what the content and emotion of the voice are, etc.).

[0085] Deep neural networks may include, but are not limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), autoencoders, generative adversarial networks (GANs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), Q networks, U networks, Siamese networks, etc.

[0086] In various embodiments, the network function may include an autoencoder, which may be a type of artificial neural network that outputs output data similar to input data.

[0087] An autoencoder may include at least one hidden layer, and an odd number of hidden layers may be positioned between the input and output layers. The number of nodes in each layer may be reduced from the number of nodes in the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically from the bottleneck layer to the output layer (symmetrical to the input layer). The nodes of the dimensionality reduction layer and the dimensionality restoration layer may or may not be symmetrical. In addition, the autoencoder may perform nonlinear dimensionality reduction. The number of input and output layers may correspond to the number of sensors remaining after preprocessing the input data. In the autoencoder structure, the number of nodes in the hidden layer included in the encoder may have a structure that decreases as it moves away from the input layer. The number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and the decoder) may be maintained above a certain number (e.g., more than half of the number of nodes in the input layer) because too small a number may not convey sufficient information.

[0088] In various embodiments, the artificial intelligence model may be a diffusion model that learns correlations between the first medical image and the second medical image and correlations between the style and style key of the second medical image.

[0089] For example, a diffusion model may be a model that learns the correlation between the first medical image and the second medical image and the correlation between the style of the second medical image and the style key by learning data that uses a specific first medical image and a specific style key as input data and a specific second medical image generated by transforming the specific first medical image and reflecting the style corresponding to the specific style key as correct data, so that a preset loss function is minimized.

[0090] At this time, the preset loss function for the diffusion model can be expressed as in the following mathematical expression 1.

[0091] <Mathematical Formula 1>

[0092]

[0093] Here, the above is the expected value, above is a latent variable at a specific point in time Intermediate state image in the above The first medical image is the initial image, is the style key, above is the actual noise and the above may be noise predicted by the diffusion model.

[0094] Meanwhile, the diffusion model may be a Brownian Bridge Diffusion Model (BBDM), and when the diffusion model is a Brownian Bridge Diffusion Model, a preset loss function for the diffusion model may be expressed as in the following mathematical expression 2.

[0095] <Mathematical Formula 2>

[0096]

[0097] Here, the above is the expected value, above The second medical image is the target image, is a latent variable at a specific point in time Intermediate state image in the above is the weight, above is the interpolation coefficient, above is the time-step noise coefficient, The first medical image is the initial image, is the style key, above is the actual noise and the above may be noise predicted by the diffusion model.

[0098] The Brownian Bridge Diffusion Model (BBDM) is based on the Brownian Bridge, a stochastic process. Here, the Brownian Bridge can refer to a random movement that begins at a specific starting point (e.g., 0) and is forced to reach a predetermined endpoint (e.g., 1) after a certain amount of time.

[0099] Diffusion is a process in which values ​​change over time and are distributed probabilistically. The Brownian Bridge Diffusion Model (BBDM) is characterized by modeling data by utilizing this diffusion process.

[0100] The Brownian Bridge Diffusion Model (BBDM) can be designed to ensure that a stochastic process follows specific starting and ending points. For example, when converting CT data to MRI data in medical imaging, the data can be transformed based on the CT image (starting point) and the MRI image (ending point). The Brownian Bridge Diffusion Model (BBDM) ensures that the data evolves gradually over time, and that all intermediate states are smoothly connected.

[0101] In one embodiment, when the computing device (100) obtains a histogram of a medical image to be generated using a first medical image as a style key, the computing device (100) can generate a second medical image having a histogram corresponding to the style key by transforming the first medical image with the style key as a condition through a diffusion model.

[0102] In addition, when there are multiple first medical images, the computing device (100) can generate multiple second medical images in which the style corresponding to the style key is identically reflected by transforming the multiple first medical images with the style key as a condition through a diffusion model.

[0103] That is, the computing device (100) can generate a second medical image with a style key as a condition, thereby generating a second medical image in which a user's desired style is reflected based on the first medical image, as illustrated in FIG. 4, and can generate a plurality of second medical images with matching styles, thereby resolving the style mismatch problem of 3D volume data. Hereinafter, with reference to FIG. 5, a hardware configuration of the computing device (100) that performs a medical image conversion method using a style condition-based diffusion model according to various embodiments of the present disclosure will be described.

[0104] FIG. 5 is a diagram illustrating a hardware configuration of a medical image conversion device using a style condition-based diffusion model according to one embodiment of the present disclosure.

[0105] Referring to FIG. 5, a computing device (100) according to one embodiment of the present disclosure may include one or more processors (110), a memory (120) for loading a computer program (151) executed by the processor (110), a bus (130), a communication interface (140), and a storage (150) for storing the computer program (151). Here, only components related to the embodiment of the present disclosure are illustrated in FIG. 5. Therefore, a person skilled in the art to which the present disclosure pertains may understand that other general components may be included in addition to the components illustrated in FIG. 5.

[0106] The processor (110) controls the overall operation of each component of the computing device (100). The processor (110) may be configured to include a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), or any other type of processor well known in the art of the present disclosure.

[0107] Additionally, the processor (110) may perform operations for at least one application or program for executing a method according to embodiments of the present disclosure, and the computing device (100) may have one or more processors.

[0108] In various embodiments, the processor (110) may further include a Random Access Memory (RAM, not shown) and a Read-Only Memory (ROM, not shown) that temporarily and / or permanently store signals (or data) processed within the processor (110). In addition, the processor (110) may be implemented in the form of a system on chip (SoC) that includes at least one of a graphics processing unit, RAM, and ROM.

[0109] The memory (120) stores various data, commands, and / or information. The memory (120) can load a computer program (151) from the storage (150) to execute methods / operations according to various embodiments of the present disclosure. When the computer program (151) is loaded into the memory (120), the processor (110) can perform the method / operation by executing one or more instructions constituting the computer program (151). The memory (120) may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

[0110] The bus (130) provides a communication function between components of the computing device (100). The bus (130) may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

[0111] The communication interface (140) supports wired and wireless Internet communication of the computing device (100). Furthermore, the communication interface (140) may support various communication methods other than Internet communication. To this end, the communication interface (140) may be configured to include a communication module well known in the technical field of the present disclosure. In some embodiments, the communication interface (140) may be omitted.

[0112] The storage (150) can non-temporarily store a computer program (151). When performing a medical image conversion process using a style condition-based diffusion model and a 3D volume data conversion process using a multi-slice learned diffusion model through a computing device (100), the storage (150) can store various information necessary to provide the medical image conversion process using a style condition-based diffusion model and the 3D volume data conversion process using a multi-slice learned diffusion model.

[0113] Storage (150) may be configured to include non-volatile memory such as ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), flash memory, a hard disk, a removable disk, or any form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0114] The computer program (151) may include one or more instructions that, when loaded into the memory (120), cause the processor (110) to perform a method / operation according to various embodiments of the present disclosure. That is, the processor (110) may perform the method / operation according to various embodiments of the present disclosure by executing the one or more instructions.

[0115] In one embodiment, the computer program (151) may include one or more instructions for performing a method for converting a medical image using a style condition-based diffusion model, the method including the steps of: obtaining a first medical image; obtaining a style key indicating a style of a medical image to be generated; and generating a second medical image in which a style corresponding to the obtained style key is reflected by converting the obtained first medical image based on the obtained style key using a pre-learned artificial intelligence model.

[0116] The steps of a method or algorithm described in connection with the embodiments of the present disclosure may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present disclosure pertains.

[0117] The components of the present disclosure may be implemented as programs (or applications) and stored on a medium to be executed in conjunction with a computer, which is hardware. The components of the present disclosure may be implemented as software programs or software elements. Similarly, the embodiments may be implemented in a programming or scripting language such as C, C++, Java, or an assembler, including various algorithms implemented as a combination of data structures, processes, routines, or other programming components. Functional aspects may be implemented as algorithms that are executed on one or more processors.

[0118] As described above, those skilled in the art will appreciate that the present disclosure can be implemented in other specific forms without altering the technical spirit or essential characteristics thereof. Therefore, the above-described embodiments should be understood as illustrative in all respects and not restrictive. The scope of the present disclosure is defined by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included within the scope of the present disclosure.

[0119] The features and advantages described in this specification are not exhaustive, and many additional features and advantages will become apparent to those skilled in the art upon review of the drawings, specification, and claims. Furthermore, it should be noted that the language used in this specification has been primarily selected for readability and instructional purposes, and may not be intended to delineate or circumscribe the subject matter of the present disclosure.

[0120] The above description of the embodiments of the present disclosure has been presented for illustrative purposes. It is not intended to limit the present disclosure to the precise form disclosed, nor is it intended to be exhaustive. Those skilled in the art will appreciate that numerous modifications and variations are possible in light of the above disclosure.

[0121] Therefore, the scope of this disclosure is not limited by the detailed description, but is defined by any claims of the application based on this description. Accordingly, the disclosure of embodiments of this disclosure is illustrative and does not limit the scope of this disclosure, which is set forth in the following claims.

Claims

1. In a method performed by a computing device, Step of acquiring a first medical image; A step of obtaining a style key indicating the style of the medical image to be generated; and A step of generating a second medical image in which a style corresponding to the acquired style key is reflected by transforming the acquired first medical image based on the acquired style key through a learned artificial intelligence model, The above-mentioned artificial intelligence model is, A diffusion model that learns the correlation between the first medical image and the second medical image and the correlation between the style and style key of the second medical image. A medical image transformation method using a style-condition-based diffusion model.

2. In paragraph 1, The first medical image obtained above is a CT image extracted corresponding to one plane from 3D CT data, The second medical image obtained above is generated by converting the CT image, and is an MRI image corresponding to one of the planes. A medical image transformation method using a style-condition-based diffusion model.

3. In paragraph 1, The style key obtained above is, Including at least one of an intensity histogram representing the brightness distribution of a medical image to be generated using the first medical image, a cumulative histogram representing the cumulative distribution of brightness values, and a differential histogram representing the rate of change between brightness values. A medical image transformation method using a style-condition-based diffusion model.

4. In paragraph 1, The above diffusion model is, The Brownian Bridge Diffusion Model (BBDM) A medical image transformation method using a style-condition-based diffusion model.

5. In paragraph 1, The above diffusion model is, A model that is trained to minimize a preset loss function by using learning data that uses a specific first medical image and a specific style key as input data, and a specific second medical image generated by transforming the specific first medical image and reflecting a style corresponding to the specific style key as correct data. A medical image transformation method using a style-condition-based diffusion model.

6. In paragraph 5, The above preset loss function is, It is expressed as the following mathematical formula 1, A medical image transformation method using a style-condition-based diffusion model. <Mathematical Formula 1> Here, the above is the expected value, above is a latent variable at a specific point in time Intermediate state image in the above The first medical image is the initial image, is the style key, above is the actual noise and the above is the noise predicted by the diffusion model 7. In paragraph 5, The above preset loss function is, If the above diffusion model is a Brownian bridge diffusion model, it is expressed as in the following mathematical expression 2. A medical image transformation method using a style-condition-based diffusion model. <Mathematical Formula 2> Here, the above is the expected value, above The second medical image is the target image, is a latent variable at a specific point in time Intermediate state image in the above is the weight, above is the interpolation coefficient, above is the time-step noise coefficient, The first medical image is the initial image, is the style key, above is the actual noise and the above is the noise predicted by the diffusion model 8. In paragraph 1, The step of generating the second medical image is as follows: If the acquired style key is a histogram of the medical image to be generated, a step of generating a second medical image having a histogram corresponding to the acquired style key by transforming the acquired first medical image with the acquired style key as a condition, A medical image transformation method using a style-condition-based diffusion model.

9. In paragraph 1, The step of generating the second medical image is as follows: In the case where the first medical images obtained above are multiple, a step of generating multiple second medical images in which a style corresponding to the obtained style key is identically reflected by converting the multiple first medical images based on the acquired style key as a condition, A medical image transformation method using a style-condition-based diffusion model.

10. Processor; network interface; memory; and A computer program loaded into the above memory and executed by the above processor, The above computer program, Instructions for acquiring the first medical image; An instruction for obtaining a style key indicating the style of a medical image to be generated; and An instruction for generating a second medical image in which a style corresponding to the acquired style key is reflected by transforming the acquired first medical image based on the acquired style key through a pre-learned artificial intelligence model, wherein the pre-learned artificial intelligence model is a diffusion model in which a correlation between a first medical image and a second medical image and a correlation between a style of the second medical image and a style key are learned. Medical image transformation device using a style condition-based diffusion model.

11. Step of acquiring the first medical image; A step of obtaining a style key indicating the style of the medical image to be generated; and A computer-readable non-transitory recording medium having recorded thereon instructions for executing a method for converting a medical image using a style condition-based diffusion model, the method comprising: converting the acquired first medical image by a condition of the acquired style key, through a pre-learned artificial intelligence model, wherein the pre-learned artificial intelligence model is a diffusion model in which a correlation between a first medical image and a second medical image and a correlation between a style of the second medical image and a style key are learned; generating a second medical image in which a style corresponding to the acquired style key is reflected;

Images