Medical image processing device, medical image processing method, and program

Abstract

To provide a medical image processing device, medical image processing method, and program that utilize a machine learning model to generate high-quality medical images while also ensuring transparency and explainability during the generation process.SOLUTION: A medical image processing device of this embodiment includes an acquisition unit, a generation unit, an image quality adjustment unit, and a calculation unit. The acquisition unit acquires an input image to be processed. The generation unit uses a trained model to generate a reference image from the input image. The image quality adjustment unit adjusts the image quality of the input image. The calculation unit compares an image-quality adjusted image, which is the input image whose image quality has been adjusted, with the reference image and calculates a difference in image quality between the two images. The trained model is a machine learning model trained based on a training dataset containing two unpaired images. The image quality adjustment unit readjusts the image quality of the input image based on the difference in image quality.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The embodiments disclosed in this specification and the drawings relate to a medical image processing device, a medical image processing method, and a program. [Background technology]

[0002] One of the requirements for optimizing image quality in medical image diagnostic systems (also called medical image diagnostic devices) is to make an image generated by one system (hereinafter referred to as an input image) closer in image quality to an image generated by another system (hereinafter referred to as a target image).

[0003] If the input image and the target image are in a paired relationship, the parameters of the machine learning model can be optimized based on well-known reference-type indices such as MAD (Most Apparent Distortion) and SSIM (Structural SIMilarity) to bring the image quality of the input image closer to that of the target image.

[0004] The term "pair" here refers to a state in which an image in the source domain (i.e., input image) and a corresponding image in the target domain (i.e., target image) are paired with each other pixel by pixel.

[0005] However, in reality, the paired relationship between an input image and a target image is a very special situation, so it is difficult to use the paired input image and target image as a training dataset for learning a machine learning model.

[0006] Furthermore, when using a machine learning model to generate an image with reduced noise (an image with better image quality) from an input image, there are problems such as the process by which the machine learning model performs its calculations not being fully explained, and the behavior in unexpected situations being unclear. In other words, image generation using a machine learning model may lack transparency and explainability.

[0007] On the other hand, instead of using reference-based indices, it is possible to optimize image quality using well-known non-reference-based indices such as RISQUE (Blind / Referenceless Image Spatial Quality Evaluator), but their application to medical images has not been fully studied. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] International Publication No. 2020 / 070834 [Patent Document 2] Japanese Patent Publication No. 2020-44162 [Patent Document 3] Japanese Patent Application Publication No. 2018-206382 Summary of the Invention [Problem to be solved by the invention]

[0009] The problem to be solved by the embodiments disclosed in this specification and the drawings is to generate medical images with high image quality using a machine learning model, while also ensuring transparency and explainability in the generation process. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described below can also be positioned as other problems. [Means for solving the problem]

[0010] A medical image processing apparatus according to an embodiment includes an acquisition unit, a generation unit, an image quality adjustment unit, and a calculation unit. The acquisition unit acquires an input image to be processed. The generation unit generates a reference image from the input image using a trained model. The image quality adjustment unit adjusts the image quality of the input image. The calculation unit compares an image quality adjustment image, which is the input image whose image quality has been adjusted, with the reference image, and calculates an image quality difference, which is the difference in image quality between the image quality of the image quality adjustment image and the image quality of the reference image. The trained model is a machine learning model trained based on a training dataset including training input images, which are input images for training, and training target images, which are target images for training, and which are not paired with the training input images. The image quality adjustment unit readjusts the image quality of the input image based on the image quality difference. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram showing an example of the arrangement of a medical image processing apparatus 100 according to a first embodiment. [Figure 2] 3 is a flowchart showing the flow of a series of processes by the processing circuitry 120 of the medical image processing apparatus 100 according to the first embodiment. [Figure 3] FIG. 2 is a diagram schematically showing the flow of a series of processes in the processing circuitry 120 of the medical image processing apparatus 100 according to the first embodiment. [Figure 4] FIG. 1 is a diagram for explaining a trained model MDL according to the first embodiment. [Figure 5] FIG. 11 is a diagram showing an example of a screen of a display 113a. [Figure 6] FIG. 10 is a diagram for explaining a trained model MDL according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, a medical image processing apparatus, a medical image processing method, and a program according to an embodiment will be described with reference to the drawings.

[0013] (First embodiment) [Configuration of medical image processing device] 1 is a diagram showing an example of the configuration of a medical image processing apparatus 100 according to the first embodiment. For example, the medical image processing apparatus 100 may be incorporated into a medical image diagnostic apparatus (also called a modality), or may be prepared separately from the medical image diagnostic apparatus. The medical image diagnostic apparatus may include, for example, an ultrasound diagnostic apparatus, an X-ray CT (Computed Tomography) apparatus, an MRI (Magnetic Resonance Imaging) apparatus, an X-ray apparatus, etc.

[0014] The medical image processing apparatus 100 may be a single device, or may be a system in which a plurality of devices connected via a communication network NW operate in cooperation with each other.

[0015] The communication network NW may refer to any information and communication network that uses telecommunications technology. For example, the communication network NW may include wireless / wired LANs such as hospital backbone LANs (Local Area Networks), the Internet, telephone communication networks, optical fiber communication networks, cable communication networks, satellite communication networks, etc.

[0016] That is, the medical image processing apparatus 100 may be realized by a plurality of computers (processors) included in a distributed computing system or a cloud computing system.

[0017] As shown in the figure, for example, the medical image processing apparatus 100 includes a communication interface 111, an input interface 112, an output interface 113, a memory 114, and a processing circuit 120.

[0018] The communication interface 111 communicates with external devices via the communication network NW. The communication interface 111 includes, for example, a network interface card (NIC) and an antenna for wireless communication.

[0019] The input interface 112 receives various input operations from an operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuitry 120 .

[0020] For example, the input interface 112 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch panel, etc. The input interface 112 may be, for example, a user interface that accepts audio input from a microphone, etc. If the input interface 112 is a touch panel, the input interface 112 may also have the display function of a display 113a included in the output interface 113, which will be described later.

[0021] In this specification, the input interface 112 is not limited to an interface having physical operation parts such as a mouse, keyboard, etc. For example, an example of the input interface 112 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

[0022] The output interface 113 includes, for example, a display 113a, a speaker 113b, etc. The display 113a displays various types of information.

[0023] For example, the display 113a displays images generated by the processing circuitry 120, a GUI (Graphical User Interface) for receiving various input operations from an operator, etc. For example, the display 113a is an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL (Electro Luminescence) display, etc. The speaker 113b outputs information input from the processing circuitry 120 as sound.

[0024] The memory 114 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory), a flash memory, a hard disk, or an optical disk. These non-transitory storage media may also be realized by other storage devices connected via a communication network NW, such as a NAS (Network Attached Storage) or an external storage server device.

[0025] The memory 114 may also include a non-transitory storage medium such as a ROM (Read Only Memory), a register, etc. The memory 114 stores programs executed by the hardware processor of the processing circuit 120, various calculation results by the processing circuit 120, model definition data, etc.

[0026] The model definition data is information (program or algorithm) that defines the trained model MDL, which will be described later.

[0027] The processing circuit 120 includes, for example, an acquisition function 121, a generation function 122, an image quality adjustment function 123, a calculation function 124, and an output control function 125. The acquisition function 121 is an example of an "acquisition unit," the generation function 122 is an example of a "generation unit," the image quality adjustment function 123 is an example of an "image quality adjustment unit," and the calculation function 124 is an example of a "calculation unit."

[0028] The processing circuitry 120 realizes these functions by, for example, a hardware processor (computer) executing a program stored in the memory 114 (storage circuitry).

[0029] The hardware processor in the processing circuit 120 refers to a circuit such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD) or a Complex Programmable Logic Device (CPLD), or a Field Programmable Gate Array (FPGA)).

[0030] Instead of storing the program in the memory 114, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes its functions by reading and executing the program embedded in the circuit. The program may be stored in the memory 114 in advance, or may be stored in a non-transitory storage medium such as a DVD or CD-ROM, and installed into the memory 114 from the non-transitory storage medium when the non-transitory storage medium is inserted into a drive device (not shown) of the medical image processing apparatus 100.

[0031] The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function, or multiple components may be integrated into a single hardware processor to realize each function.

[0032] [Processing flow of medical image processing equipment] A series of processes performed by the processing circuitry 120 of the medical image processing apparatus 100 will be described below with reference to Fig. 2 and Fig. 3. Fig. 2 is a flowchart showing the flow of a series of processes performed by the processing circuitry 120 of the medical image processing apparatus 100 according to the first embodiment. Fig. 3 is a diagram schematically showing the flow of a series of processes performed by the processing circuitry 120 of the medical image processing apparatus 100 according to the first embodiment.

[0033] First, the acquisition function 121 acquires an input image IMG-IN (step S100). The input image IMG-IN is a medical image obtained by scanning a subject with a medical image diagnostic device (modality). The subject is typically a human being, but is not limited to this, and may be another animal such as a dog or cat, or may be a plant.

[0034] For example, if the medical image diagnostic device is an ultrasound diagnostic device, the input image IMG-IN is an ultrasound image. Similarly, if the medical image diagnostic device is an X-ray CT device, the input image IMG-IN is a CT image. If the medical image diagnostic device is an MRI device, the input image IMG-IN is an MR image. If the medical image diagnostic device is an X-ray device, the input image IMG-IN is an X-ray image.

[0035] Next, the generation function 122 reads the model definition data from the memory 114, and generates a reference image IMG-GEN from the input image IMG-IN using the trained model MDL defined by the model definition data (step S102).

[0036] The reference image IMG-GEN is a medical image that the trained model MDL outputs in response to an input image IMG-IN, and is the medical image that is referenced during the image quality comparison process described below.

[0037] The trained model MDL may be implemented, for example, by Cycle-Consistent Adversarial Networks (CycleGAN) or a model derived from CycleGAN (e.g., Cycle-MedGAN). The trained model MDL may be implemented using other machine learning models such as support vector machines, decision trees, random forests, and logistic regression instead of neural networks such as CycleGAN. In the following, the trained model MDL will be described as being implemented by CycleGAN as an example.

[0038] When the trained model MDL is implemented by a neural network such as CycleGAN, the model definition data includes, for example, connection information about how the units in each layer of the neural network, such as the input layer, one or more hidden layers (intermediate layers), and output layer, are connected to each other, and weight information about the connection coefficients assigned to the data input and output between the connected units.

[0039] The connection information includes, for example, the number of units included in each layer, information specifying the type of unit to which each unit is connected, the activation function that realizes each unit, and gates provided between units in the hidden layer.

[0040] The activation functions that realize the units may be, for example, ReLU (Rectified Linear Unit) functions, ELU (Exponential Linear Units) functions, clipping functions, sigmoid functions, step functions, hyperbolic tangent functions, identity functions, etc. The gates selectively pass or weight data transmitted between units depending on, for example, the value (e.g., 1 or 0) returned by the activation functions.

[0041] The connection coefficients include, for example, weights assigned to output data when data is output from a unit in a hidden layer of a neural network to a unit in a deeper layer, and may also include bias components specific to each layer.

[0042] 4 is a diagram illustrating the trained model MDL according to the first embodiment. For example, when the trained model MDL is implemented using CycleGAN, as shown in the figure, the trained model MDL includes a first generative model GEN1 and a second generative model GEN2. The first generative model GEN1 and the second generative model GEN2 are implemented by combining an encoder and a decoder.

[0043] These first and second generative models GEN1 and GEN2 are trained based on a training dataset consisting of two unpaired images.

[0044] As described above, a pair refers to a state in which an image in the source domain (i.e., input image) and a corresponding image in the target domain (i.e., target image) are paired with each other pixel by pixel.

[0045] For example, a medical image before specific image processing and a medical image after specific image processing are in a paired relationship. When the medical image is an ultrasound image, an ultrasound image generated from ultrasound signal data before BF (Beam Forming) processing according to a certain BF method and an ultrasound image generated from the same ultrasound signal data before BF processing according to another BF method are in a paired relationship. Similarly, an ultrasound B-mode image and a color Doppler, contrast, elastography, or attenuation image of the same time phase are in a paired relationship. When the medical image is an MR image, a T1-weighted image and a T2-weighted image of the same original MR scan data are also in a paired relationship.

[0046] In this embodiment, instead of using such paired images as a training data set, we use unequal images as a training data set, i.e., a source domain image (i.e., input image) and a target domain image (i.e., target image) that are pixel-wise unpaired.

[0047] For example, the first generative model GEN1 and the second generative model GEN2 are trained using a training dataset consisting of a training input image IMG-IN(A) and a training target image IMG-IN(B). The training input image IMG-IN(A) is an input image of a source domain A prepared for training. The training target image IMG-IN(B) is a target image of a target domain B prepared for training, and is an unpaired image of the training input image IMG-IN(A).

[0048] [Forward image transformation process] A training input image IMG-IN(A) is input to the first generative model GEN1. In response to this, the first generative model GEN1 converts the training input image IMG-IN(A) into an image that is close to the target domain B and outputs it. Hereinafter, the image that is close to the target domain B converted from the training input image IMG-IN(A) by the first generative model GEN1 will be referred to as a generated image IMG-GEN(B).

[0049] The second generative model GEN2 receives the generated image IMG-GEN(B) output by the first generative model GEN1. In response, the second generative model GEN2 converts the generated image IMG-GEN(B) into an image closer to the source domain A and outputs it. Hereinafter, the image that has been converted by the first generative model GEN1 into an image closer to the target domain B, and then further converted by the second generative model GEN2 into an image closer to the original source domain A, will be referred to as the generated image IMG-CYC(A).

[0050] [Reverse image conversion process] Meanwhile, the second generative model GEN2 is also input with the training target image IMG-IN(B). In response to this, the second generative model GEN2 converts the training target image IMG-IN(B) into an image that is close to the source domain A and outputs it. Hereinafter, the image that is close to the source domain A and that is converted from the training target image IMG-IN(B) by the second generative model GEN2 will be referred to as the generated image IMG-GEN(A).

[0051] The first generative model GEN1 also receives as input the generated image IMG-GEN(A) output by the second generative model GEN2. In response to this, the first generative model GEN1 converts the generated image IMG-GEN(A) into an image that is close to the target domain B and outputs it. Hereinafter, the image that has been converted by the second generative model GEN2 into an image that is close to the source domain A, and then further converted by the first generative model GEN1 into an image that is close to the original target domain A, will be referred to as the generated image IMG-CYC(B).

[0052] The parameters (weighting coefficients and bias components) of the first generative model GEN1 and the second generative model GEN2 described above are adjusted based on the losses in the forward image transformation process and the backward image transformation process. As described above, the forward image transformation process is a cyclic process in which a training input image IMG-IN(A) is transformed into a generated image IMG-GEN(B), which is an image close to the target domain B, and then the generated image IMG-GEN(B) is transformed into a generated image IMG-CYC(A), which is an image close to the source domain A. The backward image transformation process is a cyclic process in which a training target image IMG-IN(B) is transformed into a generated image IMG-GEN(A), which is an image close to the source domain A, and then the generated image IMG-GEN(A) is transformed into a generated image IMG-CYC(B), which is an image close to the target domain B.

[0053] For example, the parameters (weighting coefficients and bias components) of the first generative model GEN1 and the second generative model GEN2 are adjusted to minimize the overall loss, which is the sum of the losses in the forward image translation process and the backward image translation process. Such an overall loss is called the cycle consistency loss and the adversarial loss.

[0054] By training the first generative model GEN1 and the second generative model GEN2 using a training data set consisting of two such unpaired images, it is possible to generate a reference image IMG-GEN that is not paired pixel-by-pixel with the input image IMG-IN in the process of S102. More specifically, of the trained first generative model GEN1 and second generative model GEN2, the generation function 122 inputs the input image IMG-IN to the first generative model GEN1, thereby generating an unpaired reference image IMG-GEN that is in a domain different from the source domain of the input image IMG-IN.

[0055] 2 and 3, the image quality adjustment function 123 adjusts the image quality of the input image IMG-IN using an image filter (step S104). Hereinafter, the input image IMG-IN whose image quality has been adjusted using the image filter will be referred to as an image quality adjusted image IMG-OUT.

[0056] The image filter is a filter for smoothing the input image IMG-IN to reduce noise and for emphasizing features such as edges, for example, edges of specific biological tissues such as blood vessels.

[0057] Next, the calculation function 124 compares the reference image IMG-GEN generated using the learned model MDL with the image quality adjusted image IMG-OUT, and calculates the difference between the image quality of the reference image IMG-GEN and the image quality of the image quality adjusted image IMG-OUT (hereinafter referred to as the image quality difference) (step S106).

[0058] Next, the calculation function 124 determines whether the image quality difference is equal to or less than a threshold value (step S108). The threshold value may be, for example, a value at which the image quality difference can be considered to be sufficiently small.

[0059] If it is determined that the difference in image quality exceeds the threshold, the image quality adjustment function 123 changes all or some of the multiple image quality indices referenced when adjusting the image quality of the input image IMG-IN so as to reduce the difference in image quality (step S110). This readjusts the image quality of the input image IMG-IN.

[0060] The image quality index is the degree of smoothing of the image filter, the degree of feature emphasis, etc. The image quality index may be interpreted as the parameter of the image filter.

[0061] When the image quality of the input image IMG-IN is readjusted due to a change in the image quality index, the image quality difference between the reference image IMG-GEN and the image quality adjusted image IMG-OUT is calculated again. In this manner, the image quality adjustment of the input image IMG-IN is repeated until the image quality difference becomes equal to or less than the threshold value.

[0062] On the other hand, if it is determined that the image quality difference is equal to or less than the threshold value, the output control function 125 outputs the image quality adjusted image IMG-OUT that has been compared with the reference image IMG-GEN (step S112).

[0063] For example, the output control function 125 may display the image quality adjusted image IMG-OUT on the display 113a. The output control function 125 may also transmit the image quality adjusted image IMG-OUT to an external device (for example, a computer used by a medical professional such as a doctor or technician) via the communication interface 111. The output control function 125 may also output a reference image IMG-GEN in addition to the image quality adjusted image IMG-OUT. This completes the processing of this flowchart.

[0064] According to the first embodiment described above, the processing circuitry 120 of the medical image processing apparatus 100 acquires an input image IMG-IN to be processed. The processing circuitry 120 generates a reference image IMG-GEN from the input image IMG-IN using the trained model MDL.

[0065] The trained model MDL is a machine learning model (e.g., CycleGAN) that has been trained in advance using a training dataset consisting of training input images IMG-IN(A) and training target images IMG-IN(B). The training input images IMG-IN(A) are input images of a source domain A prepared for training. The training target images IMG-IN(B) are target images of a target domain B prepared for training, and are unpaired images of the training input images IMG-IN(A).

[0066] The processing circuit 120 adjusts the image quality of the input image IMG-IN, compares the image quality-adjusted image IMG-OUT, which is the input image IMG-IN with the reference image IMG-GEN generated using the trained model MDL, and calculates the image quality difference between the images. The processing circuit 120 readjusts the image quality of the input image IMG-IN based on the image quality difference.

[0067] In this way, by adjusting the parameters (image quality index) of the image filter so that the image quality of the image IMG-OUT generated through the image filter approaches the image quality of the reference image IMG-GEN generated by a machine learning model such as CycleGAN, it is possible to generate medical images with good image quality. Furthermore, since the final output is not a reference image IMG-GEN generated by a machine learning model whose process is a black box, but rather an image IMG-OUT generated through a conventionally known image filter, convincing explanations based on conventional knowledge can be provided to medical professionals and patients. In other words, transparency and explainability can be ensured.

[0068] (Modification of the first embodiment) Below, a description will be given of a modified example of the first embodiment. In the description of the first embodiment above, it has been described that the image quality adjustment function 123 changes the image quality index (parameters of the image filter) so as to reduce the image quality difference when it is determined that the image quality difference exceeds the threshold, but this is not limited to this. For example, the image quality adjustment function 123 may change the scan conditions of the medical image diagnostic device so as to reduce the image quality difference.

[0069] When the medical image diagnostic device is an ultrasound diagnostic device, the scan conditions include, for example, whether the zoom function is on or off, a frame rate value, an image quality level, a dynamic range value, a brightness value, etc. The scan conditions may further include the type of ultrasound probe used for scanning, whether a stress echo test is performed, etc.

[0070] Furthermore, when the medical image diagnostic device is an X-ray CT device, the scan conditions include, for example, the scan region, the scan method, the conditions for collecting detection data or projection data, and the conditions for reconstructing a CT image.

[0071] In this way, the image quality of the input image IMG-IN may be adjusted retroactively using an image filter after the input image IMG-IN has been acquired, or the image quality of the input image IMG-IN may be adjusted in advance by changing the scanning conditions before or during the acquisition of the input image IMG-IN.

[0072] In the above description of the first embodiment, the image quality of the input image IMG-IN is automatically adjusted until the image quality difference is equal to or less than a threshold value, but this is not limiting. For example, the image quality adjustment function 123 may adjust the image quality of the input image IMG-IN based on an image quality index preset by a user such as a doctor or technician. In other words, the image quality of the input image IMG-IN may be manually adjusted in response to a user request.

[0073] FIG. 5 is a diagram illustrating an example of the screen of the display 113a. As illustrated, the display 113a may display a GUI (Graphical User Interface) that allows the user to select whether to automatically or manually adjust the image quality of the input image IMG-IN. For example, the GUI may display the current input image IMG-IN, for which the image quality difference is to be calculated, and a reference image IMG-GEN generated by a machine learning model side by side. For example, when the user operates button B1 (automatic adjustment button) on the GUI, the image quality difference between these images is calculated. If the image quality difference exceeds a threshold, the image quality index (parameters of the image filter) is automatically changed, and the image quality of the input image IMG-IN is automatically readjusted. On the other hand, when the user operates button B2 (manual adjustment button) on the GUI, the image quality of the input image IMG-IN is readjusted based on the image quality index set by the user. In this way, the user's operation may intervene in the adjustment of the image quality of the input image IMG-IN. In other words, the image quality of the input image IMG-IN may be adjusted interactively.

[0074] (Second embodiment) The second embodiment will be described below. In the above-described first embodiment, the trained model MDL is described as being implemented by CycleGAN. In contrast, the second embodiment differs from the first embodiment in that the trained model MDL is implemented by a diffusion model. The following description will focus on the differences from the first embodiment, and will omit a description of the points in common with the first embodiment. In the description of the second embodiment, the same parts as in the first embodiment will be described with the same reference numerals.

[0075] The generation function 122 according to the second embodiment generates a reference image IMG-GEN from an input image IMG-IN using a diffusion model. At this time, the generation function 122 controls the characteristics of the reference image IMG-GEN based on the characteristics (e.g., edges, segmentation, etc.) of the input image IMG-IN.

[0076] 6 is a diagram for explaining the trained model MDL according to the second embodiment. The trained model MDL according to the second embodiment is implemented using a diffusion model. The diffusion model is used for image generation, similar to CycleGAN.

[0077] In particular, the diffusion model may be a latent diffusion model that diffuses latent variables converted from the input image IMG-IN by an arbitrary encoder in a latent space.

[0078] The diffusion model may be provided with a conditional mechanism that incorporates feature vectors encoded by other encoders through cross-attention. In this embodiment, a control network (also called a stable diffusion control network) is applied as the conditional mechanism. The control network is a model that applies the stable diffusion model, which is a type of GAN.

[0079] First, the image quality adjustment function 123 according to the second embodiment performs edge detection processing (e.g., Canny Filter, etc.) on the input image IMG-IN to generate an input image IMG-IN with enhanced edges. Hereinafter, the input image IMG-IN with enhanced edges will be referred to as an edge-enhanced image IMG-IN#.

[0080] Next, the image quality adjustment function 123 according to the second embodiment inputs the input image IMG-IN to the diffusion model, which is the trained model MDL, and also inputs the edge-enhanced image IMG-IN# to a control net provided as a conditional mechanism for the diffusion model. This causes the diffusion model to generate a reference image IMG-GEN with controlled edges according to the edge-enhanced image IMG-IN# input to the control net. On the reference image IMG-GEN generated in this way, biological tissue and the like are rendered along the edges of the edge-enhanced image IMG-IN#. As a result, it is possible to prevent the trained model MDL from generating a medical image (reference image IMG-GEN) that appears unnatural to the human eye.

[0081] According to the second embodiment described above, even when a diffusion model is applied as the trained model MDL, medical images with good image quality can be generated, as in the first embodiment, and further, transparency and explainability can be ensured in the generation process.

[0082] Furthermore, according to the second embodiment, when a control net is applied to the conditional mechanism of the diffusion model, it is possible to prevent the generation of medical images (reference images IMG-GEN) that appear unnatural to the human eye.

[0083] Although several embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the invention and its equivalents as defined in the claims. [Explanation of symbols]

[0084] 100... medical image processing device, 111... communication interface, 112... input interface, 113... output interface, 114... memory, 120... processing circuit, 121... acquisition function, 122... generation function, 123... image quality adjustment function, 124... calculation function, 125... output control function

Claims

1. an acquisition unit that acquires an input image to be processed; a generation unit that generates a reference image from the input image using a trained model; an image quality adjustment unit that adjusts the image quality of the input image; a calculation unit that compares an image quality adjusted image, which is the input image whose image quality has been adjusted, with the reference image, and calculates an image quality difference, which is the difference in image quality between the image quality adjusted image and the image quality of the reference image; The trained model is a machine learning model trained based on a training dataset including training input images that are input images for training, and training target images that are target images for training, the training target images being unpaired with the training input images; the image quality adjustment unit readjusts the image quality of the input image based on the image quality difference; Medical imaging equipment.

2. the image quality adjustment unit adjusts the image quality of the input image by changing all or some of a plurality of image quality indices that are referenced when adjusting the image quality of the input image so that the image quality difference becomes smaller. The medical image processing device according to claim 1 .

3. the input image is acquired by scanning a subject with a medical image diagnostic device; the image quality adjustment unit adjusts the image quality of the input image by changing the scan conditions of the medical image diagnostic apparatus so as to reduce the image quality difference. The medical image processing device according to claim 1 .

4. the image quality adjustment unit further adjusts the image quality of the image quality adjusted image in response to a user request. The medical image processing device according to claim 1 or 2.

5. The training of the trained model involves: a forward transformation process that transforms the training input images into the training target images; an inverse transformation process that transforms the training target images into the training input images; and an adjustment process for adjusting parameters of the machine learning model based on the forward loss and the backward loss. The medical image processing device according to claim 1 or 2.

6. The trained model is CycleGAN or a derivative model of the CycleGAN, The medical image processing device according to claim 5 .

7. The generation unit controls a feature of the reference image based on a feature of the input image when generating the reference image from the input image using the trained model. The medical image processing device according to claim 1 or 2.

8. the trained model is a diffusion model that includes a control net as a conditional mechanism, the diffusion model generates the reference image in which the characteristics are controlled according to the control net; The medical image processing device according to claim 7 .

9. A computer-assisted medical image processing method, comprising: obtaining an input image to be processed; generating a reference image from the input image using the trained model; adjusting the image quality of the input image; comparing an image quality adjusted image, which is the input image whose image quality has been adjusted, with the reference image, and calculating an image quality difference, which is a difference in image quality between the image quality adjusted image and the image quality of the reference image; The trained model is a machine learning model trained based on a training dataset including training input images that are input images for training, and training target images that are target images for training, the training target images being unpaired with the training input images; and further comprising: readjusting the image quality of the input image based on the image quality difference. Medical image processing methods.

10. A program to be executed by a computer, obtaining an input image to be processed; generating a reference image from the input image using the trained model; adjusting the image quality of the input image; comparing an image quality adjusted image, which is the input image whose image quality has been adjusted, with the reference image, and calculating an image quality difference, which is a difference in image quality between the image quality adjusted image and the image quality of the reference image; The trained model is a machine learning model trained based on a training dataset including training input images that are input images for training, and training target images that are target images for training, the training target images being unpaired with the training input images; and further comprising: readjusting the image quality of the input image based on the image quality difference. program.

Images