Liver MRI image enhancement method based on Unet network-diffusion model

Through the Unet network and diffusion model based on the convolutional diffusion model, combined with dynamic condition coding and attention mechanism, virtual enhanced MRI images of arterial, portal and delayed stages of colorectal cancer liver metastasis were generated, solving the difficulty of displaying ordinary MRI images. The subtle features and the potential harm problems of Pumeixian contrast agents achieve high-quality virtual enhanced image generation and reduce medical costs.

CN118898547AInactive Publication Date: 2024-11-05HANGZHOU DIANZI UNIV

Patent Information

Application Number
CN202410922294.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-10
Publication Date
2024-11-05
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the diagnosis of liver metastasis of colorectal cancer, ordinary MRI images are difficult to clearly show the subtle characteristics of the lesions and blood perfusion. Moreover, when using Pumeixian contrast agent, there is potential harm and high cost to the liver and kidneys, making it difficult to generate arterial phase and phylum. MRI enhanced images of pulse phase and delay phase.

Method used

Using the Unet network based on the convolutional diffusion model, the MRI-type Pumeixian enhancement images of different phases are generated by inputting liver straight-scan MRI images, and the improved Unet network and diffusion model are used for preliminary and re-enhancement, combining dynamic conditional coding and Attention mechanism generates virtually enhanced MRI images of the arterial phase, portal phase, and delay phase.

Benefits of technology

Without using Pumeixian contrast agent, the generated virtual enhancement MRI image quality is excellent, avoiding the potential harm of the contrast agent to the human body, reducing the medical cost of patients, and improving the imaging quality and robustness of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118898547A_ABST
    Figure CN118898547A_ABST
Patent Text Reader

Abstract

The invention discloses a liver MRI image enhancement method based on a Unet network-diffusion model. The method comprises the following steps: acquiring different-phase MRI images of a liver part and a corresponding image enhanced by a Pumex display, and outputting a preliminary enhanced image through a Unet network model; and allowing the model to dynamically adjust parameters of the model according to condition information in different time steps through a diffusion model by using a dynamic condition coding method, so as to output a final PiMI enhanced MRI (Magnetic Resonance Imaging) image. According to the method, primary enhancement and secondary enhancement are respectively carried out, the improved Unet network model is used for carrying out primary feature enhancement on the liver MRI image, and the enhanced MRI image with slightly poor resolution is generated. And then the preliminarily enhanced MRI image is enhanced again by using a diffusion model, so that the problems of blurred boundary feeling and low resolution of a medical image are solved, and the imaging quality of the generated different phase virtual enhanced MRI image is more excellent.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of medical image processing and relates to a liver MRI image enhancement method based on a Unet network-diffusion model. Background Art

[0002] Colorectal cancer is a common malignant tumor of the digestive tract, and its morbidity and mortality rates are high worldwide. Colorectal cancer liver metastasis is one of the most common distant metastases in patients with colorectal cancer, and this metastasis seriously affects the patient's prognosis and quality of life. MRI (magnetic resonance imaging) plays a vital role in the diagnosis and management of colorectal cancer liver metastasis. With its high resolution and soft tissue contrast, MRI images can clearly display the anatomical structure and lesion characteristics of the liver, such as the location, size, number of tumors, and their relationship with surrounding tissues. In addition, MRI can also evaluate the reserve of liver function and the resectability of tumors, providing an important basis for clinical decision-making. Therefore, MRI images have irreplaceable value in the diagnosis, staging, and formulation of treatment plans for colorectal cancer liver metastasis.

[0003] In the diagnosis of colorectal cancer liver metastasis, although plain scan MRI can provide basic anatomical information, it is often not sufficient to clearly show the subtle features of the lesions and blood perfusion. In order to more accurately assess the nature and extent of the lesions, intravenous contrast-enhanced MRI is usually required to provide additional contrast. Ordinary contrast agents are mainly metabolized by the kidneys, so for patients with underlying kidney disease, such as diabetes, heart disease, or chronic kidney disease, the risk of kidney damage after contrast agent use may increase. High-dose contrast agent gadolinium deposition in the brain can cause adverse reactions to the central nervous system and harm to patients. Specific reactions are unrelated to the contrast agent dose, and even low-dose contrast agents may cause allergic reactions.

[0004] As a new type of liver-specific contrast agent, phenytoin has a high clinical application value. It has unique advantages in the diagnosis of colorectal cancer liver metastasis. After entering the liver through the bloodstream, it can be selectively taken up by the organic anion transporting polypeptide 1 (OATP1) on the liver cell membrane, producing a significant signal difference between normal liver tissue and diseased tissue. This characteristic makes phenytoin more sensitive and specific in showing liver lesions, especially in detecting small liver metastases and distinguishing benign and malignant lesions, and can quantitatively evaluate liver function. However, as a high-end contrast agent, phenytoin still needs to be excreted by the liver and kidneys in a 1:1 ratio, which may cause harm to the liver and kidneys of the human body, especially patients with liver and kidney problems, and its high cost increases the economic burden on patients. Therefore, avoiding the use of contrast agents while obtaining enhanced liver MRI images has very important clinical and social significance.

[0005] Since the liver is the only organ in the body that is supplied by both arteries and veins, and blood supply is provided by the portal vein and the hepatic artery, the MRI enhanced image of the liver is usually obtained by collecting images of three phases: the arterial phase, the portal phase, and the delayed phase, in order to more comprehensively observe the blood flow of the liver and its surrounding tissues, which helps to diagnose liver diseases more accurately, can more easily discover potential lesions, and reduce the possibility of misdiagnosis and missed diagnosis. However, the generation of images of these three phases is difficult and easily affected by liver lesions. At the same time, precise operation and coordination are required in multiple aspects such as injection of contrast agent, scanning time, and control of the patient's breathing and heartbeat, which causes a lot of trouble for doctors and technicians. In order to solve this problem, the present invention considers generating MRI enhanced images of different phases for different blood supply phases, and proposes a liver PEM-enhanced MRI enhancement method that can generate different phases of the arterial phase, portal phase, and delayed phase. Summary of the invention

[0006] The purpose of the present invention is to address the risks and cost issues of phenobarbital in clinical use, and propose a method and system for MRI image enhancement of colorectal cancer liver metastasis based on a convolution diffusion model, which can generate MRI phenobarbital-enhanced images of different phases by inputting liver plain scan MRI images without using phenobarbital.

[0007] In a first aspect, the present invention provides a method for enhancing MRI images of colorectal cancer liver metastases based on a convolution diffusion model, the method comprising the following steps:

[0008] Step S1, obtaining an undeveloped enhanced MRI image of the liver, and corresponding enhanced MRI images of different phases; the undeveloped enhanced MRI image includes different phases, specifically, a plain scan arterial phase (T1WI) image, a portal phase (T2WI) image, and a delayed phase (ADC) image; for the enhanced real MRI image, it is also necessary to additionally classify and mark the images of the arterial phase, portal phase, and delayed phase; the enhancement uses a contrast agent called promexin;

[0009] Step S2, preprocessing the two groups of MRI images;

[0010] Step S3, using the preprocessed undeveloped and enhanced MRI image groups of different phases and the corresponding preprocessed developed and enhanced MRI image groups as training labels to construct a first data set;

[0011] Step S4: construct an improved Unet network and train it using the first data set; the output is a preliminary enhanced MRI image.

[0012] The improved Unet network is to replace all convolutional layers except the last convolutional layer of the last decoding layer in the Unet network with improved residual modules;

[0013] The improved residual module includes two 3×3 convolutional layers, a standardization layer, a group normalization layer, and a SiLU activation layer, and the number of residual modules is reduced layer by layer during the downsampling process to obtain more underlying features.

[0014] Step S5, the improved Unet network takes the preprocessed and undeveloped enhanced MRI image groups of different phases of the first data set as input, and outputs a preliminary enhanced MRI image group; the preliminary enhanced MRI image group and the corresponding preprocessed and developed enhanced MRI image group are used as training labels to construct a second data set;

[0015] Step S6: building a diffusion model and training it using the second data set;

[0016] The network architecture of the diffusion model adopts the ResUNet model, which consists of two parts: a forward denoising process q and a reverse denoising process p.

[0017] In the forward noise addition process q, Gaussian noise data conforming to the standard normal distribution is gradually added to the preliminary enhanced MRI image x0 To pure noise image x T ;

[0018] The ResUNet network of the diffusion model consists of a modified ResNet encoder and a UNet decoder. The ResNet encoder consists of three convolutional stages, each of which contains multiple residual blocks, the number of which corresponds to the configuration in ResNet34. Each residual block consists of two convolutional blocks, each of which contains a group normalization layer, a SiLU activation layer, and a convolutional layer. In addition, the residual block also receives a temporal embedding, which is processed through a linear layer, a SiLU activation layer, and another linear layer. The processed result is added to the output of the first convolutional block to form a residual connection.

[0019] In the reverse denoising process p of the diffusion model, the pure noise image x T The time t is input to the ResNet encoder, and the sinusoidal position code is generated according to the time t, and the image x T Combined to get the predicted noise z θ ; Then calculate the predicted noise z θ With the added real noise The loss is calculated and the stochastic gradient descent algorithm is implemented by the Adam optimizer to update the value of the weight θ until the ResUNet training is completed;

[0020] In the reverse denoising process, the trained ResUNet model is used to predict the conditional probability distribution p of the reverse diffusion process. θ (xt-1 |x t );

[0021] In this way, the pure noise data x is gradually transformed into T Restore to x0 which is close to the real data distribution;

[0022] q(x t |x0) represents the forward diffusion process from the original data x0 to the x at time step t t The conditional probability distribution of p predicted by the ResUNet model θ (x t-1 |x t ) to approximate q(x t |x0) inverse process, so as to restore the original data x0 as accurately as possible.

[0023] The ResNet encoder of the diffusion model includes four layers of noise coding layers; each layer of noise coding layers contains multiple series-connected improved residual blocks, the number of which corresponds to the configuration in ResNet34, namely 3, 4, 6, and 3 respectively; each residual block consists of two convolutional blocks, each of which contains a convolutional layer, a group normalization layer, and a SiLU activation layer; in addition, the residual block also receives a time embedding, which is processed by a linear layer, a SiLU activation layer, and another linear layer; the processed result is added to the output of the second convolutional block to form a residual connection;

[0024] The convolution layer in the second convolution block performs convolution processing on the output of the first convolution block, adds it to the input of the residual block, and then inputs it to the group normalization layer and SiLU activation layer for processing, and finally obtains the output of the second convolution block;

[0025] The real enhanced MRI image is embedded into the ResUNet encoder as an additional input I0 using dynamic conditional coding. The specific implementation process is:

[0026] Obtain the conditional feature map of I0 through ResNet encoding The output of the first noise coding layer has the same shape as x t Noise image feature map Then the first conditional encoding layer and Attention fusion is performed at each scale of . Specifically, first, the conditional feature map and noise characteristic diagram Perform LN (layer normalization) separately to obtain the normalized feature map and Then the two normalized feature maps are multiplied element by element to obtain the affinity map, and finally the affinity map is combined with the conditional feature map Multiply to enhance the attention area and obtain the conditional feature map

[0027] The second conditional coding layer converts the noise image feature map output by the second noise coding layer and conditional feature plots Attention fusion is performed at each scale to obtain the conditional feature map

[0028] Output and conditional feature map of the third noise coding layer After residual connection, it is input into the fourth noise coding layer.

[0029] The attention fusion adopts the attention mechanism The formula is as follows:

[0030]

[0031] in stands for element-wise multiplication, k is the index of the layer; this operation is applied to the middle two stages, each of which is a convolution stage implemented according to ResNet34.

[0032] The use of dynamic conditional coding can effectively solve the problem of poor model training effect caused by low contrast of MRI images and the difficulty in distinguishing the lesion area and background inherent in medical images, thereby improving the training effect and generating more realistic images.

[0033] Step S7: using the trained improved Unet network and diffusion model to enhance the liver MRI image and generate virtual enhanced liver MRI images of different phases.

[0034] In a second aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described.

[0035] In a third aspect, the present invention provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the described method is implemented.

[0036] The beneficial effects of the present invention are:

[0037] The present invention uses two models for preliminary enhancement and secondary enhancement respectively, and uses the improved Unet network model to perform preliminary feature enhancement on the liver MRI image to generate an enhanced MRI image with slightly poor resolution. Preliminary enhancement can effectively overcome the disadvantage that the conditional prior is the only given condition in the conditional diffusion model. Then the diffusion model is used to enhance the preliminary enhanced MRI image again. In order to overcome the blurred boundary sense and low resolution of medical images, a dynamic conditional coding is introduced in each step, and the image coding features of the current step are compared with the coding features of the conditional original image using the attention mechanism. The image quality of the virtual enhanced MRI images in different phases (arterial phase, portal phase and delayed phase) is better, and it has unexpected effect on eliminating image artifacts. The experimental results show that it has also achieved good results in the external test set, proving that the model has good robustness.

[0038] Using the Unet network based on the improved residual block to perform preliminary enhancement on the original MRI image can better mine the underlying features of the image; using the loss function that combines the perceptual loss function and the structural loss function can better improve the robustness of the model; using the Adam optimizer to update the parameters of the model effectively accelerates the convergence speed of the loss. The dynamic conditional coding and the improved ResNet encoder are used to significantly improve the accuracy of the image output by the diffusion model; the attention module is introduced in the reverse denoising of the diffusion model to further improve the similarity of the output image, so that the final output MRI virtual enhanced image is more close to the real MRI enhanced image.

[0039] The present invention can generate virtual enhanced MRI images of Promexin in different phases according to the user's plain scan liver MRI image without using the Promexin contrast agent, thereby avoiding the possible harm to the human body caused by the use of contrast agents and greatly reducing the cost of medical treatment for patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a flowchart of image virtual enhancement of the present invention;

[0042] Figure 2 It is a schematic diagram of the improved Unet model structure;

[0043] Figure 3It is a schematic diagram of the diffusion model structure;

[0044] Figure 4 Schematic diagram of the improved Unet residual block structure; (a) is a traditional Unet residual block, and (b) is an improved Unet residual block of the present invention;

[0045] Figure 5 It is a schematic diagram of the improved ResNet residual block structure. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0047] This embodiment is a possible implementation based on Python. The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them.

[0048] Figure 1 The present invention is a flow chart of a multi-phase MRI prometastatic enhancement image enhancement method for colorectal cancer liver metastasis based on the Unet network-diffusion model, wherein the steps include the following:

[0049] Step S201, collecting MRI images of the liver, including plain scan T1WI images, T2WI images, ADC images, and MRI images of different phases enhanced with promecon contrast agent, and classifying and arranging the promecon enhanced images according to arterial phase, portal phase, and delayed phase;

[0050] Step S202, preprocessing the two groups of images, wherein the preprocessing includes image registration, resampling, maximum and minimum normalization, non-local mean filtering denoising, and random cropping;

[0051] Specifically, using the coordinate information of the DICOM image, taking the T1WI enhanced by PUMIX as the benchmark, coordinate registration was performed on the plain scan T1WI image, T2WI image, and ADC image respectively; then the pixel intervals of all images were uniformly resampled to l×l×1 using a linear interpolation algorithm; using maximum and minimum normalization, the brightness of the pixels was normalized within the range of [0,1] to ensure that the grayscale values ​​of all MRI images were on the same scale for subsequent analysis and comparison; then the image was denoised using a non-local average filter, which calculated the grayscale value similarity between pixels as a weight and performed a weighted average filter, which could better preserve the details and structure of the image; finally, the center position was retained according to the size of 160×160 for random cropping. The preprocessed images can effectively eliminate the differences in images caused by equipment, parameters, patients, etc., making it convenient for the subsequent model training to converge stably.

[0052] Step S203, taking the T1WI image, T2WI image, and ADC image preprocessed in step S2 as input, and the PUMEC MRI image preprocessed in step S2 as a label, constructing a data set and building a Unet network model.

[0053] Specifically, the Unet network structure is an improved Unet network, such as Figure 2 It includes 5 encoder-decoder layers. The improvement is that all convolutional layers except the last convolutional layer of the last decoding layer of the Unet network are replaced with improved residual modules. The residual module can introduce skip connections to alleviate the gradient vanishing problem in deep neural networks, thereby allowing the model to reach a deeper level.

[0054] like Figure 4 The improved residual module includes two 3×3 convolutional layers, group normalization, and SiLU activation layers. The output of the first 3×3 convolutional layer is processed by group normalization and SiLU activation layers and then input to the second 3×3 convolutional layer; the output of the second 3×3 convolutional layer is added to the original input of the first 3×3 convolutional layer and then processed by group normalization and SiLU activation layers; during the downsampling process, the improved residual module reduces the number of residual modules layer by layer according to the decreasing relationship of 5, 3, 2, 1, 1 to obtain more underlying features, which is conducive to restoring the details of the generated image. Normalization is the same as step S2, and the formula of the SiLU activation function is f(x) = x * sigmoid(x), where sigmoid(x) = 1 / (1+exp(-x)).

[0055] In order to improve the perceptual quality and accuracy of the images generated by the improved Unet network, the loss function of the improved Unet network is:

[0056]

[0057] Where L P represents the perceptual loss function, where x represents the input image, y represents the label image, and F i (x), F i (y) represents the feature vector of image x and y at the i-th feature layer, and N is the number of feature layers. The perceptual similarity between the input image and the label image is measured by calculating the distance between the feature representations of multiple feature layers in the trained neural network. This distance L P The smaller it is, the more perceptually similar the input image and the target image are.

[0058] L SSIM =||1-SSIM(y,f θ (x))∣∣1

[0059] Where L SSIM represents the structural loss function, y represents the label image, and f θ (x) is the output image; the SSIM function expression is μ x and μ y represents the mean, σ x and σ y represents the variance, σ xy represents the covariance, C1, C1 is a small constant to prevent the numerator and denominator from being zero at the same time, and |||()||1 represents the L1 norm. L SSIM It is an indicator used to measure the similarity between two image blocks. It combines information from three aspects: brightness, contrast, and structure. Specifically, L SSIM The closer the value of L is to 1, the more similar the two image blocks are in brightness, contrast, and structure; conversely, SSIM The closer the value is to -1, the less similar the two image blocks are.

[0060] During the Unet network training process, the Adam optimizer is used to implement the stochastic gradient descent algorithm to optimize the model parameters. Compared with the traditional gradient descent, this algorithm can speed up the training process. The epoch is set to 1000 and the batch is set to 16.

[0061] The output of the Unet network model is a group of liver MRI images of different phases that are initially enhanced. The initially enhanced images are still relatively blurry and the resolution needs to be improved. Therefore, a diffusion model is built to further enhance the above-mentioned group of initially enhanced images.

[0062] Step S204, to further enhance the MRI images, the preliminary enhanced MRI image group output by the Unet network is used as input, the pre-processed pleuromide enhanced MRI image group is used as training labels, a data set is constructed, and a diffusion model is built.

[0063] Specifically, Figure 3 The diffusion model consists of two parts: a forward denoising process q and a backward denoising process p.

[0064] In the forward noise addition process q, Gaussian noise data conforming to the standard normal distribution is gradually added to the preliminary enhanced MRI image x0 To pure noise image x T ; is from x0 to x T This is the forward process of gradual noise addition, which gradually adds Gaussian noise from the original image x0 to the pure noise image x T , the pure noise images generated are used as training sets. The forward process can be regarded as a Markov process. It can be expressed as in The noise is adjusted using a nonlinear scheduler.

[0065] In the reverse denoising process p of the diffusion model, the pure noise image x T The time t is input to the ResNet encoder, and the sinusoidal position code is generated according to the time t, and the image x T Combined to get the predicted noise z θ ; Then calculate the predicted noise z θ With the added real noise The loss is calculated and the stochastic gradient descent algorithm is implemented by the Adam optimizer to update the value of the weight θ until the ResUNet training is completed; it is expressed as:

[0066]

[0067] p θ (x 0:T-1 |x T ) indicates that given the current noise level x t In the case of the previous step x t-1 The conditional probability distribution of this process aims to gradually remove the noise and transform the variable from the pure noise distribution p θ (x T ) is converted back to the original data distribution p θ (x0); where θ is the parameter of the diffusion model, p θ (x t-1 |x t ) represents the conditional probability distribution learned in the reverse diffusion process. The trained ResUNet model is used to predict the conditional probability distribution p of the reverse diffusion process. θ (x t-1 |x t ), the distribution can be expressed as:

[0068]

[0069] Among them, μ θ is the noise data x at the current time step according to the ResUNet model. t and the mean parameter predicted at time step t, while β θ is the variance parameter associated with time step t, and I represents the identity matrix, ensuring that all dimensions have the same variance;

[0070] In this way, the pure noise data x is gradually transformed into T Restore to x0 which is close to the real data distribution;

[0071] q(x t |x0) represents the forward diffusion process from the original data x0 to the x at time step t t The conditional probability distribution of p predicted by the ResUNet model θ (x t-1 |x t ) to approximate q(x t |x0) inverse process, so as to restore the original data x0 as accurately as possible.

[0072] The denoising sampling of the diffusion model is based on the ResUNet network architecture, which consists of a ResNet encoder and a standard UNet decoder. The ResNet encoder part includes four layers of noise coding layers; each layer of noise coding layer contains multiple series-connected improved residual modules. The number of these residual blocks is based on the structural design of ResNet34, and the specific numbers are 3, 4, 6, and 3 respectively. This design makes it easier for the network to learn the identity mapping and is easier to optimize. Figure 5 In the improved residual block, the time embedding t is introduced to guide the network to denoise at different time steps during the diffusion process. Each residual block consists of two convolutional blocks, each of which contains a convolutional layer, a group normalization layer, and a SiLU activation layer; in addition, the residual block also receives the time embedding (Time Embedding), which is processed by a linear layer, a SiLU activation layer, and another linear layer; the processed result is added to the output of the second convolutional block to form a residual connection; and then the result is passed to the subsequent encoding stage. Finally, the feature map generated by the encoder is passed to a standard convolutional decoder, which gradually upsamples these feature maps to generate the final enhanced image. This network structure combines the residual learning mechanism of ResNet and the encoder-decoder structure of ResUNet, enabling the model to effectively denoise and generate high-quality images during the diffusion process.

[0073] The convolution layer in the second convolution block performs convolution processing on the output of the first convolution block, adds it to the input of the residual block, and then inputs it to the group normalization layer and SiLU activation layer for processing, and finally obtains the output of the second convolution block;

[0074] The real enhanced MRI image is embedded into the model as an additional input I0 using dynamic conditional encoding. The conditional feature map of I0 is obtained by ResNet encoding. and x with the same shape t Noise image feature map Then and Each scale is fused through the attention mechanism The formula is as follows

[0075]

[0076] Specifically, firstly, the conditional feature map and noise characteristic diagram Perform LN (layer normalization) separately to obtain the normalized feature map and Then the two normalized feature maps are multiplied element by element to obtain the affinity map, and finally the affinity map is combined with the conditional feature map Multiply to enhance the attention area and obtain the conditional feature map in Represents element-by-element multiplication, and k is the index of the layer. In this way, the problem of poor model training effect caused by low contrast of MRI images and the difficulty in distinguishing the lesion area from the background inherent in medical images can be solved, thereby improving the training effect and generating more realistic images.

[0077] The diffusion model is trained according to the standard process of DDPM (Denoising Diffusion Probabilistic Models), and its loss function can be expressed as:

[0078]

[0079] Where E represents the mathematical expectation, represents the noise added to the original data x0, z θ is the noise predicted by the model, is a coefficient related to the time step t and is used to control the amount of noise added. i Represents conditional information. This loss function measures the actual noise added and the noise z predicted by the model θ The difference between them is measured using the squared Euclidean distance.

[0080] The diffusion model optimizer is the same as the convolutional network model. The Adam optimizer is used to implement the stochastic gradient descent algorithm to optimize the model parameter θ. This algorithm can speed up the training process compared to the traditional gradient descent. The epoch is set to 1000 and the batch is set to 16.

[0081] For MRI plethysmographic enhanced image data of different phases, the models are trained and saved respectively.

[0082] After inputting the user's liver plain scan MRI image, the model can output MRI MRI enhanced images in three different phases: arterial phase, portal phase, and delayed phase.

[0083] Step S205, performing quality evaluation on the generated virtual enhanced image, the specific method is as follows:

[0084] Calculate the mean absolute error, the formula is: Where Y represents the real enhanced MRI image, represents the virtual enhanced MRI image, ΩY represents the number of images, and the smaller the MAE value, the better the quality of the generated image;

[0085] Calculate its peak signal-to-noise ratio (PSNR), the formula is Where Y represents the real enhanced MRI image, Represents a virtual enhanced MRI image, MAX I It is the maximum possible pixel value in the image. MSE represents the mean square error. The larger the PSNR value, the better the quality of the generated image.

[0086] Calculate its structural similarity (SSIM), the formula is Where Y represents the real enhanced MRI image, represents the virtual enhanced MRI image, μ represents the mean, σ represents the variance, Represents Y and The covariance of SSIM is in the range of [0,1]. The closer it is to 1, the higher the similarity of the generated images.

[0087] Calculate the no-reference image evaluation index (NIQE), first the formula of the MVG model is The distance formula of NIQE is: v1, v2 and Σ1, Σ2 are the mean and variance of the Gaussian mixture model of the natural image and the image to be tested, respectively. The smaller the NIQE value, the better the quality of the generated image.

[0088] In order to verify the effectiveness and advantages of the model, the U-DDPM of the present invention, GAN, and ResUNet were trained in the same hardware and software environment with the same hyperparameters, and the output results of the three models were compared and analyzed using the above objective evaluation indicators. From the objective quantitative analysis of image quality evaluation indicators, among the three models, the improved U-DDPM model of the present invention performed best in this task, and achieved better values ​​in each indicator, verifying the effectiveness and superiority of the model improvement. The experimental results are shown in Table 1-2.

[0089] Table 1 Mean values ​​of indicators of each model in the internal test set A

[0090]

[0091] Table 2 Mean values ​​of indicators of each model in the external test set B

[0092]

[0093] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A liver MRI image enhancement method based on Unet network-diffusion model, characterized in that The method comprises the following steps: S1: Acquire MRI images of the liver at different phases without enhancement and the corresponding MRI images at different phases with enhancement; S2: preprocessing operations are performed on all acquired MRI images; S3: using the MRI image groups of different phases without development enhancement after preprocessing and the corresponding MRI image groups with development enhancement after preprocessing as training labels to construct a first data set; S4: Build an improved Unet network model, train it using the first data set, and output a preliminary enhanced MRI image; S5: The improved Unet network takes the preprocessed and unenhanced MRI image groups of different phases of the first data set as input, and outputs a preliminary enhanced MRI image group; the preliminary enhanced MRI image group and the corresponding preprocessed and enhanced MRI images of different phases are used as training labels to construct a second data set; S6: Build a diffusion model and train it using the second dataset; S7: Liver MRI image enhancement is achieved using the trained improved Unet network and diffusion model to generate virtual enhanced liver MRI images of different phases.

2. The method according to claim 1, characterized in that: The images of different phases in step S1 include plain scan T1WI images, T2WI images, and ADC images.

3. The method according to claim 1, characterized in that The improved U-net network is to replace the other convolutional layers except the last convolutional layer of the last decoding layer in the U-net network with the improved residual module; The improved residual module is cascaded in sequence and includes two 3×3 convolutional layers connected in series, a group normalization layer, and a SiLU activation layer.

4. The method according to claim 1, characterized in that The loss function of the improved Unet network is: Where L P represents the perceptual loss function, x represents the input image, y represents the label image, F i (x), F i (y) represents the feature vectors of images x and y at the i-th layer, respectively, and N is the number of layers of the improved Unet network; L SSIM =||1-SSIM(y,f θ (x))||1 Where L SSIM represents the structural loss function, f θ (x) is the output image.

5. The method according to claim 1, characterized in that The diffusion model includes two parts: a forward denoising process q and a reverse denoising process p, and its architecture is an improved ResUNet model, including a ResNet encoder and a UNet decoder; In the forward noise addition process q, Gaussian noise data conforming to the standard normal distribution is gradually added to the preliminary enhanced MRI image x0. To pure noise image x T ; In the reverse denoising process p, the pure noise image x T The time t is input to the ResNet encoder, and the sinusoidal position code is generated according to the time t, and the image x T Combined to get the predicted noise z θ ; Then calculate the predicted noise zθ and the added real noise The loss is calculated and the stochastic gradient descent algorithm is implemented by the Adam optimizer to update the value of the weight θ until the ResUNet training is completed; it is expressed as: p θ (x 0:T-1 |x T ) indicates that given the current noise level x t In the case of the previous step x t-1 The conditional probability distribution of this process aims to gradually remove the noise and transform the variable from the pure noise distribution p θ (x T ) is converted back to the original data distribution p θ (x0); where θ is the parameter of the diffusion model, p θ (x t-1 |x t ) represents the conditional probability distribution learned during the reverse diffusion process.

6. The method according to claim 5, characterized in that: In the reverse denoising process, the trained ResUNet model is used to predict the conditional probability distribution p of the reverse diffusion process. θ (x t-1 |x t ), the distribution can be expressed as: Among them, μ θ is the noise data x at the current time step according to the ResUNet model. t and the mean parameter predicted at time step t, while β θ is the variance parameter associated with time step t, and I represents the identity matrix, ensuring that all dimensions have the same variance; In this way, the pure noise data x is gradually transformed into T Restore to x0 which is close to the real data distribution; q(x t |x0) represents the forward diffusion process from the original data x0 to the x at time step t t The conditional probability distribution of p predicted by the ResUNet model θ (x t-1 |x t ) to approximate q(x t |x0) inverse process, so as to restore the original data x0 as accurately as possible.

7. The method according to claim 5, characterized in that The ResNet encoder of the diffusion model includes four layers of noise coding layers; each layer of noise coding layers contains multiple modified residual blocks connected in series; each residual block consists of two convolutional blocks, each of which contains a convolutional layer, a group normalization layer, and a SiLU activation layer; in addition, the residual block also receives a temporal embedding, which is processed through a linear layer, a SiLU activation layer, and another linear layer; the processed result is added to the output of the second convolutional block to form a residual connection; The convolution layer in the second convolution block performs convolution processing on the output of the first convolution block, adds it to the input of the residual block, and then inputs it to the group normalization layer and SiLU activation layer for processing, and finally obtains the output of the second convolution block; The real enhanced MRI image is embedded into the ResUNet encoder as an additional input I0 using dynamic conditional coding. The specific implementation process is: Obtain the conditional feature map of I0 through ResNet encoding The output of the first noise coding layer has the same shape as x t Noise image feature map Then the first conditional encoding layer and Attention fusion is performed at each scale of . Specifically, first, the conditional feature map and noise characteristic diagram Perform LN (layer normalization) separately to obtain the normalized feature map and Then the two normalized feature maps are multiplied element by element to obtain the affinity map, and finally the affinity map is combined with the conditional feature map Multiply to enhance the attention area and obtain the conditional feature map The second conditional coding layer converts the noise image feature map output by the second noise coding layer and conditional feature plots Attention fusion is performed at each scale to obtain the conditional feature map Output and conditional feature map of the third noise coding layer After residual connection, it is input into the fourth noise coding layer.

8. The method according to claim 7, characterized in that Attention fusion uses attention mechanism The formula is as follows: in stands for element-wise multiplication, and k is the index of the layer.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 8.

10. A computing device, comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 8 is implemented.

Citation Information

Patent Citations

  • Colorectal cancer liver metastasis MRI image enhancement method based on generative adversarial network

    CN115908333A

  • Weak light enhancement method based on adaptive sparse attention diffusion model

    CN117649368A

  • SAR image generation method based on de-noising diffusion probability model

    CN118230191A

  • Liver CT automatic segmentation method based on deep shape learning

    WO2023071531A1

Cited By

  • Physical intelligent magnetic resonance diffusion weighted image denoising and quantifying method

    CN119251074A

  • Low-dose CT image denoising generalization method based on diffusion model

    CN120525755A

  • A low-dose CT image denoising generalization method based on a diffusion model

    CN120525755B

  • Primary liver cancer subtype image intelligent identification method

    CN122223453A