Image Processing Method, Apparatus, Device, Medium and Product

Abstract

Embodiments of the present invention disclose an image processing method, apparatus, device, medium and product. Among them, the method includes: obtaining a three-dimensional magnetic resonance image to be processed, performing image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; respectively inputting the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices with preliminarily improved image quality, and recombining the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; inputting the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; the three-dimensional image enhancement model is trained based on a latent diffusion generative model. The technical solution of the embodiments of the present invention solves the problem that the current algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of medical image processing based on artificial intelligence, and in particular, to an image processing method, apparatus, device, medium and product. Background Art

[0002] Magnetic Resonance Images (MRI) often suffer from various forms of degradation, including degradation caused by noise, artifacts (such as ghosting caused by motion), and intensity inhomogeneity, etc., which will affect the quality of the processed images. Before MRI is used for clinical analysis, image processing is required to improve the image quality.

[0003] However, in the process of implementing the present invention, it is found that there are at least the following technical problems in the prior art:

[0004] At present, traditional methods for image enhancement processing of MRI are prone to losing image details, with high computational complexity and limited effects. And image enhancement methods based on deep learning are more targeted at two-dimensional MRI, and for three-dimensional MRI, spatial structure information will be lost, and the image enhancement processing effect is also not good. Summary of the Invention

[0005] Embodiments of the present invention provide an image processing method, apparatus, device, medium and product, which can directly process three-dimensional magnetic resonance images, enhance image detail processing, and improve the effect of image enhancement.

[0006] In a first aspect, embodiments of the present invention provide an image processing method, which includes:

[0007] Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0008] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with preliminarily improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0009] Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0010] In a second aspect, embodiments of the present invention further provide an image processing apparatus, which includes:

[0011] An image acquisition module, configured to acquire a three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0012] A first image enhancement module, configured to input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with preliminarily improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0013] A second image enhancement module, configured to input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, which includes:

[0015] One or more processors;

[0016] A memory, configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method provided in any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image processing method provided in any embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the image processing method provided in any embodiment of the present invention.

[0020] The embodiments in the above invention have the following advantages or beneficial effects:

[0021] In an embodiment of the present invention, by obtaining a three-dimensional magnetic resonance image to be processed and performing image slicing on the three-dimensional magnetic resonance image to be processed, corresponding original magnetic resonance image slices are obtained; the original magnetic resonance image slices are respectively input into a pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices with initially improved image quality, and the new magnetic resonance image slices are recombined to obtain a new three-dimensional magnetic resonance image to be processed; the new three-dimensional magnetic resonance image to be processed is input into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model. The technical solution of the embodiment of the present invention solves the problem that the current algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement. Description of the Drawings

[0022] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present invention;

[0023] Figure 2 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0024] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0025] Figure 4 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the framework of an image enhancement model in an application example of an image processing method provided by an embodiment of the present invention;

[0027] Figure 6 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

[0028] Figure 7 is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed Embodiments

[0029] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only for explaining the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings, rather than all the structures.

[0030] Figure 1The flowchart of an image processing method provided by an embodiment of the present invention. This embodiment is applicable to the scenario of image enhancement processing using MRI, especially for the case of three-dimensional MRI image enhancement processing. This method can be executed by an image processing device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0031] As Figure 1 shown, the image processing method of this embodiment includes the following steps:

[0032] S110. Obtain the three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain the corresponding original magnetic resonance image slices.

[0033] The three-dimensional magnetic resonance image to be processed can be the magnetic resonance imaging result of any region of interest, an image that requires image quality improvement.

[0034] During the magnetic resonance imaging process, the magnetic resonance image may degenerate due to reasons such as noise, motion artifacts, and / or signal inhomogeneity, resulting in unsatisfactory image quality. Therefore, in order to make the magnetic resonance imaging have a clearer visual effect during downstream use and not affect the reading and analysis of the image by the readers, image enhancement processing is required to improve the image quality.

[0035] Performing image slicing on the three-dimensional magnetic resonance image to be processed can be to perform image slicing in any direction in the three-dimensional image space to obtain the corresponding original magnetic resonance image slices. For example, the original magnetic resonance image slices can be coronal plane image slices, sagittal plane image slices, or horizontal plane image slices.

[0036] S120. Input the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain the new magnetic resonance image slices with the image quality initially improved, and recombine the new magnetic resonance image slices to obtain the new three-dimensional magnetic resonance image to be processed.

[0037] In this embodiment, the image quality improvement is divided into two stages. In the first stage, a two-dimensional image slice rough enhancement model is used to remove large-scale degradation; in the second stage, a three-dimensional fine control latent diffusion generation model is used to restore the missing details.

[0038] In S120, it corresponds to the first stage of image quality improvement. The pre-trained two-dimensional image enhancement model can be a pre-trained image encoder and decoder. Its training samples can be low-quality magnetic resonance sample images, and the learning label is the corresponding high-quality magnetic resonance sample image of the sample.

[0039] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with a preliminary improvement in image quality. Further, according to the image sequence of the original magnetic resonance image slices, the new magnetic resonance image slices can be recombined to obtain a new three-dimensional magnetic resonance image to be processed.

[0040] S130. Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image.

[0041] In S130, it corresponds to the second stage of image quality improvement.

[0042] Among them, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model. The new three-dimensional magnetic resonance image to be processed can be used as a control signal for controlling the diffusion process of the latent diffusion generative model, enabling the three-dimensional image enhancement model to generate a very detailed and rich-detail target three-dimensional magnetic resonance image. Thus, the image quality of the three-dimensional magnetic resonance image to be processed is improved.

[0043] The technical solution of this embodiment obtains a three-dimensional magnetic resonance image to be processed, performs image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; inputs the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with a preliminary improvement in image quality, and recombines the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; inputs the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; among them, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0044] Figure 2 It is a flowchart of another image processing method provided by an embodiment of the present invention. This embodiment and the image processing method in the above embodiment belong to the same inventive concept, and further describes the process of preliminarily improving the two-dimensional image quality in a single image direction. This method can be executed by an image processing device, and the device can be implemented in a software and / or hardware manner and integrated into a computer device with application development functions.

[0045] As Figure 2 shown, the image processing method of this embodiment includes the following steps:

[0046] S210. Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices.

[0047] S220. Input the original magnetic resonance image slices into the pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder.

[0048] Among them, the variational autoencoder (VAE) is the two-dimensional image enhancement model. The encoder of the variational autoencoder uses multiple residual Swin Transformer blocks (RSTB) to extract the depth features of each original magnetic resonance image slice, and the decoder of the variational autoencoder upsamples these depth features to the original image space to obtain the processed image result with improved quality.

[0049] In this step, input the original magnetic resonance image slices into the pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder. Among them, the slice image features are the depth features of higher dimensions extracted by the encoder of the variational autoencoder.

[0050] S230. Input the slice image features into the decoder of the variational autoencoder for feature decoding to obtain the new magnetic resonance image slices with preliminarily improved image quality.

[0051] The new magnetic resonance image slices are that the decoder of the variational autoencoder upsamples the slice image features to the original image space to obtain the processed image result with improved quality.

[0052] Furthermore, in an optional implementation manner, the training process of the two-dimensional image enhancement model (i.e., the variational autoencoder) includes the following steps:

[0053] First, obtain the three-dimensional magnetic resonance sample images that meet the preset image quality standard, that is, the high-quality sample images that can be used as sample learning labels.

[0054] Then, a preset image degradation processing model is used to degrade the three-dimensional magnetic resonance sample image to obtain a degraded three-dimensional magnetic resonance sample image, and the degraded three-dimensional magnetic resonance sample image is subjected to image slicing to obtain corresponding degraded magnetic resonance sample image slices. Among them, the preset image degradation processing model includes any one and a combination of multiple of a noise simulation algorithm model, a motion artifact simulation algorithm model, and an intensity non-uniformity simulation algorithm model. That is to say, a three-dimensional magnetic resonance sample image can have 7 degradation processing methods, such as noise simulation degradation, motion artifact simulation degradation, intensity non-uniformity simulation degradation, noise simulation degradation and motion artifact simulation degradation, motion artifact simulation degradation and intensity non-uniformity simulation degradation, noise simulation degradation and intensity non-uniformity simulation degradation, and noise simulation degradation, motion artifact simulation degradation, and intensity non-uniformity simulation degradation. Among them, the motion artifact simulation degradation algorithm model can have different motion artifact simulation strategies. This can enrich the model training samples to the greatest extent.

[0055] Subsequently, the degraded magnetic resonance sample image slices can be respectively input into the two-dimensional image enhancement model to be trained to obtain two-dimensional model output reference images, and the two-dimensional model output reference images are combined to form corresponding three-dimensional magnetic resonance reference images.

[0056] Finally, the learning loss of the two-dimensional image enhancement model to be trained can be calculated based on the three-dimensional magnetic resonance reference image and the three-dimensional magnetic resonance sample image, and the parameters of the two-dimensional image enhancement model to be trained are updated based on the learning loss to realize the training of the two-dimensional image enhancement model. The learning loss can include mean squared error (MSE) loss and structural similarity (SSIM) loss.

[0057] S240. Recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0058] After the new three-dimensional magnetic resonance image to be processed is processed, the structural information in a single direction is restored, most of the low-quality information is removed, and the image quality is improved but there are fewer details. The details of the new three-dimensional magnetic resonance image to be processed can be further enriched through S250.

[0059] S250. Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image.

[0060] Among them, the three-dimensional image enhancement model is a model trained based on the latent diffusion generative model. It can adopt the U-Net architecture, which has a symmetric encoder-decoder structure. It can effectively capture the context information and details of the new three-dimensional magnetic resonance image to be processed, and use skip connections to retain the feature information at different levels, so as to generate very detailed and rich-output.

[0061] The technical solution of this embodiment is to obtain a three-dimensional magnetic resonance image to be processed, perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; input the original magnetic resonance image slices into a pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder; input the slice image features into the decoder of the variational autoencoder for feature decoding to obtain new magnetic resonance image slices with preliminary improved image quality; recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0062] Figure 3 It is a flowchart of another image processing method provided by an embodiment of the present invention. This embodiment and the image processing method in the above embodiment belong to the same inventive concept, and further illustrate the process of three-dimensional image enhancement processing. This method can be executed by an image processing device, and the device can be implemented in a software and / or hardware manner and integrated in a computer device with application development functions.

[0063] As Figure 3 shown, the image processing method of this embodiment includes the following steps:

[0064] S310. Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices.

[0065] S320. Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with preliminary improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0066] S330. Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model, and perform feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain image encoding features in the latent space.

[0067] Among them, the image encoding module of the three-dimensional image enhancement model can be the encoder of a pre-trained variational autoencoder. By downsampling the new three-dimensional magnetic resonance image to be processed, feature encoding is performed to extract higher-dimensional features, thereby obtaining the image encoding features in the latent space.

[0068] The encoder of the variational autoencoder downsamples the new three-dimensional magnetic resonance image to be processed. For example, it can be an 8-fold downsampling encoding process.

[0069] S340: Add noise to the image encoding features to obtain the noisy image encoding features, and input the noisy image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result in the latent space.

[0070] The image diffusion generation module of the three-dimensional image enhancement model performs the encoding and decoding processes of the image based on the U-Net structure to achieve denoising diffusion and obtain the image generation result in the latent space.

[0071] S350: Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

[0072] The image decoding module can be a decoding module corresponding to the image encoding module, such as the decoder of a variational autoencoder.

[0073] The technical solution of this embodiment obtains the three-dimensional magnetic resonance image to be processed, performs image slicing on the three-dimensional magnetic resonance image to be processed to obtain the corresponding original magnetic resonance image slices; inputs the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain the new magnetic resonance image slices with the image quality initially improved, and recombines the new magnetic resonance image slices to obtain the new three-dimensional magnetic resonance image to be processed; inputs the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model, performs feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain the image encoding features in the latent space; add noise to the image encoding features to obtain the noisy image encoding features, and input the noisy image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result in the latent space; input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0074] Figure 4The flowchart of another image processing method provided by an embodiment of the present invention. This embodiment and the image processing method in the above embodiment belong to the same inventive concept, and further illustrate the processing process of three-dimensional MRI image enhancement. This method can be executed by an image processing device, which can be implemented in software and / or hardware and integrated in a computer device with application development functions.

[0075] As Figure 4 shown, the image processing method of this embodiment includes the following steps:

[0076] S410. Obtain the three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices.

[0077] S420. Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0078] S430. Input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result.

[0079] S440. Perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain a corresponding gradient map, input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a second image encoding result.

[0080] The gradient map of the new three-dimensional magnetic resonance image to be processed is a derivative image obtained after processing the data of the new three-dimensional magnetic resonance image to be processed. The MRI device uses a gradient magnetic field to encode spatial positions. Hydrogen protons at different positions have different resonance frequencies or phases due to the action of the gradient field, and are thus distinguished for imaging. The gradient map focuses on this change information caused by the gradient field. Through a specific algorithm, it extracts the rate of change of the signal intensity with spatial position within the imaging region, that is, the gradient information, and visualizes it. For example, for the new three-dimensional magnetic resonance image to be processed, through mathematical operations such as Fourier transform, the complex time-domain signal can be transformed into a frequency-domain signal, and further the components reflecting the spatial frequency change can be extracted from it to construct a gradient map, showing the rapid change of the signal at the boundary between tissues. The image characteristics of the gradient map are prominent boundaries, high-resolution details, and also have guiding significance for clinical applications.

[0081] In this embodiment, in order to make the image generated by the image diffusion generation module a more realistic three-dimensional magnetic resonance image, some feature encoding information is added as the diffusion control information of the image diffusion generation module. The gradient map is input into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the second image encoding result. The second image encoding result will also be input into the image diffusion generation module.

[0082] Among them, the second image encoding module can be a variational autoencoder with the same structure as the first image encoding module but with optimized parameters.

[0083] S450. Input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset-dimensional feature information.

[0084] The third image encoding module can be a conditional encoder for extracting the feature information of the new three-dimensional magnetic resonance image to be processed. This feature information is information related to the region of interest in the three-dimensional magnetic resonance image to be processed. For example, if the three-dimensional magnetic resonance image to be processed is a three-dimensional magnetic resonance image of the head, the preset-dimensional feature information can be parameters related to the head such as the age, gender, ventricular volume, and / or brain volume corresponding to the three-dimensional magnetic resonance image to be processed.

[0085] The encoding module can be trained according to the feature information to be extracted to obtain the target third image encoding module.

[0086] S460. Concatenate the first image encoding result, the second image encoding result, and the preset-dimensional feature information to obtain the image encoding feature in the latent space.

[0087] S470. In each denoising time step of the image diffusion generation module, input the noise image encoding feature into the conditional control sub-module in the image diffusion generation module; and, concatenate the noise information in the noise image encoding feature with the preset-dimensional feature information to obtain the concatenation result, and input the concatenation result into the U-net sub-module.

[0088] Among them, the image diffusion generation module includes a conditional control sub-module and a pre-trained U-net sub-module, where the conditional control sub-module has the same structure as the encoder of the U-net sub-module.

[0089] S480. Input the encoding feature processed by the conditional control sub-module for the noise image encoding feature into the decoder in the U-net sub-module, and decode it together with the encoding result of the encoder in the U-net sub-module to obtain the image generation result in the latent space.

[0090] The U-Net sub-module has a symmetric encoder and decoder. The encoder is responsible for downsampling the encoded features of the input noisy image to gradually extract higher-level and more abstract features while reducing the spatial resolution; the decoder then performs upsampling to gradually restore the low-resolution feature map to the size of the original input image and reconstruct the image using the features extracted by the encoder during this process. This encoder-decoder structure enables the model to effectively capture the high-level features and details of the image and handle image information well in both the forward and reverse processes of the diffusion model.

[0091] This step implements skip connections, which directly connect the lower layers of the encoder in the conditional control sub-module and the encoder in the U-net sub-module to the higher layers of the decoder, enabling the decoder in the U-net sub-module to directly utilize the features of the corresponding layers in the encoder when reconstructing the image. In the diffusion model, especially during the process of gradually upgrading the image from a low-resolution state to a high-resolution state, skip connections help retain the details of the image, avoid information loss and blurring, and make the generated image clearer and more accurate.

[0092] In a preferred embodiment, in each noise reduction time step, the generation result of the image diffusion generation module can also be sampled and updated to obtain the best sampling data. For example, sample the generation result of the image in the latent space. Then, decode the sampled data to obtain the image decoding result in the image space corresponding to the new three-dimensional magnetic resonance image to be processed. Among them, the decoder for decoding the sampled data can be the decoder corresponding to the first image encoding module. Furthermore, the normalized cross-correlation loss between the image decoding result and the new three-dimensional magnetic resonance image to be processed can be calculated; based on the normalized cross-correlation loss, the sampled data is gradient-updated to obtain the best sampling result in the corresponding noise reduction time step. This sampling result can be used as the input in the next noise reduction time step for the next noise reduction process. The process of gradient-updating the sampled data can be achieved through a generation guidance network, and the generation guidance network is composed of the decoder corresponding to the first image encoding module and the gradient updater for calculating the normalized cross-correlation loss.

[0093] After completing the denoising process through step-by-step iteration, the generation result of the image in the latent space can be obtained.

[0094] S490. Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

[0095] Among them, the image decoding module can be the decoder corresponding to the first image encoding module.

[0096] The technical solution of this embodiment is to obtain a three-dimensional magnetic resonance image to be processed, perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result; perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain a corresponding gradient map, input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a second image encoding result; input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset-dimensional feature information; splice the first image encoding result, the second image encoding result and the preset-dimensional feature information to obtain an image encoding feature in the latent space; in each denoising time step of the image diffusion generation module, input the noise image encoding feature into the conditional control sub-module in the image diffusion generation module; and splice the noise information in the noise image encoding feature with the preset-dimensional feature information to obtain a splicing result, and input the splicing result into the U-net sub-module; input the encoding feature processed by the conditional control sub-module for the noise image encoding feature into the decoder in the U-net sub-module, and perform decoding together with the encoding result of the encoder in the U-net sub-module to obtain an image generation result in the latent space; input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain a target three-dimensional magnetic resonance image. The technical solution of the embodiment of the present invention solves the problem that the current algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0097] In a specific embodiment, the three-dimensional magnetic resonance image to be processed is a head three-dimensional magnetic resonance image. To implement head three-dimensional magnetic resonance image enhancement processing, first, the publicly available Alzheimer's Disease Neuroimaging Initiative dataset is used to train the two-dimensional image enhancement model and the three-dimensional image enhancement model.

[0098] Figure 5 Shows a two-stage coarse-to-fine 3D MRI enhancement network framework.

[0099] In the Alzheimer's Disease Neuroimaging Initiative dataset, 211 T1-weighted MRI scan datasets scanned at 3T were selected. These data were used for the degradation model and the enhancement model, and they were divided into a training set, a validation set and a test set, which were 146, 31 and 31 respectively.

[0100] First, register our data rigidly using a preset rigid registration tool into a preset standard image space. The preset standard image space can be the space corresponding to an MRI image generated by an application technique or model based on the Latent Diffusion Model (LDM) in brain imaging or brain science research, with a size of [160, 224, 160].

[0101] After that, apply a degradation model to randomly apply one or more of three degradation methods to the MRI image, including adding artifacts, Gaussian noise, and intensity inhomogeneity. This degradation method makes each high-quality image (Ground Truth) correspond to seven different degraded images, thus forming image pairs for training. Then normalize all the images to between [0, 1].

[0102] Among them, the MRI degradation model is used to better simulate the artifact problems in reality, such as noise, motion artifacts, and non-uniform intensity distribution.

[0103] Specifically, in noise simulation, according to the fact that MRI noise follows a Rician distribution, which means that both the real part and the imaginary part are affected by Gaussian noise with the same variance. The noise image can be simulated by the following formula: . Where I HQ is the original image, I LQ is the image after degradation processing, µ and σ are the mean and standard deviation of the Gaussian noise (default values: 0, 0.02), and the signal-to-noise ratio is set to 5 to replicate the real MRI noise level.

[0104] In motion artifact simulation, to simulate the motion artifacts of brain MRI, three different phase-encoding line sampling strategies can be adopted. For example, independent Gaussian sampling is used to simulate a single motion artifact; segmented instantaneous sampling is used to simulate intermittent artifacts; segmented constant sampling, where the standard deviation of the Gaussian distribution is set to 10 voxels, and the number of segments (k) is randomly determined as an integer between 1 and 8. During the construction of the dataset, one of the above artifact types can be randomly applied to 0 - 20% of the phase-encoding lines.

[0105] In intensity inhomogeneity simulation, it can be assumed that the MRI data has a dimension of D × H × w, and the coil center is randomly selected , for each voxel , the Euclidean distance to is normalized to . Then calculate the coil sensitivity mask :[[]]END]] , where α represents the intensity controlling the coil effect. Apply this mask to the original image: This simulates the intensity inhomogeneity caused by coil positioning and enhances the robustness of the model to such variations in real MRI images.

[0106] After the samples are prepared, the network can be trained. For the first-stage training, first slice along the transverse plane (Transverse Plane / Axial Plane), and 160 slices of 160*224 are obtained for each three-dimensional brain MRI image. Then, the results of the first-stage training are stitched together to obtain a three-dimensional brain MRI image with preliminary quality improvement to continue the second-stage training. Both the trained two-dimensional image enhancement model and the three-dimensional image enhancement model are trained using the Adam optimizer. For the two-dimensional image enhancement network, it is trained for 8000 steps with a batch size of 96 and a decay rate of 5e-5. In the three-dimensional image enhancement model, in the image encoding network (Conditional Extraction Network, CENet corresponding to the first image encoding module and the second image encoding module), the variational autoencoder network is used, trained for 8000 steps with a batch size of 3 and a decay rate of 1e-5. The cond extraction network (Cond encoder corresponding to the third image encoding module) uses the architecture of 3DResnet18, trained for 18000 steps with a batch size of 3 and a decay rate of 1e-5. For the conditional control network (Conditional Control Network, CCNet, corresponding to the conditional control sub-module in the image diffusion generation module), BrainLdm is used as the generation prior, and the control network is trained for 12000 steps with a decay rate of 1e-4 and a batch size of 3. For the final generative guidance network (Generative Guidance Network, GGNet, corresponding to the network for optimizing the sampled data in the noise reduction time steps), 1000 steps of sampling are used. The first 900 steps are for normal sampling generation, and the last 100 steps use the Normalized Cross-Correlation Loss (NCCLoss) to constrain each step of generation.

[0107] Furthermore, in the first stage, a designed 2D network is used to recover the structural information of the three-dimensional brain MRI image in a single direction, removing most of the low-quality information and generating a high-quality but less detailed image. The 2D network adopts the structure of a Variational Autoencoder (VAE). Its encoder uses multiple Residual Swin Transformer Blocks (RSTB) to extract deep features, and the decoder upsamples these features to the original image space.

[0108] The network parameters are trained by optimizing the following joint loss function, including the Mean Squared Error (MSE) loss and the Structural Similarity Index (SSIM) loss: LSSIM = 1−SSIM(I HQ , I ref ). Among them, I HQ is the high-quality image, I LQ is the low-quality image, and I ref is the reference image generated by the 2D network. This output serves as the preliminary image optimization result for the subsequent diffusion model.

[0109] In the second stage, the prior knowledge of the pre-trained diffusion model and the output I ref from the first stage are utilized to further improve the accuracy and realism of MRI image reconstruction. The network consists of three main components: the Conditional Extraction Network (CENet), the Conditional Control Network (CCNet), and the Generative Guidance Network (GGNet).

[0110] The pre-trained BrainLDM model has the ability to generate realistic T1-weighted brain MRI images. It includes an Autoencoder and a diffusion model containing multiple UNet denoisers, which can effectively recover image details.

[0111] The Conditional Extraction Network (CENet) uses the following three key conditions to guide the generation of the diffusion process: the output I ref of the first stage, the gradient map of I ref , and four brain-related parameters (age, gender, ventricular volume, brain volume). Specifically, through the VAE encoder E pre-trained by BrainLDM, I refis mapped to the latent space. Then, the gradient map of I ref is processed by an encoder with the same structure as the VAE encoder but optimized for I ref to preserve edge information. Finally, a conditional encoder is designed to encode I ref into the conditional variable C (four brain-related parameters) to assist the main network in generating more realistic images. After training, all the parameters of the conditional encoder are kept frozen.

[0112] Conditional Control Network (CCNet) To ensure the correct direction of image generation, a conditional control network (CCNet, Figure 5 the red part in ) based on ControlNet is designed. A trainable copy of the pre-trained UNet encoder is constructed, and I ref , the gradient map of I ref , the four conditional variables C, and the noise are concatenated as z' t and then input into the network, as shown in the following formula: .

[0113] The output of CCNet is added to the result of the pre-trained UNet decoder. During training, only the parameters of CCNet are updated.

[0114] Generation Guidance Network (GGNet) To further improve the fidelity of the generated images and make the denoising process more consistent with the structure, a generation guidance network (GGNet) is designed. At each sampling step, UNet first predicts the noise, then removes the predicted noise from z' t to obtain a clean latent variable, which decodes z0 into , and calculates the difference between and using the Normalized Cross-Correlation Loss (NCCLoss). The latent variable z0 is updated by gradient descent, and finally a reconstructed image guided by prior knowledge and is generated . The formula for NCCLoss is as follows: . Where and are the means of and respectively.

[0115] Figure 6Schematic diagram of a structure of an image processing apparatus provided by an embodiment of the present invention. This embodiment is applicable to scenarios of image enhancement processing using MRI, especially for the case of three-dimensional MRI image enhancement processing. The image processing apparatus can be implemented in software and / or hardware and integrated into a computer terminal device with application development functions.

[0116] As Figure 6 shown, the image processing apparatus includes: an image acquisition module 510, a first image enhancement module 520, and a second image enhancement module 530.

[0117] Among them, the image acquisition module 510 is used to acquire a three-dimensional magnetic resonance image to be processed, perform image slicing processing on the three-dimensional magnetic resonance image to be processed, and obtain corresponding original magnetic resonance image slices; the first image enhancement module 520 is used to input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively, obtain new magnetic resonance image slices with a preliminary improvement in image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; the second image enhancement module 530 is used to input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; among them, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0118] The technical solution of this embodiment is to acquire a three-dimensional magnetic resonance image to be processed, perform image slicing processing on the three-dimensional magnetic resonance image to be processed, and obtain corresponding original magnetic resonance image slices; input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively, obtain new magnetic resonance image slices with a preliminary improvement in image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; among them, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and the effects are not good at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0119] In an alternative embodiment, the first image enhancement module 520 is specifically used for:

[0120] Input the original magnetic resonance image slices into a pre-trained variational autoencoder respectively, and extract slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder;

[0121] Input the slice image features into the decoder of the variational autoencoder for feature decoding to obtain new magnetic resonance image slices with a preliminary improvement in image quality.

[0122] In an alternative embodiment, the image processing device further includes a model training module for training a model to obtain a two-dimensional image enhancement model. The training process of the two-dimensional image enhancement model includes:

[0123] Obtain three-dimensional magnetic resonance sample images that meet the preset image quality standards;

[0124] Use a preset image degradation processing model to degrade the three-dimensional magnetic resonance sample images to obtain degraded three-dimensional magnetic resonance sample images, and perform image slicing on the degraded three-dimensional magnetic resonance sample images to obtain corresponding degraded magnetic resonance sample image slices;

[0125] Input the degraded magnetic resonance sample image slices into the two-dimensional image enhancement model to be trained respectively to obtain two-dimensional model output reference images, and form corresponding three-dimensional magnetic resonance reference images from the two-dimensional model output reference images; wherein, the two-dimensional image enhancement model is a variational autoencoder

[0126] Calculate the learning loss of the two-dimensional image enhancement model to be trained based on the three-dimensional magnetic resonance reference images and the three-dimensional magnetic resonance sample images, and update the parameters of the two-dimensional image enhancement model to be trained based on the learning loss to realize the training of the two-dimensional image enhancement model.

[0127] In an alternative embodiment, the preset image degradation processing model includes any one and a combination of multiple of a noise simulation algorithm model, a motion artifact simulation algorithm model, and an intensity non-uniformity simulation algorithm model.

[0128] In an alternative embodiment, the second image enhancement module 530 is specifically configured to:

[0129] Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model, and perform feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain image encoding features in the latent space;

[0130] Add noise to the image encoding features to obtain noise image encoding features, and input the noise image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain an image generation result in the latent space;

[0131] Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

[0132] In an alternative embodiment, the second image enhancement module 530 is specifically configured to:

[0133] Input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result;

[0134] Perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain the corresponding gradient map, and input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the second image encoding result;

[0135] Input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset-dimensional feature information;

[0136] Concatenate the first image encoding result, the second image encoding result, and the preset-dimensional feature information to obtain the image encoding feature in the latent space.

[0137] In an alternative embodiment, the image diffusion generation module includes a conditional control sub-module and a pre-trained U-net sub-module, where the conditional control sub-module has the same structure as the encoder of the U-net sub-module. Correspondingly, the second image enhancement module 530 can also be specifically configured to:

[0138] In each noise reduction time step of the image diffusion generation module, input the noise image encoding feature into the conditional control sub-module in the image diffusion generation module; and,

[0139] Concatenate the noise information in the noise image encoding feature with the preset-dimensional feature information to obtain a concatenation result, and input the concatenation result into the U-net sub-module;

[0140] Input the encoding feature processed by the conditional control sub-module for the noise image encoding feature into the decoder in the U-net sub-module, and perform decoding together with the encoding result of the encoder in the U-net sub-module to obtain the image generation result in the latent space.

[0141] In an alternative embodiment, the second image enhancement module 530 is specifically configured to:

[0142] In each noise reduction time step, sample the generation result of the image diffusion generation module to obtain sampling data;

[0143] Decode the sampling data to obtain the image decoding result in the image space corresponding to the new three-dimensional magnetic resonance image to be processed;

[0144] Calculate the normalized cross-correlation loss between the image decoding result and the new three-dimensional magnetic resonance image to be processed;

[0145] Based on the normalized cross-correlation loss, perform gradient update on the sampling data to obtain the noise reduction result in the corresponding noise reduction time step.

[0146] In an alternative embodiment, the three-dimensional magnetic resonance image to be processed is a three-dimensional magnetic resonance image of the head.

[0147] The image processing device provided by the embodiments of the present invention can execute the image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0148] Figure 7 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Figure 7 It shows a block diagram of an exemplary computer device 12 suitable for implementing the embodiments of the present invention. Figure 7 The displayed computer device 12 is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as intelligent controllers, servers, mobile phones and other terminal devices.

[0149] Such as Figure 7 As shown, the computer device 12 is presented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0150] The bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the multiple bus structures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0151] The computer device 12 typically includes a variety of computer system-readable media. These media can be any available media accessible by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0152] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 can be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, commonly referred to as a "hard disk drive"). Although Figure 7Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) can be provided. In these cases, each drive can be connected to the bus 18 through one or more data medium interfaces. The system memory 28 may include at least one program product having a set (such as at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0153] A program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in the system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 42 generally perform the functions and / or methods in the embodiments described in the present invention.

[0154] The computer device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 22. In addition, the computer device 12 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although Figure 7 not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0155] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28. For example, it implements the image processing method provided by the embodiments of the present invention, and the method includes:

[0156] Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0157] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0158] Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0159] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the image processing method provided in any embodiment of the present invention. The method includes:

[0160] Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0161] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0162] Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0163] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.

[0164] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0165] The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0166] The computer program code for performing the operations of the present invention may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., by connecting through the Internet using an Internet service provider).

[0167] Those of ordinary skill in the art should understand that the various modules or steps of the present invention described above may be implemented using a general-purpose computing device. They may be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Optionally, they may be implemented using program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them may be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0168] The embodiments of the present disclosure also provide a computer program product, including a computer program, which when executed by a processor, implements the image processing method provided in any one of the embodiments of the present disclosure.

[0169] In the process of implementing a computer program product, computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, alternatively, can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0170] Note that the above is only a preferred embodiment of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image processing method, characterized in that, Including: Obtain the three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain the corresponding original magnetic resonance image slices; Input the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain the new magnetic resonance image slices with the image quality initially improved, and recombine the new magnetic resonance image slices to obtain the new three-dimensional magnetic resonance image to be processed; Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on the latent diffusion generative model; Wherein, the step of inputting the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image includes: Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model, and perform feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain the image encoding features in the latent space; Add noise to the image encoding features to obtain the noisy image encoding features, and input the noisy image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result in the latent space; Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image; Wherein, the step of performing feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain the image encoding features in the latent space includes: Input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the first image encoding result; Perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain the corresponding gradient map, input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the second image encoding result; Input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model to extract at least one preset-dimensional feature information; Stitch the first image encoding result, the second image encoding result and the preset-dimensional feature information to obtain the image encoding features in the latent space.

2. The method according to claim 1, wherein The step of inputting the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain the new magnetic resonance image slices with the image quality initially improved includes: Input the original magnetic resonance image slices into the pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder; Input the slice image features into the decoder of the variational autoencoder for feature decoding to obtain the new magnetic resonance image slices with the image quality initially improved.

3. The method according to claim 2, wherein The training process of the two-dimensional image enhancement model includes: Obtain the three-dimensional magnetic resonance sample images that meet the preset image quality standards; The three-dimensional magnetic resonance sample image is degraded by using a preset image degradation processing model to obtain a degraded three-dimensional magnetic resonance sample image, and the degraded three-dimensional magnetic resonance sample image is subjected to image slicing processing to obtain corresponding degraded magnetic resonance sample image slices; The degraded magnetic resonance sample image slices are respectively input into a two-dimensional image enhancement model to be trained to obtain two-dimensional model output reference images, and the two-dimensional model output reference images are combined to form corresponding three-dimensional magnetic resonance reference images; wherein, the two-dimensional image enhancement model is the variational autoencoder The learning loss of the two-dimensional image enhancement model to be trained is calculated according to the three-dimensional magnetic resonance reference image and the three-dimensional magnetic resonance sample image, and the parameters of the two-dimensional image enhancement model to be trained are updated based on the learning loss to realize the training of the two-dimensional image enhancement model.

4. The method according to claim 3, wherein The preset image degradation processing model includes any one or a combination of a noise simulation algorithm model, a motion artifact simulation algorithm model, and an intensity inhomogeneity simulation algorithm model.

5. The method according to claim 1, characterized in that The image diffusion generation module includes a conditional control sub-module and a pre-trained U-net sub-module, wherein the conditional control sub-module has the same structure as the encoder of the U-net sub-module; Correspondingly, the inputting the noise image encoded feature into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result in the latent space includes: In each noise reduction time step of the image diffusion generation module, inputting the noise image encoded feature into the conditional control sub-module in the image diffusion generation module; and, Performing information splicing on the noise information in the noise image encoded feature and the preset dimension feature information to obtain a splicing result, and inputting the splicing result into the U-net sub-module; Inputting the encoded feature processed by the conditional control sub-module for the noise image encoded feature into the decoder in the U-net sub-module, and decoding it together with the encoding result of the encoder in the U-net sub-module to obtain the image generation result in the latent space.

6. The method according to claim 5, wherein The method further includes: In each noise reduction time step, sampling the generation result of the image diffusion generation module to obtain sampling data; Decoding the sampling data to obtain an image decoding result in the image space corresponding to the new three-dimensional magnetic resonance image to be processed; Calculating the normalized cross-correlation loss between the image decoding result and the new three-dimensional magnetic resonance image to be processed; Performing gradient update on the sampling data based on the normalized cross-correlation loss to obtain the noise reduction result in the corresponding noise reduction time step.

7. According to the method described in any one of claims 1-6, characterized in that, The three-dimensional magnetic resonance image to be processed is a head three-dimensional magnetic resonance image.

8. An image processing apparatus, characterized in that, Including: An image acquisition module, configured to acquire a three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; A first image enhancement module, configured to input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively, obtain new magnetic resonance image slices with a preliminary improvement in image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; A second image enhancement module, configured to input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model; Specifically, the second image enhancement module is configured to: Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model, and perform feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain image encoding features in the latent space; Overlay noise on the image encoding features to obtain noise image encoding features, and input the noise image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain an image generation result in the latent space; Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image; The second image enhancement module can also be configured to: Input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result; Perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain a corresponding gradient map, input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a second image encoding result; Input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset-dimensional feature information; Concatenate the first image encoding result, the second image encoding result, and the preset-dimensional feature information to obtain image encoding features in the latent space.

9. A computer device, characterized in that, The computer device includes: One or more processors; A memory, configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image processing method according to any one of claims 1-7.

11. A computer program product comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image processing method according to any one of claims 1-7.

Images