Image processing method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses an image processing method and device, equipment, a medium and a product, and the method comprises the steps: obtaining a to-be-processed three-dimensional magnetic resonance image, carrying out the image fragmentation processing of the to-be-processed three-dimensional magnetic resonance image, and obtaining corresponding original magnetic resonance image slices; respectively inputting the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices of which the image quality is preliminarily improved, and recombining the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; inputting the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; the three-dimensional image enhancement model is obtained based on training of a potential diffusion generation model. According to the technical scheme provided by the embodiment of the invention, the problems of complexity and poor effect of an algorithm for improving the quality of the three-dimensional magnetic resonance image at present are solved, the three-dimensional magnetic resonance image can be directly processed, and the image enhancement effect is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of medical image processing based on artificial intelligence, and in particular, to an image processing method, apparatus, device, medium, and product. Background Art

[0002] Magnetic Resonance Images (MRI) often suffer from various forms of degradation, including degradation caused by noise, artifacts (such as ghosts caused by motion), and intensity inhomogeneity, which can affect the quality of the processed images. Before MRI is used for clinical analysis, image processing is required to improve the image quality.

[0003] However, in the process of implementing the present invention, it is found that there are at least the following technical problems in the prior art:

[0004] Currently, traditional methods for image enhancement processing of MRI are prone to losing image details, have high computational complexity, and limited effects. Image enhancement methods based on deep learning are more targeted at two-dimensional MRI, and for three-dimensional MRI, spatial structure information will be lost, and the image enhancement processing effect is also not good. Summary of the Invention

[0005] Embodiments of the present invention provide an image processing method, apparatus, device, medium, and product, which can directly process three-dimensional magnetic resonance images, enhance image detail processing, and improve the image enhancement effect.

[0006] In a first aspect, embodiments of the present invention provide an image processing method, which includes:

[0007] Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0008] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with preliminarily improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0009] Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0010] In a second aspect, embodiments of the present invention further provide an image processing apparatus, which includes:

[0011] An image acquisition module, configured to acquire a three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0012] A first image enhancement module, configured to input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively, obtain new magnetic resonance image slices with preliminarily improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0013] A second image enhancement module, configured to input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, which includes:

[0015] One or more processors;

[0016] A memory, configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method provided in any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the image processing method provided in any embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the image processing method provided in any embodiment of the present invention.

[0020] The embodiments in the above-mentioned invention have the following advantages or beneficial effects:

[0021] In an embodiment of the present invention, by obtaining a three-dimensional magnetic resonance image to be processed and performing image slicing processing on the three-dimensional magnetic resonance image to be processed, corresponding original magnetic resonance image slices are obtained; the original magnetic resonance image slices are respectively input into a pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices with a preliminary improvement in image quality, and the new magnetic resonance image slices are recombined to obtain a new three-dimensional magnetic resonance image to be processed; the new three-dimensional magnetic resonance image to be processed is input into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement. Description of the Drawings

[0022] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present invention;

[0023] Figure 2 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0024] Figure 3 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0025] Figure 4 is a flowchart of another image processing method provided by an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the image enhancement model framework in an application example of an image processing method provided by an embodiment of the present invention;

[0027] Figure 6 is a schematic structural diagram of an image processing device provided by an embodiment of the present invention;

[0028] Figure 7 is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed Embodiments

[0029] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings, rather than all the structures.

[0030] Figure 1The flowchart of an image processing method provided by an embodiment of the present invention. This embodiment is applicable to the scenario of image enhancement processing using MRI, especially for the case of three-dimensional MRI image enhancement processing. This method can be executed by an image processing device, which can be implemented in the form of software and / or hardware and integrated in a computer device with application development functions.

[0031] As Figure 1 shown, the image processing method of this embodiment includes the following steps:

[0032] S110. Obtain the three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices.

[0033] The three-dimensional magnetic resonance image to be processed can be the magnetic resonance imaging result of any region of interest, and the image whose image quality needs to be improved.

[0034] During the magnetic resonance imaging process, the magnetic resonance image may be degraded due to reasons such as noise, motion artifacts, and / or signal inhomogeneity, and the image quality is not ideal. Therefore, in order to make the magnetic resonance imaging have a clearer visual effect during downstream use and not affect the reading and analysis of the image by the image readers, image enhancement processing needs to be performed to improve the image quality.

[0035] Performing image slicing processing on the three-dimensional magnetic resonance image to be processed can be to perform image slicing in any direction in the three-dimensional image space to obtain corresponding original magnetic resonance image slices. For example, the original magnetic resonance image slices can be coronal plane image slices, sagittal plane image slices, or horizontal plane image slices.

[0036] S120. Input the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with the image quality initially improved, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0037] In this embodiment, the image quality improvement is divided into two stages. In the first stage, a two-dimensional image slice rough enhancement model is used to remove large-scale degradation; in the second stage, a three-dimensional fine control latent diffusion generation model is used to restore the missing details.

[0038] In S120, it corresponds to the first stage of image quality improvement. The pre-trained two-dimensional image enhancement model can be a pre-trained image encoder and decoder. Its training samples can be low-quality magnetic resonance sample images, and the learning label is the corresponding high-quality magnetic resonance sample image of the sample.

[0039] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality. Further, the new magnetic resonance image slices can be recombined according to the image sequence of the original magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0040] S130. Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image.

[0041] In S130, it corresponds to the second stage of image quality improvement.

[0042] Among them, the three-dimensional image enhancement model is a model trained based on the latent diffusion generation model. The new three-dimensional magnetic resonance image to be processed can be used as a control signal for controlling the diffusion process of the latent diffusion generation model, enabling the three-dimensional image enhancement model to generate a very detailed and rich-detail target three-dimensional magnetic resonance image. Thus, the image quality of the three-dimensional magnetic resonance image to be processed is improved.

[0043] The technical solution of this embodiment obtains a three-dimensional magnetic resonance image to be processed, performs image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; inputs the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombines the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; inputs the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image; among them, the three-dimensional image enhancement model is a model trained based on the latent diffusion generation model. The technical solution of the embodiment of the present invention solves the problem that the current algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0044] Figure 2 It is a flowchart of another image processing method provided by an embodiment of the present invention. This embodiment and the image processing method in the above embodiment belong to the same inventive concept, and further describes the process of initially improving the two-dimensional image quality in a single image direction. This method can be executed by an image processing device, and the device can be implemented in a software and / or hardware manner and integrated into a computer device with application development functions.

[0045] As Figure 2 shown, the image processing method of this embodiment includes the following steps:

[0046] S210. Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices.

[0047] S220. Input the original magnetic resonance image slices into the pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder.

[0048] Among them, the variational autoencoder (VAE) is the two-dimensional image enhancement model. The encoder of the variational autoencoder uses multiple residual Swin Transformer blocks (RSTB) to extract the depth features of each original magnetic resonance image slice, and the decoder of the variational autoencoder upsamples these depth features to the original image space to obtain the processed image result with improved quality.

[0049] In this step, input the original magnetic resonance image slices into the pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder. Among them, the slice image features are the depth features of higher dimensions extracted by the encoder of the variational autoencoder.

[0050] S230. Input the slice image features into the decoder of the variational autoencoder for feature decoding to obtain the new magnetic resonance image slices with the initially improved image quality.

[0051] The new magnetic resonance image slices are that the decoder of the variational autoencoder upsamples the slice image features to the original image space to obtain the processed image result with improved quality.

[0052] Furthermore, in an optional implementation manner, the training process of the two-dimensional image enhancement model (i.e., the variational autoencoder) includes the following steps:

[0053] First, obtain the three-dimensional magnetic resonance sample images that meet the preset image quality standards, that is, the high-quality sample images that can be used as sample learning labels.

[0054] Then, a preset image degradation processing model is used to degrade the three-dimensional magnetic resonance sample image to obtain a degraded three-dimensional magnetic resonance sample image, and the degraded three-dimensional magnetic resonance sample image is sliced to obtain corresponding degraded magnetic resonance sample image slices. Among them, the preset image degradation processing model includes any one and a combination of multiple of a noise simulation algorithm model, a motion artifact simulation algorithm model, and an intensity inhomogeneity simulation algorithm model. That is to say, a three-dimensional magnetic resonance sample image can have 7 degradation processing methods, such as noise simulation degradation, motion artifact simulation degradation, intensity inhomogeneity simulation degradation, noise simulation degradation and motion artifact simulation degradation, motion artifact simulation degradation and intensity inhomogeneity simulation degradation, noise simulation degradation and intensity inhomogeneity simulation degradation, and noise simulation degradation, motion artifact simulation degradation, and intensity inhomogeneity simulation degradation. Among them, the motion artifact simulation degradation algorithm model can have different motion artifact simulation strategies. This can enrich the model training samples to the greatest extent.

[0055] Subsequently, the degraded magnetic resonance sample image slices can be respectively input into the two-dimensional image enhancement model to be trained to obtain two-dimensional model output reference images, and the two-dimensional model output reference images are combined to form corresponding three-dimensional magnetic resonance reference images.

[0056] Finally, the learning loss of the two-dimensional image enhancement model to be trained can be calculated based on the three-dimensional magnetic resonance reference image and the three-dimensional magnetic resonance sample image, and the parameters of the two-dimensional image enhancement model to be trained are updated based on the learning loss to achieve the training of the two-dimensional image enhancement model. The learning loss can include the Mean Squared Error (MSE) loss and the Structural Similarity Index (SSIM) loss.

[0057] S240. Recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0058] After the new three-dimensional magnetic resonance image to be processed is processed, the structural information in a single direction is restored, most of the low-quality information is removed, and the image quality is improved but there are fewer details. The details of the new three-dimensional magnetic resonance image to be processed can be further enriched through S250.

[0059] S250. Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image.

[0060] Among them, the three-dimensional image enhancement model is a model trained based on the latent diffusion generative model. It can adopt the U-Net architecture, which has a symmetric encoder-decoder structure. It can effectively capture the context information and details of the new three-dimensional magnetic resonance image to be processed, and use skip connections to retain the feature information at different levels, so as to generate very detailed and rich-output.

[0061] In the technical solution of this embodiment, by obtaining the three-dimensional magnetic resonance image to be processed and performing image slicing on the three-dimensional magnetic resonance image to be processed, the corresponding original magnetic resonance image slices are obtained; the original magnetic resonance image slices are respectively input into the pre-trained variational autoencoder, and the slice image features in the original magnetic resonance image slices are extracted through the encoder of the variational autoencoder; the slice image features are input into the decoder of the variational autoencoder for feature decoding to obtain new magnetic resonance image slices with initially improved image quality; the new magnetic resonance image slices are recombined to obtain a new three-dimensional magnetic resonance image to be processed; the new three-dimensional magnetic resonance image to be processed is input into the pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image. The technical solution of the embodiment of the present invention solves the problems that the current algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0062] Figure 3 It is a flowchart of another image processing method provided by an embodiment of the present invention. This embodiment and the image processing method in the above embodiment belong to the same inventive concept, and further illustrate the process of three-dimensional image enhancement processing. This method can be executed by an image processing device, and the device can be implemented in a software and / or hardware manner and integrated in a computer device with application development functions.

[0063] As Figure 3 shown, the image processing method of this embodiment includes the following steps:

[0064] S310. Obtain the three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain the corresponding original magnetic resonance image slices.

[0065] S320. Respectively input the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0066] S330. Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model, and perform feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain the image encoding features in the latent space.

[0067] Among them, the image encoding module of the three-dimensional image enhancement model can be the encoder of a pre-trained variational autoencoder. By downsampling the new three-dimensional magnetic resonance image to be processed, feature encoding is performed to extract higher-dimensional features, thereby obtaining the image encoding features in the latent space.

[0068] The encoder of the variational autoencoder downsamples the new three-dimensional magnetic resonance image to be processed. For example, it can be an 8-fold downsampling encoding process.

[0069] S340. Add noise to the image encoding features to obtain the noisy image encoding features, and input the noisy image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result in the latent space.

[0070] The image diffusion generation module of the three-dimensional image enhancement model performs the encoding and decoding processes of the image based on the U-Net structure to achieve denoising diffusion and obtain the image generation result in the latent space.

[0071] S350. Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

[0072] The image decoding module can be a decoding module corresponding to the image encoding module, such as the decoder of a variational autoencoder.

[0073] The technical solution of this embodiment obtains the three-dimensional magnetic resonance image to be processed, performs image slicing on the three-dimensional magnetic resonance image to be processed to obtain the corresponding original magnetic resonance image slices; inputs the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain the new magnetic resonance image slices with the image quality initially improved, and recombines the new magnetic resonance image slices to obtain the new three-dimensional magnetic resonance image to be processed; inputs the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model, performs feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain the image encoding features in the latent space; adds noise to the image encoding features to obtain the noisy image encoding features, and inputs the noisy image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result in the latent space; inputs the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0074] Figure 4The flowchart of another image processing method provided by an embodiment of the present invention. This embodiment and the image processing method in the above embodiment belong to the same inventive concept, and further illustrate the processing process of three-dimensional MRI image enhancement. This method can be executed by an image processing device, which can be implemented in software and / or hardware and integrated into a computer device with application development functions.

[0075] As Figure 4 shown, the image processing method of this embodiment includes the following steps:

[0076] S410. Obtain the three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain the corresponding original magnetic resonance image slices.

[0077] S420. Input the original magnetic resonance image slices into the pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with preliminary improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed.

[0078] S430. Input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the first image encoding result.

[0079] S440. Perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain the corresponding gradient map, input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the second image encoding result.

[0080] The gradient map of the new three-dimensional magnetic resonance image to be processed is a derivative image obtained after processing the data of the new three-dimensional magnetic resonance image to be processed. The MRI device uses a gradient magnetic field to encode spatial positions. Hydrogen protons at different positions have different resonance frequencies or phases due to the action of the gradient field, and are thus distinguished for imaging. The gradient map focuses on this change information caused by the gradient field. Through a specific algorithm, it extracts the rate of change of the signal intensity with spatial position within the imaging region, that is, the gradient information, and visualizes it. For example, for the new three-dimensional magnetic resonance image to be processed, through mathematical operations such as Fourier transform, the complex time-domain signal can be transformed into a frequency-domain signal, and further the components reflecting the spatial frequency change can be extracted from it to construct a gradient map, showing the rapid change of the signal at the boundary between tissues. The image characteristics of the gradient map are prominent boundaries, high-resolution details, and also have guiding significance for clinical practice.

[0081] In this embodiment, in order to make the image generated by the image diffusion generation module a more realistic three-dimensional magnetic resonance image, some feature encoding information is added as the diffusion control information of the image diffusion generation module. The gradient map is input into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain the second image encoding result. The second image encoding result will also be input into the image diffusion generation module.

[0082] Among them, the second image encoding module can be a variational autoencoder with the same structure as the first image encoding module but with optimized parameters.

[0083] S450. Input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset dimension feature information.

[0084] The third image encoding module can be a conditional encoder for extracting the feature information of the new three-dimensional magnetic resonance image to be processed. This feature information is information related to the region of interest in the three-dimensional magnetic resonance image to be processed. For example, if the three-dimensional magnetic resonance image to be processed is a three-dimensional magnetic resonance image of the head, the preset dimension feature information can be parameter information related to the head such as the age, gender, ventricle volume, and / or brain volume corresponding to the three-dimensional magnetic resonance image to be processed.

[0085] The encoding module can be trained according to the feature information to be extracted to obtain the target third image encoding module.

[0086] S460. Concatenate the first image encoding result, the second image encoding result, and the preset dimension feature information to obtain the image encoding feature in the latent space.

[0087] S470. In each denoising time step of the image diffusion generation module, input the noise image encoding feature into the conditional control sub-module in the image diffusion generation module; and, concatenate the noise information in the noise image encoding feature with the preset dimension feature information to obtain a concatenation result, and input the concatenation result into the U-net sub-module.

[0088] Among them, the image diffusion generation module includes a conditional control sub-module and a pre-trained U-net sub-module, where the conditional control sub-module has the same structure as the encoder of the U-net sub-module.

[0089] S480. Input the encoding feature processed by the conditional control sub-module for the noise image encoding feature into the decoder in the U-net sub-module, and decode it together with the encoding result of the encoder in the U-net sub-module to obtain the image generation result in the latent space.

[0090] The U-Net sub-module has a symmetric encoder and decoder. The encoder is responsible for downsampling the encoded features of the input noisy image to gradually extract higher-level and more abstract features while reducing the spatial resolution; the decoder then performs upsampling to gradually restore the low-resolution feature map to the size of the original input image, and uses the features extracted by the encoder to reconstruct the image during this process. This encoding and decoding structure enables the model to effectively capture the high-level features and details of the image and handle image information well in both the forward and backward processes of the diffusion model.

[0091] This step implements skip connections, which directly connect the lower layers of the encoder in the conditional control sub-module and the encoder in the U-net sub-module to the higher layers of the decoder, enabling the decoder in the U-net sub-module to directly utilize the features of the corresponding layers in the encoder when reconstructing the image. In the diffusion model, especially during the process of gradually enhancing the image from a low-resolution state to a high-resolution state, skip connections help retain the details of the image, avoid information loss and blurring, and make the generated image clearer and more accurate.

[0092] In a preferred implementation, in each denoising time step, the generation result of the image diffusion generation module can also be sampled and updated to obtain the best sampled data. For example, sample the generation result of the image in the latent space. Then, decode the sampled data to obtain the image decoding result in the image space corresponding to the new three-dimensional magnetic resonance image to be processed. Among them, the decoder for decoding the sampled data can be the decoder corresponding to the first image encoding module. Furthermore, the normalized cross-correlation loss between the image decoding result and the new three-dimensional magnetic resonance image to be processed can be calculated; the sampled data is gradient-updated based on the normalized cross-correlation loss to obtain the best sampling result in the corresponding denoising time step. This sampling result can be used as the input in the next denoising time step for the next denoising process. The process of gradient-updating the sampled data can be implemented by a generation guidance network, which is composed of the decoder corresponding to the first image encoding module and the gradient updater for calculating the normalized cross-correlation loss.

[0093] After completing the denoising process through step-by-step iteration, the generation result of the image in the latent space can be obtained.

[0094] S490. Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

[0095] Among them, the image decoding module can be the decoder corresponding to the first image encoding module.

[0096] The technical solution of this embodiment is to obtain a three-dimensional magnetic resonance image to be processed, perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result; perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain a corresponding gradient map, input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a second image encoding result; input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset-dimensional feature information; splice the first image encoding result, the second image encoding result and the preset-dimensional feature information to obtain an image encoding feature in the latent space; in each noise reduction time step of the image diffusion generation module, input the noise image encoding feature into the conditional control sub-module in the image diffusion generation module; and splice the noise information in the noise image encoding feature with the preset-dimensional feature information to obtain a splicing result, and input the splicing result into the U-net sub-module; input the encoding feature processed by the conditional control sub-module for the noise image encoding feature into the decoder in the U-net sub-module, and perform decoding together with the encoding result of the encoder in the U-net sub-module to obtain an image generation result in the latent space; input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and have poor effects at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0097] In a specific embodiment, the three-dimensional magnetic resonance image to be processed is a three-dimensional magnetic resonance image of the head. To implement the enhancement processing of the three-dimensional magnetic resonance image of the head, first, the data of the publicly available Alzheimer's Disease Neuroimaging Initiative dataset is used to train the two-dimensional image enhancement model and the three-dimensional image enhancement model.

[0098] Figure 5 Shows a two-stage coarse-to-fine 3DMRI enhancement network framework.

[0099] In the Alzheimer's Disease Neuroimaging Initiative dataset, 211 t1-weighted MRI scan datasets scanned at 3T are selected. These data are used for the degradation model and the enhancement model, and they are divided into a training set, a validation set and a test set, which are 146, 31 and 31 respectively.

[0100] First, register our data rigidly using a preset rigid registration tool and map it into a preset standard image space. The preset standard image space can be the space corresponding to the MRI images generated by the application technology or model of the Latent Diffusion Model (LDM) in brain imaging or brain science research, with a size of [160, 224, 160].

[0101] After that, use a degradation model to randomly apply one or more of three degradation methods to the MRI images, including adding artifacts, Gaussian noise, and intensity inhomogeneity. This degradation method makes each high-quality image (Ground Truth) correspond to seven different degraded images, thus forming image pairs for training. Then normalize all the images to the range of [0, 1].

[0102] Among them, the MRI degradation model is used to better simulate the artifact problems in reality, such as noise, motion artifacts, and non-uniform intensity distribution.

[0103] Specifically, in noise simulation, since MRI noise follows a Rician distribution, it means that both the real part and the imaginary part are affected by Gaussian noise with the same variance. The noise image can be simulated by the following formula: . Where I HQ is the original image, I LQ is the image after degradation processing, μ and σ are the mean and standard deviation of Gaussian noise (default values: 0, 0.02), and the signal-to-noise ratio is set to 5 to replicate the real MRI noise level.

[0104] In motion artifact simulation, to simulate the motion artifacts in brain MRI, three different phase-encoding line sampling strategies can be adopted. For example, independent Gaussian sampling is used to simulate a single motion artifact; segmented instantaneous sampling is used to simulate intermittent artifacts; segmented constant sampling, where the standard deviation of the Gaussian distribution is set to 10 voxels, and the number of segments (k) is randomly determined as an integer between 1 and 8. During the construction of the dataset, one of the above artifact types can be randomly applied to 0 - 20% of the phase-encoding lines.

[0105] In intensity inhomogeneity simulation, it can be assumed that the MRI data has a dimension of D × H × w, and the coil center is randomly selected , for each voxel , the Euclidean distance to is normalized to . Then calculate the coil sensitivity mask : , where α represents the intensity controlling the coil effect. Apply this mask to the original image: This simulates the intensity inhomogeneity caused by coil positioning and enhances the robustness of the model to such variations in real MRI images.

[0106] After the samples are prepared, the network can be trained. For the first-stage training, first slice along the transverse plane (Transverse Plane / Axial Plane), and obtain 160 slices of 160*224 for each three-dimensional brain MRI image. Then, splice the results of the first-stage training to obtain a three-dimensional brain MRI image with preliminary quality improvement to continue the second-stage training. Both the trained two-dimensional image enhancement model and the three-dimensional image enhancement model are trained using the Adam optimizer. For the two-dimensional image enhancement network, use a batch size of 96 and a decay rate of 5e-5 for 8000 steps of training. In the three-dimensional image enhancement model, in the image encoding network (Conditional Extraction Network, CENet corresponding to the first image encoding module and the second image encoding module), the variational autoencoder network is used, with a batch size of 3 and a decay rate of 1e-5 for 8000 steps of training. The cond extraction network (Cond encoder corresponding to the third image encoding module) uses the architecture of 3DResnet18, with a batch size of 3 and a decay rate of 1e-5 for 18000 steps of training. For the conditional control network (Conditional Control Network, CCNet, corresponding to the conditional control sub-module in the image diffusion generation module), use BrainLdm as the generation prior, and use a decay rate of 1e-4 for the control network, with a batch size of 3 for 12000 steps of training. For the final generative guidance network (Generative Guidance Network, GGNet, corresponding to the network for optimizing the sampled data in the noise reduction time steps), use 1000 steps of sampling. The first 900 steps are for normal sampling generation, and the last 100 steps use the Normalized Cross-Correlation Loss (NCCLoss) to constrain each step of generation.

[0107] Furthermore, in the first stage, a designed 2D network is used to recover the structural information of the three-dimensional brain MRI image in a single direction, removing most of the low-quality information and generating a high-quality but less detailed image. The 2D network adopts the structure of a Variational Autoencoder (VAE). Its encoder uses multiple Residual Swin Transformer Blocks (RSTB) to extract deep features, and the decoder upsamples these features to the original image space.

[0108] The network parameters are trained by optimizing the following joint loss function, including Mean Squared Error (MSE) loss and Structural Similarity Index (SSIM) loss: LSSIM = 1−SSIM(I HQ , I ref ). Among them, I HQ is the high-quality image, I LQ is the low-quality image, and I ref is the reference image generated by the 2D network. This output serves as the preliminary image optimization result for the subsequent diffusion model.

[0109] In the second stage, the prior knowledge of the pre-trained diffusion model and the output I ref from the first stage are utilized to further improve the accuracy and realism of MRI image reconstruction. The network consists of three main components: a Conditional Extraction Network (CENet), a Conditional Control Network (CCNet), and a Generative Guidance Network (GGNet).

[0110] The pre-trained BrainLDM model has the ability to generate realistic T1-weighted brain MRI images. It includes an Autoencoder and a diffusion model containing multiple UNet denoisers, which can effectively recover image details.

[0111] The Conditional Extraction Network (CENet) uses the following three key conditions to guide the generation of the diffusion process: the output I ref of the first stage, the gradient map of I ref , and four brain-related parameters (age, gender, ventricle volume, brain volume). Specifically, through the VAE encoder E pre-trained by BrainLDM, I refMap it to the latent space. Then, I ref 's gradient map is processed by an encoder with the same structure as the VAE encoder but optimized for I ref to preserve edge information. Finally, a conditional encoder is designed to encode I ref into the conditional variable C (four brain-related parameters) to assist the main network in generating more realistic images. After training, all parameters of the conditional encoder remain frozen.

[0112] Conditional Control Network (CCNet) To ensure the correct direction of image generation, a conditional control network based on ControlNet (CCNet, Figure 5 the red part in ) is designed. A trainable copy of the pre-trained UNet encoder is constructed, and I ref , I ref 's gradient map, the four conditional variables C, and noise are concatenated into z' t and then input into the network. The formula is as follows: .

[0113] The output of CCNet is added to the result of the pre-trained UNet decoder. During training, only the parameters of CCNet are updated.

[0114] Generation Guidance Network (GGNet) To further improve the fidelity of the generated images and make the denoising process more consistent with 's structure, the Generation Guidance Network (GGNet) is designed. At each sampling step, UNet first predicts the noise, then removes the predicted noise from z' t to obtain a clean latent variable, which decodes z 0 into , and calculates the difference between and using the Normalized Cross-Correlation Loss (NCCLoss). The latent variable z 0 is updated by gradient descent, and finally a reconstructed image guided by prior knowledge and is generated . The formula for NCCLoss is as follows: . Where and are the means of and respectively.

[0115] Figure 6A structural schematic diagram of an image processing device provided by an embodiment of the present invention. This embodiment is applicable to the scenario of image enhancement processing using MRI, especially for the case of three-dimensional MRI image enhancement processing. The image processing device can be implemented in software and / or hardware and integrated into a computer terminal device with application development functions.

[0116] As Figure 6 shown, the image processing device includes: an image acquisition module 510, a first image enhancement module 520, and a second image enhancement module 530.

[0117] Among them, the image acquisition module 510 is used to acquire the three-dimensional magnetic resonance image to be processed, perform image slicing processing on the three-dimensional magnetic resonance image to be processed, and obtain the corresponding original magnetic resonance image slices; the first image enhancement module 520 is used to input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively, obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; the second image enhancement module 530 is used to input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image; among them, the three-dimensional image enhancement model is a model trained based on the latent diffusion generative model.

[0118] The technical solution of this embodiment is to acquire the three-dimensional magnetic resonance image to be processed, perform image slicing processing on the three-dimensional magnetic resonance image to be processed, and obtain the corresponding original magnetic resonance image slices; input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively, obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain the target three-dimensional magnetic resonance image; among them, the three-dimensional image enhancement model is a model trained based on the latent diffusion generative model. The technical solution of the embodiment of the present invention solves the problems that the algorithms for improving the quality of three-dimensional magnetic resonance images are complex and the effects are not good at present, and can directly process three-dimensional magnetic resonance images to improve the effect of image enhancement.

[0119] In an optional implementation manner, the first image enhancement module 520 is specifically used for:

[0120] Input the original magnetic resonance image slices into a pre-trained variational autoencoder respectively, and extract the slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder;

[0121] Input the slice image features into the decoder of the variational autoencoder for feature decoding to obtain new magnetic resonance image slices with initially improved image quality.

[0122] In an alternative embodiment, the image processing apparatus further includes a model training module for training a model to obtain a two-dimensional image enhancement model. The training process of the two-dimensional image enhancement model includes:

[0123] Obtain three-dimensional magnetic resonance sample images that meet the preset image quality standards;

[0124] Use a preset image degradation processing model to degrade the three-dimensional magnetic resonance sample images to obtain degraded three-dimensional magnetic resonance sample images, and perform image slicing on the degraded three-dimensional magnetic resonance sample images to obtain corresponding degraded magnetic resonance sample image slices;

[0125] Input the degraded magnetic resonance sample image slices into the two-dimensional image enhancement model to be trained respectively to obtain two-dimensional model output reference images, and form corresponding three-dimensional magnetic resonance reference images from the two-dimensional model output reference images; wherein, the two-dimensional image enhancement model is a variational autoencoder

[0126] Calculate the learning loss of the two-dimensional image enhancement model to be trained based on the three-dimensional magnetic resonance reference images and the three-dimensional magnetic resonance sample images, and update the parameters of the two-dimensional image enhancement model to be trained based on the learning loss to realize the training of the two-dimensional image enhancement model.

[0127] In an alternative embodiment, the preset image degradation processing model includes any one and a combination of multiple of a noise simulation algorithm model, a motion artifact simulation algorithm model, and an intensity non-uniformity simulation algorithm model.

[0128] In an alternative embodiment, the second image enhancement module 530 is specifically configured to:

[0129] Input the new three-dimensional magnetic resonance image to be processed into the pre-trained three-dimensional image enhancement model, and perform feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain image encoding features in the latent space;

[0130] Overlay noise on the image encoding features to obtain noise image encoding features, and input the noise image encoding features into the image diffusion generation module of the three-dimensional image enhancement model to obtain an image generation result in the latent space;

[0131] Input the image generation result into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

[0132] In an alternative embodiment, the second image enhancement module 530 is specifically configured to:

[0133] Input the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result;

[0134] Perform image processing on the new three-dimensional magnetic resonance image to be processed to obtain a corresponding gradient map, and input the gradient map into the second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a second image encoding result;

[0135] Input the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model for feature information extraction to obtain at least one preset-dimensional feature information;

[0136] Concatenate the first image encoding result, the second image encoding result, and the preset-dimensional feature information to obtain an image encoding feature in the latent space.

[0137] In an alternative embodiment, the image diffusion generation module includes a conditional control sub-module and a pre-trained U-net sub-module, where the conditional control sub-module has the same structure as the encoder of the U-net sub-module. Correspondingly, the second image enhancement module 530 can also be specifically configured to:

[0138] In each noise reduction time step of the image diffusion generation module, input the noise image encoding feature into the conditional control sub-module in the image diffusion generation module; and,

[0139] Concatenate the noise information in the noise image encoding feature with the preset-dimensional feature information to obtain a concatenation result, and input the concatenation result into the U-net sub-module;

[0140] Input the encoding feature processed by the conditional control sub-module for the noise image encoding feature into the decoder in the U-net sub-module, and perform decoding together with the encoding result of the encoder in the U-net sub-module to obtain an image generation result in the latent space.

[0141] In an alternative embodiment, the second image enhancement module 530 is specifically configured to:

[0142] In each noise reduction time step, sample the generation result of the image diffusion generation module to obtain sampling data;

[0143] Decode the sampling data to obtain an image decoding result in the image space corresponding to the new three-dimensional magnetic resonance image to be processed;

[0144] Calculate the normalized cross-correlation loss between the image decoding result and the new three-dimensional magnetic resonance image to be processed;

[0145] Based on the normalized cross-correlation loss, perform gradient update on the sampling data to obtain a noise reduction result in the corresponding noise reduction time step.

[0146] In an alternative embodiment, the three-dimensional magnetic resonance image to be processed is a three-dimensional magnetic resonance image of the head.

[0147] The image processing apparatus provided by the embodiments of the present invention can execute the image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0148] Figure 7 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Figure 7 It shows a block diagram of an exemplary computer device 12 suitable for implementing the embodiments of the present invention. Figure 7 The displayed computer device 12 is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as intelligent controllers, servers, mobile phones and other terminal devices.

[0149] Such as Figure 7 As shown, the computer device 12 is presented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0150] The bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus structures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0151] The computer device 12 typically includes a variety of computer system-readable media. These media can be any available media accessible by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0152] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 can be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, commonly referred to as a "hard disk drive"). Although Figure 7Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., CD-ROM, DVD-ROM or other optical media) can be provided. In these cases, each drive can be connected to the bus 18 through one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the various embodiments of the present invention.

[0153] A program / utility 40 having a set (at least one) of program modules 42 can be stored, for example, in the system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment. The program modules 42 generally perform the functions and / or methods in the embodiments described in the present invention.

[0154] The computer device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 22. Moreover, the computer device 12 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although Figure 7 not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0155] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28, such as implementing the image processing method provided by the embodiments of the present invention. The method includes:

[0156] Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0157] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0158] Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0159] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the image processing method provided in any embodiment of the present invention. The method includes:

[0160] Obtain a three-dimensional magnetic resonance image to be processed, and perform image slicing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices;

[0161] Input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with initially improved image quality, and recombine the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed;

[0162] Input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein, the three-dimensional image enhancement model is a model trained based on a latent diffusion generative model.

[0163] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.

[0164] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0165] The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0166] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0167] Those of ordinary skill in the art should understand that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Optionally, they can be implemented using program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0168] The embodiments of the present disclosure also provide a computer program product, including a computer program, which when executed by a processor, implements the image processing method provided in any one of the embodiments of the present disclosure.

[0169] In the process of implementing the computer program product, computer program code for performing the operations of the present disclosure can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, alternatively, can be connected to an external computer (e.g., by using an Internet service provider to connect through the Internet).

[0170] Note that the above is only a preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. An image processing method, characterized in that: include: Acquiring a three-dimensional magnetic resonance image to be processed, and performing image slicing processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; Inputting the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model respectively to obtain new magnetic resonance image slices with preliminary image quality improvement, and recombining the new magnetic resonance image slices to obtain new three-dimensional magnetic resonance images to be processed; The new three-dimensional magnetic resonance image to be processed is input into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein the three-dimensional image enhancement model is a model trained based on a potential diffusion generation model.

2. The method according to claim 1, characterized in that The step of inputting the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices with preliminarily improved image quality comprises: Inputting the original magnetic resonance image slices into a pre-trained variational autoencoder respectively, and extracting slice image features in the original magnetic resonance image slices through the encoder of the variational autoencoder; The slice image features are input into the decoder of the variational autoencoder for feature decoding to obtain a new magnetic resonance image slice with preliminary image quality improvement.

3. The method according to claim 2, characterized in that The training process of the two-dimensional image enhancement model includes: Acquire a three-dimensional magnetic resonance sample image that meets a preset image quality standard; Using a preset image degradation processing model to perform degradation processing on the three-dimensional magnetic resonance sample image to obtain a degraded three-dimensional magnetic resonance sample image, and performing image slicing processing on the degraded three-dimensional magnetic resonance sample image to obtain corresponding degraded magnetic resonance sample image slices; The degraded magnetic resonance sample image slices are respectively input into the two-dimensional image enhancement model to be trained to obtain a two-dimensional model output reference image, and the two-dimensional model output reference image is composed of a corresponding three-dimensional magnetic resonance reference image; wherein the two-dimensional image enhancement model is the variational autoencoder The learning loss of the two-dimensional image enhancement model to be trained is calculated according to the three-dimensional magnetic resonance reference image and the three-dimensional magnetic resonance sample image, and the parameters of the two-dimensional image enhancement model to be trained are updated based on the learning loss to realize the training of the two-dimensional image enhancement model.

4. The method according to claim 3, characterized in that The preset image degradation processing model includes any one of a noise simulation algorithm model, a motion artifact simulation algorithm model and an intensity non-uniformity simulation algorithm model, and a combination of multiple ones.

5. The method according to claim 1, characterized in that The step of inputting the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image comprises: Inputting the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model, performing feature encoding through an image encoding module of the three-dimensional image enhancement model, and obtaining image encoding features in a latent space; superimposing noise on the image coding feature to obtain a noise image coding feature, and inputting the noise image coding feature into an image diffusion generation module of the three-dimensional image enhancement model to obtain an image generation result of the latent space; The image generation result is input into the image decoding module of the three-dimensional image enhancement model for image decoding to obtain the target three-dimensional magnetic resonance image.

6. The method according to claim 5, characterized in that The step of performing feature encoding through the image encoding module of the three-dimensional image enhancement model to obtain image encoding features of the latent space includes: Inputting the new three-dimensional magnetic resonance image to be processed into the first image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a first image encoding result; Performing image processing on the new three-dimensional magnetic resonance image to be processed to obtain a corresponding gradient map, and inputting the gradient map into a second image encoding module of the three-dimensional image enhancement model for feature encoding to obtain a second image encoding result; Inputting the new three-dimensional magnetic resonance image to be processed into the third image encoding module of the three-dimensional image enhancement model to extract feature information, so as to obtain at least one preset dimension feature information; The first image encoding result, the second image encoding result and the preset dimensional feature information are concatenated to obtain image encoding features of a latent space.

7. The method according to claim 6, characterized in that The image diffusion generation module includes a condition control submodule and a pre-trained U-net submodule, wherein the structure of the encoder of the condition control submodule is the same as that of the U-net submodule; Correspondingly, the step of inputting the noise image coding feature into the image diffusion generation module of the three-dimensional image enhancement model to obtain the image generation result of the latent space includes: In each denoising time step of the image diffusion generation module, the noise image encoding feature is input into the condition control submodule in the image diffusion generation module; and, The noise information in the noise image coding feature is concatenated with the preset dimension feature information to obtain a concatenated result, and the concatenated result is input into the U-net submodule; The coding features processed by the conditional control submodule for the coding features of the noise image are input into the decoder in the U-net submodule, and decoded together with the coding results of the encoder in the U-net submodule to obtain the image generation result of the latent space.

8. The method according to claim 7, characterized in that The method further comprises: In each denoising time step, sampling the generated result of the image diffusion generating module to obtain sampling data; Decoding the sampled data to obtain an image decoding result in an image space corresponding to the new three-dimensional magnetic resonance image to be processed; Calculating a normalized cross-correlation loss between the image decoding result and the new three-dimensional magnetic resonance image to be processed; The sampled data is gradient updated based on the normalized cross-correlation loss to obtain a denoising result in the corresponding denoising time step.

9. The method according to any one of claims 1 to 8, characterized in that: The three-dimensional magnetic resonance image to be processed is a three-dimensional magnetic resonance image of the head.

10. An image processing device, characterized in that: include: An image acquisition module is used to acquire a three-dimensional magnetic resonance image to be processed, and perform image slice processing on the three-dimensional magnetic resonance image to be processed to obtain corresponding original magnetic resonance image slices; A first image enhancement module is used to input the original magnetic resonance image slices into a pre-trained two-dimensional image enhancement model to obtain new magnetic resonance image slices with preliminary image quality improvement, and reorganize the new magnetic resonance image slices to obtain a new three-dimensional magnetic resonance image to be processed; The second image enhancement module is used to input the new three-dimensional magnetic resonance image to be processed into a pre-trained three-dimensional image enhancement model to obtain a target three-dimensional magnetic resonance image; wherein the three-dimensional image enhancement model is a model trained based on a potential diffusion generation model.

11. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the image processing method as described in any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the image processing method according to any one of claims 1 to 9.

Images