An image super-resolution reconstruction method and system

CN117522690BActive Publication Date: 2026-09-15SHANDONG NORMAL UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311554686.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-20
Publication Date
2026-09-15
Estimated Expiration
2043-11-20

AI Technical Summary

Technical Problem

尽管如此,基于生成对抗网络的医学超分辨率方法仍然会出现伪影并且容易发生模式崩溃,这对于优化和捕获整个数据分布具有挑战性

Benefits of technology

[0023](1) This invention discloses an image super-resolution reconstruction method and system. The scheme utilizes the advantages of the diffusion model to solve the mode collapse problem in medical super-resolution models based on generative adversarial networks. Furthermore, to recover more high-frequency details, the scheme first synthesizes residual images instead of directly synthesizing high-resolution MRI images. Before introducing the residual images into the diffusion model, a vector quantization variational autoencoder is used to convert the residual images into discrete image labels with shorter sequence lengths, effectively reducing computational complexity and accelerating model convergence. Finally, a Transformer architecture is introduced to model the relationship between low-resolution MRI images and residual images, effectively improving the fidelity of the reconstructed images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117522690B_ABST
    Figure CN117522690B_ABST
Patent Text Reader

Abstract

The present disclosure provides an image super-resolution reconstruction method and system, comprising: acquiring a prostate MRI image to be reconstructed, and obtaining a first resolution MRI image and a second resolution MRI image through corresponding preprocessing; obtaining a residual image based on the difference between the first resolution MRI image and the second resolution MRI image; wherein the second resolution is lower than the first resolution; taking the residual image as the input of a pre-trained vector quantization variational autoencoder to obtain a discrete image label; and based on the obtained discrete image label, using a pre-trained diffusion model to perform discrete diffusion to obtain a reconstructed image label; and based on the reconstructed image label and the second resolution MRI image, using a pre-trained image decoding model to obtain a super-resolution reconstruction result of the MRI image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This disclosure belongs to the field of super-resolution technology in image processing, and specifically relates to an image super-resolution reconstruction method and system. Background Technology

[0002] High-resolution magnetic resonance imaging (MRI) is a leading medical imaging technique for accurately depicting the anatomy and physiological processes of the prostate. Compared to other imaging modalities such as computed tomography (CT), MRI does not use ionizing radiation, a significant advantage that allows it to provide detailed images of tissue structures. Utilizing pulse sequences, MRI systems can be configured to deliver multi-contrast images, including T1, T2, and proton density (PD) weighted images, which can reveal important physiological and pathological features. However, obtaining high-resolution prostate MRI is often challenging in clinical applications due to factors such as scanning equipment cost, limited scan time, and patient non-cooperation. Furthermore, noise interference and undersampling can degrade image quality during medical image transmission and storage. Therefore, improving the quality and resolution of post-acquisition images has attracted considerable research attention. Super-resolution is a promising technique for improving MRI quality without requiring hardware upgrades to synthesize high-resolution MRI from low-resolution MRI.

[0003] Recent advances in deep learning have significantly improved the performance of medical super-resolution models. These deep learning methods have gained popularity in super-resolution tasks due to their powerful ability to produce natural-looking outputs. Some studies have shown that Generative Adversarial Networks (GANs) outperform other machine learning algorithms in medical super-resolution. GANs use perceptual loss functions, including adversarial and content losses, to train the target for the GAN task. GAN-based medical super-resolution methods are a pioneering technique for overcoming the oversmoothing problem. They combine content loss (such as L1 and L2) and adversarial loss to generate HR images with improved visual quality. Nevertheless, GAN-based medical super-resolution methods still suffer from artifacts and are prone to mode collapse, which poses challenges for optimization and capturing the entire data distribution. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this disclosure provides an image super-resolution reconstruction method and system. The proposed scheme leverages the advantages of diffusion models to solve the mode collapse problem in medical super-resolution models based on generative adversarial networks. Furthermore, to recover more high-frequency details, the scheme first synthesizes residual images instead of directly synthesizing high-resolution MRI images. Before introducing the residual images into the diffusion model, a vector quantization variational autoencoder is used to convert the residual images into short-sequence discrete image labels, effectively reducing computational complexity and accelerating model convergence. Finally, a Transformer architecture is introduced to model the relationship between low-resolution MRI images and residual images, effectively improving the fidelity of the reconstructed images.

[0005] According to a first aspect of the present disclosure, an image super-resolution reconstruction method is provided, comprising:

[0006] Acquire MRI images of the prostate to be reconstructed, and obtain first-resolution MRI images and second-resolution MRI images through appropriate preprocessing;

[0007] A residual image is obtained based on the difference between the first-resolution MRI image and the second-resolution MRI image; wherein the second resolution is lower than the first resolution.

[0008] The residual image is used as the input to a pre-trained vector quantization variational autoencoder to obtain discrete image labels; and based on the obtained discrete image labels, a pre-trained diffusion model is used to perform discrete diffusion to obtain reconstructed image labels.

[0009] Based on the reconstructed image labels and second-resolution MRI images, a pre-trained image decoding model is used to obtain super-resolution reconstruction results of MRI images.

[0010] Furthermore, the image decoding model includes several sequentially connected Transformer blocks and a Softmax layer, wherein each Transformer block is equipped with a cross-attention mechanism that includes second-resolution MRI image information.

[0011] Furthermore, each Transformer block is equipped with an adaptive normalization layer to inject the current time step into the image decoding model.

[0012] Furthermore, the reconstructed image labels and the second-resolution MRI image are used in a pre-trained image decoding model, specifically: the reconstructed image labels are input into the first Transformer block via an adaptive normalization layer; and the second-resolution MRI image features extracted by the encoder are directly input into each Transformer block.

[0013] Furthermore, the corresponding preprocessing specifically involves: obtaining a first-resolution MRI image through image enhancement based on the prostate MRI image to be reconstructed; and performing degradation processing on the first-resolution MRI image to obtain a second-resolution MRI image of the same size as the first-resolution MRI image.

[0014] Furthermore, the method obtains the long-range dependency between the second-resolution MRI image and the synthetic imaging by modeling the relationship between the second-resolution MRI image and the residual image, thereby realizing the construction of super-resolution images.

[0015] According to a second aspect of the present invention, an image super-resolution reconstruction system is provided, comprising:

[0016] The data acquisition unit is used to acquire the MRI image of the prostate to be reconstructed, and to obtain the first resolution MRI image and the second resolution MRI image through corresponding preprocessing.

[0017] A residual image acquisition unit is used to obtain a residual image based on the difference between a first-resolution MRI image and a second-resolution MRI image; wherein the second resolution is lower than the first resolution.

[0018] A discrete processing unit is used to obtain discrete image labels by using the residual image as input to a pre-trained vector quantization variational autoencoder; and to perform discrete diffusion using a pre-trained diffusion model based on the obtained discrete image labels to obtain reconstructed image labels.

[0019] The super-resolution reconstruction unit is used to obtain super-resolution reconstruction results of MRI images based on reconstructed image labels and second-resolution MRI images, using a pre-trained image decoding model.

[0020] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the memory, wherein the processor executes the program to implement the image super-resolution reconstruction method described above.

[0021] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the aforementioned image super-resolution reconstruction method.

[0022] The above one or more technical solutions have the following beneficial effects:

[0023] (1) This invention discloses an image super-resolution reconstruction method and system. The scheme utilizes the advantages of the diffusion model to solve the mode collapse problem in medical super-resolution models based on generative adversarial networks. Furthermore, to recover more high-frequency details, the scheme first synthesizes residual images instead of directly synthesizing high-resolution MRI images. Before introducing the residual images into the diffusion model, a vector quantization variational autoencoder is used to convert the residual images into discrete image labels with shorter sequence lengths, effectively reducing computational complexity and accelerating model convergence. Finally, a Transformer architecture is introduced to model the relationship between low-resolution MRI images and residual images, effectively improving the fidelity of the reconstructed images.

[0024] (2) The proposed solution does not generate a high-resolution image at once, but synthesizes an image by reverse iterative denoising, thereby preserving more texture and detail information of the high-resolution image; therefore, the proposed solution can produce higher resolution and quality output, thereby helping to improve the accuracy of doctors' diagnosis.

[0025] (3) The proposed scheme improves the stability and convergence speed of network training by synthesizing residual images instead of directly generating high-resolution MRI images, enabling the model to focus more on restoring high-frequency details of high-resolution prostate MRI.

[0026] (4) The proposed scheme integrates a transformer architecture to model the relationship between low-resolution MRI images and residual images, which can capture the long-range dependency between low-resolution MRI images and synthetic imaging and restore the high-frequency details of high-resolution MRI.

[0027] (5) The masking and replacement guidance strategy provided by the scheme makes the network easy to identify and simplifies the backpropagation process. Furthermore, random token replacement forces the network to understand the global context, rather than just focusing on the [M] token. Attached Figure Description

[0028] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0029] Figure 1 This is a schematic diagram of the overall process of an image super-resolution reconstruction method described in the embodiments of this disclosure;

[0030] Figure 2 This is a detailed structural diagram of the basic block of the low-resolution encoder described in the embodiments of this disclosure;

[0031] Figure 3This is a detailed structural diagram of the dense block of the low-resolution encoder described in the embodiments of this disclosure;

[0032] Figure 4 These are visual restoration effect images of different scanning perspectives in the PROSTATEx dataset described in this embodiment of the disclosure;

[0033] Figure 5 This is a qualitative comparison image of the different methods described in the embodiments of this disclosure on a prostate diagnostic dataset, magnified 4 times.

[0034] Figure 6 This is a qualitative comparison chart of the different methods described in the embodiments of this disclosure on the PROSTATEx dataset, magnified 4 times.

[0035] Figure 7 This is a schematic diagram showing the PSNR values ​​of the method described in this embodiment at different mask rates. Detailed Implementation

[0036] Those skilled in the art will understand that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0037] It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as here.

[0038] To facilitate understanding of this disclosure, the following detailed explanation and description are provided with reference to the accompanying drawings and specific embodiments. These specific embodiments do not constitute a limitation on the embodiments of this disclosure.

[0039] Those skilled in the art should understand that the accompanying drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily essential for implementing this disclosure.

[0040] Terminology Explanation:

[0041] PSNR: Peak Signal-to-Noise Ratio

[0042] SSIM: Structural Similarity

[0043] MSE: Mean Squared Error

[0044] LPIPS: Learning to perceive image patch similarity

[0045] Example 1

[0046] The purpose of this embodiment is to provide an image super-resolution reconstruction method.

[0047] An image super-resolution reconstruction method includes:

[0048] Acquire MRI images of the prostate to be reconstructed, and obtain first-resolution MRI images and second-resolution MRI images through appropriate preprocessing;

[0049] A residual image is obtained based on the difference between the first-resolution MRI image and the second-resolution MRI image; wherein the second resolution is lower than the first resolution.

[0050] The residual image is used as the input to a pre-trained vector quantization variational autoencoder to obtain discrete image labels; and based on the obtained discrete image labels, a pre-trained diffusion model is used to perform discrete diffusion to obtain reconstructed image labels.

[0051] Based on the reconstructed image labels and second-resolution MRI images, a pre-trained image decoding model is used to obtain super-resolution reconstruction results of MRI images.

[0052] In a specific implementation, the image decoding model includes several sequentially connected Transformer blocks and a Softmax layer, wherein each Transformer block is equipped with a cross-attention mechanism that includes second-resolution MRI image information.

[0053] In practice, each Transformer block is equipped with an adaptive normalization layer, which is used to inject the current time step into the image decoding model.

[0054] In a specific implementation, the reconstructed image labels and the second-resolution MRI image are used with a pre-trained image decoding model. Specifically, the reconstructed image labels are input into the first Transformer block via an adaptive normalization layer; and the second-resolution MRI image features extracted by the encoder are directly input into each Transformer block.

[0055] In a specific implementation, the corresponding preprocessing is as follows: based on the prostate MRI image to be reconstructed, a first resolution MRI image is obtained through image enhancement; the first resolution MRI image is degraded to obtain a second resolution MRI image of the same size as the first resolution MRI image.

[0056] In specific implementation, the method obtains the long-range dependency between the second-resolution MRI image and the synthetic imaging by modeling the relationship between the second-resolution MRI image and the residual image, thereby realizing the construction of super-resolution images.

[0057] In this embodiment, the prostate MRI super-resolution reconstruction method and system based on the discrete residual diffusion model are used to generate high-fidelity, high-resolution MRI images to assist in the detection of prostate diseases. Specifically, the method includes the following steps:

[0058] like Figure 1 As shown, firstly, low-resolution MRI images and their corresponding high-resolution MRI image pairs are obtained through preprocessing. In this embodiment, the residual image is quantized using an image discretization module. Specifically, given the residual image, we use a pre-trained vector quantization variational autoencoder to obtain the discrete image token z∈U. M Where U is the codebook storing image tokens, M = h·w represents the token sequence length, and S is the codebook size of the vector quantization variational autoencoder. The entire framework treats it as the maximization of the conditional transition distribution q(z|c), where c refers to the image token of the low-resolution MRI.

[0059] The aforementioned method maximizes the probability q(z|c) of the diffusion model, a novel approach that yields compelling quality in image synthesis. In this work, we propose integrating a conditionally variable discrete diffusion process into high-resolution prostate MRI synthesis. Instead of directly synthesizing high-resolution MRI, our method utilizes residual prediction to estimate the difference between the high-resolution MRI and the upsampled low-resolution MRI. This difference is referred to as the input residual image. The generated residual image is added to the upsampled low-resolution MRI to reconstruct the synthesized image. Therefore, the goal of the scheme described in this embodiment is to estimate the distribution of conditional transitions at each diffusion time step during the diffusion process.

[0060] The diffusion model described is an image generation model consisting of a separate low-resolution encoder and a denoising image decoder. The latter is responsible for performing denoising diffusion operations on discrete image tags. In the initial stage of the inference phase, all image tags are either masked or randomly processed. The masked tags here function similarly to tags in a masked generative model. The denoising diffusion process progressively estimates the probability density of image tags based on the input text. In each step, the diffusion image decoder utilizes the contextual information of all tags predicted in the previous step to estimate a new probability density distribution, and uses this distribution to predict the tags in the current step.

[0061] The following details the discrete diffusion process inspired by masked language modeling and how to train a denoised image decoder to reverse the forward diffusion process:

[0062] Essentially, the forward diffusion process gradually degrades the residual image data z0 through a Markov chain, for example, by randomly replacing some labels. The reverse process extracts noise from the latent variable z0. TInitially, noise is gradually removed to recover the initial latent variable z0. This is done by extracting the latent variable z0 from the distribution q(z0). t |z t-1 This is achieved by sampling to denoise the latent variables. During the inference phase, the initial latent variable z0 is unknown. We use a denoising network to estimate the conditional transfer distribution p. θ (z t-1 |z t c), which depends on the global context.

[0063] More specifically, weighing a single image tag, using This indicates that z0 is located at position i. This marker corresponds to an entry in the specified codebook and is represented by an index value ranging from 1 to S, i.e. To avoid confusion, we will simplify the notation and remove the superscript 'i' in the following discussion. From z t-1 To z t The transition probabilities are expressed using the matrix [W]. ml =q(z) t =m|z t-1 =l)∈R S*S Defined by [the specific definition / method]. The entire diffusion process of the marker sequence can be represented as:

[0064]

[0065] Here, h(z) is a column vector of uniform length. This vector has a value of 1 only in the entry corresponding to z. State z t The probability distribution is given by vector W t v(z t-1 )Decide.

[0066] Furthermore, an important feature is that the posterior distribution of this diffusion process can be easily calculated, i.e.,

[0067]

[0068] Design the transition matrix W t This is a key aspect of the discrete diffusion model, so care must be taken to ensure that the reverse network can effectively recover the signal from the noise.

[0069] The guiding strategy for masking (i.e., masking) and replacement. To address the problem of uniform diffusion, we draw inspiration from MLM and propose destroying tokens by randomly masking certain markers. This allows the reverse network to accurately identify damaged regions. More precisely, we add a special token called [M], which gives each token a discrete state (S+1). The mask diffusion process can be described as follows: each regular token has a chance φT to be replaced by the [M] token, and a probability of uniform diffusion occurring.

[0070] Therefore, we can convert the transfer matrix into,

[0071]

[0072] The advantages of this masking and replacement transformation are:

[0073] 1) Makes the network easier to identify and simplifies the back diffusion process.

[0074] 2) Compared to previous mask-only methods, we theoretically prove that in addition to the labeled mask, it also needs to include a small amount of uniform noise; otherwise, when z... t When z = 0, we obtain a normal posterior.

[0075] 3) Random token substitution forces the network to understand the global context, rather than just focusing on the [M] token.

[0076] We train a denoising network p θ (z t-1 |z t c) to estimate the posterior distribution q(z) t-1 |z t The problem of reversing the forward diffusion process is solved by using z0. The denoising network is optimized by minimizing the variational lower bound (VLB):

[0077] L vlb =L0+L1+…+L T-1 +L T ,

[0078] L0 = -log p θ (z0|z1,z),

[0079] L t-1 =D KL (q(z t-1 |z t ,z0)||p θ (z t-1 ||z t ,c))

[0080] L T =D KL (q(z T |z0)||p(z T )),

[0081] In this embodiment, all experiments were conducted on two datasets: the prostate diagnostics dataset and the PROSTATEx dataset. The prostate diagnostics dataset included data from 92 participants, while the PROSTATEx dataset included data from 346 participants. These datasets contained multi-slice MRI images of all patients diagnosed with prostate cancer, with each participant undergoing multiple scans, each scan using multiple slice images. For the SR task, we used data from 84 participants from the prostate diagnostics dataset and 300 participants from the PROSTATEx dataset for training, with the remaining patient samples used for model performance evaluation. All samples in the datasets were in DICOM format. First, the DICOM images from these datasets were converted to PNG format, and the corresponding HR images were downsampled to generate LR images. Then, the input images were cropped to 224×224 and further downsampled to 28×28 or 56×56 (height×width). Finally, the detection results and corresponding data of the system described in this embodiment are as follows: Figure 5 and Figure 6 And as shown in Table 1 below:

[0082] Table 1. Comparison of PSNR, SSIM, MSE, and LPIPS on prostate and prostate diagnostic datasets.

[0083]

[0084] Table 1 shows the scores for various metrics, including PSNR, SSIM, MSE, and LPIPS, on different datasets at 4x magnification. It can be seen that our method achieves state-of-the-art results on different datasets. For the PROSTATEx dataset, our model significantly improves PSNR and SSIM at a 4x scale factor, with PSNR increasing from 21.68 dB to 21.71 dB and SSIM from 0.71 to 0.76. These results demonstrate the effectiveness of deep learning networks in MRI SR, surpassing traditional methods. The ability to directly apply deep learning networks without preprocessing also yields satisfactory results, highlighting the potential of deep networks in this field. Table 1 shows the evaluation results, indicating that our method outperforms previous methods on various evaluation metrics, producing high-quality, high-resolution MRI images. Specifically, the reconstructed texture is sharper than previous PSNR-driven methods. Unlike generative adversarial network-based methods, our system produces more natural results without any artifacts and does not require additional modules such as discriminators.

[0085] Furthermore, the results show that our model achieves a high LPIPS score, indicating higher perceptual similarity. From the perspective of the deep learning-based architecture, our system reconstructs medical details more accurately. These findings confirm the robustness of the design and the architecture's ability to more accurately reconstruct medically relevant details crucial for prostate cancer diagnosis, with significant margins.

[0086] This example validates the importance of residual prediction and the total diffusion step. To examine the impact of the total diffusion step T and the effectiveness of residual prediction, we conducted an ablation study, as shown in Table 2. From the results in rows 1, 2, 4, and 5, it is clear that image quality improves with increasing total diffusion steps. However, a larger total diffusion step leads to slower inference speed, so we chose T=100 as the default setting after balancing. The results from rows 1 and 3 demonstrate that residual prediction significantly improves both image quality and training speed, thus showing its effectiveness.

[0087] Table 2 Ablation Study Results (T and Res represent the total diffusion step size and residual prediction, respectively)

[0088]

[0089] This example demonstrates the importance of masking and replacement guidance strategies. We investigated the impact of these strategies on the performance of our method on the PROSTATEx dataset. To evaluate the effectiveness, we set different final mask ratios (φT). Figure 7 As shown, our method performs best when the final mask rate is set to 0.3. When the final mask rate is greater than 0.3, the method may be prone to error accumulation, while when the final mask rate is less than 0.3, the network may have difficulty identifying regions that require more attention.

[0090] Example 2

[0091] The purpose of this embodiment is to provide an image super-resolution reconstruction system.

[0092] An image super-resolution reconstruction system, comprising:

[0093] The data acquisition unit is used to acquire the MRI image of the prostate to be reconstructed, and to obtain the first resolution MRI image and the second resolution MRI image through corresponding preprocessing.

[0094] A residual image acquisition unit is used to obtain a residual image based on the difference between a first-resolution MRI image and a second-resolution MRI image; wherein the second resolution is lower than the first resolution.

[0095] A discrete processing unit is used to obtain discrete image labels by using the residual image as input to a pre-trained vector quantization variational autoencoder; and to perform discrete diffusion using a pre-trained diffusion model based on the obtained discrete image labels to obtain reconstructed image labels.

[0096] The super-resolution reconstruction unit is used to obtain super-resolution reconstruction results of MRI images based on reconstructed image labels and second-resolution MRI images, using a pre-trained image decoding model.

[0097] The system described in this embodiment corresponds to the method described in Embodiment 1. Its technical details have been described in detail in Embodiment 1, so they will not be repeated here.

[0098] Example 3

[0099] The purpose of this embodiment is to provide an electronic device.

[0100] This embodiment provides an electronic device, including a memory and a processor, as well as computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, they implement an image super-resolution reconstruction method as described in Embodiment 1.

[0101] Example 4

[0102] The purpose of this embodiment is to provide a computer-readable storage medium.

[0103] This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the image super-resolution reconstruction method described in Embodiment 1.

[0104] The system described in Embodiment 2 and the steps involved in Embodiments 3 and 4 correspond to the method described in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this disclosure.

[0105] Those skilled in the art will understand that the modules or steps described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. This disclosure is not limited to any particular combination of hardware and software.

[0106] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

[0107] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. An image super-resolution reconstruction method, characterized in that, include: Acquire MRI images of the prostate to be reconstructed, and obtain first-resolution MRI images and second-resolution MRI images through appropriate preprocessing; A residual image is obtained based on the difference between the first-resolution MRI image and the second-resolution MRI image; wherein the second resolution is lower than the first resolution. The residual image is used as the input to a pre-trained vector quantization variational autoencoder to obtain discrete image labels; and based on the obtained discrete image labels, a pre-trained diffusion model is used to perform discrete diffusion to obtain reconstructed image labels. Based on the reconstructed image labels and second-resolution MRI images, a pre-trained image decoding model is used to obtain super-resolution reconstruction results of MRI images.

2. The image super-resolution reconstruction method as described in claim 1, characterized in that, The image decoding model includes several sequentially connected Transformer blocks and a Softmax layer, wherein each Transformer block is equipped with a cross-attention mechanism that includes second-resolution MRI image information.

3. The image super-resolution reconstruction method as described in claim 2, characterized in that, Each Transformer block has an adaptive normalization layer that injects the current time step into the image decoding model.

4. The image super-resolution reconstruction method as described in claim 3, characterized in that, The reconstructed image labels and the second-resolution MRI images are used with a pre-trained image decoding model. Specifically, the reconstructed image labels are input into the first Transformer block through an adaptive normalization layer; and the second-resolution MRI image features extracted by the encoder are directly input into each Transformer block.

5. The image super-resolution reconstruction method as described in claim 1, characterized in that, The corresponding preprocessing specifically involves: obtaining a first-resolution MRI image through image enhancement based on the prostate MRI image to be reconstructed; and performing degradation processing on the first-resolution MRI image to obtain a second-resolution MRI image of the same size as the first-resolution MRI image.

6. The image super-resolution reconstruction method as described in claim 1, characterized in that, The method describes a modeling approach that models the relationship between the second-resolution MRI image and the residual image to obtain the long-range dependency between the second-resolution MRI image and the synthetic imaging, thereby enabling the construction of a super-resolution image.

7. An image super-resolution reconstruction system, characterized in that, include: The data acquisition unit is used to acquire the MRI image of the prostate to be reconstructed, and to obtain the first resolution MRI image and the second resolution MRI image through corresponding preprocessing. A residual image acquisition unit is used to obtain a residual image based on the difference between a first-resolution MRI image and a second-resolution MRI image; wherein the second resolution is lower than the first resolution. A discrete processing unit is used to obtain discrete image labels by using the residual image as input to a pre-trained vector quantization variational autoencoder; and to perform discrete diffusion using a pre-trained diffusion model based on the obtained discrete image labels to obtain reconstructed image labels. The super-resolution reconstruction unit is used to obtain super-resolution reconstruction results of MRI images based on reconstructed image labels and second-resolution MRI images, using a pre-trained image decoding model.

8. The image super-resolution reconstruction system as described in claim 7, characterized in that, The image decoding model includes several sequentially connected Transformer blocks and a Softmax layer, wherein each Transformer block is equipped with a cross-attention mechanism that includes second-resolution MRI image information.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and running thereon, characterized in that, When the processor executes the program, it implements an image super-resolution reconstruction method as described in any one of claims 1-6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements an image super-resolution reconstruction method as described in any one of claims 1-6.

Citation Information

Patent Citations

  • Image generation method and device, model training method and device and electronic equipment

    CN115880506A

  • Generating videos using sequences of generative neural networks

    DE202023101550U1