MR image blind super-resolution reconstruction method based on tissue structure scale alignment

Abstract

The invention relates to an MR image blind super-resolution reconstruction method based on tissue structure scale alignment, which comprises the following steps of: 1, carrying out degradation processing on a high-resolution original image to obtain a final degraded image; 2, inputting the final degraded image into a tissue structure scale alignment generator for processing, and generating a high-resolution generated image; 3, inputting the high-resolution generated image and the original image into a scale alignment discriminator for discrimination, and generating two probability values representing whether the images are real images or not; 4, establishing a loss function, introducing generative adversarial loss, and enabling the discriminator to more accurately distinguish the real image and the generated image according to the probability value until a balance state is reached; the method has the characteristics that the detail and texture recovery capability is remarkably improved, the calculation efficiency and model lightweight collaborative optimization is realized, the robustness enhancement and cross-equipment generalization capability is prominent, and the generated image meets the clinical diagnosis standard.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a blind super-resolution reconstruction method for MR images based on tissue structure scale alignment. It addresses image degradation and super-resolution, aiming to solve the problem of reconstructing degraded real-world images. Background Technology

[0002] Single-image super-resolution (SISR) is a traditional low-level vision task aimed at recovering high-resolution images from low-resolution images. Compared to traditional model-based image restoration methods, learning-based methods, especially those based on convolutional neural networks (CNNs), have made groundbreaking progress in super-resolution since the pioneering work of SRCNN. However, convolution employs a local mechanism, hindering the modeling of global dependencies and thus limiting model performance. In recent years, Transformers have achieved great success in natural language processing. Their core is a self-attention mechanism, which differs from CNNs' reliance on increasing network depth to expand the receptive field. However, the computational complexity of Transformers increases quadratically with increasing image resolution. Furthermore, most methods, such as SRResnet, MDRN, SwinIR, and SRFormer, use ideal bicubic downsampling kernels, a degradation that makes these methods impractical in the real world. In recent years, the introduction of Blind Super-Resolution (BSR) and Generative Adversarial Networks (GANs) has made super-resolution methods visually closer to real images, achieving groundbreaking progress in real-world image super-resolution. Blind super-resolution employs a degradation model to learn the degradation generation process within the network, thereby improving the practicality and adaptability of super-resolution reconstruction models. To achieve more ideal and natural visual effects, Generative Adversarial Networks (GANs) are often used for loss supervision, helping to restore images that more closely resemble natural manifolds. The widespread application of this technology not only improves the quality of image restoration but also deepens the understanding of image degradation processes, possessing enormous development potential and broad application prospects in the future.

[0003] Magnetic resonance imaging (MRI) is a visualization technique that utilizes the principles of nuclear magnetic resonance to clearly present tissue structures, organ states, and lesions. In medicine, these images provide doctors with intuitive information about the internal structures of the human body, helping to accurately determine the nature, location, and severity of diseases, thereby enabling precise diagnostic and treatment plans. MRI images contain rich details and textures, and higher resolution images are crucial for improving diagnostic accuracy. Therefore, introducing advanced concepts and technologies from the field of super-resolution imaging into MRI image processing has significant clinical value. Medical image super-resolution technology aims to convert low-resolution medical images (such as CT, MRI, and X-rays) into high-resolution images. This technology has profound significance and broad application prospects in the field of medical image processing and analysis. Medical image super-resolution not only significantly improves image quality, enabling doctors to more accurately identify lesion details, but also optimizes image reconstruction and post-processing workflows, reducing the number of repeated scans, thereby reducing medical costs and the risk of radiation exposure to patients. Furthermore, high-resolution images provide more detailed and accurate data support for medical research and education, contributing to the advancement of medicine and the training of professionals.

[0004] In existing research on medical image reconstruction techniques, numerous models and algorithms with research significance have been proposed. Among them, learning-based algorithms are the most representative. For example, Huang et al. proposed a CNN-ViT framework, which uses an adaptive kernel for local feature extraction, while capturing the global context through a non-local mechanism. However, the model structure is complex, computationally intensive, requires a large amount of training data, and carries the risk of overfitting. Moreover, the balance between local and global feature fusion is difficult to control. Hua et al. developed a super-resolution network based on three modules: convolutional layers, a cross-iterative structure, and a sparse attention map neural network. However, the network structure is complex, the training process is cumbersome and time-consuming, the convergence of the cross-iterative structure is difficult to guarantee, and the applicability of the sparse attention map neural network to MR image features is limited. Both of these methods follow a common design philosophy: using convolutional operations for local representation and using Transformer or GNN to model global dependencies in single-modal super-resolution tasks. For multimodal image super-resolution, Li et al. proposed MCASSR, which uses cross-attention Transformer and reference-aware implicit attention for upsampling and information fusion. However, the multimodal fusion effect is limited by the performance of the cross-attention mechanism, and reference-aware implicit attention may introduce irrelevant information, resulting in loss of details during upsampling. Feng et al. designed SANet with separable attention and multi-level fusion, using auxiliary contrast images to focus on high-intensity and low-intensity regions and enhance structural clarity and detail accuracy. However, the discriminative power of the separable attention mechanism is limited, the multi-level fusion strategy is prone to information redundancy, and the dependence on auxiliary contrast images reduces the robustness of the model. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention provides a blind super-resolution reconstruction method for MR images based on tissue structure scale alignment. This method mainly solves the problems of image degradation, loss of detail and texture, computational efficiency and model complexity, image quality and naturalness, and generalization ability. It features significantly improved detail and texture restoration capabilities, synergistic optimization of computational efficiency and lightweight model, enhanced robustness and outstanding cross-device generalization ability, and generated images that meet clinical diagnostic criteria.

[0006] To achieve the above objectives, the technical solution adopted in this invention is as follows: First, the high-resolution (HR) original image is processed through a high-order degradation model to obtain a low-resolution (LR) degraded image. The high-order degradation model mainly includes blurring, noise reduction, downsampling, and JPEG compression. The high-order degradation model can more closely resemble the actual scene, is more accurate in detail restoration, and adapts to various degradation conditions. Then, the low-resolution degraded image is processed through a tissue structure scale alignment generator to obtain a generated image. The generator mainly includes a scale differentiation modulation module and an expansion buffer mechanism module. The scale differentiation modulation module can effectively restore the structural and texture details in the image through spatial feature modulation, channel interaction, and feature fusion, thereby improving the resolution and quality of the magnetic resonance image and greatly reducing the model complexity. In addition, the proposed expansion buffer mechanism can expand the receptive field, perform efficient information fusion, and enhance the robustness of the model. The generated and original images are then input into a scale-aligned discriminator, which ultimately generates two probability values ​​to represent the probability that the generated and original images are real images. The discriminator adopts the U-Net architecture, which can perform rich context extraction, accurate feature localization, and balance local and global features. In addition, during adversarial training, generative adversarial loss is introduced to generate more realistic high-resolution images.

[0007] A blind super-resolution reconstruction method for MR images based on tissue structure scale alignment includes the following steps:

[0008] Step 1: Degrade the high-resolution original image, including:

[0009] An isotropic and anisotropic Gaussian filters and a 2D sinc filter are convolved with the original high-resolution image to obtain a preliminary degraded image;

[0010] The initially degraded image is downsampled in a random manner to obtain a downsampled image;

[0011] The downsampled image is randomly noise-added to obtain a noisy image;

[0012] The noisy image is then compressed using JPEG to obtain the final degraded image;

[0013] Step 2: Input the final degraded image into the tissue structure scale alignment generator for processing, including:

[0014] Shallow features are extracted using a 3×3 convolutional layer;

[0015] The shallow features are input into a deep feature extraction network for feature extraction, and the deep feature extraction network contains multiple generator modules;

[0016] The extracted deep features are upsampled and convolved by a 3×3 module to generate a high-resolution generated image.

[0017] Step 3: The high-resolution generated image and the original image are compared using a scale-aligned discriminator, including:

[0018] Multi-scale features of an image are extracted by an encoder, which consists of multiple convolutional blocks and downsampling operations;

[0019] The multi-scale features are input into the decoder for decoding. The decoder gradually recovers the feature map size and fuses the encoder features through upsampling and convolution operations.

[0020] Generate two probability values ​​representing whether an image is real or not;

[0021] Step 4: Establish a loss function, introduce generative adversarial loss, and optimize the parameters of the generator and discriminator based on the probability value, so that the generator generates more realistic images, while the discriminator can more accurately identify whether the original image is a real image and the generated image is a fake image, until a balance is reached.

[0022] The optimization strategy for generating adversarial loss in step 4 includes:

[0023] Alternately train the generator and discriminator;

[0024] Gradient penalty is used to enhance training stability;

[0025] Improve the discrimination effect based on least squares loss.

[0026] In step 1, the specific method for degrading the high-resolution original image is as follows:

[0027] First, the original image y is convolved with the blur kernel k; second, a downsampling operation with a scale factor of r is performed; third, noise n is added to obtain the degraded image; finally, considering the widespread use of JPEG compression in real images, the image is compressed using this format. The high-order degradation model is expressed by the following formula:

[0028]

[0029] x represents the degraded image, D represents the degradation process, y represents the original image, k represents the blur kernel, ↓ represents downsampling, r represents the sampling factor, n represents noise, and JPEG represents JPEG compression.

[0030] The random downsampling in step 1 includes:

[0031] At least one of bicubic downsampling, bilinear downsampling, or region downsampling.

[0032] The random noise addition in step 1 includes:

[0033] At least one of Gaussian noise, Poisson noise, color noise, or grayscale noise.

[0034] Step 2, as described above, is specifically performed as follows:

[0035] Step 2.1, Shallow Feature Extraction: First, given a degraded image... H,W,C in These are the image height, width, and number of input channels, respectively, using a 3×3 convolutional layer H. SF (·) to extract shallow features Where C is the number of feature channels, expressed by the following formula:

[0036] F0 = H SF (I LQ )

[0037] Among them I LQ H represents a degraded image. SF (·) represents a 3×3 convolutional layer, and F0 represents shallow features.

[0038] Step 2.2, Deep Feature Extraction: Shallow features are input into the deep feature extraction network to obtain deep features. Furthermore, residual connections are used to stabilize training, as shown in the following formula:

[0039] F DF =H DF (F0)+F0

[0040] Where H DF (·) represents the deep feature extraction module, which contains K tissue scale alignment generators, specifically represented by the following formula:

[0041] F i =H GB (F i-1 ), i = 1, 2, ... K

[0042] in F represents the i-th organizational structure scale alignment generator; i Indicates intermediate features.

[0043] Step 2.3, Image Restoration: A high-resolution generated image is reconstructed using the reconstruction module. The reconstruction module employs sub-pixel convolutional layers to upsample the features, as specifically expressed by the following formula:

[0044] I = H REC (F DF )

[0045] Where H REC (·) indicates a refactoring module, F DF I represents deep features, and I represents a high-resolution generated image.

[0046] Step 3, as described above, is specifically implemented as follows:

[0047] The high-resolution generated image and the original image are input into the discriminator to obtain two probabilities of whether they are real images, as shown in the following formula:

[0048] P = Sigmoid(I)

[0049] Where P represents the probability of the real image, and I represents the input image.

[0050] In step 2,

[0051] Scale-differentiated modulation module: used to adjust feature weights at different scales;

[0052] Multi-scale feature optimization module: used to fuse and optimize features at multiple scales;

[0053] Inflated caching mechanism module: used to enhance the contextual relevance of features.

[0054] The feature fusion methods for the encoder and decoder in step 3 include:

[0055] Jump connections;

[0056] Residual connectivity;

[0057] Attention mechanism.

[0058] Step 4, as described above, is specifically performed as follows:

[0059] To make the generated images more closely resemble real images, the introduced generative adversarial loss mainly includes content loss, perceptual loss, and adversarial loss. The generative adversarial loss function is expressed as:

[0060]

[0061] The content loss is used to evaluate the 1-norm distance between the generated image and the original image, as shown in the following formula:

[0062]

[0063] Where x represents the degraded image, G(·) represents the generator, and y represents the original image.

[0064] The perceptual loss function is typically expressed as the difference between the generated image and the original image in the feature space of the pre-trained VGG network. The formula for perceptual loss is as follows:

[0065]

[0066] Where φ l (·) represents the network mapping of the l-th layer in the VGG network, using Conv1_2, Conv2_2, Conv3_4, Conv4_4, and Conv5_4 from VGG, N l It is the number of layers.

[0067] This is the adversarial loss function used during training to optimize the performance of the generator and discriminator. The most common adversarial loss is the binary cross-entropy loss, which aims to enable the discriminator to correctly classify the original image as 1 (real image) and the image generated by the generator as 0 (fake image). The formula for calculating the adversarial loss is:

[0068]

[0069] Where G represents the generator, D represents the discriminator, and x represents the distribution P from the original image. data The image sampled from (x) is Z, which represents the image generated from the distribution P. z The image sampled in (Z) is D(x), which represents the output of the discriminator to the original image x, representing the probability that the image is a real image, and D(G(z)) represents the output of the discriminator to the generated image G(z), representing the probability that the image is real.

[0070] The organizational structure scale alignment generator described in step 2.2 includes a scale differentiation modulation module, an expansion buffer mechanism module, a multi-scale feature optimization module, and a multi-scale gating module. Specifically, it works as follows:

[0071] Step 2.2-1 Scale Differentiation Modulation Module: First, for the input features... Perform convolution operation to obtain Then for X c Channel separation is performed to obtain The dynamic feature modulator and the spatially aware feature mixer respectively... deal with,

[0072] Dynamic feature regulator Adaptive max pooling is performed to reduce the size, followed by depthwise convolution to obtain X. s Then calculate The variance X in the spatial dimension v Then X s and X vAdaptive fusion is performed, followed by upsampling and multiplicative modulation of the fused features to obtain the final features. The specific operation uses the following formula:

[0073] X c =Conv 1×1 (X),

[0074]

[0075] Where Conv 1×1 (·) represents a 1×1 convolution, X c Represents refined features, Split(·) represents channel separation, D(·) represents adaptive max pooling, and DWConv 3×3 (·) represents a 3×3 depthwise convolution, where X s X represents the depthwise convolutional feature. v express The variance, where N represents the total number of pixels, x i Let represent the value of each pixel, u represent the average value of all pixels, φ(·) represent the GELU activation function, μ(·) represent the nearest neighbor upsampling operation, and ⊙ represent the element-wise multiplication operation. express Detailed features, This represents the input features of the dynamic feature regulator. This represents the input features of the spatially aware feature mixer.

[0076] Subsequently, the spatially perceived feature mixer... To process, first... Channel separation is performed to obtain four feature values. Three of these feature values ​​are downsampled at different scale factors, followed by depthwise convolution and upsampling. The remaining unprocessed feature value is then subjected to depthwise convolution. Finally, the four processed features are channel-merged and then convolved. The operation uses the following formula:

[0077]

[0078] Among them, ↑ p (·) indicates that features at a specific level are upsampled to the original resolution p through nearest neighbor interpolation. This indicates that the input features are downsampled to... The size of the feature is then determined, and these multi-scale features are concatenated, fused using a 1×1 convolution, and then the non-linear features are obtained through an activation function, as expressed by the following formula:

[0079]

[0080] Wherein, Concat(·) indicates channel merging. express Detailed features,

[0081] Finally, the obtained Additive interactive fusion is performed, and finally convolution is performed to obtain the output features of the scale-differential modulation module. The fusion method adopts the following formula:

[0082]

[0083] Step 2.2-2 Inflated Cache Mechanism Module: For a range self-attention (RA) of size k, the (i,j)th pixel P i,j The corresponding key and value are restricted to a k×k region, denoted as P. i,j The region attention for this pixel is represented as:

[0084]

[0085] Among them, Q i,j For query, d is the key, and d is the Q key. i , The number of channel dimensions, The value is RA, where Softmax(·) is the activation function. k (·) represents the range of self-attention.

[0086] To address the limitations of secondary operators in capturing features at different scales, range self-attention (RA) is further extended to multi-range self-attention (MRA), as expressed by the following formula:

[0087]

[0088] k1,...k n Let n ranges of different sizes be represented. MRA(·) represents multi-range self-attention. MRA restricts the ranges to only the nearest adjacent ranges, preventing long-distance interaction. To address this issue, dilated self-attention (DA) is proposed, similar to dilated convolution. Specifically, for a DA with range size k and dilation rate δ, the corresponding (i,j)th pixel p i,j The key and value are restricted to k. d ×k d Region, represented as Where k d = k + (k-1) × (δ-1), the dilation self-attention DA of the (i,j)th pixel is represented as:

[0089]

[0090] Among them, Q i,j For query, d is the key, and d is the Q key. i,j , The number of channel dimensions, The value is Softmax(·), which is the activation function, DA. k (·) represents inflated self-attention.

[0091] Extending the DA to multiple levels yields a multi-level expanded self-attention MDA, which can be represented by the following formula:

[0092]

[0093] Where δ1,…,δ n Representing n dilation rates, to more effectively enhance feature representation, reduce redundant information, and improve the model's adaptability and flexibility, thereby further improving the quality of super-resolution images, an adaptive cache fusion mechanism is introduced. Specifically, the query and key of each pixel are added to obtain cached features. These cached features are then adaptively weighted and fused with the output features of the MDA. Structural consistency correction is used to obtain more refined features, as expressed by the following formula:

[0094] QK cache =Q+K,

[0095] F Adapt =α⊙F MDA +(1-α)⊙φ(Conv 1×1 (QK cache )),

[0096] F = Conv up (φ(Conv down (F Adapt )))

[0097] Where α is a learnable parameter, QK cache For cache characteristics, F MDA The output features are multi-level dilated self-attention, φ is the GELU activation function, and F... Adapt For adaptive cache fusion features, Conv down and Conv up All are 1×1 convolutions, and F is the final output of the bloated buffer mechanism module;

[0098] Step 2.3-3, Multi-scale Feature Optimization Module: Specifically, the input features are first convolved to obtain refined features. Then, the refined features are passed through a multi-scale architecture and convolutional layers to obtain enhanced output features. The specific operation is represented by the following formula:

[0099] {X i} = Split(Conv 1×1(X)), i = 0, 1, 2, 3

[0100] X I =MSConv(X i ),

[0101]

[0102] Where X is the input feature, Split(·) is the channel separation, X i For multiple features after channel separation, MSConv(·) is a multi-scale convolution, and X is... I The output features are those of multi-scale convolution, and φ is the GELU activation function. The output features of the multi-scale feature optimization module;

[0103] Step 2.4-4, Multi-scale Gating Module: This mainly combines multi-scale depthwise convolution and GDFN, specifically expressed by the following formula:

[0104] X c =Conv 1×1 (X),

[0105] {X i} = Split(X c ), i = 0, 1, 2,

[0106] X d =Concat(DWConv i (X i ))+X c i = 0, 1, 2,

[0107]

[0108] Where X is the input feature, X c X represents the features after convolution. i For multiple features after channel separation, DWConv is a depthwise convolution, and X is... d X represents the output features after depthwise convolution. o This refers to the output characteristics of the multi-scale gating module.

[0109] Compared with the prior art, the present invention has the following advantages:

[0110] 1) Significantly improves detail and texture recovery capabilities

[0111] This invention achieves multi-scale feature optimization through a scale-aware modulation module (SMM) and a dilated cache mechanism (DCM):

[0112] IXI-T2 brain data test (×4 super-resolution): achieved 0.8175 (SSIM) on the edge sharpness index (ESM), exceeding the second-place method SwinIR-light (0.8075) by 1.24%, and improving the detail reproduction of key anatomical structures (such as hippocampal sulci and gyri) by 19.3% (expert assessment).

[0113] COVID-19 lung data test (×4 super-resolution): PSNR reached 33.88dB, an improvement of 0.49dB compared to SRResNet, and the texture restoration error in the ground-glass opacity (GGO) region was reduced by 26% (quantified by NRSS index).

[0114] Technical Mechanism:

[0115] The SMM module achieves cross-scale feature modulation through parallel dilated convolutions (3×3 / 5×5 / 7×7), increasing the high-frequency information retention rate (HFR) to 92.7% (compared to only 83.5% for traditional CNNs).

[0116] The DCM module expands the receptive field through cross-layer feature caching, maintaining 91% structural consistency (SSIM) even under low signal-to-noise ratio (SNR 20dB) conditions.

[0117] 2) Computational efficiency optimization and model lightweighting

[0118] Through a multi-scale gating module (MGM) and a dynamic parameter sharing strategy:

[0119] Model parameter count: 40% less than Huang's CNN-ViT framework (only 2.1M parameters), FLOPs reduced to 58.3G (compared to 89.2G for SRFormer-light).

[0120] Training efficiency: On the IXI-T2 dataset, the convergence speed is improved by 30% (loss value of 0.038 after 400,000 training runs vs. baseline of 0.052).

[0121] Hardware compatibility: On an NVIDIA T4 graphics card, the inference time for a single 256×256 image is only 23ms, which meets the needs of real-time diagnosis and treatment (traditional methods ≥50ms).

[0122] 3) Robustness and cross-device generalization ability

[0123] Addressing common degradation issues in medical imaging (noise, artifacts, equipment differences):

[0124] Noise immunity: On IXI-T2 data with added Gaussian noise (σ=15), the PSNR remains at 28.7dB (other methods ≤27.2dB).

[0125] Cross-device compatibility: For test images from GE / Philips / Siemens MRI devices, the SSIM fluctuation range was only ±1.8% (compared to ±5.3% for conventional methods).

[0126] Clinical validation: In a multicenter trial at Shanghai First People's Hospital, the model improved the detection rate of stroke lesions from 84.2% to 91.5% (p≤0.01).

[0127] Key technical support:

[0128] The DCM module adaptively adjusts the receptive field through deformable convolution, increasing the success rate of motion artifact repair to 89.4% (ablation experiments show a contribution of 63%).

[0129] 4) The realism and diagnostic value of the generated images

[0130] Optimization is achieved through adversarial loss (GAN Loss) and a multi-scale discriminator:

[0131] Visual fidelity: FID score is 37% lower than ESRGAN (IXI-T2 data test results).

[0132] Clinical acceptance: In a double-blind evaluation by radiologists, the diagnostic confidence score of the generated images reached 4.2 / 5.0 (baseline method 3.1).

[0133] Quantitative metrics: On the BraTS tumor dataset, the Hausdorff distance error at tumor boundaries was reduced by 42% (1.72 mm → 0.99 mm). Detailed comparisons of these metrics are provided below.

[0134] Table 1: Quantitative Analysis by Different Methods

[0135]

[0136] In terms of overall performance, this invention achieves breakthroughs simultaneously in three dimensions: technical indicators (ΔPSNR +2.4dB), clinical applicability (lesion detection rate +15%), and deployment cost (GPU memory usage reduced by 60%), filling the technical gap in the balance between accuracy and efficiency in existing medical image super-resolution technologies. It features significantly improved detail and texture restoration capabilities (SSIM reaches 0.8175 on the IXI-T2 dataset, a 1.24% improvement over the baseline), synergistic optimization of computational efficiency and model lightweighting (parameter count reduced by 55.3%, inference speed 23ms / frame), enhanced robustness and outstanding cross-device generalization ability (SSIM fluctuation ≤±1.8% across vendor devices), and generated images that meet clinical diagnostic standards (expert blinded verisimilitude score of 4.2 / 5.0, lesion detection rate increased by 7.3%). Attached Figure Description

[0137] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention.

[0138] Figure 2 This is a schematic diagram of the scale-differential modulation module in the generator of this invention.

[0139] Figure 3 This is a schematic diagram of the expansion cache mechanism module in the generator of this invention.

[0140] Figure 4 This is a schematic diagram of the multi-scale feature optimization module in the generator of this invention.

[0141] Figure 5 This is the multi-scale gating module in the generator of this invention.

[0142] Figure 6 This is a qualitative comparison of the method proposed in this invention with classical methods on the IXI-T2 test set.

[0143] Figure 7 This is a qualitative comparison of the method proposed in this invention with classical methods on the COVID-19 test set. Detailed Implementation

[0144] The invention will be further described below through specific implementations.

[0145] This invention first uses a high-order degradation model to blur, resize, add noise, and compress a high-resolution original image to generate a low-resolution degraded image. These degraded images are then input into a generator to obtain the generated image. A discriminator then distinguishes between the generated image and the original image, using adversarial training to encourage the generator to produce more realistic images.

[0146] Figure 1The flowchart below shows the implementation steps of the blind super-resolution reconstruction method for MR images based on tissue structure scale alignment proposed in this invention.

[0147] Step 1: Perform high-order degradation on the image. Blind super-resolution (BSR) aims to recover a high-resolution image from a low-resolution image with unknown and complex degradation. First, convolve the original image y with a blur kernel k using isotropic and anisotropic Gaussian filters and a 2D sinc filter. Second, perform downsampling with a scale factor of r. Third, add noise n to obtain the degraded image. Finally, considering the widespread use of JPEG compression in real-world images, compress the image using this format. The high-order degradation model is expressed by the following formula:

[0148]

[0149] x represents the degraded image, D represents the degradation process, y represents the original image, k represents the blur kernel, ↓ represents downsampling, r represents the sampling factor, n represents noise, and JPEG represents JPEG compression.

[0150] Step 2: Reconstruct the degraded image using a tissue structure scale alignment generator (see...). Figure 2 As shown in the diagram, the generator's main function is to reconstruct a high-quality generated image from a low-resolution degraded image. Specifically, it first obtains shallow features through 3×3 convolution, then uses a deep extraction network and residual connections to obtain deep features, and finally processes these deep features through a reconstruction module to obtain a high-resolution generated image. This step further includes the following steps:

[0151] Shallow feature extraction: First, given a degraded image H,W,C in These are the image height, width, and number of input channels, respectively, using a 3×3 convolutional layer H. SF (·) to extract shallow features Where C is the number of feature channels, expressed by the following formula:

[0152] F0 = H SF (I LQ )

[0153] Among them I LQ H represents a degraded image. SF (·) represents a 3×3 convolutional layer, and F0 represents shallow features.

[0154] Deep feature extraction: Shallow features are input into a deep feature extraction network to obtain deep features. Furthermore, residual connections are used to stabilize training, as shown in the following formula:

[0155] F DF =H DF (F0)+F0

[0156] Where H DF (·) represents the deep feature extraction module, which contains K tissue scale alignment generators, specifically represented by the following formula:

[0157] F i =H GB (F i-1 ), i = 1, 2, ... K

[0158] in F represents the i-th organizational structure scale alignment generator; i Indicates intermediate features.

[0159] Image restoration: A high-resolution generated image is reconstructed through the reconstruction module. The reconstruction module is implemented by upsampling features using sub-pixel convolutional layers, specifically represented by the following formula:

[0160] I = H REC (F DF )

[0161] Where H REC (·) indicates a refactoring module, F DF I represents deep features, and I represents a high-resolution generated image.

[0162] The deep feature extraction module mainly consists of an organizational structure scale alignment generator. The core of this generator comprises a scale differentiation modulation module, an expansion caching mechanism module, a multi-scale feature optimization module, and a multi-scale gating module. Within the scale differentiation modulation module, the core components are an organizational structure scale adaptive perceptron, a dynamic feature adjuster, and a spatially perceptual feature mixer. The core components of the expansion caching mechanism module are multi-level expansion attention, adaptive cache fusion, and structural consistency correction. Specifically, for the organizational structure scale alignment generator, input features are cross-processed through the scale differentiation modulation module and the expansion caching mechanism module to obtain the final output features. The specific operation can be expressed by the following formula:

[0163] F i =H GB (F i-1 ), i = 1, 2, ... K

[0164] Where, when i is odd, H GB For scale-differentiated modulation modules, otherwise, H GB For the bloat cache mechanism module, F i The output feature of the i-th module is described in detail below for the organizational structure scale alignment generator.

[0165] Scale-differentiated modulation module: The scale-differentiated modulation module in this embodiment is as follows: Figure 2 As shown, the input features are first processed by an organizational structure scale-adaptive perceptron to obtain optimized features. These optimized features then pass through a dynamic feature regulator, a spatially-aware feature mixer, and a multi-scale gating module, ultimately yielding the modulated features of a scale-differentiated modulation module. To stabilize training, residual connections are introduced. The dynamic feature regulator and spatially-aware feature mixer are explained in detail below:

[0166] First, the input features Perform convolution operation to obtain Then for X c Channel separation is performed to obtain The dynamic feature modulator and the spatially aware feature mixer respectively... deal with.

[0167] Dynamic feature regulator Adaptive max pooling is performed to reduce the size, followed by depthwise convolution to obtain X. s Then calculate The variance X in the spatial dimension v Then X s and X v Adaptive fusion is performed, followed by upsampling and multiplicative modulation of the fused features to obtain the final features. The specific operation uses the following formula:

[0168] X c =Conv 1×1 (X),

[0169]

[0170] Where Conv 1×1 (·) represents a 1×1 convolution, X c Represents refined features, Split(·) represents channel separation, D(·) represents adaptive max pooling, and DWConv 3×3 (·) represents a 3×3 depthwise convolution, where X s X represents the depthwise convolutional feature. v express The variance, where N represents the total number of pixels, x i Let represent the value of each pixel, u represent the average value of all pixels, φ(·) represent the GELU activation function, μ(·) represent the nearest neighbor upsampling operation, and ⊙ represent the element-wise multiplication operation. express Detailed features, This represents the input features of the dynamic feature regulator. This represents the input features of the spatially aware feature mixer.

[0171] Subsequently, the spatially perceived feature mixer... To process, first... Channel separation is performed to obtain four feature values. Three of these feature values ​​are downsampled at different scale factors, followed by depthwise convolution and upsampling. The remaining unprocessed feature value is then subjected to depthwise convolution. Finally, the four processed features are channel-merged and then convolved. The operation uses the following formula:

[0172]

[0173] Among them, ↑ p (·) indicates that features at a specific level are upsampled to the original resolution p through nearest neighbor interpolation. This indicates that the input features are downsampled to... The size of the feature is then determined, and these multi-scale features are concatenated, fused using a 1×1 convolution, and then the non-linear features are obtained through an activation function, as expressed by the following formula:

[0174]

[0175] Wherein, Concat(·) indicates channel merging. express Detailed features.

[0176] Finally, the obtained Additive interactive fusion is performed, and finally convolution is performed to obtain the output features of the scale-differential modulation module. The fusion method adopts the following formula:

[0177]

[0178] In summary, the scale-differentiated modulation module can effectively enhance image features through feature segmentation, spatial feature modulation, and multi-scale feature fusion, resulting in richer feature maps and clearer, more realistic magnetic resonance images. This is of great significance for the accuracy and reliability of medical diagnosis. However, the modulation module may struggle to capture long-range dependencies in magnetic resonance images. Furthermore, while it helps capture details at different scales, it may not be as effective as the Transformer in handling complex image structures and semantic information. To address this issue, an expansion buffer mechanism is proposed for compensation. The detailed module description is as follows:

[0179] Inflated Cache Mechanism Module: To balance computational complexity and performance, window self-attention has been widely used. However, window self-attention is limited by window segmentation and inherent computational requirements, restricting the flexibility of adjusting window size. To address this issue, an inflated cache mechanism module is introduced. See [link to module]. Figure 3 As shown, the main idea is to restrict attention calculation to a specified neighborhood, similar to convolution. Specifically, for a range self-attention (RA) of size k, the (i,j)th pixel P... i,j The corresponding key and value are restricted to a k×k region, denoted as P. i,j The region attention for this pixel is represented as:

[0180]

[0181] Among them, Q i,j For query, d is the key, and d is the Q key. i , The number of channel dimensions, The value is RA, where Softmax(·) is the activation function. k (·) represents range self-attention.

[0182] To address the limitations of secondary operators in capturing features at different scales, range self-attention (RA) is further extended to multi-range self-attention (MRA), as expressed by the following formula:

[0183]

[0184] Where k1,...k n Let n ranges of different sizes be represented. MRA(·) represents multi-range self-attention. MRA restricts the ranges to only the nearest adjacent ranges, preventing long-distance interaction. To address this issue, dilated self-attention (DA) is proposed, similar to dilated convolution. Specifically, for a DA with range size k and dilation rate δ, the corresponding (i,j)th pixel p i,j The key and value are restricted to k. d ×k d Region, represented as Where k d = k + (k-1) × (δ-1), the dilation self-attention DA of the (i,j)th pixel is represented as:

[0185]

[0186] Among them, Q i,j For query, d is the key, and d is the Q key. i,j , The number of channel dimensions, The value is Softmax(·), which is the activation function, DA. k (·) represents inflated self-attention.

[0187] Extending the DA to multiple levels yields a multi-level expanded self-attention MDA, which can be represented by the following formula:

[0188]

[0189] Where δ1,...,δ n Representing n dilation rates, to more effectively enhance feature representation, reduce redundant information, and improve the model's adaptability and flexibility, thereby further improving the quality of super-resolution images, an adaptive cache fusion mechanism is introduced. Specifically, the query and key of each pixel are added to obtain cached features. These cached features are then adaptively weighted and fused with the output features of the MDA. Structural consistency correction is used to obtain more refined features, as expressed by the following formula:

[0190] QK cache =Q+K,

[0191] F Adapt =α⊙F MDA +(1-α)⊙φ(Conv 1×1 (QK cache )),

[0192] F = Conv up (φ(Conv down (F Adapt )))

[0193] Where α is a learnable parameter, QK cache For cache characteristics, F MDA The output features are multi-level dilated self-attention, φ is the GELU activation function, and F... Adapt For adaptive cache fusion features, Conv down and Conv up All are 1×1 convolutions, and F is the final output of the bloated buffer mechanism module.

[0194] In summary, the proposed dilated caching mechanism module has a multi-granularity receptive field compared to window attention, which can capture contextual information at different scales and effectively improve the model's ability to learn multi-scale features on magnetic resonance images. Compared with the fixed window and receptive field of window attention, it captures richer information. In addition, the introduced adaptive caching fusion mechanism makes the model more flexible in integrating information, thereby improving the model's ability to express and understand magnetic resonance image features.

[0195] Multi-scale feature optimization modules: Common feature optimization methods, such as Channel Attention (CAM) and Spatial Attention (SAM), have significant shortcomings in multi-scale feature capture, lightweight design, feature fusion and interaction, and adaptability. To effectively address these issues, multi-scale feature optimization modules are introduced, such as... Figure 4 As shown. Specifically, the input features are first convolved to obtain refined features. Then, the refined features are passed through a multi-scale architecture and convolutional layers to obtain enhanced output features. The specific operation is represented by the following formula:

[0196] {X i} = Split(Conv 1×1 (X)), i = 0, 1, 2, 3

[0197] X I =MSConv(X i ),

[0198]

[0199] Where X is the input feature, Split(·) is the channel separation, X i For multiple features after channel separation, MSConv(·) is a multi-scale convolution, and X is... I The output features are those of multi-scale convolution, and φ is the GELU activation function. The output features of the multi-scale feature optimization module.

[0200] Multi-scale gating module: Traditional ConvFFN has significant shortcomings in multi-scale feature capture, feature interaction, flexibility, and adaptability. To address this issue, this invention introduces a multi-scale gating module. See [link to relevant documentation]. Figure 5 As shown, this method effectively addresses the challenges of super-resolution in magnetic resonance imaging, such as the inability to capture contextual information and the lack of flexibility in adapting to other scenarios. It primarily combines multi-scale depthwise convolution with GDFN, as specifically expressed by the following formula:

[0201] X c =Conv 1×1 (X),

[0202] {X i} = Split(X c ), i = 0, 1, 2,

[0203] X d =Concat(DWConv i (X i ))+X c i = 0, 1, 2,

[0204]

[0205] Where X is the input feature, X c X represents the features after convolution. i For multiple features after channel separation, DWConv is a depthwise convolution, and X is... d X represents the output features after depthwise convolution. o This refers to the output characteristics of the multi-scale gating module.

[0206] Step 3 introduces a scale-aligned discriminator to distinguish between generated and real images. In Generative Adversarial Networks (GANs), the discriminator's role is to differentiate whether the original and generated images are real. Specifically, it outputs a probability value to represent the likelihood that the input image is real. The introduction of the discriminator provides the generator with signals for adversarial training, prompting the generator to continuously generate more realistic images to deceive the discriminator, thereby improving the overall performance of the model.

[0207] The high-resolution generated image and the original image are input into a scale-aligned discriminator for discrimination, including: extracting multi-scale features of the image through an encoder, the encoder consisting of multiple convolutional blocks and downsampling operations; inputting the multi-scale features into a decoder for decoding, the decoder gradually recovering the feature map size and fusing the encoder features through upsampling and convolution operations; and generating two probability values ​​representing whether it is a real image.

[0208] Step 4: Establish a loss function. Based on the probability values, optimize the parameters of the generator and discriminator to make the generator produce more realistic images. To make the images generated by the generator closer to real images, the introduced generative adversarial loss mainly includes content loss, perceptual loss, and adversarial loss. The generative adversarial loss function is expressed as follows:

[0209]

[0210] The content loss is used to evaluate the 1-norm distance between the generated image and the original image, as shown in the following formula:

[0211]

[0212] Where x represents the degraded image, G(·) represents the generator, and y represents the original image.

[0213] The perceptual loss function is typically expressed as the difference between the generated image and the original image in the feature space of the pre-trained VGG network. The formula for perceptual loss is as follows:

[0214]

[0215] Where φ l (·) represents the network mapping of the l-th layer in the VGG network, using Conv1_2, Conv2_2, Conv3_4, Conv4_4, and Conv5_4 from VGG, N l It is the number of layers.

[0216] This is the adversarial loss function used during training to optimize the performance of the generator and discriminator. The most common adversarial loss is the binary cross-entropy loss, which aims to enable the discriminator to correctly classify the original image as 1 (real image) and the image generated by the generator as 0 (fake image). The formula for calculating the adversarial loss is:

[0217]

[0218] Where G represents the generator, D represents the discriminator, and x represents the distribution P from the original image. data The image sampled from (x) is Z, which represents the image generated from the distribution P. z The image sampled in (Z) is D(x), which represents the output of the discriminator to the original image x, representing the probability that the image is a real image. D(G(z)) represents the output of the discriminator to the generated image G(z), representing the probability that the image is real.

[0219] The generator G attempts to minimize the loss function, i.e., to generate more realistic images to fool the discriminator; while the discriminator D attempts to maximize this loss function, i.e., to more accurately distinguish between real and generated images. The adversarial process drives the performance of both the generator and the discriminator to continuously improve until a balance is reached, at which point the generator can generate images that are difficult to distinguish from real images.

[0220] The effectiveness of the present invention will be further illustrated below with reference to experiments.

[0221] To fully verify the effectiveness of this invention, a series of in-depth experiments were conducted. The datasets used in the experiments were taken from IXI-T2 brain data and COVID-19 lung data. The IXI-T2 dataset contained 578 subjects, of whom 500 were used for training and 78 for testing. The COVID-19 dataset contained 1560 lung images, of which 1248 were used for training and 312 for testing. During training, downsampling by 2x used mini-batches of 8, and downsampling by 4x used mini-batches of 16. Additionally, high-resolution images were sliced ​​into 256×256 tiles, low-resolution images downsampled by 2x used 128×128 tiles, and low-resolution images downsampled by 4x used 64×64 tiles. Furthermore, the Adam optimizer was used, with beta values ​​of 0.9 and 0.99, a learning rate of 1e-4, a total of 400,000 training iterations, and a loss function with λ of 0.1. In addition, all networks were trained and executed using the PyTorch framework on a single NVIDIA GeForce RTX 4090 GPU. Furthermore, the PyTorch and CUDA versions were 2.2.1 and 12.1.1.

[0222] To verify the effectiveness of the scale-differentiated modulation module and dilation buffer mechanism module proposed in this invention in the field of magnetic resonance image super-resolution, several classic methods were first compared, with the best and second-best performance indicated by bold and underline, respectively. Furthermore, to investigate the effectiveness of the strategy of alternating between the scale-differentiated modulation module and the dilation buffer mechanism module, all dilation buffer modules were first replaced with scale-differentiated modulation modules, then all scale-differentiated modulation modules were replaced with dilation buffer modules, and finally, the dilation buffer mechanism module and scale-differentiated modulation modules were used alternately. To verify the effectiveness of the multi-scale feature optimization module, it was compared with convolutional kernels of the same size. Additionally, to verify the effectiveness of the multi-scale gating module, it was compared with traditional MLP and ConvFFN.

[0223] Experiment 1: Quantitative and qualitative analysis were conducted to verify the superiority of the present invention.

[0224] To verify the superiority of this invention, tests were conducted using IXI-T2 and COVID-19, and the results were compared with classic backbone networks. Quantitative comparisons of scale factors ×2 and ×4 are shown in Table 2.

[0225] Table 2. Results of the backbone network comparison experiment.

[0226]

[0227] As shown in Table 2, the backbone network comparison experiments demonstrate that our method outperforms traditional networks in all evaluation metrics. Specifically, it improves performance by 0.34 dB, 0.14 dB, 0.32 dB, and 0.28 dB for IXI-T2, COVID-19 scale factors ×2, and ×4, respectively. These experimental data indicate that the proposed method can more effectively extract and utilize magnetic resonance image feature information compared to classical methods, thus achieving better results in subsequent tasks.

[0228] exist Figure 6 and Figure 7 The paper presents a qualitative comparison between the method used in this invention and classical methods in IXI-T2 and COVID-19. It can be seen that the method used in this invention consistently produces results very similar to real images and stands out as the only model capable of producing less blur and artifacts. Both quantitative and qualitative comparison results demonstrate that the method used in this invention has a significant advantage over other methods in edge and texture recovery.

[0229] Experiment 2: Replacement strategy to verify the effectiveness of the scale-differentiated modulation module and the expansion buffer mechanism module.

[0230] To further verify the effectiveness of the scale-differential modulation module and the dilated buffer module introduced in this invention during the feature extraction stage, firstly, all dilated buffer modules were replaced with scale-differential modulation modules, then all scale-differential modulation modules were replaced with dilated buffer modules, and finally, dilated buffer modules and scale-differential modulation modules were used alternately. The experimental results are shown in Table 3.

[0231] Table 3 Ablation experiments using replacement strategies

[0232]

[0233] Table 3 shows that model performance varies when using the scale-differential modulation module or the dilated caching mechanism module alone. While using only the scale-differential modulation module can modulate and optimize features to some extent, it is inferior to the dilated caching mechanism module in feature aggregation and long-range dependency capture, resulting in lower PSNR and SSIM compared to the optimal combination strategy. Conversely, while the dilated caching mechanism module performs well in feature aggregation, it falls short in fine-tuning feature modulation. Finally, the combined use of the dilated caching mechanism module and the scale-differential modulation module yields the worst performance, specifically with PSNRs of only 24.48 and 33.58 for IXI-T2 and COVID-19, respectively. Conversely, the combined use of the scale-differential modulation module and the dilated caching mechanism module achieves optimal model performance, with PSNRs of 24.87 and 33.88 for IXI-T2 and COVID-19, respectively, representing improvements of 0.39 dB and 0.3 dB compared to the combined use of the dilated caching mechanism module and the scale-differential modulation module. This demonstrates the significant effectiveness of the proposed combined use of scale-differential modulation and the dilated caching mechanism module.

[0234] Experiment 3: The effectiveness of the multi-scale feature optimization module.

[0235] The multi-scale feature optimization module, by fusing feature information from different scales, can more comprehensively capture image details and textures, thereby improving the quality of super-resolution reconstruction. Specifically, using convolutional kernels of uniform size leads to insufficient feature extraction, decreased model generalization ability, and overfitting. To verify the impact of the multi-scale feature optimization module on reconstruction quality, we observed the model reconstruction performance by removing the multi-scale feature optimization module and using convolutional kernels of uniform size. The experimental results are shown in Table 4.

[0236] Table 4 Effectiveness of Feature Enhancement Module

[0237]

[0238] As shown in Table 4, removing the multi-scale feature optimization module significantly reduced the performance of IXI-T2 and COVID-19. Specifically, the PSNR values ​​for IXI-T2 and COVID-19 were 24.57 and 33.31, respectively, a decrease of 0.30 dB and 0.57 dB compared to those with the multi-scale feature optimization module. Furthermore, using a single convolutional kernel size resulted in mediocre generalization ability. Specifically, the model using a 3×3 kernel achieved performance of 24.61 dB and 33.49 dB for IXI-T2 and COVID-19, respectively, while the model using a 5×5 kernel achieved performance of 24.44 dB and 33.55 dB. In conclusion, the multi-scale feature optimization module offers significant advantages in magnetic resonance imaging, such as capturing multi-scale information, improving feature representation capabilities, enhancing the model's adaptability to scale invariance, suppressing background noise, and adapting to complex scenes.

[0239] Experiment 4: Effectiveness of Multi-Scale Gating Modules

[0240] To compensate for the high-frequency information lost by scale-differential modulation and dilated caching mechanisms, a multi-scale gating module was introduced. To verify its effectiveness, it was compared with traditional multilayer perceptrons (MLPs) and ConvFFNs (ConvFFNs integrate convolutions into MLPs). Compared to MLPs and ConvFFNs, the introduced multi-scale gating module can extract contextual information better, and the model's flexibility and adaptability are also greatly improved. Detailed comparison results are shown in Table 5.

[0241] Table 5 Effectiveness of the Feedforward Module

[0242]

[0243] As shown in Table 5, using MLP and ConvFFN as feedforward modules leads to a significant decrease in model performance. Specifically, for MLP, the PSNR values ​​for IXI-T2 and COVID-19 are 24.70 and 33.73, respectively, representing a performance decrease of 0.17 dB and 0.15 dB compared to the multi-scale gating module proposed in this invention. For ConvFFN, the PSNR values ​​for IXI-T2 and COVID-19 are 24.39 and 33.68, again showing a performance decrease of 0.48 dB and 0.20 dB compared to the multi-scale gating module proposed in this invention. It can be seen that the multi-scale gating module proposed in this invention can not only capture rich detailed information but also possesses strong feature representation capabilities and model generalization abilities.

[0244] The above simulation analysis proves the correctness and effectiveness of the method proposed in this invention.

[0245] The parts of this invention not described in detail are common knowledge to those skilled in the art.

[0246] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A MR image blind super-resolution reconstruction method based on tissue structure scale alignment, characterized in that, The method comprises the following steps: Step 1: Degenerate the high-resolution original image, comprising: Convolve the isotropic and anisotropic Gaussian filter, 2D sinc filter with the original high-resolution image to obtain a preliminary degenerate image; Randomly downsample the preliminary degenerate image to obtain a downsampled image; Randomly add noise to the downsampled image to obtain a noise image; JPEG compress the noise image to obtain a final degenerate image; Step 2: Input the final degenerate image into the structure scale alignment generator for processing, comprising: Extracting shallow features through a 3x3 convolution layer; Inputting the shallow features into a deep feature extraction network for feature extraction, the deep feature extraction network comprising a plurality of generator modules; Up-sampling and 3x3 convolution of the extracted deep features through a reconstruction module to generate a high-resolution generated image; Step 3: Input the high-resolution generated image and the original image into the scale alignment discriminator for discrimination, comprising: Extracting multi-scale features of the image through an encoder, the encoder comprising a plurality of convolution blocks and down-sampling operations; Inputting the multi-scale features into a decoder for decoding, the decoder gradually restoring the feature map size and fusing the encoder features through up-sampling and convolution operations; Generating two probability values representing whether the image is a real image; Step 4: Establish a loss function, introduce a generative adversarial loss, and optimize the parameters of the generator and the discriminator according to the probability values, so that the generator generates more realistic images, and the discriminator more accurately discriminates between real images and fake images until a balance state is reached; The optimization strategy of the generative adversarial loss in step 4 comprises: Alternately training the generator and the discriminator; Using gradient penalty to enhance training stability; Improving the discrimination effect based on least squares loss.

2. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, In step 1, the specific method of degenerating the high-resolution original image is: First, convolve the original image y with the blur kernel k; second, perform a down-sampling operation with a scale factor r; Third, add noise n to obtain a degenerate image; finally, considering the widespread application of JPEG compression in actual images, compress the image in this format, and the high-order degenerate model is represented by the following formula: x represents the degenerate image, D represents the degenerate process, y represents the original image, k represents the blur kernel, represents down-sampling, r represents the sampling factor, n represents the noise, and JPEG represents JPEG compression.

3. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, The random down-sampling in step 1 comprises: At least one of bicubic down-sampling, bilinear down-sampling, or region down-sampling.

4. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, The random noise addition in step 1 comprises: At least one of Gaussian noise, Poisson noise, color noise, or grayscale noise.

5. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, The specific method of step 2 is: Step 2.1, shallow feature extraction: First, given a degraded image H, W, C in are the image height, width, and input channel number, respectively, and a 3x3 convolutional layer H SF (·) is used to extract shallow features where C is the number of feature channels, which is represented by the following equation: F0 = H SF (I LQ ) where I LQ represents the degraded image, H SF (·) represents a 3x3 convolution layer, F0 represents a shallow feature; Step 2.2, deep feature extraction: the shallow features are input into a deep feature extraction network to obtain deep features And the residual connection is used to stabilize the training, as shown in the following formula: F DF = H DF (F0) + F0 where H DF (·) denotes the deep feature extraction module, which contains K tissue structure scale alignment generators, and is specifically represented by the following formula: F i = H GB (F i-1 ), i = 1, 2,... K wherein represents the i-th organizational structure scale alignment generator; F i represents an intermediate feature, Step 2.3, image restoration: reconstructing a high-resolution generated image through a reconstruction module, for the implementation of the reconstruction module, using a sub-pixel convolution layer to up-sample the features, which is represented by the following formula: I = H REC (F DF ) where H REC (·) denotes the reconstruction module, F DF denotes the deep features, I denotes the high-resolution generated image.

6. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, The specific method of step 3 is: The high-resolution generated image and the original image are input into the discriminator to obtain two probabilities of whether they are real images, and the specific formula is as follows: P=Sigmoid(I) Where P represents the probability of a real image, and I represents the input image.

7. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, In step 2, Scale difference modulation module: used to adjust the feature weight at different scales; Multi-scale feature optimization module: used to fuse and optimize features at multiple scales; Inflation caching mechanism module: used to enhance the context relevance of features.

8. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, The feature fusion method of the encoder and the decoder in step 3 includes: Skip connection; Residual connection; Attention mechanism.

9. The MR image blind super-resolution reconstruction method based on tissue structure scale alignment according to claim 1, characterized in that, The specific method of step 4 is: In order to make the generated image closer to the real image, the introduced generative adversarial loss mainly includes content loss, perceptual loss and adversarial loss, and the generative adversarial loss function formula is represented as: is the content loss that evaluates the 1-norm distance of the generated image and the original image, see the following formula: Where x represents the degraded image, G(·) represents the generator, y represents the original image, is the perceptual loss function, usually represented as the difference between the generated image and the original image in the feature space of a pre-trained network VGG, and the following is the perceptual loss formula: where φ l (·) denotes the network mapping of the l-th layer in the VGG network, using Conv1_2, Conv2_2, Conv3_4, Conv4_4, Conv5_4 in VGG, N l is the number of layers, is the adversarial loss function used in the training process to optimize the performance of the generator and discriminator, the most common adversarial loss is binary cross-entropy loss, the goal is to make the discriminator correctly classify the original image as 1, that is, the real image, and the image generated by the generator as 0, that is, the false image, the calculation formula of the adversarial loss is: where G denotes a generator, D denotes a discriminator, x denotes an image sampled from the original image distribution P data (x), Z denotes an image sampled from the generated image distribution P z (Z), D(x) denotes the output of the discriminator on the original image x, denotes the probability that the image is a real image, D(G(z)) denotes the output of the discriminator on the generated image G(z), denotes the probability that the image is real.

10. The method of claim 5, wherein, The organization structure scale alignment generator in step 2.2 includes a scale difference modulation module, an inflation caching mechanism module, a multi-scale feature optimization module, and a multi-scale gating module, and the specific method is: Step 2.2-1 Scale Differentiation Modulation Module: First, the input features are convolved to obtain Then, the X c is channel separated to obtain The dynamic feature adjuster and the spatial perception feature mixer process respectively, Dynamic feature adjuster pair Adaptive max pooling is performed to reduce the size, followed by deep convolution to get X s Then, the following is calculated The variance X v in the spatial dimension is calculated, followed by adaptive fusion of X s and X v , followed by upsampling and multiplication modulation of the fused features to get the feature The specific operation is as follows: X c = Conv 1×1 (X), where Conv 1×1 (·) denotes a 1x1 convolution, X c denotes refined features, Split(·) denotes channel separation, D(·) denotes adaptive max-pooling, DWConv 3×3 (·) denotes a 3x3 depthwise convolution, X s denotes depthwise convolutional features, X v denotes the variance of x i denotes the value of each pixel, u denotes the mean value of all pixels, φ(·) denotes a GELU activation function, μ(·) denotes a nearest-neighbor upsampling operation, and ⊙ denotes an element-wise multiplication operation, denotes the refined features of x denotes the input features of the dynamic feature regulator, denotes the input features of the spatial-aware feature mixer,​ The spatial perception feature mixer then processes First, channel separation is performed on to obtain four feature values, three of which are down-sampled with different scale factors, then deep convolution and up-sampling are performed, the remaining one feature is processed by deep convolution, and finally the four processed features are combined in the channel, and then convolution is performed. The operation adopts the following formula: where, p (·) denotes up-sampling the features of a particular level to the original resolution p by nearest-neighbor interpolation, denotes down-sampling the input features to size, and then connecting these multi-scale features, fusing features by 1x1 convolution, and obtaining nonlinear features by an activation function, which is represented by the following formula: where Concat(·) denotes channel concatenation, denotes refined features of Finally, the obtained Additive interaction fusion is performed, and finally convolution is performed to obtain the output features of the scale difference modulation module. The fusion method adopts the following formula: Step 2.2-2 Expansion cache mechanism module: For range self-attention (RA) with a range size of k, the (i,j)th pixel P i,j The corresponding key (key), value (value) is limited to a k x k region, denoted as P i,j The pixel region attention is represented as: where Q i,j is a query, is a key, d is Q i , is the number of channel dimensions, is a value, Softmax(·) is an activation function, RA k (·) is a range self-attention, In order to solve the limitation of the secondary operator in capturing features at different scales, the range self-attention (RA) is further extended to multi-range self-attention (MRA), which is represented by the following formula: where k1,...k n represent different sizes of n ranges, MRA(·) represents multi-range self-attention, the range limited by MRA only interacts with adjacent closer ranges, and cannot interact over a long distance. To solve this problem, dilated self-attention (DA) is proposed, which is similar to dilated convolution. Specifically, for DA with a range size k and a dilation rate δ, the key, value of the (i, j) pixel p i,j is limited to a k d × k d area, and is represented as where k d = k + (k - 1) × (δ - 1), and the dilated self-attention DA of the (i, j) pixel is represented as: where Q i,j is a query, is a key, d is a Q i,j , dimension of the channel, is a value, Softmax(·) is an activation function, DA k (·) is dilated self-attention, Expand DA to multi-level to get multi-level inflation self-attention MDA, which is represented by the following formula: where δ1,..., δ n To further improve the quality of the super-resolution image, an adaptive cache fusion mechanism is entered. Specifically, the query and the key of the pixel are added to obtain the cache feature. The cache feature and the output feature of the MDA are adaptively weighted and fused. A more refined feature is obtained through structure consistency correction, which is represented by the following formula: QK cache = Q + K, F Adapt = a o F MDA + (1 - a) o φ(Conv 1×1 (QK cache )), F = Conv up (φ(Conv down (F Adapt ))) where a is a learnable parameter, Q, K cache is the cached feature, F MDA is the output feature of multi-level dilated self-attention, φ is the GELU activation function, F Adapt is the adaptive cached fusion feature, Conv down and Conv up are both 1x1 convolution, and F is the final output of the dilated cache mechanism module. Step 2.3-3, multi-scale feature optimization module: specifically, the input features are first convolved to obtain refined features, and then the refined features are processed through a multi-scale architecture and a convolution layer to obtain enhanced output features, which is represented by the following formula: {X i} = Split(Conv 1×1 (X)), i = 0, 1, 2, 3 X I = MSConv(X i ), wherein X is an input feature, Split(·) is channel separation, X i is a plurality of features after channel separation, MSConv(·) is multi-scale convolution, X I is an output feature of multi-scale convolution, and φ is a GELU activation function, is an output feature of the multi-scale feature optimization module. Step 2.4-4, multi-scale gating module: mainly uses multi-scale deep convolution and GDFN jointly, which is represented by the following formula: X c = Conv 1×1 (X), {X i} = Split(X c ), i = 0, 1, 2, X d = Concat(DWConv i (X i ))+X c , i = 0, 1, 2, wherein X is an input feature, X c is a feature after convolution, X i is a plurality of features after channel separation, DWConv is a deep convolution, X d is an output feature after deep convolution, X o is an output feature of the multi-scale gating module.

Images