Frequency domain-texture joint learning-based arbitrary-scale MRI super-resolution reconstruction method

Abstract

The invention discloses an arbitrary-scale MRI (Magnetic Resonance Imaging) super-resolution reconstruction method based on frequency domain-texture joint learning, which relates to the technical field of medical image super-resolution reconstruction and comprises the following steps of: 1, constructing a data set; step 2, constructing a collaborative feature extraction module MB-CFE; step 3, constructing a texture injection type implicit up-sampling module TIM; 4, adding the image domain feature and the frequency domain feature element by element to obtain a joint feature, inputting the joint feature and the texture feature into a TIM module, completing pixel reconstruction of any real number multiple in a periodic space, and outputting a high-resolution image; 5, constructing a mixed loss function, and training the network; step 6, inputting an image to be reconstructed into the network, and outputting a high-resolution reconstructed image; according to the frequency domain-texture joint learning-based arbitrary-scale MRI super-resolution reconstruction method provided by the invention, the problems of insufficient texture detail recovery, single super-resolution scale and the like in the prior art are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image super-resolution reconstruction, in particular to a frequency domain-texture joint learning arbitrary scale MRI super-resolution reconstruction method. BACKGROUND

[0002] Magnetic resonance imaging (MRI) technology has become a core imaging method for clinical analysis of anatomical and pathological features due to its inherent advantages of no ionizing radiation and high soft tissue contrast. For example, in brain tumor, precise diagnosis and treatment require clear distinction between tumor infiltration margin, necrotic core and edema zone; however, high-resolution scanning is time-consuming and has high risk of motion-induced artifacts, so low-resolution fast sampling is forced to be used in clinic, resulting in missing of k-space high-frequency data and reduction of spatial resolution. Therefore, under the premise of not increasing the scanning time, implementing super-resolution reconstruction on MRI images to restore high-frequency details has become an urgent need to improve the diagnostic accuracy.

[0003] Existing models have achieved certain effects in the task of MRI super-resolution reconstruction, but there are still key problems such as insufficient recovery of texture details and single super-resolution scale that have not been solved. SUMMARY

[0004] The purpose of the present application is to provide a frequency domain-texture joint learning arbitrary scale MRI super-resolution reconstruction method to solve the problems of insufficient recovery of texture details and single super-resolution scale in the prior art.

[0005] To achieve the above purpose, the present application provides a frequency domain-texture joint learning arbitrary scale MRI super-resolution reconstruction method, comprising the following steps: Step 1, constructing a data set, dividing the data set into a training set and a test set; Step 2, constructing a collaborative feature extraction module MB-CFE, including an image domain branch, a frequency domain branch and a texture branch, obtaining image domain features, frequency domain features and texture features, extracting complementary features from spatial structure, frequency distribution and local details respectively, and realizing more robust feature fusion; Step 3, constructing a texture-injected implicit upsampling module TIM; Step 4, adding the image domain features and the frequency domain features element by element to obtain joint features, and inputting the joint features and the texture features into the TIM module together, completing pixel reconstruction of any real multiple in the periodic space and outputting a high-resolution image; Step 5, constructing a hybrid loss function to train the network composed of the MB-CFE module and the TIM module; Step 6, inputting a low-resolution MRI image to be reconstructed into the trained network to output a high-resolution reconstructed image.

[0006] Therefore, the application adopts the arbitrary scale MRI super-resolution reconstruction method of the above frequency domain-texture joint learning, constructs a multi-branch cooperative feature extraction module (MB-CFE), deploys the image domain, frequency domain and texture three branches in parallel, and couples the TIM implicit upsampling, so that 1.1~30.9x arbitrary real super-resolution can be completed in a single forward inference, and the limitations of fixed integer multiples and multiple model switching are no longer limited; the TIM module modulates and decodes through periodic coordinate modeling and cell constraints, ensures the zero drift of macro structures such as brain regions and tumor boundaries, and at the same time independently injects high-frequency textures, which significantly relieves the "structure smoothing-texture loss" split; the texture branch adopts a 1-to-64 progressive channel design, the parameter amount is reduced compared with the conventional U-Net, the implicit MLP decoding replaces the traditional interpolation, the inference speed is improved, a single 12 GB GPU can process 512x512x3 slices in real time, and the clinical real-time demand is met; during the training stage, 0.7 L1 and 0.3 LPIPS mixed loss are used to constrain the pixel accuracy and perceptual quality at the same time, and the continuous ratio can be freely switched without retraining.

[0007] The technical solutions of the application will be further described in detail below with the help of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 The overall flowchart of the arbitrary scale MRI super-resolution reconstruction method of the frequency domain-texture joint learning of the application is shown in the figure. Figure 2 The overall architecture of the network in the embodiment of the application is shown in the figure. Figure 3 The architecture of the image domain branch in the embodiment of the application is shown in the figure. Figure 4 The architecture of the RCANBlock block in the embodiment of the application is shown in the figure. Figure 5 The architecture of the frequency domain branch in the embodiment of the application is shown in the figure. Figure 6 The architecture of the texture branch in the embodiment of the application is shown in the figure. Figure 7 The test results of the embodiment of the application on the BraTS data set are shown in the figure. Figure 8 The test results of different resolutions of the embodiment of the application on the BraTS data set are shown in the figure, including error maps. Figure 9 The comparison figure of the texture extraction method of the embodiment of the application and the mainstream method Swin-Transformer at x2 ratio is shown in the figure. Figure 10 The comparison figure of the embodiment of the application under different loss functions is shown in the figure. DETAILED DESCRIPTION

[0009] The following detailed description of embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application claimed, but merely represents selected embodiments of the application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0010] Referring to Figures 1-2 , the method for arbitrary scale MRI super-resolution reconstruction of frequency domain-texture joint learning includes the following steps: Step 1, constructing a data set, dividing the data set into a training set and a test set.

[0011] S11, select the BraTS 2021 public data set, download 1251 registered 3D imaging data from the multi-modal MRI benchmark library, each registered 3D imaging data contains four registered 3D imaging modalities, respectively T1, T1ce, T2 and FLAIR; the T2 modality is taken as the super-resolution reconstruction target; the T2 modality contains 1251 samples, each sample is three-dimensional data with a dimension of 240x240x155, and the voxel is 1m 3 , isotropic; the sample number and spatial dimension of the T1, T1ce and FLAIR modalities are consistent with those of the T2 modality, which are used for subsequent experimental verification; S12, each sample in the T2 modality is sequentially split into 155 240x240 axial slices along the depth direction, obtaining HR slices; bilinear interpolation is used for downsampling, wherein the scaling factor , to generate corresponding 120x120 LR slices; S13, pair the 240x240 HR slices and the corresponding 120x120 LR slices one by one to form a one-to-one registered LR-HR two-dimensional slice pair, obtaining a data set; divide the data set into a training set and a test set according to a ratio of 8:2, and at the same time, make all slices of the same patient only belong to the training set or the test set to prevent data leakage.

[0012] Step 2, construct a collaborative feature extraction module MB-CFE, including an image domain branch, a frequency domain branch and a texture branch, to obtain image domain features, frequency domain features and texture features, extract complementary features from spatial structure, frequency distribution and local details, and realize more robust feature fusion.

[0013] S21, referring to Figures 3-4 , an image domain branch is constructed by a residual channel attention network (RCAN) to accurately extract the structure-semantic features of the slice under the T2 modality; the image domain branch includes 2 convolution layers, 4 RCANBlocks and a global residual; S211, the initial convolutional layer maps the input LR slice to a 64-dimensional feature map with the same spatial size by 64 3x3 convolution kernels, for extracting the underlying texture and edge information, represents the input low-resolution MRI slice, represents the real number field, representing the numerical type of the data as a real number, represents the height of the slice, represents the width of the slice; S212, a deep residual channel attention unit is constructed by sequentially stacking 4 RCANBlock blocks to enhance the feature expression step by step; each RCANBlock block sequentially performs 3x3 convolution, ReLU, 3x3 convolution, adaptive average pooling, two layers of 1x1 convolution (64→4→64), and Sigmoid to generate channel weights. After multiplying the channel weights with the local features channel by channel, the deep features are output by adding the input residual of the RCANBlock block, realizing channel-level attention enhancement of key structures such as tumor edges in the image; S213, the deep features output after stacking the 4 RCANBlock blocks are uniformly integrated by 3x3 convolution to correct the detail loss in the stacking process, keeping the feature dimension and spatial resolution unchanged; S214, global residual connection is introduced to add the feature map obtained in S211 and the integrated output obtained in S213 element by element to obtain image domain features , which are texture-semantics complementary rich representations, ensuring complete transmission of high-frequency details to subsequent reconstruction modules and effectively alleviating the gradient degradation problem; S22, please refer to Figure 5 S221, the input LR slice is processed by 3x3 convolution to extract features in the frequency domain (k-space) and return to the image domain by inverse Fourier transform; the k-space high-frequency gain mechanism is mainly used to explicitly strengthen the tumor edge and fine texture, complementing the image domain branch; S222, the real and imaginary parts of the complex matrix are stacked along the channel to form ​​​​​​The tensor is expanded to 128 dimensions, and then split into two parts according to the "first 64 dimensions - last 64 dimensions", which serve as the real and imaginary parts of 64 sets of complex features, respectively, thereby enhancing the local spectral correlation. S223. Perform inverse Fourier transform (IFFT) on each of the 64 groups of complex features, and remap the enhanced spectrum back to the image domain to obtain... Frequency domain characteristics Since each group of convolutional kernels focuses on different frequency directions and bandwidths in k-space, the 64 feature maps after inverse transformation each highlight edges and tumor boundaries at different scales and in different directions, forming a rich high-frequency detail basis, which complements the semantic features of the image domain branch and provides key frequency domain priors for subsequent fusion and reconstruction. S23, please refer to Figure 6 The texture branch includes a differential texture extraction unit, a U-Net network unit, and a lesion attention unit; texture maps are generated through a Gaussian difference mechanism, and a lesion attention mechanism is embedded based on the U-Net structure to accurately capture high-frequency texture and detail features of T2 slices; S231. Construct a differential texture extraction unit to process the input LR slices. Perform Gaussian difference, through An adaptive 5×5 Gaussian kernel performs Gaussian blurring, which is then subtracted pixel-by-pixel from the input image to obtain a texture map that focuses on high-frequency details, used to highlight lesion edges and microstructures. The standard deviation of the Gaussian kernel, also known as the smoothing coefficient of Gaussian blur, is expressed as: ; In the formula, For the output texture map , This is a Gaussian blur function; this operation can effectively separate the low-frequency structural contours and high-frequency texture details of an image, laying the foundation for the subsequent extraction of key texture features such as lesion edges and micro-tissues. S232. Construct a U-Net network unit with a mirror structure of "three-level encoding-three-level decoding"; The encoder's logical flow is as follows: In the first-level encoder, a 3×3 convolution with padding=1 and ReLU are used to... The texture map is losslessly mapped to It captures the finest texture primitives at full resolution; in the second-level encoder, it compresses the space to [a specific value] using 3×3 convolutions with stride=2 and ReLU. And expand the channels to 32, the output feature dimension is This achieves "spatial redundancy reduction + local context aggregation"; in the third-level encoder, it preserves space redundancy through 3×3 convolutions with padding=1 and ReLU. Size, increase the number of channels to 64, the output feature dimension is Further abstracting high-frequency texture patterns completes the "compression-expansion" feature extraction at the encoding end; where padding is the number of pixels to fill the edge region of the input feature map before the convolution operation, and stride is the number of pixels to move each time the convolution kernel slides on the input feature map; The decoder performs inverse recovery in a symmetrical manner. The process logic is as follows: In the first-stage decoder, the features are upsampled back using a 3×3 transposed convolution (stride=2, padding=1, output_padding=1, where output_padding is the number of extra pixels added to the edges of the generated output feature map after the transposed convolution operation) and ReLU. By reducing the number of channels to 32, the output feature dimension is... The first stage quickly reconstructs the geometric structure; the second stage decoder compresses the channels to 16 using 3×3 convolution and ReLU for fine-tuning edge details; the third stage decoder adjusts the channels back to 64 using 3×3 convolution and ReLU, and outputs the final product. Texture feature map; Thus, the network discards spatial redundancy and expands channel capacity layer by layer during the encoding stage, and restores spatial details and refines channel semantics layer by layer during the decoding stage. This not only preserves deep high-frequency abstraction, but also has complete spatial resolution, providing a high-quality feature base with both details and semantics for the subsequent lesion attention module. S233. Enhance key region texture features through lesion attention units: This involves enhancing the decoder output... Using the texture feature map as input, local correlation features are extracted through a 3×3 convolution kernel with padding=1; the local correlation features are compressed into a larger value through adaptive max pooling. The channel-level response vectors are used to focus on the strong response features of the lesion region; the channel weight distribution is adjusted by a 1×1 convolution kernel, and the channel attention weights in the 0-1 interval are generated by combining the sigmoid activation function; finally, the channel attention weights are multiplied by the channel dimension of the texture feature map output by the decoder to obtain the texture features. This enables the differential enhancement of texture features in lesion areas and the suppression of redundant features in background areas, thereby improving the clinical relevance of texture features.

[0014] Step 3: Construct the Texture Injection Implicit Upsampling Module (TIM).

[0015] S31, Based on target super-resolution scale Calculate the mesh size of the SR slice ,in For any real number and , Indicates the height of the SR slice. This indicates the width of the SR slice, determines the mesh size, and generates a normalized global coordinate matrix. ,in This represents the horizontal coordinate of a pixel in an SR slice. Representing the vertical coordinates of pixels in an SR slice, the global coordinate matrix C serves as the "baseline map" for all subsequent feature calculations, ensuring that any... The uniqueness of the pixel spatial location in the lower HR slice and the consistency with the global structure avoid anatomical area shifts in scenarios such as brain MRI; S32, Regarding the global coordinate matrix Perform a cycle mapping to convert infinite coordinates into repeating cells; the calculation expression is: ; ; In the formula, express The local coordinate offset of the direction, the result is The relative offset ratio within "integer coordinate units"; express The local coordinate offset of the direction, in The local relative offset ratio of the direction; and ; This function represents the floor function, which takes the largest integer not greater than the input value (e.g., floor(2.7)=2, floor(-0.3)=-1); it normalizes the period of the coordinates, provides a uniform period scale for high-frequency parameterization, and suppresses boundary jumps. S33, Through The periodic activation function maps the coordinates Embedded into a 64-dimensional implicit space, we obtain a coordinate embedding vector. (Extended to 64 dimensions), the calculation expression is: ; In the formula, For frequency coefficients, This embedding vector can enhance the coordinates and subsequent frequency intensity. The characteristic correlation improves the accuracy of periodic structure representation; S34. Define the cell unit: The cell unit size is 4×4 pixels. The cell unit size is 8×8 pixels; local normalized coordinates are generated within each cell unit. The local normalized coordinates are mapped to cell features through a fully connected layer (2D input coordinates, 64D output features). The feature can constrain the periodic deviation of the local region under the action of the frequency parameter, and avoid structural distortion; S35, by bilinear interpolation Extended to the size of the SR slice , provide detail constraints for local pixels, and global coordinate matrix ( ) Generate a complementary architecture of the "global-local" dual coordinate system; Lay the foundation for local accuracy for subsequent regional differentiation of amplitude weight ( ) And texture feature superposition, avoid "local blur" caused by global periodic features.

[0016] Step 4, add the image domain feature and the frequency domain feature element by element to obtain the joint feature, and input the joint feature and the texture feature into the TIM module, complete the pixel reconstruction of any real multiple in the periodic space and output the high-resolution image.

[0017] S41, add the image domain feature and the frequency domain feature element by element to obtain the fusion feature , the calculation expression is: ; In the formula, The output shape is , realize the lossless complement of spatial semantics and k-space frequency information in the same representation space, and serve as the main input branch of the TIM module; The texture feature does not enter the early fusion, but is sent into the TIM module in parallel with , avoid diluting high-frequency textures by global features, and realize "structure framework + texture details" joint learning reconstruction; S42, TIM module periodic modulation and arbitrary real size reconstruction, call global coordinate matrix and cell features for pixel-level coupling; S421, extract frequency control parameters: perform two groups of 3x3 convolution (padding=1) on the fusion feature respectively to obtain frequency intensity and amplitude weight , the calculation expressions are respectively: ; ; In the formula, used to define the "frequency intensity map" of the periodic feature, 64-channel fusion feature map is mapped to 64-channel frequency intensity map with the same channel number through 3x3 convolution with padding=1, that is, the output shape is ; The amplitude map providing "dynamic weights" for periodic features, the 64-channel fused feature map is expanded to a 128-channel amplitude weight map through another padding=1 3x3 convolution, i.e., the output shape is ; S422, periodic feature generation and weighting; S4221, up-sampling: if , the frequency intensity and the amplitude weight are bilinearly interpolated to the size of the SR slice, obtaining and , the spatial size is expanded from the of the LR slice to the of the SR slice; if , cubic interpolation is used; S4222, basic periodic feature: through the cosine trigonometric function and the sine trigonometric function , even and odd frequency components are captured respectively, the coordinates generated in step 3 are embedded into the global coordinate matrix and the up-sampled frequency intensity is pixel-by-pixel multiplied to obtain the position-weighted frequency feature, the position-weighted frequency feature and the cell feature are pixel-by-pixel added to obtain the 64-dimensional periodic feature after local-global modulation, the 64-dimensional periodic feature is respectively subjected to and operations to generate 64-dimensional even frequency component and 64-dimensional odd frequency component , after splicing and , the 128-dimensional basic periodic feature is obtained, the calculation expressions are respectively: ; ; ; In the formula, is used to normalize the periodic range; represents a feature splicing operation; and are ; S4223, amplitude weight weighting: the up-sampled amplitude weight is element-wise multiplied with the basic periodic feature , the periodic features in different regions are differentially adjusted to enhance the regional differential periodicity, and the calculation expression is: ; In the formula, The final amplitude-weighted periodic feature is represented as: is the result of the basic periodic feature modulated by the amplitude weight, which retains the frequency domain / position information and integrates the "strength weight" of the texture; S423, calculate the joint feature: combine the texture feature with the periodic feature through 3x3 convolution to expand to 128 dimensions and interpolate to the size of the SR slice, obtaining ; add and pixel by pixel to realize the fusion of "structure + texture", and output the 128-dimensional joint feature , which has both global periodic structure and local texture details, and the calculation expression is: ; S424, HR image reconstruction (pixel-level mapping): convert the 128-dimensional joint feature into a single-channel SR image through "convolution dimension reduction + fully connected mapping" to complete the final reconstruction; S4241, convolution dimension reduction: compress the 128-dimensional joint feature to 64 dimensions through a 3x3 convolution layer with padding=1, obtaining feature map, which reduces the computational load while retaining key features, and outputs S4242, pixel-wise MLP: perform two layers of 1x1 convolution (64→16→1) on the reduced feature map, independently regress a single-channel grayscale value for each pixel position, realize point-to-point mapping from LR feature to HR space, and output high-resolution image , where is a real number, representing the number of channels 1.

[0018] Step 5, construct a hybrid loss function to train the network composed of the MB-CFE module and the TIM module, please refer to Figure 2 .

[0019] S51, construct a hybrid loss function by combining the norm loss function and the perceptual loss function LPIPS, then the hybrid loss is calculated as: ; In the formula, represents the norm loss function, also known as the mean absolute error loss; represents the perceptual loss function; The core of the loss is to calculate the average of the absolute difference between the model prediction value (super-resolution result) and the true label (high-resolution image). Assuming that the model output super-resolution result is: ; wherein, is the total pixels of the image, is the single pixel value; the real high-resolution image label is: ; wherein, is the real value of the single pixel corresponding to the high-resolution reference image; The mathematical formula of the loss is: ; The perceptual distance is calculated after extracting the features by using the pre-trained VGG-16; the medical scene gives priority to guaranteeing the structural accuracy, so the weight is 0.7; the LPIPS weight is 0.3, which is used to consider the visual perception quality; S52, setting the training strategy: using the Adam optimizer, the initial learning rate is 1x10 -4 , and the decay is 0.5 times at the 40th epoch and the 60th epoch; the early stopping patience is set to 8 epochs, and the minimum PSNR improvement threshold is 0.05 dB; the model weight and the visualized result are saved once every 2 epochs; S53, iterative optimization process: inputting the low-resolution image and the corresponding high-resolution reference image into the network, calculating the hybrid loss after obtaining the reconstruction result by forward, and performing reverse propagation; synchronously recording the batch PSNR and SSIM, and accumulating the epoch average value for monitoring convergence and triggering early stopping, until completing the whole training period.

[0020] Step 6, inputting the low-resolution MRI image to be reconstructed into the trained network, and outputting the high-resolution reconstructed image.

[0021] Stability verification Through the method of the application, the input low-resolution image is subjected to two times super-resolution reconstruction, and the enlarged 2SR image is obtained, and the 2SR image is compared with the corresponding high-resolution HR reference image, please refer to Figure 7 , the 2SR image is highly consistent with the reference image in maintaining the sharpness of the tumor edge and the texture details, which verifies the effectiveness of the application in real clinical data.

[0022] In order to evaluate the stability of the application under any real magnification, please refer to Table 1, which shows the reconstruction quality indicators of the application under different magnifications. In the interval of 1~30.9x, the PSNR only gently decreases from 37.01 dB to 34.13 dB, the SSIM is always greater than or equal to 0.955, the LPIPS is less than or equal to 0.071, the MSE increase is less than twice, and there is no sawtooth and no block effect. Please refer to Figure 8, further give the absolute error graph (|SR-HR|) of the same slice at ×2, ×3.5, ×30.9 three magnifications; observe Figure 8 It can be seen that the error distribution is uniform and no visible artifacts are present. Figure 8 Table 1 collectively with Table 1 confirms that the error of the application is controllable in a wide scale range, the structure is faithful, and the texture is real, providing a high-reliability, no-retraining, plug-and-play MRI super-resolution practical solution for clinical use.

[0023] Table 1 Image reconstruction quality indicators of the application at different magnifications

[0024] Performance comparison and verification Select the same case slice of BraTS 2021 test set, and test the existing pre-trained model under the same conditions. Please refer to Table 2, which gives the comparison results of PSNR, SSIM and LPIPS at ×2, ×3, ×4, ×8 integer magnifications, wherein the loss function adopts L1 and LPIPS loss weighted fusion; at ×30.9 magnification, only the application can output single forward, and the rest need multiple interpolation splicing, so it is not included. The data in Table 2 shows that the application is significantly better than EDSR and Real-ESRGAN at all comparable magnifications, and still maintains 34.13 dB PSNR at the extreme 30.9x, verifying its wide scale, high fidelity and clinical practical value.

[0025] Table 2 Comparison of super-resolution performance at different integer magnifications on BraTS 2021 verification set

[0026] Network configuration comparison and verification In the design of the texture extraction branch, the application adheres to the principle of "lightweight first": using a progressive 1→64 channel U-Net to replace the current mainstream Swin-Transformer, and embedding a lesion attention gate. Since the loss function used by Swin-Transformer is generally L1, L1 loss function is used for testing here. Please refer to Table 3, the data in Table 3 shows that in the texture extraction subtask, the lightweight U-Net is sufficient to meet the requirements, taking into account speed, memory and interpretability, and is better than Swin-Transformer.

[0027] Please refer to Figure 9In Swin, significant bright red spots appear at the lesion edges (easily misinterpreted as false positives), while U-Net shows a uniform yellow color overall, offering superior subjective fidelity. Correspondingly, please refer to Table 4. Objective indicators show that U-Net's PSNR is 0.42–0.8 dB lower than Swin's (average approximately 0.68 dB) at magnifications of ×2 to ×6; the SSIM difference is less than 0.3%; and although the MSE is 9.7%–20.3% higher than Swin's, U-Net achieves superior visual fidelity with less than half the computational cost. Therefore, the U-Net branch provides a preferred lightweight solution that balances efficiency and accuracy for super-resolution MRI at any scale.

[0028] Table 3 Comparison of different network configurations for texture extraction branch

[0029] Table 4 Comparison of Objective Indicators for Texture Branch Networks

[0030] Loss function optimization effect verification The network configuration comparison above has confirmed that, under the same L1 supervision, the lightweight U-Net's PSNR is only about 0.42~0.8 dB lower than Swin-Transformer (average about 0.68 dB), while the number of parameters is reduced by half. To compensate for this slight difference without introducing any additional network parameters, this experiment verifies that while keeping the U-Net skeleton unchanged, the training loss is upgraded from L1 to a joint constraint of 0.7L1 + 0.3LPIPS. See Table 5; the new loss increases PSNR by 0.22~0.26 dB at ×3, ×4, and ×5 ratios, decreases MSE by about 5%, and maintains SSIM above 0.964. Meanwhile, please refer to... Figure 10 The error heatmap shows a significant reduction in high-brightness artifacts at the lesion edges, with visual smoothness superior to that of Swing. Thanks to the perceptual constraints of LPIPS, U-Net achieves objective metrics comparable to Swing and superior subjective quality without increasing parameters or latency, verifying that the L1+LPIPS combined loss can provide a dual-excellent solution for super-resolution MRI at any scale, offering both "metrics" and "visual" advantages.

[0031] Table 5. Comparison of reconstruction performance of different loss functions at various magnifications.

[0032] Therefore, the application adopts the arbitrary scale MRI super-resolution reconstruction method of the above frequency domain-texture joint learning, constructs a multi-branch cooperative feature extraction module (MB-CFE), deploys three branches of the image domain, the frequency domain and the texture in parallel, and couples the TIM implicit upsampling, so that 1.1~30.9x arbitrary real super-resolution can be completed in a single forward inference, and the application is no longer limited to the limitations of fixed integer multiples and multi-model switching; the TIM module modulates and decodes through periodic coordinate modeling and cell constraints, ensures zero drift of macro structures such as brain regions and tumor boundaries, and simultaneously independently injects high-frequency textures, which significantly relieves the "structure smoothing-texture loss" split; the texture branch adopts a 1-to-64 progressive channel design, the parameter quantity is reduced by more than 40% compared with the conventional U-Net, the implicit MLP decoding replaces the traditional interpolation, and the inference speed is improved by not less than 35%, a single 12 GB GPU can process 512x512x3 slices in real time, and the clinical real-time demand is met; during the training stage, the 0.7 L1 and 0.3 LPIPS mixed loss is used to constrain the pixel accuracy and perceptual quality at the same time, and the continuous ratio can be freely switched without retraining.

[0033] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application and not to limit them, although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand: the technical solutions of the present application can still be modified or replaced by equivalents, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the present application.

Claims

1. An arbitrary scale MRI super-resolution reconstruction method of frequency domain-texture joint learning, characterized in that, The method comprises the following steps: Step 1, constructing a data set, dividing the data set into a training set and a test set; Step 2, constructing a collaborative feature extraction module MB-CFE, including an image domain branch, a frequency domain branch and a texture branch, to obtain image domain features, frequency domain features and texture features; Step 3, constructing a texture injection type implicit upsampling module TIM; Step 4, element-wise adding the image domain features and the frequency domain features to obtain joint features, and inputting the joint features and the texture features into the TIM module, to complete pixel reconstruction of any real multiple in the period space and output a high-resolution image; Step 5, constructing a hybrid loss function to train the network composed of the MB-CFE module and the TIM module; Step 6, inputting a low-resolution MRI image to be reconstructed into the trained network to output a high-resolution reconstructed image.

2. The method of claim 1, wherein, The specific process of step 1 is as follows: S11, downloading from a multi-modal MRI benchmark library Example registered 3D imaging data, each example registered 3D imaging data containing four registered 3D imaging modalities, respectively, T1, T1ce, T2 and FLAIR; taking the T2 modality as the super-resolution reconstruction target; the T2 modality contains samples, each sample being three-dimensional data with a dimension of 240x240x155 and a voxel of 1m 3 , isotropic; The sample number and spatial dimension of the T1, T1ce and FLAIR modalities are consistent with those of the T2 modality; S12, each sample in the T2 modality is sequentially split into 155 axial slices of 240x240 in the depth direction, obtaining HR slices; down-sampling is performed by bilinear interpolation, wherein the scaling factor , to generate the corresponding 120x120 LR slices; S13, pairing the 240*240 HR slices and the corresponding 120*120 LR slices one by one to form a one-to-one registered LR-HR two-dimensional slice pair, and obtaining a data set; dividing the data set into a training set and a test set according to a ratio of 8:2, and at the same time, making all slices of the same patient belong to the training set or the test set to prevent data leakage.

3. The method of claim 2, wherein, The specific process of step 2 is as follows: S21, constructing an image domain branch through a residual channel attention network to extract the structure-semantic features of the slice under the T2 modality; the image domain branch comprises two convolution layers, four RCANBlock and a global residual; S211, the initial convolutional layer maps the input LR slice to a 64-dimensional feature by 64 3x3 convolutional kernels The feature map keeps the spatial dimension unchanged, extracting the low-level texture and edge information. represents the input low-resolution MRI slice, represents the real number field, represents the height of the slice, represents the width of the slice;​ S212, constructing a deep residual channel attention unit by sequentially stacking four RCANBlock blocks to enhance the feature expression step by step; Each RCANBlock block sequentially performs 3*3 convolution, ReLU, 3*3 convolution, adaptive average pooling, two layers of 1*1 convolution and Sigmoid to generate channel weights, multiplies the channel weights with the local features channel by channel, and then adds the residual error of the input of the RCANBlock block to output the deep features, so as to realize the channel-level attention enhancement of the key structures in the image; S213, uniformly integrating the deep features output by the four RCANBlock blocks through 3*3 convolution to correct the detail loss in the stacking process and keep the feature dimension and spatial resolution unchanged; S214, introducing global residual connection, element-wise adding the feature map obtained in S211 and the integrated output obtained in S213 to obtain an image domain feature ; S22, frequency domain branch extracts features from the input LR slice performing a two-dimensional fast Fourier transform, extracting features in the frequency domain by convolution, returning to the image domain by inverse Fourier transform; S221. Slice the input LR. Perform a two-dimensional fast Fourier transform to map the gray intensity distribution in the image domain to k-space data in the frequency domain, and obtain a complex matrix in k-space; S222, stack the real part and the imaginary part of the complex matrix along the channel stack Tensor, the dimension of the tensor is increased to 128 by 128 3x3 convolutions in the real domain, and the dimension of the tensor is increased to 128 by 128 3x3 convolutions in the real domain Tensor, the dimension of the tensor is increased to 128 by 128 3x3 convolutions in the real domain, and the dimension of the tensor is increased to 128 by 128 3x3 convolutions in the real domain S223, inverse Fourier transform is performed on the 64 groups of complex features one by one, and the enhanced frequency spectrum is remapped back to the image domain to obtain the frequency domain features ; S23, the texture branch comprises a differential texture extraction unit, a U-Net network unit and a lesion attention unit; S231, construct a differential texture extraction unit for the input LR slice performing a Gaussian difference by performing a Gaussian blur operation by an adaptive 5x5 Gaussian kernel, and then subtracting the input image pixel by pixel to obtain a texture map, wherein denotes the standard deviation of the Gaussian kernel; the expression is: ； wherein is the output texture, is a Gaussian blur function; S232, constructing a U-Net network unit with a mirror structure of "three-level encoding-three-level decoding"; The logic flow of the encoder is as follows: in the first level encoder, the texture map of is losslessly mapped to by 3*3 convolution with padding=1 and ReLU, and the texture primitive is captured at full resolution; in the second level encoder, the space is compressed to by 3*3 convolution with stride=2 and ReLU, and the channel is expanded to 32, and the output feature dimension is ; in the third level encoder, the size of is kept by 3*3 convolution with padding=1 and ReLU, and the channel is upgraded to 64, and the output feature dimension is ; wherein padding represents the number of pixels filled in the edge area of the input feature map before the convolution operation, and stride represents the number of pixels moved each time the convolution kernel slides on the input feature map. The decoder inversely recovers in a symmetrical manner, and the flow logic is as follows: in the first-level decoder, the features are up-sampled back to the original size through 3*3 transpose convolution and ReLU , the channel is reduced to 32, and the output feature dimension is , the geometry is reconstructed; in the second-level decoder, the channel is compressed to 16 through 3*3 convolution and ReLU, and the edge details are adjusted; in the third-level decoder, the channel is adjusted back to 64 through 3*3 convolution and ReLU, and the texture feature map of is output. S233, Enhance texture features through lesion attention units: This involves enhancing the texture features output by the decoder. Using the texture feature map as input, local correlation features are extracted through a 3×3 convolution kernel with padding=1; the local correlation features are then compressed using an adaptive max pooling operation. The channel-level response vector is generated; the channel weight distribution is adjusted using a 1×1 convolution kernel, and the channel attention weights in the 0-1 interval are generated by combining the Sigmoid activation function; the channel attention weights are multiplied by the channel dimension of the texture feature map output by the decoder to obtain the texture features. .

4. The method of claim 3, wherein, The specific process of step 3 is as follows: S31, determining a grid size based on the target super-resolution scale Computing the grid size of the SR slice wherein is an arbitrary real number and , denotes the height of the SR slice, denotes the width of the SR slice, determining a grid size, and generating a normalized global coordinate matrix wherein denotes the horizontal coordinate of a pixel in the SR slice, denotes the vertical coordinate of a pixel in the SR slice; S32, to the global coordinate matrix Performing period mapping, converting infinite coordinates into repeating units; the calculation expression is: ； ； wherein represents a local coordinate offset in the direction of represents a local coordinate offset in the direction of represents a floor function, and ; S33、by The periodic activation function maps the coordinates to a 64-dimensional latent space, obtaining a coordinate embedding vector The calculation expression is: ； In the formula, are frequency coefficients, ; S34, define cell units: when the cell unit size is 4x4 pixels, when the cell unit size is 8x8 pixels; generate local normalized coordinates inside each cell unit map the local normalized coordinates to cell features through a fully connected layer ; S35. The global-local bi-coordinate system is generated by bilinear interpolation of the global coordinates of the SR slice vertices. Extension to the size of the SR slice , the global coordinate matrix The complementary architecture of the "global-local" bi-coordinate system is generated.

5. The method of claim 4, wherein, The specific process of step 4 is as follows: S41、the image domain feature and the frequency domain feature element-wise addition to obtain the fusion feature The calculation expression is: ； In the formula, The output shape is ; S42, TIM module periodic modulation and reconstruction of arbitrary real number size, call global coordinate matrix with cell features pixel-level coupling is performed; S421, extracting frequency control parameters; S422, generating and weighting period features; S423, calculating joint features; S424, HR image reconstruction.

6. The arbitrary scale MRI super-resolution reconstruction method of frequency domain-texture joint learning according to claim 5, characterized in that, In S421: on the fused features performing two 3x3 convolutions with padding=1 each, resulting in frequency intensities and amplitude weights with the following expressions respectively: ； 。 7. The arbitrary scale MRI super-resolution reconstruction method of frequency domain-texture joint learning according to claim 6, characterized in that, In S422: S4221, up-sampling: if the frequency intensity and the amplitude weight are bilinearly interpolated to the size of the SR slice, resulting in and the spatial size is expanded from the of the LR slice to the of the SR slice; if tricubic interpolation is used. S4222, base period feature: through cosine trigonometric function and sine trigonometric function Capture even and odd frequency components respectively, embed the coordinates generated in step 3 into the global coordinate matrix and the frequency intensity after upsampling Pixel by pixel multiplication, get position weighted frequency feature, combine position weighted frequency feature and cell feature Pixel by pixel addition, get 64-dimensional period feature after local-global modulation, respectively execute and Operation to generate 64-dimensional even frequency component and 64-dimensional odd frequency component After splicing and Get 128-dimensional base period feature The calculation expressions are respectively: ； ； ； In the formula, for normalizing the period range; denotes a feature stitching operation; S4223, amplitude weight weighting: the up-sampled amplitude weight with the base period characteristics multiply by element, the expression is calculated as: ； In the formula, represents the period feature after amplitude weighting.

8. The arbitrary scale MRI super-resolution reconstruction method of frequency domain-texture joint learning according to claim 7, characterized in that, In S423: texture features are upsampled to 128 dimensions by 3x3 convolution and interpolated to the size of the SR slice, resulting in ; the and are added pixel by pixel, and the 128-dimensional joint feature is output, with the calculation expression being: 。 9. The method of claim 8, wherein, In S424, the 128-dimensional joint features are converted into a single-channel SR image through "convolution dimension reduction + full connection mapping" to complete the reconstruction; S4241、Convolutional Dimension Reduction: 128-dimensional joint features are compressed to 64-dimensional by a 3x3 convolutional layer with padding=1, resulting in feature maps; S4242, Pixel-wise MLP: on the reduced dimensionality The feature map is sequentially performed with two layers of 1x1 convolution, outputting high-resolution images wherein denotes a channel number of 1.

10. The method of any scale MRI super-resolution reconstruction with frequency-texture joint learning according to claim 9, characterized in that, The specific process of step 5 is as follows: S51、 by The mixed loss function is constructed by the norm loss function and the perceptual loss function LPIPS. Then the mixed loss The calculation expression of the mixed loss function is as follows: ； In the formula, denotes norm loss function; denotes perception loss function; S52, set training strategy: use Adam optimizer, initial learning rate 1x10 -4 At 40th epoch and 60th epoch, decay by 0.5 times; set early stop patience to 8 epochs, minimum PSNR improvement threshold to 0.05 dB; save model weights and visualization results every 2 epochs; S53、Iterative optimization process: input low-resolution image and corresponding high-resolution reference image into the network, calculate the hybrid loss and back propagation after forward reconstruction result; record batch PSNR and SSIM synchronously, and accumulate epoch average value for monitoring convergence and triggering early stop until completing the whole training period.

Images