Multi-modal MRI reconstruction method based on iterative refinement and k-space post correction

Abstract

The application discloses a multi-modal MRI reconstruction method based on iterative refinement and K-space post-correction, which comprises the following steps: collecting an undersampled target modality image and a fully sampled auxiliary modality image, and constructing a magnetic resonance image pair; constructing a multi-modal MRI reconstruction network model, including an image domain iterative refinement network, a K-space post-correction module and an adaptive fusion module; inputting the magnetic resonance image pair into the image domain iterative refinement network to extract image features and output image domain reconstruction features; mapping the image domain reconstruction features to a frequency domain by using the K-space post-correction module to obtain frequency domain correction features; performing weighted combination on the image domain reconstruction features and the frequency domain correction features in the adaptive fusion module to form dual-domain fusion features; and obtaining a predicted reconstruction image result. The application realizes efficient and accurate magnetic resonance image reconstruction through cross-modal structure alignment and frequency domain joint optimization by using deep learning technology.

Description

Technical Field

[0001] This invention relates to the fields of image processing and magnetic resonance imaging technology, specifically to a multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction. Background Technology

[0002] Magnetic resonance imaging (MRI) plays a crucial role in modern clinical diagnosis due to its superior soft tissue contrast, high spatial resolution, and lack of ionizing radiation. However, traditional MRI scans require complete acquisition of K-space (frequency domain) data, making the data acquisition process extremely time-consuming and prone to introducing motion artifacts. To improve scanning efficiency, K-space undersampling techniques are commonly used clinically to accelerate imaging. However, high undersampling inevitably introduces aliasing artifacts and leads to the loss of anatomical details.

[0003] In recent years, deep learning-based multimodal MRI reconstruction methods have gradually become mainstream. These methods significantly improve the reconstruction quality of accelerated imaging by utilizing high-resolution anatomical information from other modalities (such as proton density-weighted images, PD) to guide the reconstruction of the target modality (such as T2-weighted images). However, existing multimodal reconstruction methods still face two major challenges: First, due to different scanning protocols and the motion of the scanned object, spatial positional offsets (spatial misalignment) inevitably exist between different modalities. Traditional single-pass feature fusion methods cannot progressively align cross-modal structures, leading to the accumulation of registration errors in the network. Second, many existing deep learning reconstruction methods only perform black-box mapping in the image domain, ignoring the physical property that the MRI signal itself is acquired in K-space, lacking direct correction of frequency distribution, resulting in blurred high-frequency details in the reconstruction results at high speedup ratios.

[0004] Therefore, this invention proposes a multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction, aiming to achieve high-quality, high-fidelity accelerated MRI reconstruction by dynamically correcting cross-modal spatial offsets and combining K-space for high-frequency posterior correction. Summary of the Invention

[0005] The purpose of this invention is to provide a multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction, which can effectively extract information from undersampled MRI images and perform accurate, clear, and high-quality reconstruction.

[0006] To achieve the above functions, this invention designs a multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction, performing the following steps S1-S5 to complete the magnetic resonance image reconstruction:

[0007] Step S1: Acquire undersampled magnetic resonance images and corresponding fully sampled magnetic resonance images. Use the undersampled magnetic resonance images as target modal images and the corresponding fully sampled magnetic resonance images as auxiliary modal images to form magnetic resonance image pairs. Divide the magnetic resonance image pairs into training and test sets according to a preset ratio.

[0008] Step S2: Construct a multimodal MRI reconstruction network model based on iterative refinement and K-space posterior correction. The multimodal MRI reconstruction network model includes an image domain iterative refinement network, a K-space posterior correction module, and an adaptive fusion module.

[0009] The network is iteratively refined in the input image domain using magnetic resonance images. The auxiliary modal images are spatially aligned to the target modal image. Features from both the target and auxiliary modal images are then extracted and iteratively refined to output image domain reconstruction features. A K-space posterior correction module maps the image domain reconstruction features to the frequency domain, capturing inherent properties and artifact patterns to obtain frequency domain corrected features. In the adaptive fusion module, the image domain reconstruction features and frequency domain corrected features are weighted and combined to form dual-domain fusion features. Residual connections are then performed to obtain the predicted magnetic resonance image reconstruction results.

[0010] Step S3: Design the total loss function for optimizing the multimodal MRI reconstruction network model, and preset the parameters of the multimodal MRI reconstruction network model;

[0011] Step S4: Input the magnetic resonance images in the training set into the multimodal MRI reconstruction network model, and use the Adam optimizer to iteratively optimize and update the parameters of the multimodal MRI reconstruction network model according to the total loss function.

[0012] Step S5: Train and test the multimodal MRI reconstruction network model until it reaches the preset convergence condition. Use the trained multimodal MRI reconstruction network model to achieve end-to-end reconstruction from undersampled MRI images to complete MRI images.

[0013] As a preferred technical solution of the present invention: for the magnetic resonance image pair formed in step S1, the magnetic resonance image pair is divided into a training set and a test set in a ratio of 0.9:0.1.

[0014] As a preferred technical solution of the present invention, the specific steps of step S2 are as follows:

[0015] Step S2.1: In the image domain iterative thinning network, let the first... The input for each iteration is the target modality image. and auxiliary modal images First, the residual registration module concatenates the two data points along the channel dimension. After initial convolutional feature mixing, the residual prediction module is used to predict the two-dimensional displacement field. With confidence mask :

[0016] ;

[0017] In the formula, This indicates a channel-level concatenation operation. The kernel size is Convolution operation, These are the learnable parameters for the residual prediction module. This indicates the operation of the residual prediction module.

[0018] The residual prediction module contains residual blocks, and the calculation formula for single-layer feature transfer is as follows:

[0019] ;

[0020] In the formula, The input features for the residual block are... For the corresponding output features, For activation function, For the output mapping layer of the residual block;

[0021] Next, image alignment is performed using a spatial transformation network: Let the pixel coordinate vector of the target modality image be... Its corresponding auxiliary modal image coordinate vector The matrix transformation formula is as follows:

[0022] ;

[0023] Subsequently, auxiliary modal images Grid resampling and bilinear interpolation are performed, and local deformation weights are modulated using a confidence mask to obtain the aligned auxiliary modal image. The calculation formula for the interpolation transformation is as follows:

[0024] ;

[0025] In the formula, , For spatial pixel discrete indexing of auxiliary modal images, Indicates the confidence level mask. Represents the pixel coordinates in the target modal image. Represents the pixel coordinates in the auxiliary modality image;

[0026] Step S2.2: Transfer the target modality image Aligned auxiliary modal image Two Swin-Transformer UNet-based backbone networks are input respectively. The two backbone networks form the target branch and the auxiliary branch, respectively. The two branches have the same structure but independent parameters. The intermediate features of the target branch are output separately. Intermediate features of auxiliary branches The calculation formula is as follows:

[0027] ;

[0028] ;

[0029] In the formula, and These are the backbone networks for the target branch and auxiliary branches, respectively. and Here are the network parameters for the target branch and the auxiliary branch, respectively; the backbone network calculates multi-head self-attention based on the moving window mechanism to extract the corresponding intermediate features;

[0030] Subsequently, the obtained intermediate features will be extracted. and Alignment is performed in both spatial and channel dimensions, and pixel-by-pixel subtraction is carried out to obtain a cross-modal difference map. Convolution is then used to extract the correction residuals. The superimposed image onto the target branch yields the iterated target modal image. As input for the next round; after completing all After one iteration, the target modal images output from all iterations are fused to obtain the image domain reconstruction features. The process is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] In the formula, and These represent the kernel size as follows: and Convolution operations; This represents a channel-level splicing operation. express The target modal image output in each iteration;

[0035] Step S2.3: In the K-space posterior correction module, reconstruct features from the image domain. Complex data are obtained by mapping to the frequency domain using Fourier transform. The real and imaginary parts are decomposed and concatenated to form a two-channel real tensor. Shallow features are extracted using a frequency domain feature extractor. The output features are then input into the backbone network. Reorganized into complex tensors The frequency domain corrected features are obtained by inverse Fourier transform. The calculation formula is as follows:

[0036] ;

[0037] ;

[0038] ;

[0039] In the formula, Use the LeakyReLU activation function; The parameter is Feature mapping network; and These represent the output features of the backbone network, respectively. The real and imaginary parts, The imaginary unit;

[0040] Step S2.4: In the adaptive fusion module, reconstruct features in the image domain. and frequency domain correction features As input, the two are first concatenated along the channel dimension, and then the dual-domain fused features are obtained through a convolutional mixing block. Subsequently, a channel attention mechanism was introduced to calculate the adaptive weight coefficients of each channel. The calculation formula is as follows:

[0041] ;

[0042] ;

[0043] ;

[0044] In the formula, Use the GELU activation function; This represents the channel descriptor after global average pooling. and These are the height and width of the image, respectively. and These are the pixel indices of the image in the height and width directions, respectively; and These are the parameters for the fully connected layer. Use the Sigmoid activation function;

[0045] Finally, the adaptive weighting coefficients are multiplied channel-by-channel by the dual-domain fusion features, and then combined with the image domain reconstruction features. Perform residual connections to output the final magnetic resonance image reconstruction result. :

[0046] ;

[0047] In the formula, This indicates a channel-by-channel multiplication operation.

[0048] As a preferred technical solution of the present invention, the specific steps of step S3 are as follows:

[0049] Constructing a total loss function for optimizing a multimodal MRI reconstruction network model The total loss function is composed of image domain deep supervised loss. Frequency domain consistency loss And the network parameter regularization terms are jointly constructed;

[0050] Image Domain Deep Supervision Loss The calculation formula is as follows:

[0051] ;

[0052] In the formula, For the first The target modal image output by the round of iterations in coordinates Pixel value at that location, These are the pixel values ​​of the corresponding auxiliary modality image. and These are the height and width of the image, respectively. This represents the total number of iterations.

[0053] Frequency domain consistency loss The calculation formula is as follows:

[0054] ;

[0055] In the formula, The coordinates output by the K-space posterior correction module Frequency domain data at that location, The coordinates are obtained by Fourier transform of the target modal image. The actual frequency domain data at that location, This is a binarized undersampling mask matrix. Indicates the undersampling mask matrix in coordinates The value at each sampling point; where the value at a sampling point is 1, and the value at an unsampled point is 0.

[0056] The total loss function The final calculation formula is as follows:

[0057] ;

[0058] In the formula, Represents the set of all learnable parameters. For each of the individual weight parameters, and These are the hyperparameters for controlling frequency domain consistency and the hyperparameters for weight decay, respectively.

[0059] As a preferred embodiment of the present invention, the specific steps of step S4 are as follows:

[0060] Step S4.1: Initialize the parameters in the multimodal MRI reconstruction network model; select magnetic resonance image pairs from the training set and input them into the multimodal MRI reconstruction network model to obtain the predicted magnetic resonance image reconstruction results;

[0061] Step S4.2: Based on the total loss function constructed in step S3, calculate the gradient using the backpropagation algorithm, and use the Adam optimizer to update the learnable parameters in the multimodal MRI reconstruction network model;

[0062] Step S4.3: For all magnetic resonance image pairs in the entire training set, repeat the operations of steps S4.1 and S4.2 until the preset number of iterations of training are completed.

[0063] As a preferred embodiment of the present invention, the specific steps of step S5 are as follows:

[0064] Step S5.1: Input the magnetic resonance image pairs in the test set into the multimodal MRI reconstruction network model, and calculate the total loss function value and the preset image quality evaluation index;

[0065] Step S5.2: If the total loss function value decreases to below the preset threshold, or the change in the image quality assessment index is lower than the preset threshold, then the multimodal MRI reconstruction network model is determined to have converged, and step S5.3 is executed; otherwise, return to step S4 to continue training.

[0066] Step S5.3: Input the target modal image to be processed and the corresponding auxiliary modal image into the converged multimodal MRI reconstruction network model, and output the magnetic resonance image reconstruction results.

[0067] The present invention also designs a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction.

[0068] The present invention also designs a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction.

[0069] Beneficial effects: Compared with the prior art, the advantages of the present invention include:

[0070] This invention proposes a multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction. It fully utilizes the structural prior information of auxiliary modalities, reduces the negative impact of spatial migration on reconstruction quality, and constructs a joint optimization mechanism in both the image and K-space domains, adaptively balancing the contributions of features from different domains to the final image. The image domain iterative refinement network and its internal residual registration module dynamically align cross-modal structures and progressively refine features; the K-space posterior correction module captures inherent properties in the frequency domain, corrects frequency distribution, and restores high-frequency details; and the adaptive fusion module uses a channel attention mechanism to weightedly combine image and frequency domain features to form a high-quality reconstruction result. The reconstruction achieves high fidelity and good artifact suppression, enabling clear and accurate multimodal MRI image reconstruction. Attached Figure Description

[0071] Figure 1 This is a flowchart of a multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction provided according to an embodiment of the present invention;

[0072] Figure 2 This is a schematic diagram of the overall architecture of the multimodal MRI reconstruction network model provided in an embodiment of the present invention;

[0073] Figure 3 This is a schematic diagram of an image domain iterative thinning network provided according to an embodiment of the present invention;

[0074] Figure 4 This is a schematic diagram of the K-space posterior correction module provided according to an embodiment of the present invention;

[0075] Figure 5 This is a schematic diagram of an adaptive fusion module provided according to an embodiment of the present invention;

[0076] Figure 6 This is a comparison diagram of the visual effects of different reconstruction methods under Cartesian sampling according to an embodiment of the present invention;

[0077] Figure 7 This is a comparison diagram of the visual effects of different reconstruction methods under radial sampling, provided by an embodiment of the present invention. Detailed Implementation

[0078] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0079] The multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction provided in this embodiment of the invention refers to... Figure 1 Perform the following steps S1-S5 to complete the magnetic resonance image reconstruction:

[0080] Step S1: Acquire undersampled magnetic resonance images and corresponding fully sampled magnetic resonance images. Use the undersampled magnetic resonance images as target modal images and the corresponding fully sampled magnetic resonance images as auxiliary modal images to form magnetic resonance image pairs. Divide the magnetic resonance image pairs into training and test sets according to a preset ratio.

[0081] Step S2: Construct a multimodal MRI reconstruction network model based on iterative refinement and K-space posterior correction. The multimodal MRI reconstruction network model includes an image domain iterative refinement network, a K-space posterior correction module, and an adaptive fusion module.

[0082] The network is iteratively refined in the input image domain using magnetic resonance images. The auxiliary modal images are spatially aligned to the target modal image. Features from both the target and auxiliary modal images are then extracted and iteratively refined to output image domain reconstruction features. A K-space posterior correction module maps the image domain reconstruction features to the frequency domain, capturing inherent properties and artifact patterns to obtain frequency domain corrected features. In the adaptive fusion module, the image domain reconstruction features and frequency domain corrected features are weighted and combined to form dual-domain fusion features. Residual connections are then performed to obtain the predicted magnetic resonance image reconstruction results.

[0083] Reference Figure 2 The specific steps of step S2 are as follows:

[0084] Step S2.1: During multimodal magnetic resonance imaging, due to involuntary subject movement, tissue geometric deformation under different contrasts, and differences in scanning parameters, spatial positional deviations inevitably exist between the target modality and the auxiliary modality. Traditional single-shot registration methods often struggle to handle complex non-rigid deformations and are prone to accumulating and amplifying registration errors in subsequent networks. Therefore, referring to... Figure 3 In this embodiment, a lightweight residual registration module is embedded in each iteration of the image domain iterative refinement network to achieve step-by-step, dynamic, and precise alignment of cross-modal anatomical structures.

[0085] Reference Figure 3 In the image domain iterative thinning network, let the first... The input for each iteration is the target modality image. and auxiliary modal images First, the residual registration module concatenates the two data points along the channel dimension. After initial convolutional feature mixing, the residual prediction module is used to predict the two-dimensional displacement field. With confidence mask :

[0086] ;

[0087] In the formula, This indicates a channel-level concatenation operation. The kernel size is Convolution operation, These are the learnable parameters for the residual prediction module. This indicates the operation of the residual prediction module.

[0088] The residual prediction module consists of multiple stacked residual blocks. By introducing an identity mapping, it effectively alleviates the gradient vanishing problem during deep network training and ensures the stability of the displacement field prediction process.

[0089] The formula for calculating single-layer feature transfer in a residual block is:

[0090] ;

[0091] In the formula, The input features for the residual block are... For the corresponding output features, For activation function, This is the output mapping layer for the residual block.

[0092] Next, a fully differentiable image alignment is performed using a spatial transformation network. The spatial transformation network can spatially address each pixel of the target grid based on the predicted two-dimensional displacement field, establishing a dense coordinate mapping relationship between the source image and the target image.

[0093] First, a coordinate mapping relationship is established based on the two-dimensional displacement field. Let the pixel coordinate vector of the target modal image be... Its corresponding auxiliary modal image coordinate vector The matrix transformation formula is as follows:

[0094] ;

[0095] Subsequently, auxiliary modal images To further improve the robustness of subsequent feature fusion, grid resampling and bilinear interpolation are performed. This step innovatively combines the predicted confidence mask with the interpolation resampling. To modulate local deformation weights, thereby suppressing misguidance in poorly registered regions and obtaining aligned auxiliary modal images. The calculation formula for the interpolation transformation is as follows:

[0096] ;

[0097] In the formula, , For spatial pixel discrete indexing of auxiliary modal images, Indicates the confidence level mask. Represents the pixel coordinates in the target modal image. This represents the pixel coordinates in the auxiliary modality image.

[0098] Step S2.2: After completing cross-modal spatial alignment, deep feature extraction and progressive fusion are required. Traditional convolutional neural networks, due to their limited receptive field, struggle to establish global self-similarity of non-local anatomical structures in magnetic resonance images.

[0099] Target modality image Aligned auxiliary modal image Two backbone networks are input, with the backbone network structure being Swin-Transformer U-Net. This backbone network combines the multi-scale local context aggregation capability of U-Net with the global long-distance dependency modeling capability of Swin-Transformer based on the moving window mechanism to achieve hierarchical feature extraction and recovery. The two backbone networks form a target branch and an auxiliary branch, respectively. The two branches have the same structure but independent parameters, and output the intermediate features of the target branch. Intermediate features of auxiliary branches The calculation formula is as follows:

[0100] ;

[0101] ;

[0102] In the formula, and These are the backbone networks for the target branch and auxiliary branches, respectively. and Let be the network parameters for the target branch and the auxiliary branch, respectively.

[0103] The Swin-Transformer Unet backbone network adopted in the initial stage The image patch embedding layer, with its initial channel dimension The value is set to 96. In the encoder section, the number of heads in the multi-head self-attention mechanism at each stage is configured as {2, 4, 8}, and the corresponding moving window sizes are set as {1, 2, 8}. The bottleneck stage of the backbone network consists of two consecutive Swin-Transformer blocks with 16 attention heads and a window size of 8. In the symmetrical decoder section, the backbone network performs three 2x upsampling operations sequentially, and finally restores the feature map to the original image resolution through a 4x upsampling layer. To achieve global dependency modeling in the feature processing at each of the above levels, the core computational unit of the backbone network, the Swin-Transformer block, performs multi-head self-attention operations based on the moving window mechanism. The specific calculation formula within its local window is as follows:

[0104] ;

[0105] In the formula, , , These represent the query matrix, key matrix, and value matrix within the local window, respectively. Representing feature dimension, This represents the relative position offset parameter; This represents the self-attention operation. This represents the Softmax function.

[0106] Subsequently, given that the features of the auxiliary branch contain complete prior structural information, while the features of the target branch suffer from aliasing artifacts and missing information, this step will extract the obtained intermediate features. and Alignment is performed in both spatial and channel dimensions, and pixel-by-pixel subtraction is used to calculate the spatial feature difference between the two modalities, thereby accurately locating the missing information and artifact distribution of the target modality. A cross-modal difference map is then obtained, and convolution is used to extract the corrected residuals. The superimposed image onto the target branch yields the iterated target modal image. This serves as the input for the next round; through the extraction and superposition of cross-modal residuals, the network can accurately remove aliasing artifacts in the target modality and progressively repair missing high-frequency anatomical details. After completing all... After one iteration, the target modal images output from all iterations are fused to obtain the image domain reconstruction features. The process is as follows:

[0107] ;

[0108] ;

[0109] ;

[0110] In the formula, and These represent the kernel size as follows: and Convolution operations; This represents a channel-level splicing operation. express The target modal image output in each iteration. In this embodiment, the total number of iterations in the image domain is... The value is 4.

[0111] Step S2.3: Artifacts caused by undersampling manifest as structural aliasing in the image domain, but as the absence of specific high-frequency components in the corresponding frequency domain (K-space). To recover image details from a physical mechanism, refer to... Figure 4 This step introduces a K-space posterior correction module to reconstruct features in the image domain. Secondary refinement is performed at the frequency domain level.

[0112] First, features are reconstructed in the image domain using Fast Fourier Transform. Mapping to the frequency domain yields a complex tensor. To enable conventional convolutional neural networks to process complex data, it is decomposed into real and imaginary parts, and then concatenated along the channel dimension to form a realized frequency domain input tensor. .

[0113] Next, this step utilizes a frequency domain feature extractor consisting of two convolutional layers and a LeakyReLU activation function to capture the local spectral structure in the K-space and extract shallow frequency domain features. Subsequently, The data is fed into the backbone network for global dependency modeling.

[0114] Subsequently, the features output by the backbone network are changed from The convolution mapping is converted into a two-channel frequency domain output and recombined into a physically meaningful complex tensor. Finally, the inverse Fourier transform is used to remap it back to the image domain, yielding frequency-domain corrected features with restored high-frequency details. Through this a priori correction process, the network can compensate for the high-frequency energy lost during image domain reconstruction at the frequency domain level, resulting in sharper and clearer edge textures in the final reconstructed image. Its internal feature mapping process is shown in the following equation:

[0115] ;

[0116] ;

[0117] ;

[0118] In the formula, Use the LeakyReLU activation function; The parameter is Feature mapping network; and These represent the output features of the backbone network, respectively. The real and imaginary parts, It is the imaginary unit.

[0119] Step S2.4: To fully integrate the complementary information from the image domain and the frequency domain, refer to Figure 5 In the adaptive fusion module, features are reconstructed in the image domain. and frequency domain correction features As input, the two are first concatenated along the channel dimension, and then the dual-domain fused features are obtained through a convolutional mixing block. Subsequently, a channel attention mechanism was introduced to calculate the adaptive weight coefficients of each channel. The calculation formula is as follows:

[0120] ;

[0121] ;

[0122] ;

[0123] In the formula, Use the GELU activation function; This represents the channel descriptor after global average pooling; and These are the height and width of the image, respectively. and These are the pixel indices of the image in the height and width directions, respectively; and These are the parameters for the fully connected layer. This is the Sigmoid activation function.

[0124] Finally, the adaptive weighting coefficients are multiplied channel-by-channel by the dual-domain fusion features, and then combined with the image domain reconstruction features. Perform residual connections to output the final magnetic resonance image reconstruction result. :

[0125] ;

[0126] In the formula, This indicates a channel-by-channel multiplication operation.

[0127] Step S3: Design the total loss function for optimizing the multimodal MRI reconstruction network model, and preset the parameters of the multimodal MRI reconstruction network model;

[0128] The specific steps of step S3 are as follows:

[0129] Constructing a total loss function for optimizing a multimodal MRI reconstruction network model The total loss function is composed of image domain deep supervised loss. Frequency domain consistency loss And the network parameter regularization terms are jointly constructed;

[0130] Specifically, the image domain deep supervised loss is used to provide gradient guidance in each iteration. Calculate all The mean absolute error between the image domain output and the fully sampled ground truth image in each iteration, and the image domain deep supervision loss. The calculation formula is as follows:

[0131] ;

[0132] In the formula, For the first The target modal image output by the round of iterations in coordinates Pixel value at that location, These are the pixel values ​​of the corresponding fully sampled target modality ground truth image. and These are the height and width of the image, respectively. This represents the total number of iterations.

[0133] To avoid conflicts with data from physical sampling points, the frequency domain consistency loss... Frequency domain consistency loss occurs when error constraints are applied only to the corrected frequency domain data at unsampled frequency locations. The calculation formula is as follows:

[0134] ;

[0135] In the formula, The coordinates output by the K-space posterior correction module Frequency domain data at that location, The coordinates are obtained by Fourier transform of the target modal image. The actual frequency domain data at that location, This is a binarized undersampling mask matrix. Indicates the undersampling mask matrix in coordinates The value at each sampling point is 1, and the value at each unsampled point is 0.

[0136] The total loss function The final calculation formula is as follows:

[0137] ;

[0138] In the formula, Represents the set of all learnable parameters. For each of the individual weight parameters, and These are the hyperparameters for controlling frequency domain consistency and weight decay, respectively. The remaining training hyperparameters for the multimodal MRI reconstruction network model include setting the initial learning rate, the total number of iterations (Epochs), and the parameters of the Adam optimizer.

[0139] Step S4: Input the magnetic resonance images in the training set into the multimodal MRI reconstruction network model, and use the Adam optimizer to iteratively optimize and update the parameters of the multimodal MRI reconstruction network model according to the total loss function.

[0140] The specific steps of step S4 are as follows:

[0141] Step S4.1: Initialize the parameters in the multimodal MRI reconstruction network model; select magnetic resonance image pairs from the training set and input them into the multimodal MRI reconstruction network model to obtain the predicted magnetic resonance image reconstruction results;

[0142] Step S4.2: Based on the total loss function constructed in step S3, calculate the gradient using the backpropagation algorithm, and use the Adam optimizer to update the learnable parameters in the multimodal MRI reconstruction network model;

[0143] Step S4.3: For all magnetic resonance image pairs in the entire training set, repeat the operations of steps S4.1 and S4.2 until the preset number of iterations is completed. In this embodiment, the number of iterations is set to 50.

[0144] Step S5: Train and test the multimodal MRI reconstruction network model until it reaches the preset convergence condition. Use the trained multimodal MRI reconstruction network model to achieve end-to-end reconstruction from undersampled MRI images to complete MRI images.

[0145] The specific steps of step S5 are as follows:

[0146] Step S5.1: Input the magnetic resonance image pairs in the test set into the multimodal MRI reconstruction network model, and calculate the total loss function value and the preset image quality evaluation index;

[0147] Step S5.2: If the total loss function value decreases to below the preset threshold, or the change in the image quality assessment index is lower than the preset threshold, then the multimodal MRI reconstruction network model is determined to have converged, and step S5.3 is executed; otherwise, return to step S4 to continue training.

[0148] Step S5.3: Input the target modal image to be processed and the corresponding auxiliary modal image into the converged multimodal MRI reconstruction network model, and output the magnetic resonance image reconstruction results.

[0149] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction.

[0150] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction.

[0151] To demonstrate the effectiveness of this invention, comparative and ablation experiments were conducted. First, the dataset and training details are introduced. Then, comparative experimental results of different algorithms on the dataset are presented. A series of ablation experiments were then implemented to evaluate the effectiveness of the residual registration module, the K-space posterior correction module, and the adaptive fusion module. The initial learning rate of the model was set to 1e-4, and the batch size was set to 4.

[0152] The method of this invention was compared with other currently popular state-of-the-art methods in the field of medical image reconstruction, and comparative experiments were conducted on the publicly available multimodal brain MRI dataset IXI. In the experiments, Cartesian sampling and radial sampling masks were used, tested at speedup ratios of 5x, 10x, and 15x. The parameter settings of the comparison models were kept identical to the default optimal settings of their open-source code. When experimenting with each baseline model, the undersampled target modality T2 and the fully sampled auxiliary modality PD were paired as inputs, consistent with the method of this invention. The qualitative visual effect comparison images of the comparative experiments are shown below. Figure 6 and Figure 7 As shown, the quantitative results of the comparative experiments are shown in Tables 1 and 2, respectively:

[0153] Table 1. Quantitative results of the comparative experiment using Cartesian sampling

[0154]

[0155] Table 2. Quantitative results of comparative experiments using radial sampling

[0156]

[0157] To verify the effectiveness of each module in the multimodal MRI reconstruction network model, ablation experiments were conducted on the IXI dataset (Cartesian 5x speedup). The experiments used a network without the residual registration module, K-space posterior correction module, and adaptive fusion module as the base model to verify the effectiveness of each module. When verifying the effectiveness of the residual registration module and the K-space posterior correction module, the performance of the base model was compared with that of the modules with the residual registration module and the module with the K-space posterior correction module added independently. When verifying the effectiveness of the adaptive fusion module, with the first two modules enabled, the reconstruction performance of the conventional fusion method (simple stitching at the channel dimension followed by a single-layer convolution) was compared with that of the adaptive fusion module, while keeping the first two modules enabled. The quantitative results of the ablation experiments are shown in Table 3. This indicates that the corresponding module will be retained. This indicates that the corresponding module has been removed.

[0158] Table 3. Quantitative results of ablation experiments

[0159]

[0160] In the evaluation metrics, PSNR, SSIM, and RMSE represent Peak Signal-to-Noise Ratio, Structural Similarity, and Root Mean Square Error, respectively. PSNR measures the degree of image distortion by calculating the mean square error between the reconstructed image and the fully sampled real image; a higher value indicates less distortion and higher reconstruction quality. SSIM comprehensively evaluates the similarity between images from three dimensions: brightness, contrast, and structure. A value closer to 1 indicates that the structural features of the reconstructed image match the real fully sampled image more closely and are less affected by localized minor noise. Furthermore, RMSE directly quantifies the absolute deviation between the reconstructed image and the real fully sampled image at the pixel level; a lower value indicates lower reconstruction error, more complete preservation of anatomical details, and higher overall fidelity.

[0161] from Figure 6 and Figure 7 As can be seen from the visualization and error plot, this invention has significant advantages in eliminating misalignment artifacts and restoring high-frequency details. Under extreme undersampling, the contrast model generally suffers from blurred edges and structural artifacts, while the tissue boundaries reconstructed by this invention are the clearest and closely match the full-sampled ground truth. Furthermore, the contrast algorithm in the error plot retains a large number of yellow-green outlines, while this invention presents the widest dark blue area (error approaching 0), intuitively demonstrating that it effectively removes structural residuals and achieves optimal visual results in preserving anatomical structures and suppressing complex artifacts.

[0162] As shown in Tables 1 and 2, the method of this invention achieves the best performance in PSNR, SSIM, and RMSE under both Cartesian and radial sampling modes and at various speedup ratios. Taking Cartesian sampling as an example, at 5x sampling, the PSNR, SSIM, and RMSE of this invention are 41.55 dB, 0.973, and 2.241, respectively; at higher speedup ratios of 10 and 15, its PSNR is significantly improved by 1.82 dB and 2.08 dB compared to the suboptimal method, respectively. Under the radial sampling mode, where artifacts are more complex, this invention also demonstrates strong robustness, with a PSNR as high as 44.32 dB at 5x sampling; at 10x and 15x, the PSNR is significantly improved by 2.02 dB and 2.23 dB compared to the suboptimal method, respectively. In summary, this invention achieves a significant performance leap compared to current mainstream algorithms, effectively improving the accuracy and detail fidelity of multimodal MRI reconstruction under complex conditions.

[0163] As shown in Table 3, after adding the residual registration module, the PSNR and SSIM of the proposed method increased by 0.51 dB (from 39.84 dB to 40.35 dB) and 0.001 (from 0.969 to 0.970), respectively, while the RMSE decreased by 0.329 (from 2.996 to 2.667). This demonstrates that the residual registration module plays a positive role in multimodal reconstruction, effectively improving the spatial consistency of cross-modal features. Adding the K-space posterior correction module alone significantly improved the PSNR by 1.04 dB (from 39.84 dB to 40.88 dB). When the residual registration module and the K-space posterior correction module are used together, the PSNR further increases to 41.27 dB, with a particularly significant improvement in details of high-frequency edge regions. Finally, after adding the adaptive fusion module, the model's PSNR and SSIM improved by 0.28 dB (from 41.27 dB to 41.55 dB) and 0.001 (from 0.972 to 0.973), respectively, while the RMSE decreased to the optimal 2.241. In summary, the results in Table 3 demonstrate that the various modules in the proposed method complement each other, resulting in a stable performance improvement for the model.

[0164] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction, characterized in that, Perform the following steps S1-S5 to complete the magnetic resonance image reconstruction: Step S1: Acquire undersampled magnetic resonance images and corresponding fully sampled magnetic resonance images. Use the undersampled magnetic resonance images as target modal images and the corresponding fully sampled magnetic resonance images as auxiliary modal images to form magnetic resonance image pairs. Divide the magnetic resonance image pairs into training and test sets according to a preset ratio. Step S2: Construct a multimodal MRI reconstruction network model based on iterative refinement and K-space posterior correction. The multimodal MRI reconstruction network model includes an image domain iterative refinement network, a K-space posterior correction module, and an adaptive fusion module. The network is iteratively refined in the input image domain using magnetic resonance images, the auxiliary modal images are spatially aligned to the target modal images, and the features of the target modal images and auxiliary modal images are extracted and iteratively refined to output the image domain reconstructed features. The K-space posterior correction module maps the image domain reconstructed features to the frequency domain, capturing the inherent properties and artifact patterns of the frequency domain to obtain frequency domain corrected features; in the adaptive fusion module, the image domain reconstructed features and the frequency domain corrected features are weighted and combined to form dual-domain fused features; Residual connections are then performed to obtain the predicted magnetic resonance image reconstruction results; Step S3: Design the total loss function for optimizing the multimodal MRI reconstruction network model, and preset the parameters of the multimodal MRI reconstruction network model; Step S4: Input the magnetic resonance images in the training set into the multimodal MRI reconstruction network model, and use the Adam optimizer to iteratively optimize and update the parameters of the multimodal MRI reconstruction network model according to the total loss function. Step S5: Train and test the multimodal MRI reconstruction network model until it reaches the preset convergence condition. Use the trained multimodal MRI reconstruction network model to achieve end-to-end reconstruction from undersampled MRI images to complete MRI images.

2. The multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction according to claim 1, characterized in that, For the magnetic resonance image pairs formed in step S1, the magnetic resonance image pairs are divided into training set and test set according to a ratio of 0.9:0.

1.

3. The multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction according to claim 1, characterized in that, The specific steps of step S2 are as follows: Step S2.1: In the image domain iterative thinning network, let the first... The input for each iteration is the target modality image. and auxiliary modal images First, the residual registration module concatenates the two data points along the channel dimension. After initial convolutional feature mixing, the residual prediction module is used to predict the two-dimensional displacement field. With confidence mask : ； In the formula, This indicates a channel-level concatenation operation. The kernel size is Convolution operation, These are the learnable parameters for the residual prediction module. This indicates the operation of the residual prediction module; The residual prediction module contains residual blocks, and the calculation formula for single-layer feature transfer is as follows: ； In the formula, The input features for the residual block are... For the corresponding output features, For activation function, For the output mapping layer of the residual block; Next, image alignment is performed using a spatial transformation network: Let the pixel coordinate vector of the target modality image be... Its corresponding auxiliary modal image coordinate vector The matrix transformation formula is as follows: ； Subsequently, auxiliary modal images Grid resampling and bilinear interpolation are performed, and local deformation weights are modulated using a confidence mask to obtain the aligned auxiliary modal image. The formula for calculating the interpolation transformation is as follows: ； In the formula, , For spatial pixel discrete indexing of auxiliary modal images, Indicates the confidence level mask. Represents the pixel coordinates in the target modal image. Represents the pixel coordinates in the auxiliary modality image; Step S2.2: Transfer the target modality image Aligned auxiliary modal image Two Swin-Transformer UNet-based backbone networks are input respectively. The two backbone networks form the target branch and the auxiliary branch, respectively. The two branches have the same structure but independent parameters. The intermediate features of the target branch are output separately. Intermediate features of auxiliary branches The calculation formula is as follows: ； ； In the formula, and These are the backbone networks for the target branch and auxiliary branches, respectively. and Here are the network parameters for the target branch and the auxiliary branch, respectively; the backbone network calculates multi-head self-attention based on the moving window mechanism to extract the corresponding intermediate features; Subsequently, the obtained intermediate features will be extracted. and Alignment is performed in both spatial and channel dimensions, and pixel-by-pixel subtraction is carried out to obtain a cross-modal difference map. Convolution is then used to extract the correction residuals. The superimposed image onto the target branch yields the iterated target modal image. As input for the next round; after completing all After one iteration, the target modal images output from all iterations are fused to obtain the image domain reconstruction features. The process is as follows: ； ； ； In the formula, and These represent the kernel size as follows: and Convolution operations; This represents a channel-level splicing operation. express The target modal image output in each iteration; Step S2.3: In the K-space posterior correction module, reconstruct features from the image domain. Complex data are obtained by mapping to the frequency domain using Fourier transform. The real and imaginary parts are decomposed and concatenated to form a two-channel real tensor. Shallow features are extracted using a frequency domain feature extractor. The output features are then input into the backbone network. Reorganized into complex tensors The frequency domain corrected features are obtained by inverse Fourier transform. The calculation formula is as follows: ； ； ； In the formula, Use the LeakyReLU activation function; The parameter is Feature mapping network; and These represent the output features of the backbone network, respectively. The real and imaginary parts, The imaginary unit; Step S2.4: In the adaptive fusion module, reconstruct features in the image domain. and frequency domain correction features As input, the two are first concatenated along the channel dimension, and then the dual-domain fused features are obtained through a convolutional mixing block. Subsequently, a channel attention mechanism was introduced to calculate the adaptive weight coefficients of each channel. The calculation formula is as follows: ； ； ； In the formula, Use the GELU activation function; This represents the channel descriptor after global average pooling. and These are the height and width of the image, respectively. and These are the discrete pixel indices of the image in the height and width directions, respectively; and These are the parameters for the fully connected layer. Use the Sigmoid activation function; Finally, the adaptive weighting coefficients are multiplied channel-by-channel by the dual-domain fusion features, and then combined with the image domain reconstruction features. Perform residual connections to output the final magnetic resonance image reconstruction result. : ； In the formula, This indicates a channel-by-channel multiplication operation.

4. The multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction according to claim 3, characterized in that, The specific steps of step S3 are as follows: Constructing a total loss function for optimizing a multimodal MRI reconstruction network model The total loss function is composed of image domain deep supervised loss. Frequency domain consistency loss And the network parameter regularization terms are jointly constructed; Image Domain Deep Supervision Loss The calculation formula is as follows: ； In the formula, For the first The target modal image output by the round of iterations in coordinates Pixel value at that location, These are the pixel values ​​of the corresponding auxiliary modality image. and These are the height and width of the image, respectively. This represents the total number of iterations. Frequency domain consistency loss The calculation formula is as follows: ； In the formula, The coordinates output by the K-space posterior correction module Frequency domain data at that location, The coordinates are obtained by Fourier transform of the target modal image. The actual frequency domain data at that location, This is a binarized undersampling mask matrix. Indicates the undersampling mask matrix in coordinates The value at each sampling point; where the value at a sampling point is 1, and the value at an unsampled point is 0. The total loss function The final calculation formula is as follows: ； In the formula, Represents the set of all learnable parameters. For each of the individual weight parameters, and These are the hyperparameters for controlling frequency domain consistency and the hyperparameters for weight decay, respectively.

5. The multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction according to claim 4, characterized in that, The specific steps of step S4 are as follows: Step S4.1: Initialize the parameters in the multimodal MRI reconstruction network model; select magnetic resonance image pairs from the training set and input them into the multimodal MRI reconstruction network model to obtain the predicted magnetic resonance image reconstruction results; Step S4.2: Based on the total loss function constructed in step S3, calculate the gradient using the backpropagation algorithm, and use the Adam optimizer to update the learnable parameters in the multimodal MRI reconstruction network model; Step S4.3: For all magnetic resonance image pairs in the entire training set, repeat the operations of steps S4.1 and S4.2 until the preset number of iterations of training are completed.

6. The multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction according to claim 5, characterized in that, The specific steps of step S5 are as follows: Step S5.1: Input the magnetic resonance image pairs in the test set into the multimodal MRI reconstruction network model, and calculate the total loss function value and the preset image quality evaluation index; Step S5.2: If the total loss function value decreases to below the preset threshold, or the change in the image quality assessment index is lower than the preset threshold, then the multimodal MRI reconstruction network model is determined to have converged, and step S5.3 is executed; otherwise, return to step S4 to continue training. Step S5.3: Input the target modal image to be processed and the corresponding auxiliary modal image into the converged multimodal MRI reconstruction network model, and output the magnetic resonance image reconstruction results.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements each step of the multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the multimodal MRI reconstruction method based on iterative refinement and K-space posterior correction as described in any one of claims 1 to 6.

Images