Double-domain Swin Mama-based generative adversarial network MRI reconstruction method

Abstract

The invention discloses a double-domain Swin Mama-based generative adversarial network MRI (Magnetic Resonance Imaging) reconstruction method, which belongs to the technical field of image processing, and comprises the following steps: S1, collecting / sorting MRI full-sampling data, and carrying out k-space undersampling; s2, inputting under-sampling k space data into a frequency domain generator GK, processing the frequency domain data by using multiple layers of Swin Mamba Block, and converting a frequency domain completion structure into a preliminary reconstruction image I1 through inverse Fourier transform; s3, receiving the processed image, inputting the image into an image domain generator GI, and performing image restoration by using Swin Mama Block to obtain a final reconstructed image I2; s4, inputting the reconstructed image into a discriminator, and guiding the generator to update towards a vivid and complete-structure direction; and S5, mapping the repaired image back to the frequency domain generator GK again to provide synchronous constraints of global and local information, thereby improving the MRI reconstruction precision and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a generative adversarial network MRI reconstruction method based on a dual-domain Swin Mamba. Background Art

[0002] Magnetic resonance imaging (MRI) is currently an important diagnostic imaging method in medicine, capable of acquiring high-resolution images with good soft tissue contrast. However, MRI often requires long scan times, leading to patient discomfort, increased motion artifacts such as breathing or heartbeats, and potentially reduced MRI examination throughput.

[0003] To shorten scan time, undersampling is often used to reduce the number of k-space sampling points, thereby achieving fast imaging. However, undersampling can lead to severe reconstruction errors and artifacts, requiring efficient reconstruction algorithms to compensate for the lost information.

[0004] Early fast MRI reconstruction approaches often leveraged compressed sensing theory, introducing sparse priors in specific transform domains (such as wavelet and Fourier transforms) and combining them with undersampling strategies to reconstruct high-quality images. These methods performed well at relatively low speedup ratios, but as speedup increased, the images became more susceptible to noise and noticeable artifacts. Furthermore, the iterative optimization process was often time-consuming, making it difficult to meet clinical real-time requirements.

[0005] Parallel imaging (PI) methods use multiple coils to simultaneously acquire signals at different spatial locations and use sensitivity mapping to compensate for missing data. However, as the number of coils or the acceleration factor increases, PI methods face challenges such as inaccurate sensitivity, noise amplification, and reconstruction artifacts, limiting their imaging quality at high acceleration rates.

[0006] With the rapid development of deep learning, deep models based on convolutional neural networks (CNNs) and generative adversarial networks (GANs) have emerged as promising candidates for medical image reconstruction. These models can leverage massive amounts of data to learn end-to-end mappings from undersampled images to fully sampled images, significantly improving reconstruction speed and image quality. However, CNNs are limited by the receptive field size of their convolution kernels, making it difficult to simultaneously capture both long-range global dependencies and detailed structures within an image. Furthermore, traditional Transformers, due to the high complexity of their attention mechanisms when processing high-resolution images and the requirement for larger amounts of training data, also face challenges in training and computational cost.

[0007] In the field of MRI reconstruction, to better utilize information in both the image and frequency domains (k-space), some studies have proposed dual-domain joint reconstruction strategies, decomposing the image reconstruction problem into an interactive restoration process in the frequency and image domains. Other work has explored incorporating various network structures (such as U-Net and Transformer) into generative adversarial networks to simultaneously learn local and global features. However, achieving high-fidelity and robust reconstruction still faces many challenges: The recently emerged Swin Transformer significantly reduces the computational complexity of traditional self-attention on high-resolution images by segmenting the image into non-overlapping windows and performing hierarchical feature extraction at different stages, while still capturing both global and local image dependencies. Mamba (a "sequence model based on structured state space") also offers advantages in processing long sequences of data, including high efficiency, parallelization, and reduced complexity, providing new insights into long-range dependency modeling in medical imaging, particularly high-resolution MRI.

[0008] However, directly using a single Swin Transformer or Swin Mamba for MRI reconstruction may still face problems such as unstable convergence and insufficient detail restoration if there is a lack of dual constraints in the image domain and frequency domain.

[0009] In response to the above problems, the present invention designs and manufactures a generative adversarial network MRI reconstruction method based on the dual-domain Swin Mamba to overcome the above defects. Summary of the Invention

[0010] In order to solve the problems existing in the prior art, the present invention provides a generative adversarial network MRI reconstruction method based on the dual-domain Swin Mamba, which can improve the accuracy and efficiency of MRI reconstruction by performing multiple repair interactions in the frequency domain and image domain and introducing the long-distance feature capture capability based on Swin Mamba.

[0011] To achieve the above objectives, the present invention adopts the following technical solution: a dual-domain Swin Mamba-based generative adversarial network MRI reconstruction method, comprising the following steps: S1. Collect / organize full MRI sampling data and perform k-space undersampling based on the required acceleration ratio; S2. Undersampled k-space data is input to the frequency domain generator G K , frequency domain generator G K The frequency domain data is processed using multi-layer Swin MambaBlock, and then the frequency domain completion structure is inverse Fourier transformed to convert the frequency domain completion result into the preliminary reconstructed image I1; S3. Receive frequency domain generator G KProcessed image and input into image domain generator G I , use Swin MambaBlock to perform image restoration and obtain the final reconstructed image I2; S4. Image Domain Generator G I Reconstruct the image input discriminator to guide the generator to update in a realistic and structurally complete direction; S5. Map the repaired image back to the frequency domain generator G K , to provide synchronization constraints between global and local information.

[0012] Preferably, the k-space data after frequency domain processing is completed ,in represents the undersampled k-space data, represents the learnable parameters of the frequency domain generator.

[0013] Preferably, the output of the image domain generator is ,in , about to Inverse Fourier transform, represents the learnable parameters of the image domain generator; The final reconstructed image I2 is compared with the image corresponding to the real full sampling data to calculate the PSNR value.

[0014] Preferably, the discriminator adopts a patch discrimination scheme, which divides the image into multiple small blocks and judges them separately to improve the sensitivity to local artifacts.

[0015] Preferably, the frequency domain generator G K and image domain generator G I The backbone network is composed of several Swin MambaBlocks.

[0016] Preferably, the Swin Mamba Block specifically includes the following processes: (1) Swin Transformer divides the feature map into non-overlapping windows and divides the image into several sub-blocks of M×M size; (2) Calculate self-attention within the window and Mamba calculation after sequence expansion; (3) Through multiple residual connections and LayerNorm components; (4) Mapping the channels through two layers of fully connected MLP with activation function; (5) Using the residual connection again, the features processed by MLP are added to the previous features to obtain the final output features.

[0017] Preferably, Swin Mamba Block performs a sliding window / shift window operation on odd layers, that is, shifting the feature map and then splitting the window; The layered downsampling operation is performed in the Swin Mamba Block.

[0018] Preferably, after several Swin Mamba Blocks, a Patch Merging / PatchSplitting sampling operation is inserted.

[0019] Preferably, the parameters of the dual-domain generative adversarial network are optimized by training loss design: A. By calculating the frequency domain loss L K To measure the generated k-space data Compared with the real fully sampled k-space K full The difference between the two, adjust the frequency domain generator G K Learnable parameters of the multi-layer Swin Mamba Block ; B. Using image domain loss L I Based on the MSE, SSIM, and MAE indicators, the generated image is compared with the real image, and the L I Size, back-propagation adjusts the image domain generator G I The learnable parameters ; C. According to L adv Adjust the parameters of the generator and discriminator to make the generated images more "realistic" and structurally consistent; D.L K , L I and L adv According to the weights of α, β, and γ, , by minimizing , and adjust the parameters of the frequency domain generator, image domain generator and discriminator at the same time to balance the effects of frequency domain, image domain and adversarial training.

[0020] Preferably, the frequency domain loss L is calculated K In the example, we use the L1 norm Robust to outliers, using L2 norm Pay attention to the overall error size.

[0021] The invention is beneficial in that: 1. This paper implements dual constraints in the image domain and frequency domain through generator design and Swin Mamba Block structure training loss design, allowing the reconstructed image to take into account both global and local features, thereby improving the reconstruction quality.

[0022] 2. Frequency Domain Generator G of the Present Invention K The undersampled k-space data is processed, the missing information is supplemented by multiple layers of Swin Mamba Block, and the inverse Fourier transform is converted to the image domain. Image domain generator G I Receive its output, further repair details, remove artifacts, and then map it back to the frequency domain. This interaction enables dual-domain information sharing and complementarity, such as G I When repairing image details, the results are fed back to the frequency domain to provide more accurate information for subsequent reconstruction.

[0023] 3. Both generators use the Swin Mamba Block in their backbone networks. The Swin Transformer's windowed attention captures both fine-grained local features and long-range global features, while the Mamba state-space sequence model efficiently captures long-range dependencies. In the frequency domain, it better captures global and local frequency information in undersampled data; in the image domain, it helps extract image details and structural features, strengthening dual-domain constraints at the feature extraction level.

[0024] 4. Frequency domain loss L K Use L1 / L2 norm to measure the difference between the generated k-space and the real full-sampled k-space, so that the frequency domain generator can generate k-space data closer to the real one in the frequency domain. Image domain loss L I Based on indicators such as MSE, SSIM, and MAE, the generated image is compared with the real image to guide the image domain generator to generate more accurate images in the image domain. adv Through the GAN framework, the generated images are made more realistic and structurally consistent, and the comprehensive loss function adjusts each loss term, thereby strengthening the dual-domain constraints at the training level.

[0025] 5. The present invention uses dual-domain interaction between the frequency domain and the image domain. For newly undersampled k-space, the frequency domain generator uses Swin Mamba Block to initially complete the missing information of the undersampled k-space data, and the image domain generator further repairs details and removes artifacts. By complementing the global information in the frequency domain and the local texture information in the image domain, the final reconstructed image maintains overall consistency while being accurate and realistic in local texture, achieving a balance between the global and local, and effectively improving the reconstruction quality of MRI images. During MRI reconstruction, doctors can observe the details of the diseased tissue more clearly, improving the accuracy of diagnosis.

[0026] 6. During image reconstruction, this method divides the feature map into multiple windows, calculates self-attention within each window, and uses a sliding / shifting window mechanism and layered downsampling to focus on local regions. Fine structures such as gray matter / white matter boundaries, lesion outlines, and organ edges can be accurately captured, preserving rich details in the reconstructed MRI image, improving the image's detail fidelity and meeting the high clinical requirements for image detail accuracy, helping doctors more accurately observe and diagnose conditions.

[0027] 7. By serializing window features and capturing long-range dependencies using a state-space model, Mamba achieves higher computational efficiency and lower complexity in high-resolution scenarios. This means that when processing large MRI data, it can reduce computing resource consumption while maintaining effective modeling, improving reconstruction efficiency and making the algorithm more feasible for practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flowchart of a dual-domain Swin Mamba-based generative adversarial network MRI reconstruction method; Figure 2 A flow chart of the Swin Mamba Block of the present invention; Figure 3 This is a graph showing the changes in MSE, SSIM, and MAE over the training rounds; Figure 4 is the image domain loss L I Plot of changes over training rounds. DETAILED DESCRIPTION

[0029] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0030] like Figure 1 As shown, a dual-domain Swin Mamba-based generative adversarial network MRI reconstruction method includes the following steps: S1. Collect / organize full MRI sampling data and perform k-space undersampling based on the required acceleration ratio; S2. Undersampled k-space data is input to the frequency domain generator G K , frequency domain generator G K The frequency domain data is processed using a multi-layer Swin MambaBlock. Specifically, the frequency domain line-by-line (or multi-directional) sequence is expanded and state-space modeling is performed to obtain a preliminary completion of the missing information. The frequency domain completion structure is then inverse Fourier transformed and the frequency domain completion result is converted into a preliminary reconstructed image I1. This process attempts to supplement the missing information by processing the undersampled k-space data. S3. Receive frequency domain generator G KProcessed image and input into image domain generator G I , use Swin MambaBlock to perform image restoration, optimize image quality, and obtain the final reconstructed image I2; S4. Image Domain Generator G I Reconstruct the image input discriminator. The discriminator uses a lightweight multi-layer convolutional structure or a supplementary attention module to determine whether the input image or the corresponding frequency domain data is a true full-sampled image, guiding the generator to update in the direction of realistic and structural integrity. The discriminator adopts a patch discrimination scheme, dividing the image into multiple small blocks and judging them separately, thereby improving sensitivity to local artifacts. S5. Map the repaired image back to the frequency domain generator G K , using the adversarial training feedback mechanism, and the frequency domain generator G K Collaborative adjustments are made to provide synchronization constraints between global and local information.

[0031] In the above steps, the frequency domain generator G K The undersampled k-space data is processed, the missing information is supplemented by multiple layers of Swin Mamba Block, and the inverse Fourier transform is converted to the image domain. Image domain generator G I Receive its output, further repair details, remove artifacts, and then map it back to the frequency domain. This interaction enables dual-domain information sharing and complementarity, such as G I When repairing image details, the results are fed back to the frequency domain to provide more accurate information for subsequent reconstruction.

[0032] Specifically, the k-space data after frequency domain processing is completed ,in represents the undersampled k-space data, represents the learnable parameters of the frequency domain generator.

[0033] Output of the image domain generator ,in , about to Inverse Fourier transform, represents the learnable parameters of the image domain generator; The final reconstructed image I2 is compared with the image corresponding to the true fully sampled data to calculate the PSNR value, or Peak Signal-to-Noise Ratio. PSNR is a measure of image quality, often used to compare the degree of similarity between a reconstructed image and the original true image. A higher PSNR value indicates a closer reconstructed image to the original image, and thus better image quality.

[0034] ,

[0035] Where n is the number of bits in the image pixel value. For example, for an 8-bit grayscale image, n = 8, and the pixel value range is 0 - 255.

[0036] MSE (Mean Squared Error) reflects the average squared error between the corresponding pixel values ​​of the reconstructed image and the original image. A smaller MSE value indicates a smaller image difference. Substituting this into the formula, a smaller MSE value indicates a larger PSNR value, indicating higher image quality.

[0037] ,

[0038] Where N: the total number of image pixels; : The value of the i-th pixel in the original image.

[0039] : Reconstructs the value of the i-th pixel in the image. The average squared error is obtained by squaring the difference between each corresponding pixel and accumulating it, then dividing it by the total number of pixels.

[0040] The present invention uses dual-domain interaction between the frequency domain and the image domain. For newly undersampled k-space, the frequency domain generator uses Swin Mamba Block to initially complete the missing information of the undersampled k-space data, and the image domain generator further repairs details and removes artifacts. By complementing the global information in the frequency domain and the local texture information in the image domain, the final reconstructed image maintains overall consistency while being accurate and realistic in local texture, achieving a balance between the global and local, and effectively improving the reconstruction quality of MRI images. During MRI reconstruction, doctors can observe the details of the diseased tissue more clearly, improving the accuracy of diagnosis.

[0041] Frequency domain generator G K and image domain generator G I The backbone network is composed of several Swin Mamba Blocks.

[0042] Swin Mamba Block specifically includes the following processes: Figure 2 As shown: (1) Swin Transformer divides the feature map into non-overlapping windows and divides the image into several sub-blocks of M×M size; (2) Calculate self-attention within each window. The self-attention mechanism allows the model to focus on the relationship between features at different positions within the window, which helps to extract local feature information within the window. For example, in an image, each window may correspond to a local area of ​​the image, and self-attention can be used to explore the connection between pixels within this area.

[0043] After sequence expansion, Mamba performs computations by serializing window features and capturing long-range dependencies using a state-space model. Specifically, the features within each window are expanded into a sequence in a specific order, and these sequences are then input into the Mamba module. Based on the principles of the state-space model, Mamba efficiently captures long-range dependencies between different positions in the sequence. This means that even if features are distant in the original data, Mamba can detect connections between them, such as semantic connections between distant regions in an image. This is more efficient than traditional self-attention.

[0044] (3) Through multiple residual connections and LayerNorm components, this method helps the network learn and maintains the integrity of information; (4) The channels are mapped through two layers of fully connected MLP with activation functions (such as GELU). Through nonlinear transformation, the model's ability to abstractly express features is enhanced, allowing the model to learn more complex feature representations. (5) Residual connections are used again to make the network easier to train. The features processed by MLP are added to the previous features to obtain the final output features.

[0045] In the above process, the Swin Mamba Block performs a sliding / shifting window operation on odd-numbered layers. This shifts the feature map and then splits the window. This allows features originally in different windows to enter the same window. The purpose is to allow information from different windows to be integrated, thereby covering more global information and preventing the model from focusing only on local features. The Swin Mamba Block performs layered downsampling operations. Layered downsampling is to downsample feature maps at different layers of the network, reducing the resolution while increasing the receptive field, and obtaining features of different scales at different layers. This allows us to focus on local fine features while also processing global long-distance feature relationships.

[0046] The present invention preferably inserts Patch Merging / Patch Splitting sampling operations after several Swin Mamba Blocks. These operations allow the model to learn features at different scales and adapt to the requirements of different resolutions, thereby improving the model's ability to extract image features and better processing data of different sizes and complexities.

[0047] In summary, both generators use the Swin Mamba Block in their backbone networks. The Swin Transformer's windowed attention captures both fine-grained local and long-range global features, while the Mamba state-space sequence model efficiently captures long-range dependencies. In the frequency domain, it better captures global and local frequency information in undersampled data; in the image domain, it helps extract image details and structural features, strengthening dual-domain constraints at the feature extraction level.

[0048] The present invention also optimizes the parameters of the dual-domain generative adversarial network through training loss design, specifically: A. By calculating the frequency domain loss L K To measure the generated k-space data Compared with the real fully sampled k-space K full If the difference is large, then according to L K Calculate the gradient and adjust the frequency domain generator G K Learnable parameters of the multi-layer Swin Mamba Block , so that the generated k-space data is closer to the real data. Calculate the frequency domain loss L K In the example, we use the L1 norm Robust to outliers, using L2 norm Pay attention to the overall error size, and select or mix them according to your needs; B. Using image domain loss L I Based on the MSE, SSIM, and MAE indicators, the generated image is compared with the real image, and the L I Size, back-propagation adjusts the image domain generator G I The learnable parameters , optimize image detail restoration and artifact removal effects, such as Figure 3 As shown in , MSE, SSIM, and MAE change with the number of training rounds (Epochs). As the number of training rounds increases, each indicator tends to be better; Figure 4 As shown, the image domain loss L I The overall trend of changes in training rounds is a rapid decline followed by stability. The model parameters are continuously optimized and gradually converge to a stable state in the later stage. C. As the discriminator judges the authenticity of the data, the generator tries to deceive the discriminator, and the discriminator tries to identify it accurately. In this confrontation process, according to L adv Adjust the parameters of the generator and discriminator to make the generated image more "realistic" and structurally consistent. The discriminator should make L adv Maximize and improve the discrimination ability, the two compete with each other to optimize network performance; D.L K , L I and Ladv According to the weights of α, β, and γ, , by minimizing , while adjusting the parameters of the frequency domain generator, image domain generator and discriminator, balancing the effects of frequency domain, image domain and adversarial training, and minimizing , and at the same time adjust the parameters of the frequency domain generator, image domain generator and discriminator to balance the effects of frequency domain, image domain and adversarial training, so that the overall network is optimized towards generating high-quality reconstructed images.

[0049] In summary, through the training loss design, the frequency domain loss L K Use L1 / L2 norm to measure the difference between the generated k-space and the real full-sampled k-space, so that the frequency domain generator can generate k-space data closer to the real one in the frequency domain. Image domain loss L I Based on indicators such as MSE, SSIM, and MAE, the generated image is compared with the real image to guide the image domain generator to generate more accurate images in the image domain. adv Through the GAN framework, the generated images are made more realistic and structurally consistent, and the comprehensive loss function adjusts each loss term, thereby strengthening the dual-domain constraints at the training level.

[0050] It should be understood that the purpose of these embodiments is only to illustrate the present invention and is not intended to limit the scope of protection of the present invention. In addition, it should also be understood that after reading the technical content of the present invention, those skilled in the art may make various changes, modifications and / or variations to the present invention, and all of these equivalent forms also fall within the scope of protection defined by the claims appended hereto.

Claims

1. A dual-domain Swin Mamba-based generative adversarial network MRI reconstruction method, characterized by: The following steps are involved: S1. Collect / organize full MRI sampling data and perform k-space undersampling based on the required acceleration ratio; S2. Undersampled k-space data is input to the frequency domain generator G K , frequency domain generator G K The frequency domain data is processed using multi-layer Swin MambaBlock, and then the frequency domain completion structure is inverse Fourier transformed to convert the frequency domain completion result into the preliminary reconstructed image I1; S3. Receive frequency domain generator G K Processed image and input into image domain generator G I , use Swin Mamba Block to perform image restoration and obtain the final reconstructed image I2; S4. Image Domain Generator G I Reconstruct the image input discriminator to guide the generator to update in a realistic and structurally complete direction; S5. Map the repaired image back to the frequency domain generator G K , to provide synchronization constraints between global and local information.

2. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 1, characterized in that: K-space data after frequency domain processing and completion ,in represents the undersampled k-space data, represents the learnable parameters of the frequency domain generator.

3. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 1, characterized in that: Output of the image domain generator ,in , about to Inverse Fourier transform, represents the learnable parameters of the image domain generator; The final reconstructed image I2 is compared with the image corresponding to the real full sampling data to calculate the PSNR value.

4. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 1, characterized in that: The discriminator adopts a patch discrimination scheme, which divides the image into multiple small blocks and judges them separately to improve the sensitivity to local artifacts.

5. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 1, characterized in that: The frequency domain generator G K and image domain generator G I The backbone network is composed of several Swin Mamba Blocks.

6. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 1, characterized in that: The Swin Mamba Block specifically includes the following processes: (1) Swin Transformer divides the feature map into non-overlapping windows and divides the image into several sub-blocks of M×M size; (2) Calculate self-attention within the window and Mamba calculation after sequence expansion; (3) Through multiple residual connections and LayerNorm components; (4) Mapping the channels through two layers of fully connected MLP with activation function; (5) Using the residual connection again, the features processed by MLP are added to the previous features to obtain the final output features.

7. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 6, characterized in that: Swin Mamba Block will perform sliding / shifting window operations on odd layers, that is, translating the feature map and then splitting the window; The layered downsampling operation is performed in the Swin Mamba Block.

8. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 6, characterized in that: After several Swin Mamba Blocks, insert Patch Merging / Patch Splitting sampling operations.

9. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 1, characterized in that: Optimize the parameters of the dual-domain generative adversarial network through training loss design: A. By calculating the frequency domain loss L K To measure the generated k-space data Compared with the real fully sampled k-space K full The difference between the two, adjust the frequency domain generator G K Learnable parameters of the multi-layer Swin Mamba Block ; B. Using image domain loss L I Based on the MSE, SSIM, and MAE indicators, the generated image is compared with the real image, and the L I Size, back-propagation adjusts the image domain generator G I The learnable parameters ; C. According to L adv Adjust the parameters of the generator and discriminator to make the generated images more "realistic" and structurally consistent; D.L K , L I and L adv According to the weights of α, β, and γ, , by minimizing , and adjust the parameters of the frequency domain generator, image domain generator and discriminator at the same time to balance the effects of frequency domain, image domain and adversarial training.

10. The method for MRI reconstruction based on dual-domain Swin Mamba generative adversarial network according to claim 9, characterized in that: Calculate the frequency domain loss L K In the example, we use the L1 norm Robust to outliers, using L2 norm Pay attention to the overall error size.

Images