Multi-contrast MRI (Magnetic Resonance Imaging) joint reconstruction method, system, equipment and medium

Abstract

The invention discloses a multi-contrast MRI joint reconstruction method, system and device and a medium, and relates to the technical field of medical imaging and deep learning, and the method comprises the following steps: collecting under-sampling MRI image data of different contrasts, and generating an optimization objective function based on a plurality of contrasts; decomposing the optimization objective function into a first sub-problem related to an auxiliary variable and a second sub-problem related to an objective variable; and alternately solving the first sub-problem and the second sub-problem in sequence to obtain a plurality of optimal MRI reconstructed images. According to the method, feature interaction of multi-contrast data is carried out in a spatial domain and a frequency domain, so that efficient complementation and collaborative modeling of multi-contrast features are realized, and the problem of insufficient utilization of information between contrasts in a traditional method is solved. By sensing the characteristics of each contrast, targeted prompts can be generated to guide the reconstruction process. Therefore, the quality of low-quality contrast is improved, and the overall reconstruction result is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical imaging and deep learning, in particular to a multi-contrast MRI joint reconstruction method, system, device and medium. BACKGROUND

[0002] Magnetic Resonance Imaging (MRI) has become an indispensable tool in clinical diagnosis, widely recognized for its non-invasive, ultra-high spatial resolution, and excellent soft tissue contrast. Unlike other imaging techniques such as X-ray and CT (Computed Tomography), MRI does not contain ionizing radiation, making it a safe choice for repeated imaging, especially in children and elderly patients. Despite its significant advantages, MRI also faces some significant challenges, especially the longer scan time, which limits its application in certain clinical scenarios. Long scan times not only cause patient discomfort, but can also cause motion artifacts, especially in patients who are difficult to keep still. In addition, long MRI procedures are not only costly, but can also hinder the widespread use of MRI in resource-limited environments. Accelerating MRI acquisition without compromising diagnostic accuracy has become the focus of increasing numbers of researchers. One of the main methods to solve this problem is MRI image reconstruction, which aims to reconstruct high-quality images from undersampled data, significantly reducing scan time.

[0003] The emergence of deep learning methods marks a major breakthrough in the field of MRI reconstruction. These methods use powerful neural networks to learn the end-to-end mapping relationship from undersampled data to fully sampled images. The main advantage of deep learning methods is that they can learn the complex nonlinear relationship between undersampled and fully sampled images from large datasets, overcoming the limitations of traditional hand-designed methods. However, these methods usually operate as "black box" models, making it difficult to interpret them in clinical practice. With the continuous development of single-contrast MRI reconstruction technology, multi-contrast MRI reconstruction has also received increasing attention. Different contrasts can capture different aspects of tissue structure and pathology. For example, T1-weighted images may provide better anatomical details, while T2-weighted images perform well in depicting pathological regions. Multi-contrast MRI involves combining different contrast information of the same subject, such as T1-weighted images and T2-weighted images, to provide complementary diagnostic information. This approach not only takes advantage of the unique insights provided by each contrast, but also takes advantage of the fact that all contrasts capture imaging data from the same anatomical location, ensuring consistency in underlying tissue structure. By jointly utilizing multi-contrast information, multi-contrast MRI reconstruction can produce more accurate and comprehensive images that better reflect the anatomical and pathological characteristics of tissues.

[0004] Multi-contrast MRI reconstruction faces several unique challenges. Inter-contrast misalignment is one of the most prominent issues, as different contrast MRI scans often have slight misalignments due to differences in acquisition time, patient motion, and scanner settings. These misalignments can significantly degrade the quality of the reconstructed images. Traditional multi-contrast MRI reconstruction methods usually rely on registration techniques to align the images before reconstruction, but this can be computationally expensive and difficult to perform in real-time clinical settings. In recent years, deep learning models have been increasingly applied to multi-contrast reconstruction tasks, which can utilize multi-contrast information without the need for explicit pre-alignment. These multi-contrast deep learning methods aim to fully utilize the complementary information across contrasts to improve the quality of multi-contrast MRI reconstruction.

[0005] However, for existing deep learning model reconstruction methods, such as the prompt-driven MRI reconstruction method PromptMR, the main processing flow is to fill in the missing data in the k-space domain through an iterative unfolding network, and introduce a prompt module in the multi-level decoder of the network to distinguish different input types and guide the reconstruction. However, the interaction modeling between the prompt and the multi-contrast features of this type of method is still relatively simple, making it difficult to fully extract and fuse the complementary information between different contrasts, thereby showing certain limitations in the reconstruction task of multi-contrast MRI images. SUMMARY

[0006] Based on the defects of the prior art described above, the present application provides a multi-contrast MRI joint reconstruction method, system, device and medium, which solves the existing problems.

[0007] The present application adopts the following technical solutions: In a first aspect, the present application provides a multi-contrast MRI joint reconstruction method, comprising the following steps: Acquire under-sampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts; wherein the data consistency term in the original objective function of the compressed sensing algorithm is optimized to a multi-contrast data consistency term, the regularization term is optimized to a contrast-enhanced correlation regularization term, a corresponding prompt function is introduced based on each contrast, and an optimization objective function is obtained; Introduce an auxiliary variable to the optimization objective function, and decompose the optimization objective function into a first sub-problem about the auxiliary variable and a second sub-problem about the target variable, the target variable being the MRI reconstructed image corresponding to the under-sampled MRI image data of different contrasts; The first sub-problem and the second sub-problem are solved alternately to obtain a plurality of optimal MRI reconstruction images; in the solving process of the first sub-problem, the under-sampled MRI image data is encoded layer by layer to obtain a plurality of semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain a plurality of semantic enhancement feature maps; in each decoding process, the current semantic enhancement feature map is generated based on different contrasts for the corresponding contrast prompt, the frequency domain enhancement feature and the spatial enhancement feature; the contrast prompt, the frequency domain enhancement feature and the spatial enhancement feature of the current layer are fused to obtain the semantic enhancement feature map of the next layer.

[0008] Preferably, the optimization objective function is specifically as follows: ; In the formula, x i indicates the full-sampling image of the i th contrast, N is the number of contrasts, is the k spatial under-sampled MRI image data, M i is the mask operator of the i th contrast, is the Fourier transform operator, is the balance parameter, is the correlation regularization term of contrast enhancement, is the prompt function.

[0009] Preferably, the first sub-problem and the second sub-problem are solved alternately by an unfolding network, and the unfolding network comprises a plurality of unfolding modules, each unfolding module comprises a reconstruction network, a prompt module and a data consistency module, the reconstruction network comprises an encoder, a bottleneck layer and a decoder, the encoder comprises a plurality of encoding layers, the decoder comprises a plurality of decoding layers, the plurality of encoding layers are connected to the plurality of decoding layers in a skip connection manner, and each decoding layer comprises a multi-contrast interaction module and an ASSF module; The plurality of encoding layers encode the under-sampled MRI image data layer by layer to obtain a plurality of semantic feature maps with different resolutions; The bottleneck layer converges the semantic feature maps with different resolutions; The plurality of decoding layers decode the semantic feature map with the smallest resolution layer by layer to obtain a plurality of semantic enhancement feature maps; The data consistency module is used to apply a frequency domain consistency constraint to the final output semantic enhancement feature map to obtain the reconstruction image of the current unfolding module.

[0010] Preferably, each multi-contrast interactive module is configured to generate corresponding frequency domain enhanced features and spatial enhanced features based on different contrasts, each including a spatial dynamic fusion module and a frequency domain selective fusion module; The spatial dynamic fusion module includes a splicing module, a first CAB module, a down-sampling module, a first mapping module, a second CAB module, a second mapping module, and a weighting module; In the spatial dynamic fusion module, one contrast is designated as a target contrast, and the remaining contrasts are designated as auxiliary contrasts; the splicing module splices the target contrast features and the auxiliary contrast features to obtain a splicing result; The first CAB module extracts features from the splicing result to obtain spatial structure information; The down-sampling module matches the spatial structure information to the size of the target contrast to obtain a matching result; The first mapping module maps the matching result to obtain a weight map; The second CAB module extracts deep features from the target contrast features to obtain structure representation information; The second mapping module maps the structure representation information to obtain a spatial structure attention map; The weighting module dynamically weights the target contrast features, the weight map, and the spatial structure attention map to obtain a spatial enhanced feature.

[0011] Preferably, the frequency domain selective fusion module includes a global average pooling layer, a fully connected layer, a batch normalization layer, a third mapping module, and a Softmax module; In the frequency domain selective fusion module, one contrast is designated as a target contrast, and the remaining contrasts are designated as reference contrasts, and frequency domain feature maps of the target contrast and the reference contrasts are obtained; The frequency domain feature maps of the target contrast and the reference contrasts are fused to obtain a joint frequency spectrum; The global average pooling layer extracts channels from the joint frequency spectrum to obtain statistical features; The fully connected layer, the batch normalization layer, the third mapping module, and the Softmax module classify, normalize, map, and output the statistical features to obtain channel attention weights of the target contrast and the reference contrasts; The channel attention weights are weighted and fused with the corresponding frequency domain feature maps to obtain a frequency domain enhanced feature; The ASSF module fuses the frequency domain enhanced feature and the spatial enhanced feature to obtain a fusion feature.

[0012] Preferably, the prompt module includes a prompt generation unit and a prompt fusion unit; the prompt generation unit includes a one-hot encoding module, an interpolation module, and a convolution module, and the prompt fusion unit includes a splicing module, a CAB module, and a down-sampling module; A one-hot encoding module encodes the current semantic enhancement feature map x to obtain a weight vector w , the weight vector w is used to select a corresponding prompt vector p ; An interpolation module interpolates the selected prompt vector p and expands it to the same size as the current semantic enhancement feature map x ; A convolution module convolves the expanded prompt vector p to obtain a prompt feature A concatenation module concatenates the prompt feature with x to obtain a concatenation result A CAB module performs feature weighted fusion on the concatenation result to obtain a weighted fusion result A downsampling module performs downsampling processing on the weighted result to obtain a contrast prompt P corresponding to the current contrast P The contrast prompt is added to the fusion feature.

[0013] In a second aspect, the present application provides a multi-contrast MRI joint reconstruction system, comprising: A generation module is configured to acquire under-sampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts; wherein the data consistency term in the original objective function of the compressed sensing algorithm is optimized as a multi-contrast data consistency term, the regularization term is optimized as a contrast-enhanced correlation regularization term, a corresponding prompt function is introduced based on each contrast, and the optimization objective function is obtained; A decomposition module is configured to introduce an auxiliary variable to the optimization objective function, and decompose the optimization objective function into a first sub-problem about the auxiliary variable and a second sub-problem about a target variable, wherein the target variable is an MRI reconstruction image corresponding to the under-sampled MRI image data of different contrasts; A solving module is configured to sequentially solve the first sub-problem and the second sub-problem to obtain multiple optimal MRI reconstruction images; in the first sub-problem solving process, the under-sampled MRI image data is encoded layer by layer to obtain multiple semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain multiple semantic enhancement feature maps; in each decoding process, a corresponding contrast prompt, a frequency domain enhancement feature and a spatial enhancement feature are generated for the current semantic enhancement feature map based on different contrasts; the contrast prompt, the frequency domain enhancement feature and the spatial enhancement feature of the current layer are fused to obtain the semantic enhancement feature map of the next layer.

[0014] In a third aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the multi-contrast MRI joint reconstruction method when executing the program.

[0015] In a fourth aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is executable on a processor to implement the multi-contrast MRI joint reconstruction method.

[0016] Compared with the prior art, the above at least one technical solution adopted by the present application can achieve the following beneficial effects: In the present application, the data consistency term in the original objective function of the compressed sensing algorithm is optimized to a multi-contrast data consistency term, and the regularization term is optimized to a contrast-enhanced correlation regularization term, a corresponding prompt function is introduced based on each contrast, and an optimized objective function is obtained, and the present application constructs a multi-contrast joint optimization objective function, and introduces a contrast-enhanced correlation regularization term and a prompt function, effectively utilizes the complementary information between different contrast images, and decomposes the optimized objective function into two sub-problems for solving.

[0017] In the solving process, the two sub-problems are sequentially solved by unfolding the network, and the corresponding contrast prompt, frequency domain enhanced feature and spatial enhanced feature are generated based on different contrasts for the current semantic enhanced feature map; the contrast prompt, frequency domain enhanced feature and spatial enhanced feature of the current layer are fused to obtain the semantic enhanced feature map of the next layer. In the present application, the features of multi-contrast data are interacted in the spatial domain and the frequency domain, the efficient complementation and collaborative modeling of multi-contrast features are realized, and the problem of insufficient information utilization between contrasts in the traditional method is overcome. In addition, by perceiving the features of each contrast, a targeted prompt can be generated to guide the reconstruction process. Thus, the quality of low-quality contrast is improved, and the overall reconstruction result is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is the iterative flow chart of the present application; Figure 2 is the overall framework diagram of the Prompt MMR-Net of the present application; Figure 3Structure diagram of the cross-domain multi-contrast interaction module of the present application; Figure 4 Structure diagram of the spatial dynamic fusion module of the present application; Figure 5 Structure diagram of the frequency domain selective fusion module of the present application; Figure 6 Structure diagram of the contrast information perception prompt module of the present application; Figure 7 Visualization results of FLAIR in the brain data set of volunteers by different methods of the present application; Wherein, Figure 7 (a): full sampling image of the present application, Figure 7 (b): Zero-Filling reconstructed image of the present application, Figure 7 (c): Restormer reconstructed image of the present application, Figure 7 (d): MC-CDic reconstructed image of the present application, Figure 7 (e): PromptMR reconstructed image of the present application, Figure 7 (f): Prompt MMR-Net reconstructed image of the present application; Figure 8 Visualization results of T1 in the brain data set of volunteers by different methods of the present application; Wherein, Figure 8 (a): full sampling image of the present application, Figure 8 (b): Zero-Filling reconstructed image of the present application, Figure 8 (c): Restormer reconstructed image of the present application, Figure 8 (d): MC-CDic reconstructed image of the present application, Figure 8 (e): PromptMR reconstructed image of the present application, Figure 8 (f): Prompt MMR-Net reconstructed image of the present application; Figure 9 Visualization results of T2 in the brain data set of volunteers by different methods of the present application; Wherein, Figure 9 (a): full sampling image of the present application, Figure 9 (b): Zero-Filling reconstructed image of the present application, Figure 9 (c): Restormer reconstructed image of the present application, Figure 9 (d): MC-CDic reconstructed image of the present application, Figure 9 (e): PromptMR reconstructed image of the present application, Figure 9 (f): Prompt MMR-Net reconstructed image of the present application; Figure 10 Visualization results of FLAIR in the clinical brain data set of a hospital by different methods of the present application; wherein, Figure 10 (a): full-sampling image of (a): Figure 10 (b): Zero-Filling reconstructed image of (b): Figure 10 (c): Restormer reconstructed image of (c): Figure 10 (d): MC-CDic reconstructed image of (d): Figure 10 (e): PromptMR reconstructed image of (e): Figure 10 (f): Prompt MMR-Net reconstructed image of (f): Figure 11 Visualization results of T1 in hospital clinical brain dataset by different methods of the present application; wherein, Figure 11 (a): full-sampling image of (a): Figure 11 (b): Zero-Filling reconstructed image of (b): Figure 11 (c): Restormer reconstructed image of (c): Figure 11 (d): MC-CDic reconstructed image of (d): Figure 11 (e): PromptMR reconstructed image of (e): Figure 11 (f): Prompt MMR-Net reconstructed image of (f): Figure 12 Visualization results of T2 in hospital clinical brain dataset by different methods of the present application; wherein, Figure 12 (a): full-sampling image of (a): Figure 12 (b): Zero-Filling reconstructed image of (b): Figure 12 (c): Restormer reconstructed image of (c): Figure 12 (d): MC-CDic reconstructed image of (d): Figure 12 (e): PromptMR reconstructed image of (e): Figure 12 (f): Prompt MMR-Net reconstructed image of (f). DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0021] Related background: Over the years, various strategies have been developed to accelerate MRI acquisition. Compressed Sensing (CS) theory was introduced to the field of MRI reconstruction by Lustig et al., providing a theoretical basis for reconstructing high-quality images from significantly undersampled data. CS relies on the assumption that most MRI images are sparse in a suitable transform domain (such as wavelet or curvelet domain). By exploiting this sparsity, images can be reconstructed with fewer samples than required by the traditional Nyquist sampling theorem, significantly shortening the acquisition time. Although CS has been widely adopted, it is often limited by the need to manually design regularizers based on prior knowledge of image structure, and can encounter difficulties when dealing with high noise or undersampled data.

[0022] The framework of the deep learning-based sampling model and reconstruction algorithm joint optimization method is as follows: Let denote the unknown fully sampled MRI image. In the traditional MRI compressed sensing framework, denotes k the spatially undersampled image data, which is processed by the Fourier transform operator and the mask operator to obtain the fully sampled MRI image. The relationship between the fully sampled MRI image and its corresponding undersampled k spatial signal is established as follows: ; where is the measurement noise, H and W represent the height and width of the fully sampled MRI image, respectively, and represent the complex domain.

[0023] The classical compressed sensing method reconstructs by optimizing the following energy equation (original objective function): where denotes the data consistency term, is the regularization term applied to to impose prior constraints (such as sparsity), is the balance parameter that adjusts the relative weights of the two terms.

[0024] Based on this, the application provides a multi-contrast MRI joint reconstruction method, in particular, a multi-contrast MRI joint reconstruction method based on prompt enhancement. The method aims to fully exploit the complementary information between different contrast images, and improve the reconstruction quality and structural detail fidelity under the condition of undersampling. By constructing an end-to-end deep learning framework, the application effectively combines contrast adaptive guidance, cross-contrast feature fusion and frequency domain physical consistency constraints, etc. strategies, realizes the collaborative modeling and high-quality reconstruction of multi-contrast images, and is suitable for clinical and scientific research needs in various magnetic resonance imaging scenes. Specifically, the method comprises the following steps: S1: Collect undersampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts.

[0025] The input of the application is undersampled magnetic resonance image frequency domain data under different contrasts, specifically including but not limited to T1 weighted, T2 weighted and fluid attenuated inversion recovery sequence (FLAIR) and other undersampled multi-contrast images. After the contrast images are processed by different sampling masks, undersampled frequency domain data is formed.

[0026] In order to extend MRI reconstruction from single contrast to multi-contrast joint method, the correlation between multi-contrast data must be considered. In the prior art, single-contrast MRI reconstruction usually relies on single-contrast data for image reconstruction, but due to the differences between different contrast data in the acquisition process, direct reconstruction using single contrast may not be able to fully utilize all the contrast information. Therefore, in order to integrate the correlation between multi-contrast data and ensure that the information of different contrasts can be effectively fused, thereby improving the quality of the reconstructed image, the application constructs the objective function as follows: ; Wherein, is a multi-contrast data consistency term, represents the first contrast. represents a contrast-enhanced correlation regularization term for imposing prior constraints (such as sparsity, correlation between contrasts), is a balance parameter for adjusting the relative weights of the two terms.

[0027] In view of the continuous development of the prompt engineering in large-scale models and its key role in feature guidance, a prompt function is introduced for each contrast image to enhance the guidance ability of the information between contrasts, and the objective function is updated accordingly to obtain the optimization objective function: ; The construction of the optimization objective function provides a mathematical framework for joint reconstruction. By optimizing this function, multiple contrast features can be considered simultaneously, and their complementary information can be maximally utilized in the reconstruction process. Therefore, the design of the objective function not only improves the existing technology, but also is the key to realizing effective fusion and high-quality image output in multi-contrast MRI reconstruction.

[0028] S2: Introduce auxiliary variables to the optimization objective function, and decompose the optimization objective function into a first sub-problem about the auxiliary variables and a second sub-problem about the target variables.

[0029] Constructing an MRI reconstruction network: Given the effectiveness of the Half-Quadratic Splitting (HQS) method in image inverse problems, auxiliary variables are introduced, and the optimization objective function is re-expressed as follows: ; wherein, is a penalty parameter.

[0030] S3: Alternately solve the first sub-problem and the second sub-problem.

[0031] The above formula can be decomposed into two sub-problems and alternately solved: First sub-problem: update , given the reconstructed image of the th iteration, define the proximal operator according to the correlation between the contrasts.

[0032] ; Second sub-problem: update , the data consistency term is related to the differentiable Frobenius constraint, so there is a closed-form solution: ; ; wherein, denotes the position of the sampling point, denotes the set of sampling points.

[0033] Expand into a proximal operator network, and construct a multi-contrast magnetic resonance image reconstruction network Prompt MMR-Net through the alternating iteration of and .

[0034] Deep Unfolding Networks (DUNs) directly integrate domain knowledge into the network training process by mimicking the process of traditional reconstruction methods, making the learning process more interpretable. Therefore, DUNs can effectively make up for the shortcomings of purely data-driven methods and provide more accurate and stable reconstruction results. In short, the role of DUNs is to combine traditional optimization techniques with the advantages of deep learning, providing a more efficient, interpretable and accurate method for MRI image reconstruction.

[0035] Based on step S1, the present application proposes a prompt-based multi-contrast MRI reconstruction network (Prompt-based Multi-contrast MRI Reconstruction Network, hereinafter referred to as Prompt MMR-Net), as shown in Figure 2 By performing feature interaction of multi-contrast data in the spatial domain and the frequency domain, the network can utilize the complementary information between different contrasts without the need for pre-alignment. In addition, by perceiving the features of each contrast, targeted prompts can be generated to guide the reconstruction process. Thus, the quality of low-quality contrasts is improved, and the overall reconstruction result is enhanced.

[0036] Prompt MMR-Net can take multiple contrast undersampled magnetic resonance image data as input, perform joint processing and interactive fusion, and thus simultaneously reconstruct high-quality images of each contrast. As shown in Figure 1 The structure is composed of multiple functionally clear and complementary modules, mainly including a multi-contrast interactive reconstruction network (Multi-contrast Interactive Reconstruction Network, MCIR-Net), a contrast information perception prompt module (Contrast Information Perception Prompt Module, CIPP), and a data consistency module (Data Consistency Module, DCM).

[0037] MCIR-Net: MCIR-Net, as the main backbone of the overall reconstruction, adopts a U-Net-like encoder-bottleneck-decoder structure, with multi-scale and cross-contrast fusion capabilities.

[0038] The encoder includes multiple encoding layers, each layer extracts initial features by 3x3 convolution and connects CAB to enhance important features; multi-layer down-sampling operations are used to extract deep semantic information to obtain semantic feature maps of different resolutions; after down-sampling, the features are sent to the intermediate bottleneck layer to converge multi-scale feature maps.

[0039] Each decoding layer includes a cross-domain multi-contrast interaction module and an adaptive spatial-frequency fusion module (ASFF) proposed in the MMR-Mamba method. Each layer gradually recovers the spatial resolution through an upsampling operation and outputs a reconstructed feature map fused with the corresponding encoded layer feature; each decoding layer forms a skip connection with the symmetric encoding layer to retain low-level features. More importantly, each decoding layer also fuses multi-contrast supplementary information provided by the cross-domain multi-contrast interaction module (CD-MIM) module; at the same time, the prompt feature output by the CIPP module is introduced to guide the contrast enhancement.

[0040] To alleviate the problem of over-smoothing of shallow details in deep layers in the U-Net structure, the Prompt MMR-Net introduces a cross-stage information path, which separately transports the encoding and decoding features of different levels to the next stage encoder through two independent 1x1 convolution layers for additive fusion, realizing complementary fusion of shallow details and deep semantic information, and preserving key edges and texture details.

[0041] The CD-MIM module takes the intermediate feature map output by the previous decoding layer or bottleneck layer (i.e., the high-dimensional feature tensor of the corresponding layer) as input, which is a key component of the MCIR-Net for complementary fusion of multi-contrast information and runs through all decoding stages. As shown in Figure 3 The module integrates two core sub-modules: a spatial dynamic fusion module (SDFM) and a frequency selective fusion module (FSFM). The spatial dynamic fusion module SDFM adaptively adjusts the fusion weights of each contrast feature in the spatial domain (i.e., two-dimensional pixel distribution structure) of the image; while the frequency selective fusion module FSFM extracts and fuses discriminative frequency components in the frequency representation of the feature (such as the Fourier domain). Figure 3 In the formula, FFT and IFFT represent Fourier transform and inverse Fourier transform, respectively, which are used to realize round-trip conversion between the feature frequency domain and the image domain to support cross-domain feature fusion.

[0042] Referring to Figure 4, SDFM includes a splicing module, a first CAB module, a downsampling module, a first mapping module, a second CAB module, a second mapping module, and a weighting module. The first mapping module and the second mapping module both use a Sigmoid activation function. In the SDFM, each processing cycle jointly processes multiple contrasts, specifies one contrast as the current "target contrast", and the remaining contrasts as "auxiliary contrasts" to participate in fusion, thereby realizing complementary utilization of information. SDFM focuses on extracting transferable spatial features from auxiliary contrasts to enhance the representation ability of the current contrast. The network structure is designed as follows: The features of the current target contrast and the features of other auxiliary contrasts are spliced in the channel dimension through the splicing module to obtain a splicing result.

[0043] The splicing result is sent to the first CAB module to extract high-quality structural information to obtain spatial structure information; the spatial structure information is then matched to the target contrast size through the downsampling module to obtain a matching result; and the matching result is processed through the first mapping module to generate a weight map used to guide fusion.

[0044] Meanwhile, the second CAB module extracts deep features from the target contrast features to obtain structural representation information; the second mapping module maps the structural representation information to obtain a spatial structure attention map. The fusion of the two uses a dynamic weighting strategy, and the parameters learned are used to control the fusion strength of auxiliary information. Finally, the spatial enhancement features of the target contrast are output as the final output and are fed into the main network decoding layer.

[0045] Referring to Figure 5 FSFM, on the other hand, focuses on contrast interaction in the frequency domain. It includes a global average pooling layer (GAP), a fully connected layer (FC), a batch normalization layer (BN), a third mapping module, and a Softmax module. The third mapping module uses a ReLU activation function. In each round of fusion, one contrast is set as the "target contrast", and the remaining contrasts are sequentially set as "reference contrasts". The frequency domain features of the two are obtained by performing Fourier transform on the input features. The structure design includes the following steps: The frequency domain features of the target contrast and the current reference contrast are fused to obtain a joint frequency spectrum.

[0046] The joint frequency spectrum is extracted through the global average pooling layer to obtain channel-level statistical features. After the fully connected layer, the batch normalization layer, and the ReLU activation function, the Softmax generates two channel attention weights for the two contrasts respectively. These weights are used to weight the frequency domain features of the two contrasts respectively, and then the two contrasts are fused. The output frequency domain enhancement features are fed back to the main network for further reconstruction.

[0047] The two parallel sub-modules jointly constitute the CD-MIM module, which further combines the adaptive spatial-frequency fusion module (ASFF) proposed in the MMR-Mamba method, for complementary integration of multi-contrast features at spatial and frequency levels, thereby improving the reconstruction performance.

[0048] CIPP: The CIPP module is designed to introduce contrast perception capability in the decoding stage at different levels of the network, providing structured prompt information for different contrasts, referred to as PromptBlock. As shown in Figure 6 , the network structure mainly includes a prompt generation unit and a prompt fusion unit. The prompt generation unit includes a one-hot encoding module, an interpolation module, and a convolution module. The prompt fusion unit includes a splicing module, a CAB module, and a down-sampling module.

[0049] The prompt generation unit is used to configure a learnable prompt vector p for each contrast x , and the output feature (semantic enhanced feature) of each decoding layer is taken as input w , and after One-hot (one-hot encoding module) encoding, a weight vector for selecting the corresponding prompt is obtained p . The selected prompt vector x is expanded to the same size as the input feature through the interpolation module, and the spatial structure feature (prompt feature) is further extracted through 3x3 convolution (convolution module).

[0050] x The splicing module splices the above prompt feature with the output feature of each decoding layer P in the channel dimension, and then uses the channel attention module (CAB) proposed in the PromptMR method for feature weighted fusion. The weighted fusion result is down-sampled by Down (down-sampling module) to match the feature scale of the subsequent backbone network, and is taken as the contrast prompt stream into the multi-scale reconstruction path and is additively fused with the output feature of the ASFF.

[0051] DCM: The DCM module is located at the end of each reconstruction stage, which is used to impose a frequency domain consistency constraint on the network output image, ensuring that it strictly matches the original frequency domain observation data at the sampling position. This module maintains the credibility and clinical usability of the reconstructed image in a physical sense based on the undersampling mask information.

[0052] To sum up, the Prompt MMR-Net proposed in the application realizes a joint reconstruction framework of structure expression enhancement, contrast interaction optimization and physical constraint consideration through CIPP guidance, cross-stage information connection, CD-MIM fusion and DCM correction, and is suitable for high-fidelity image recovery tasks in scenes such as multi-contrast magnetic resonance imaging reconstruction.

[0053] The output of the application is a high-quality magnetic resonance image under each contrast. Through the joint modeling and training of the above multiple modules, the output image has good performance in visual detail preservation, structure restoration and noise suppression, and can meet the needs of various medical applications such as clinical image diagnosis and anatomical structure identification.

[0054] S4: End-to-end training of the reconstruction network Prompt MMR-Net, using normalized L1 norm error as the loss function of network training: ; Wherein, is the number of contrasts, is the output of the reconstruction network for the th contrast, is the corresponding fully sampled image. The gradient of the loss function with respect to the network parameters is calculated by the back propagation algorithm, and the Adam optimizer is used to optimize the network parameters.

[0055] S5: Apply the trained reconstruction network Prompt MMR-Net to MRI image reconstruction: input is the undersampled data of k-space, and output is the reconstructed magnetic resonance image.

[0056] Embodiment 2 In the numerical experiment, this embodiment uses a volunteer brain data set and a hospital clinical brain data set for experiment. The volunteer brain data includes FLAIR, T1, T2 three contrasts, the training data amount is 7661, the test data amount is 1860, and 10x2D Cartesian sampling mode is used. The hospital clinical brain data set also includes FLAIR, T1, T2 three contrasts, the training data amount is 2020, the test data amount is 324, and 8x1D Cartesian sampling mode is used, the center sampling rate is 8%, and the high frequency is randomly selected. Since the original data sizes of different contrasts collected by the data set are different, the size is aligned by filling 0 in the image domain and K-space.

[0057] In the experiments of the volunteer brain data set (Table 1) and the hospital clinical brain data set (Table 2), the Prompt MMR-Net method proposed in the application shows significant performance advantages under different contrasts (T1, T2, FLAIR) and different sampling rates.

[0058] Table 1: Comparison results of different methods on volunteer brain data

[0059] Table 2: Comparison results of different methods on hospital clinical brain data

[0060] Among them, Zero-Filling refers to a reconstruction method that directly fills the unsampled region with zeros in the undersampled k-space data and performs inverse Fourier transform to obtain the image, which belongs to the most basic prior-free fast reconstruction baseline. Restormer can capture long-distance pixel interaction and is an efficient Transformer model that can be applied to the task of restoring high-resolution images. MC-CDic is a multi-scale convolution dictionary model based on deep unfolding, which explicitly decomposes the common and difference features of multi-contrast MRI and realizes end-to-end optimization with a learnable proximity operator, can fully integrate multi-contrast complementary information in reconstruction and super-resolution tasks, and realize high-precision and interpretable MRI reconstruction. PromptMR is a full-scene MRI reconstruction method based on prompt learning, which fills the missing data in k-space and performs feature fusion in the image domain, achieving unified high-precision reconstruction of multi-contrast, dynamic and multi-view MRI data.

[0061] The specific performance is as follows: Peak Signal-to-Noise Ratio (PSNR): PSNR is a commonly used indicator to measure the error between the reconstructed image and the true image, and the higher the value, the better the reconstruction quality. In Table 1, the PSNR values of Prompt MMR-Net under T1, T2 and FLAIR contrast are 34.69, 34.75 and 32.04, respectively, all higher than other methods. In Table 2, the PSNR values are 35.58, 33.05 and 33.45, respectively, also better than other methods.

[0062] Structural Similarity Index Measure (SSIM): SSIM is used to measure the similarity of structural information between the reconstructed image and the true image, and the closer the value to 1, the closer the structural information. In Table 1, the SSIM values of Prompt MMR-Net are 0.9443, 0.9430 and 0.9066, respectively; in Table 2, the SSIM values are 0.9450, 0.9063 and 0.9149, respectively, all higher than other methods, indicating that the method performs well in preserving image structural information.

[0063] Normalized Root Mean Square Error (NRMSE): NRMSE is used to measure the error between the reconstructed image and the real image, and the lower the value, the better the reconstruction quality. In Table 1, the NRMSE values of Prompt MMR-Net are 0.0815, 0.0902 and 0.1167 respectively; in Table 2, the NRMSE values are 0.0747, 0.1155 and 0.1112 respectively, all lower than other methods, indicating that the method has significant advantages in reducing reconstruction error.

[0064] Referring to Figures 7-12 As can be seen from the figures: Detail preservation: the images reconstructed by Prompt MMR-Net can clearly preserve the detailed information of the tissue structure, especially in the anatomical structure of the brain and the pathological area, and are highly consistent with the full-sampling images.

[0065] Noise suppression: there are no obvious noise artifacts in the reconstructed images, indicating that the method can effectively suppress noise when processing undersampled data and improve image quality.

[0066] Contrast consistency: in the reconstruction of multi-contrast data (T1, T2, FLAIR), Prompt MMR-Net can fully utilize the complementary information between different contrasts to generate high-quality joint reconstructed images, verifying its effectiveness and superiority in multi-contrast MRI reconstruction.

[0067] Based on the quantitative analysis of Table 1 and Table 2 and Figures 7-12 the qualitative analysis, it can be clearly seen that the Prompt MMR-Net method proposed in the present application has significant advantages in the multi-contrast MRI joint reconstruction task. The method not only outperforms existing algorithms in numerical indicators, but also performs well in detail preservation and noise suppression of images, and can provide high-quality reconstructed images for clinical diagnosis. In addition, the method performs stably under different contrasts, has wide applicability and practicality.

[0068] Example 3 Based on the same concept, the present application also provides a multi-contrast MRI joint reconstruction system, comprising a generation module, a decomposition module and a solving module.

[0069] The generation module is used to collect undersampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts; wherein the data consistency term in the original objective function of the compressed sensing algorithm is optimized to a multi-contrast data consistency term, the regularization term is optimized to a contrast-enhanced correlation regularization term, and a corresponding prompt function is introduced based on each contrast to obtain the optimization objective function.

[0070] The decomposition module is configured to introduce auxiliary variables into the optimization objective function, decompose the optimization objective function into a first sub-problem about the auxiliary variables and a second sub-problem about target variables, and the target variables are MRI reconstruction images corresponding to the different contrast undersampled MRI image data.

[0071] The solving module is configured to sequentially solve the first sub-problem and the second sub-problem to obtain a plurality of optimal MRI reconstruction images, perform layer-by-layer encoding on the undersampled MRI image data to obtain a plurality of semantic feature maps during the solving process of the first sub-problem, perform layer-by-layer decoding on a semantic feature map with the smallest resolution to obtain a plurality of semantic enhanced feature maps, generate a corresponding contrast prompt, a frequency domain enhanced feature and a spatial enhanced feature for the current semantic enhanced feature map based on different contrasts during each decoding process, and fuse the contrast prompt, the frequency domain enhanced feature and the spatial enhanced feature of the current layer to obtain a semantic enhanced feature map of a next layer.

[0072] Embodiment 4 The application further provides a computer device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the multi-contrast MRI joint reconstruction method when executing the program.

[0073] Embodiment 5 The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executable on the processor to implement the multi-contrast MRI joint reconstruction method.

[0074] Although the preferred embodiments of the application have been described, those skilled in the art who are familiar with the basic inventive concept can make additional changes and modifications to the embodiments. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the application.

[0075] Obviously, those skilled in the art can make various modifications and changes to the application without departing from the spirit and scope of the application. Thus, if these modifications and changes of the application fall within the scope of the claims of the application and their equivalent technologies, the application also intends to include these modifications and changes.

Claims

1. A multi-contrast MRI joint reconstruction method, characterized in that, The method comprises the following steps: Collecting under-sampling MRI image data of different contrasts, generating an optimization objective function based on multiple contrasts; wherein, the data consistency term in the original objective function of the compressed sensing algorithm is optimized into a multi-contrast data consistency term, the regularization term is optimized into a contrast-enhanced correlation regularization term, and a corresponding hint function is introduced based on each contrast to obtain the optimization objective function; Introducing an auxiliary variable to the optimization objective function, decomposing the optimization objective function into a first sub-problem about the auxiliary variable and a second sub-problem about the target variable, the target variable being the MRI reconstruction image corresponding to the under-sampling MRI image data of different contrasts; Solving the first sub-problem and the second sub-problem alternately to obtain multiple optimal MRI reconstruction images; in the solving process of the first sub-problem, the under-sampling MRI image data is encoded layer by layer to obtain multiple semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain multiple semantic enhanced feature maps; in each decoding process, a corresponding contrast hint, a frequency domain enhanced feature and a spatial enhanced feature are generated for the current semantic enhanced feature map based on different contrasts; the contrast hint, the frequency domain enhanced feature and the spatial enhanced feature of the current layer are fused to obtain the semantic enhanced feature map of the next layer.

2. The multi-contrast MRI joint reconstruction method of claim 1, wherein, The optimization objective function is specifically as follows: ； wherein x i denotes the full-sampled image of the i contrast, N is the number of contrasts, is the k spatially under-sampled MRI image data, M i denotes the mask operator of the i contrast, is the Fourier transform operator, is the balancing parameter, is the contrast-enhanced correlation regularization term, is the indicator function.

3. The multi-contrast MRI joint reconstruction method of claim 1, wherein, The first sub-problem and the second sub-problem are solved alternately through an unfolding network, the unfolding network comprising multiple unfolding modules, each unfolding module comprising a reconstruction network, a hint module and a data consistency module, the reconstruction network comprising an encoder, a bottleneck layer and a decoder, the encoder comprising multiple encoding layers, the decoder comprising multiple decoding layers, the multiple encoding layers being connected to the multiple decoding layers in a skip connection manner, each decoding layer comprising a multi-contrast interaction module and an ASSF module; The multiple encoding layers encode the under-sampling MRI image data layer by layer to obtain multiple semantic feature maps with different resolutions; The bottleneck layer converges the semantic feature maps with different resolutions; The multiple decoding layers decode the semantic feature map with the smallest resolution layer by layer to obtain multiple semantic enhanced feature maps; The data consistency module is used to impose a frequency domain consistency constraint on the final output semantic enhanced feature map to obtain the reconstruction image of the current unfolding module.

4. The multi-contrast MRI joint reconstruction method of claim 3, wherein, Each multi-contrast interaction module is used to generate corresponding frequency domain enhanced features and spatial enhanced features based on different contrasts, and each multi-contrast interaction module comprises a spatial dynamic fusion module and a frequency domain selective fusion module; The spatial dynamic fusion module comprises a splicing module, a first CAB module, a down-sampling module, a first mapping module, a second CAB module, a second mapping module and a weighting module; In the spatial dynamic fusion module, one contrast is specified as a target contrast and the remaining contrasts are auxiliary contrasts; the splicing module splices the target contrast feature and the auxiliary contrast feature to obtain a splicing result; The first CAB module extracts features from the splicing result to obtain spatial structure information; The down-sampling module matches the spatial structure information to the size of the target contrast to obtain a matching result; The first mapping module maps the matching result to obtain a weight map; The second CAB module performs deep feature extraction on the target contrast feature to obtain structure representation information; The second mapping module maps the structure representation information to obtain a spatial structure attention map; The weighting module dynamically weights the target contrast feature, the weight map and the spatial structure attention map to obtain a spatial enhanced feature.

5. The multi-contrast MRI joint reconstruction method of claim 4, wherein, The frequency domain selective fusion module includes a global average pooling layer, a fully connected layer, a batch normalization layer, a third mapping module and a Softmax module; In the frequency domain selective fusion module, a contrast is specified as a target contrast, and the remaining contrasts are specified as reference contrasts, and frequency domain feature maps of the target contrast and the reference contrasts are obtained; The frequency domain feature maps of the target contrast and the reference contrasts are fused to obtain a joint frequency spectrum; The global average pooling layer extracts channels from the joint frequency spectrum to obtain statistical features; The fully connected layer, the batch normalization layer, the third mapping module and the Softmax module classify, normalize, map and output the statistical features to obtain channel attention weights of the target contrast and the reference contrasts; The channel attention weights are fused with the corresponding frequency domain feature maps to obtain frequency domain enhanced features; The ASSF module fuses the frequency domain enhanced features and the spatial enhanced features to obtain fused features.

6. The multi-contrast MRI joint reconstruction method of claim 5, wherein, The prompt module includes a prompt generation unit and a prompt fusion unit; the prompt generation unit includes a one-hot encoding module, an interpolation module and a convolution module, and the prompt fusion unit includes a concatenation module, a CAB module and a down-sampling module; One-hot encoding module x One-hot encoding is performed to obtain a weight vector w , the weight vector w for selecting the corresponding prompt vector p ; The interpolation module selects the prompt vector p The interpolation processing is performed, and is extended to the current semantic enhancement feature map x The size is consistent; the convolution module convolves the expanded prompt vector p to obtain a prompt feature The splicing module splices the prompt feature and the x obtains a splicing result. The CAB module performs feature weighting fusion on the concatenation result to obtain a weighted fusion result; The downsampling module performs downsampling processing on the weighted result to obtain a contrast prompt corresponding to the current contrast P ; contrast prompt P is added to the fusion feature.

7. A multi-contrast MRI joint reconstruction system, characterized by, The method comprises the following steps: The generating module is configured to collect under-sampled MRI image data of different contrasts, and generate an optimized objective function based on multiple contrasts; wherein a data consistency term in an original objective function of a compressed sensing algorithm is optimized as a multi-contrast data consistency term, a regularization term is optimized as a contrast-enhanced correlation regularization term, and a corresponding prompt function is introduced based on each contrast to obtain the optimized objective function; The decomposition module is configured to introduce an auxiliary variable into the optimized objective function, and decompose the optimized objective function into a first sub-problem about the auxiliary variable and a second sub-problem about a target variable, wherein the target variable is an MRI reconstruction image corresponding to the under-sampled MRI image data of different contrasts; The solving module is configured to sequentially solve the first sub-problem and the second sub-problem to obtain multiple optimal MRI reconstruction images; in the process of solving the first sub-problem, the under-sampled MRI image data is encoded layer by layer to obtain multiple semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain multiple semantic enhanced feature maps; in each decoding process, a corresponding contrast prompt, a frequency domain enhanced feature and a spatial enhanced feature are generated for the current semantic enhanced feature map based on different contrasts; the contrast prompt, the frequency domain enhanced feature and the spatial enhanced feature of the current layer are fused to obtain a semantic enhanced feature map of the next layer.

8. A computer device, comprising: The application also discloses a computer readable storage medium.

9. A computer-readable storage medium, characterized in that, The application also discloses a computer readable storage medium.

Images