A multi-contrast MRI joint reconstruction method, system, device and medium

Abstract

The application discloses a multi-contrast MRI joint reconstruction method, system, device and medium, relates to the technical field of medical imaging and deep learning, and comprises the following steps: collecting under-sampling MRI image data of different contrasts, and generating an optimization objective function based on multiple contrasts; the optimization objective function is decomposed into a first sub-problem about auxiliary variables and a second sub-problem about target variables; the first sub-problem and the second sub-problem are alternately solved in sequence, and multiple optimal MRI reconstruction images are obtained. The application realizes efficient complementary and collaborative modeling of multi-contrast features by performing feature interaction of multi-contrast data in the spatial domain and the frequency domain, and overcomes the problem of insufficient utilization of information between contrasts in the traditional method. By perceiving the features of each contrast, targeted prompts can be generated to guide the reconstruction process. Thus, the quality of low-quality contrast is improved, and the overall reconstruction result is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical imaging and deep learning, in particular to a multi-contrast MRI joint reconstruction method, system, device and medium. BACKGROUND

[0002] Magnetic Resonance Imaging (MRI) has become an indispensable tool in clinical diagnosis, widely recognized for its non-invasive, ultra-high spatial resolution, and excellent soft tissue contrast. Unlike other imaging techniques such as X-ray and CT (Computed Tomography), MRI does not contain ionizing radiation, making it a safe choice for repeated imaging, especially in children and elderly patients. Despite its significant advantages, MRI also faces some significant challenges, especially the longer scan time, which limits its application in certain clinical scenarios. Long scan times not only cause patient discomfort, but can also cause motion artifacts, especially in patients who are difficult to keep still. In addition, long MRI procedures are not only costly, but can also hinder the widespread use of MRI in resource-limited environments. Accelerating MRI acquisition without compromising diagnostic accuracy has become the focus of increasing numbers of researchers. One of the main methods to solve this problem is MRI image reconstruction, which aims to reconstruct high-quality images from undersampled data, significantly reducing scan time.

[0003] The emergence of deep learning methods marks a major breakthrough in the field of MRI reconstruction. These methods use powerful neural networks to learn the end-to-end mapping relationship from undersampled data to fully sampled images. The main advantage of deep learning methods is that they can learn the complex nonlinear relationship between undersampled and fully sampled images from large datasets, overcoming the limitations of traditional hand-designed methods. However, these methods usually operate as "black box" models, making it difficult to interpret them in clinical practice. With the continuous development of single-contrast MRI reconstruction technology, multi-contrast MRI reconstruction has also received increasing attention. Different contrasts can capture different aspects of tissue structure and pathology. For example, T1-weighted images may provide better anatomical details, while T2-weighted images perform well in depicting pathological regions. Multi-contrast MRI involves combining different contrast information of the same subject, such as T1-weighted images and T2-weighted images, to provide complementary diagnostic information. This approach not only takes advantage of the unique insights provided by each contrast, but also takes advantage of the fact that all contrasts capture imaging data from the same anatomical location, ensuring consistency in underlying tissue structure. By jointly utilizing multi-contrast information, multi-contrast MRI reconstruction can produce more accurate and comprehensive images that better reflect the anatomical and pathological characteristics of tissues.

[0004] Multi-contrast MRI reconstruction faces several unique challenges. Inter-contrast misalignment is one of the most prominent issues, as different contrast MRI scans often have slight misalignments due to differences in acquisition time, patient motion, and scanner settings. These misalignments can significantly degrade the quality of the reconstructed images. Traditional multi-contrast MRI reconstruction methods usually rely on registration techniques to align the images before reconstruction, but this can be computationally expensive and difficult to perform in real-time clinical settings. In recent years, deep learning models have been increasingly applied to multi-contrast reconstruction tasks, which can utilize multi-contrast information without the need for explicit pre-alignment. These multi-contrast deep learning methods aim to fully utilize the complementary information across contrasts to improve the quality of multi-contrast MRI reconstruction.

[0005] However, for existing deep learning model reconstruction methods, such as the prompt-driven MRI reconstruction method PromptMR, the main processing flow is to fill in the missing data in the k-space domain through an iterative unfolding network, and introduce a prompt module in the multi-level decoder of the network to distinguish different input types and guide the reconstruction. However, the interaction modeling between the prompt and the multi-contrast features of this type of method is still relatively simple, making it difficult to fully extract and fuse the complementary information between different contrasts, thereby showing certain limitations in the reconstruction task of multi-contrast MRI images. SUMMARY

[0006] Based on the defects of the prior art described above, the present application provides a multi-contrast MRI joint reconstruction method, system, device and medium, which solves the existing problems.

[0007] The present application adopts the following technical solutions:

[0008] In a first aspect, the present application provides a multi-contrast MRI joint reconstruction method, comprising the following steps:

[0009] Acquire under-sampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts; wherein the data consistency term in the original objective function of the compressed sensing algorithm is optimized to a multi-contrast data consistency term, the regularization term is optimized to a contrast-enhanced correlation regularization term, a corresponding prompt function is introduced based on each contrast, and an optimization objective function is obtained;

[0010] Introduce an auxiliary variable to the optimization objective function, and decompose the optimization objective function into a first sub-problem about the auxiliary variable and a second sub-problem about the target variable, the target variable being the MRI reconstructed image corresponding to the under-sampled MRI image data of different contrasts;

[0011] The first and second subproblems are solved alternately to obtain multiple optimal MRI reconstructed images. During the solution of the first subproblem, the undersampled MRI image data is encoded layer by layer to obtain multiple semantic feature maps. The semantic feature map with the lowest resolution is decoded layer by layer to obtain multiple semantic enhancement feature maps. During the decoding process of each layer, corresponding contrast cues, frequency domain enhancement features, and spatial enhancement features are generated for the current semantic enhancement feature map based on different contrast levels. The contrast cues, frequency domain enhancement features, and spatial enhancement features of the current layer are fused to obtain the semantic enhancement feature map of the next layer.

[0012] Preferably, the optimization objective function is as follows:

[0013] ;

[0014] In the formula, x i Indicates the first i A fully sampled image with varying contrast. N The number of contrast elements. for k Spatially undersampled MRI image data, M i For the first i A mask operator for contrast. For Fourier transform operators, For balancing parameters, For contrast enhancement, the correlation regularization term, This is the prompt function.

[0015] Preferably, the first and second subproblems are solved alternately by unfolding the network. The unfolding network includes multiple unfolding modules. Each unfolding module includes a reconstruction network, a hinting module, and a data consistency module. The reconstruction network includes an encoder, a bottleneck layer, and a decoder. The encoder includes multiple encoding layers, and the decoder includes multiple decoding layers. The multiple encoding layers are connected to the multiple decoding layers in a skip connection. Each decoding layer includes a multi-contrast interaction module and an ASSF module.

[0016] Multiple coding layers sequentially encode the undersampled MRI image data to obtain multiple semantic feature maps at different resolutions;

[0017] The bottleneck layer aggregates semantic feature maps of different resolutions;

[0018] Multiple decoding layers decode the semantic feature map with the lowest resolution layer by layer to obtain multiple semantically enhanced feature maps;

[0019] The data consistency module is used to apply frequency domain consistency constraints to the final output semantic enhancement feature map to obtain the reconstructed image of the current expansion module.

[0020] Preferably, each multi-contrast interactive module is used to generate corresponding frequency domain enhanced features and spatial enhanced features based on different contrasts, both including a spatial dynamic fusion module and a frequency domain selective fusion module;

[0021] The spatial dynamic fusion module includes a splicing module, a first CAB module, a downsampling module, a first mapping module, a second CAB module, a second mapping module, and a weighting module;

[0022] In the spatial dynamic fusion module, one contrast is designated as a target contrast, and the remaining contrasts are designated as auxiliary contrasts; the splicing module splices the target contrast features and the auxiliary contrast features to obtain a splicing result;

[0023] The first CAB module extracts features from the splicing result to obtain spatial structure information;

[0024] The downsampling module matches the spatial structure information to the target contrast size to obtain a matching result;

[0025] The first mapping module maps the matching result to obtain a weight map;

[0026] The second CAB module extracts deep features from the target contrast features to obtain structure representation information;

[0027] The second mapping module maps the structure representation information to obtain a spatial structure attention map;

[0028] The weighting module dynamically weights the target contrast features, the weight map, and the spatial structure attention map to obtain a spatial enhanced feature.

[0029] Preferably, the frequency domain selective fusion module includes a global average pooling layer, a fully connected layer, a batch normalization layer, a third mapping module, and a Softmax module;

[0030] In the frequency domain selective fusion module, one contrast is designated as a target contrast, and the remaining contrasts are designated as reference contrasts, and frequency domain feature maps of the target contrast and the reference contrasts are obtained;

[0031] The frequency domain feature maps of the target contrast and the reference contrasts are fused to obtain a joint frequency spectrum;

[0032] The global average pooling layer extracts channels from the joint frequency spectrum to obtain statistical features;

[0033] The fully connected layer, the batch normalization layer, the third mapping module, and the Softmax module classify, normalize, map, and output the statistical features to obtain channel attention weights of the target contrast and the reference contrasts;

[0034] The channel attention weight is fused with the corresponding frequency domain feature map to obtain a frequency domain enhanced feature;

[0035] The ASSF module fuses the frequency domain enhanced feature and the spatial enhanced feature to obtain a fusion feature.

[0036] Preferably, the prompt module comprises a prompt generation unit and a prompt fusion unit; the prompt generation unit comprises a one-hot encoding module, an interpolation module and a convolution module, and the prompt fusion unit comprises a splicing module, a CAB module and a down-sampling module;

[0037] The one-hot encoding module performs one-hot encoding on the current semantic enhanced feature map x to obtain a weight vector w , the weight vector w is used to select a corresponding prompt vector p ;

[0038] The interpolation module performs interpolation processing on the selected prompt vector p , and expands the prompt vector to have the same size as the current semantic enhanced feature map x ;

[0039] The convolution module performs convolution on the expanded prompt vector p to obtain a prompt feature;

[0040] The splicing module splices the prompt feature and x to obtain a splicing result;

[0041] The CAB module performs feature weighted fusion on the splicing result to obtain a weighted fusion result;

[0042] The down-sampling module performs down-sampling processing on the weighted result to obtain a contrast prompt P corresponding to the current contrast; the contrast prompt P is added to the fusion feature.

[0043] In a second aspect, the present application provides a multi-contrast MRI joint reconstruction system, comprising:

[0044] A generation module is configured to acquire under-sampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts; wherein a data consistency term in an original objective function of a compressed sensing algorithm is optimized as a multi-contrast data consistency term, a regularization term is optimized as a contrast-enhanced correlation regularization term, a corresponding prompt function is introduced based on each contrast, and the optimization objective function is obtained.

[0045] a decomposition module configured to introduce auxiliary variables into the optimization objective function, decompose the optimization objective function into a first sub-problem about the auxiliary variables and a second sub-problem about target variables corresponding to MRI reconstructed images of the under-sampled MRI image data of different contrasts;

[0046] a solving module configured to solve the first sub-problem and the second sub-problem in turn to obtain a plurality of optimal MRI reconstructed images, encode the under-sampled MRI image data layer by layer to obtain a plurality of semantic feature maps during solving the first sub-problem, decode the semantic feature map with the smallest resolution layer by layer to obtain a plurality of semantic enhanced feature maps, generate a contrast prompt, a frequency domain enhanced feature and a spatial enhanced feature corresponding to the current semantic enhanced feature map based on different contrasts during each decoding process, and fuse the contrast prompt, the frequency domain enhanced feature and the spatial enhanced feature of the current layer to obtain the semantic enhanced feature map of the next layer.

[0047] In a third aspect, the present application provides a computer device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the multi-contrast MRI joint reconstruction method when executing the program.

[0048] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the multi-contrast MRI joint reconstruction method.

[0049] Compared with the prior art, the above at least one technical solution of the present application can achieve the following beneficial effects:

[0050] In the present application, the data consistency term in the original objective function of the compressed sensing algorithm is optimized into a multi-contrast data consistency term, the regularization term is optimized into a contrast-enhanced correlation regularization term, the corresponding prompt function is introduced based on each contrast to obtain an optimization objective function, a multi-contrast joint optimization objective function is constructed, the contrast-enhanced correlation regularization term and the prompt function are introduced, the complementary information between different contrast images is effectively utilized, and the optimization objective function is decomposed into two sub-problems for solving.

[0051] In the solving process, two sub-problems are solved alternately by expanding the network, the corresponding contrast information, frequency domain enhanced feature and spatial enhanced feature are generated based on different contrasts for the current semantic enhanced feature map; the contrast information, frequency domain enhanced feature and spatial enhanced feature of the current layer are fused to obtain the semantic enhanced feature map of the next layer. Through the feature interaction of multi-contrast data in the spatial domain and the frequency domain, the efficient complementary and collaborative modeling of multi-contrast features are realized, and the problem of insufficient information utilization between contrasts in the traditional method is overcome. In addition, by perceiving the features of each contrast, targeted prompts can be generated to guide the reconstruction process. Thus, the quality of low-quality contrast is improved, and the overall reconstruction result is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0053] Figure 1 The iterative flowchart of the present application;

[0054] Figure 2 The overall framework diagram of Prompt MMR-Net of the present application;

[0055] Figure 3 The structure diagram of the cross-domain multi-contrast interaction module of the present application;

[0056] Figure 4 The structure diagram of the spatial dynamic fusion module of the present application;

[0057] Figure 5 The structure diagram of the frequency domain selective fusion module of the present application;

[0058] Figure 6 The structure diagram of the contrast information perception prompt module of the present application;

[0059] Figure 7 The visualization results of different methods in the FLAIR of the volunteer brain data set of the present application;

[0060] Among them, Figure 7 (a) of the full sampling image, Figure 7 (b) of the Zero-Filling reconstruction image, Figure 7 (c) of the Restormer reconstruction image, Figure 7 (d) of the MC-CDic reconstruction image, Figure 7of (e): PromptMR reconstructed images, Figure 7 of (f): Prompt MMR-Net reconstructed images;

[0061] Figure 8 Visualization results of T1 in the volunteer brain dataset for different methods of the present application;

[0062] wherein, Figure 8 of (a): fully sampled images, Figure 8 of (b): Zero-Filling reconstructed images, Figure 8 of (c): Restormer reconstructed images, Figure 8 of (d): MC-CDic reconstructed images, Figure 8 of (e): PromptMR reconstructed images, Figure 8 of (f): Prompt MMR-Net reconstructed images;

[0063] Figure 9 Visualization results of T2 in the volunteer brain dataset for different methods of the present application;

[0064] wherein, Figure 9 of (a): fully sampled images, Figure 9 of (b): Zero-Filling reconstructed images, Figure 9 of (c): Restormer reconstructed images, Figure 9 of (d): MC-CDic reconstructed images, Figure 9 of (e): PromptMR reconstructed images, Figure 9 of (f): Prompt MMR-Net reconstructed images;

[0065] Figure 10 Visualization results of FLAIR in the hospital clinical brain dataset for different methods of the present application;

[0066] wherein, Figure 10 of (a): fully sampled images, Figure 10 of (b): Zero-Filling reconstructed images, Figure 10 of (c): Restormer reconstructed images, Figure 10 of (d): MC-CDic reconstructed images, Figure 10 of (e): PromptMR reconstructed images, Figure 10 of (f): Prompt MMR-Net reconstructed images;

[0067] Figure 11 Visualization results of T1 in the hospital clinical brain dataset for different methods of the present application;

[0068] wherein, Figure 11 (a): full-sampling image of (a): Figure 11 (b): Zero-Filling reconstructed image of (b): Figure 11 (c): Restormer reconstructed image of (c): Figure 11 (d): MC-CDic reconstructed image of (d): Figure 11 (e): PromptMR reconstructed image of (e): Figure 11 (f): Prompt MMR-Net reconstructed image of (f):

[0069] Figure 12 Visualization results of T2 in the hospital clinical brain dataset for different methods of the present application;

[0070] wherein, Figure 12 (a): full-sampling image of (a): Figure 12 (b): Zero-Filling reconstructed image of (b): Figure 12 (c): Restormer reconstructed image of (c): Figure 12 (d): MC-CDic reconstructed image of (d): Figure 12 (e): PromptMR reconstructed image of (e): Figure 12 (f): Prompt MMR-Net reconstructed image of (f). DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0072] Related background:

[0073] Over the years, in order to accelerate MRI acquisition, people have developed various strategies. Compressed Sensing (CS) theory was introduced into the field of MRI reconstruction by Lustig et al., providing a theoretical basis for reconstructing high-quality images from significantly undersampled data. CS relies on the assumption that most MRI images are sparse in a suitable transform domain (such as wavelet or curvelet transform domain). By exploiting this sparsity, images can be reconstructed with significantly fewer samples than required by the traditional Nyquist sampling theorem, significantly shortening the acquisition time. Although CS has been widely adopted, it is usually limited to the need to manually design regularizers based on prior knowledge of image structure, and may encounter difficulties in dealing with high noise or undersampled data.

[0074] The joint optimization method framework of the sampling model based on deep learning and the reconstruction algorithm is as follows:

[0075] Let represent an unknown fully sampled MRI image. In the traditional MRI compressed sensing framework, represent k spatially undersampled image data, which is processed by a Fourier transform operator and a mask operator to obtain a fully sampled MRI image. The relationship between the fully sampled MRI image and its corresponding undersampled k spatial signal is as follows:

[0076] .

[0077] wherein, is the measurement noise, H and W represent the height and width of the fully sampled MRI image respectively, represent the complex domain.

[0078] The classical compressed sensing method realizes reconstruction by optimizing the following energy equation (original objective function):

[0079] .

[0080] wherein, represents a data consistency term, is a regularization term applied to , for imposing prior constraints (such as sparsity), is a balance parameter for adjusting the relative weight of the two terms.

[0081] Based on this, the application provides a multi-contrast MRI joint reconstruction method, in particular, a multi-contrast MRI joint reconstruction method based on prompt enhancement. The method aims to fully exploit the complementary information between different contrast images, and improve the reconstruction quality and structural detail fidelity under the condition of undersampling. By constructing an end-to-end deep learning framework, the application effectively combines contrast adaptive guidance, cross-contrast feature fusion and frequency domain physical consistency constraints, realizes the collaborative modeling and high-quality reconstruction of multi-contrast images, and is suitable for clinical and scientific research needs in various magnetic resonance imaging scenes. Specifically, the following steps are included:

[0082] S1: Collect undersampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts.

[0083] The input of the present application is under-sampled magnetic resonance image frequency domain data under different contrasts, specifically including but not limited to T1-weighted, T2-weighted and fluid attenuated inversion recovery sequence (FLAIR) and the like under-sampled multi-contrast images, each contrast image is processed by different sampling masks to form under-sampled frequency domain data.

[0084] In order to extend the MRI reconstruction from single contrast to multi-contrast joint method, the correlation between multi-contrast data must be considered. In the prior art, single contrast MRI reconstruction usually relies on single contrast data for image reconstruction, but due to the differences between different contrast data in the acquisition process, direct reconstruction using single contrast may not fully utilize all the contrast information. Therefore, in order to integrate the correlation between multi-contrast data and ensure that the information of different contrasts can be effectively fused, thereby improving the quality of the reconstructed image, the present application constructs the objective function as follows:

[0085] ;

[0086] Wherein, is a multi-contrast data consistency term, represents the first contrast. represents a contrast-enhanced correlation regularization term for imposing prior constraints (such as sparsity, correlation between contrasts), is a balance parameter for adjusting the relative weights of the two terms.

[0087] In view of the continuous development of prompt engineering in large-scale models and its key role in feature guidance, a prompt function is introduced for each contrast image to enhance the guiding ability of the information between contrasts, and the objective function is updated accordingly to obtain an optimized objective function:

[0088] ;

[0089] The construction of the optimized objective function provides a mathematical framework for joint reconstruction, and by optimizing the function, the features of multiple contrasts can be considered at the same time, and their complementary information can be maximally utilized in the reconstruction process. Therefore, the design of the objective function not only improves the prior art, but also is the key to realizing effective fusion and high-quality image output for multi-contrast MRI reconstruction.

[0090] S2: Introduce auxiliary variables to the optimized objective function, and decompose the optimized objective function into a first sub-problem about the auxiliary variables and a second sub-problem about the target variables.

[0091] Constructing MRI reconstruction network: Given the effectiveness of the Half-Quadratic Splitting (HQS) method in image inverse problems, an auxiliary variable is introduced The optimization objective function is reformulated as follows:

[0092] ;

[0093] where, is a penalty parameter.

[0094] S3: Alternately solve the first and second sub-problems.

[0095] The above equation can be decomposed into two sub-problems and solved alternately:

[0096] First sub-problem: Update , given the reconstructed image at the th iteration, the proximal operator is defined according to the correlation between the contrasts .

[0097] ;

[0098] Second sub-problem: Update , the data consistency term is related to the differentiable Frobenius constraint, so there is a closed-form solution:

[0099] ;

[0100] ;

[0101] where, denotes the position of the sampling point, denotes the set of sampling points.

[0102] Expand into a proximal operator network, and construct a multi-contrast magnetic resonance image reconstruction network Prompt MMR-Net through the alternating iteration of and .

[0103] Deep unfolding networks (DUNs) directly integrate domain knowledge into the network training process by simulating the process of traditional reconstruction methods, making the learning process more interpretable. Therefore, DUNs can effectively make up for the shortcomings of purely data-driven methods and provide more accurate and stable reconstruction results. In short, the role of DUNs is to combine the advantages of traditional optimization techniques and deep learning to provide a more efficient, interpretable, and accurate method for MRI image reconstruction.

[0104] Based on step S1, the present application proposes a prompt-based multi-contrast MRI reconstruction network (Prompt MMR-Net) as shown in Figure 2 By performing feature interaction of multi-contrast data in spatial and frequency domains, the network can utilize the complementary information between different contrasts without the need for pre-alignment. In addition, by perceiving the features of each contrast, targeted prompts can be generated to guide the reconstruction process. Thus, the quality of low-quality contrasts is improved, and the overall reconstruction result is enhanced.

[0105] Prompt MMR-Net can take multiple contrast undersampled magnetic resonance image data as input, perform joint processing and interactive fusion, and thus simultaneously reconstruct high-quality images of each contrast. As shown in Figure 1 The structure is composed of multiple functionally clear and complementary modules, mainly including a multi-contrast interactive reconstruction network (MCIR-Net), a contrast information perception prompt module (CIPP), and a data consistency module (DCM).

[0106] MCIR-Net: MCIR-Net, as the main backbone of overall reconstruction, adopts a U-Net-like encoder-bottleneck-decoder structure, with multi-scale and cross-contrast fusion capabilities.

[0107] The encoder includes multiple encoding layers, each layer extracts initial features by 3x3 convolution and connects CAB to enhance important features; multi-layer down-sampling operations are used to extract deep semantic information to obtain semantic feature maps of different resolutions; after down-sampling, the features are sent to the intermediate bottleneck layer to converge multi-scale feature maps.

[0108] Each decoding layer includes a cross-domain multi-contrast interaction module and an adaptive spatial-frequency fusion module (ASFF) proposed in the MMR-Mamba method. Each layer gradually recovers the spatial resolution through an upsampling operation and outputs a reconstructed feature map fused with the corresponding encoded layer feature; each decoding layer forms a skip connection with the symmetric encoding layer to retain low-level features. More importantly, each decoding layer also fuses multi-contrast supplementary information provided by the cross-domain multi-contrast interaction module (CD-MIM) module; at the same time, the prompt feature output by the CIPP module is introduced to guide the contrast enhancement.

[0109] To alleviate the problem of over-smoothing of shallow details in deep layers in the U-Net structure, the Prompt MMR-Net introduces a cross-stage information path, which separately transports the encoding and decoding features of different levels to the next stage encoder through two independent 1x1 convolution layers for additive fusion, realizing complementary fusion of shallow details and deep semantic information, and preserving key edges and texture details.

[0110] The CD-MIM module takes the intermediate feature map output by the previous decoding layer or bottleneck layer (i.e., the high-dimensional feature tensor of the corresponding layer) as input, which is a key component of the MCIR-Net for complementary fusion of multi-contrast information and runs through all decoding stages. As shown in Figure 3 The module integrates two core sub-modules: a spatial dynamic fusion module (SDFM) and a frequency selective fusion module (FSFM). The spatial dynamic fusion module SDFM adaptively adjusts the fusion weights of each contrast feature in the spatial domain (i.e., two-dimensional pixel distribution structure) of the image; while the frequency selective fusion module FSFM extracts and fuses discriminative frequency components in the frequency representation of the feature (such as the Fourier domain). Figure 3 In the formula, FFT and IFFT represent Fourier transform and inverse Fourier transform, respectively, which are used to realize round-trip conversion between the feature frequency domain and the image domain to support cross-domain feature fusion.

[0111] Referring to Figure 4, SDFM includes a splicing module, a first CAB module, a downsampling module, a first mapping module, a second CAB module, a second mapping module, and a weighting module. The first mapping module and the second mapping module both use a Sigmoid activation function. In the SDFM, each processing cycle jointly processes multiple contrasts, specifies one contrast as the current "target contrast", and the remaining contrasts as "auxiliary contrasts" to participate in fusion, thereby realizing complementary use of information. SDFM focuses on extracting transferable spatial features from auxiliary contrasts to enhance the representation ability of the current contrast. The network structure is designed as follows:

[0112] The features of the current target contrast and the features of other auxiliary contrasts are spliced in the channel dimension through the splicing module to obtain a splicing result.

[0113] The splicing result is sent to the first CAB module to extract high-quality structural information to obtain spatial structure information; the spatial structure information is then matched to the target contrast size through the downsampling module to obtain a matching result; and the matching result is processed through the first mapping module to generate a weight map used to guide fusion.

[0114] Meanwhile, the second CAB module extracts deep features from the target contrast features to obtain structural representation information; the second mapping module maps the structural representation information to obtain a spatial structure attention map. Both of them use a dynamic weighting strategy, and the parameters learned are used to control the fusion strength of auxiliary information. Finally, the spatial enhancement features of the target contrast are output as the spatial enhancement features of the target contrast and are transmitted to the main network decoding layer.

[0115] Referring to Figure 5 FSFM, on the other hand, focuses on contrast interaction in the frequency domain. It includes a global average pooling layer (Global Average Pooling, GAP), a fully connected layer (Fully Connected, FC), a batch normalization layer (Batch Normalization, BN), a third mapping module, and a Softmax module. The third mapping module uses a ReLU activation function. In each round of fusion, one contrast is set as the "target contrast", and the remaining contrasts are sequentially set as "reference contrasts". The frequency domain features of the two are obtained by performing Fourier transform on the input features. The structure design includes the following steps:

[0116] The frequency domain features of the target contrast and the current reference contrast are fused to obtain a joint frequency spectrum.

[0117] The joint spectrum map extracts channel-level statistical features through a global average pooling layer; after a fully connected layer, a batch normalization layer, and a ReLU activation function, two contrast ratio-specific channel attention weights are generated through Softmax; the two contrast ratio frequency domain feature maps are weighted using these weights, and then weighted fusion is performed; the output frequency domain enhancement features are fed back to the backbone network for further reconstruction.

[0118] The two parallel sub-modules jointly construct the CD-MIM module, which further combines the adaptive spatial-frequency fusion module (ASFF) proposed in the MMR-Mamba method, to complementarily integrate multi-contrast features at the spatial and frequency levels, thereby improving the reconstruction performance.

[0119] CIPP: The CIPP module is designed to introduce contrast perception capability at different decoding stages of the network, providing structured prompt information for different contrasts, referred to as PromptBlock. As shown in Figure 6 , the network structure mainly includes a prompt generation unit and a prompt fusion unit. The prompt generation unit includes a one-hot encoding module, an interpolation module, and a convolution module. The prompt fusion unit includes a concatenation module, a CAB module, and a down-sampling module.

[0120] The prompt generation unit is used to configure a learnable prompt vector p for each contrast x , and the output features (semantic enhancement features) of each decoding layer are used as input w , and after One-hot (one-hot encoding module) encoding, a weight vector for selecting the corresponding prompt is obtained p . The selected prompt vector x is expanded to the same size as the input feature through the interpolation module, and further spatial structure features (prompt features) are extracted through 3x3 convolution (convolution module).

[0121] x The concatenation module concatenates the above prompt features with the output features of each decoding layer P in the channel dimension, and then uses the channel attention module (CAB) proposed in the PromptMR method for feature weighting fusion. The weighted fusion result is down-sampled by Down (down-sampling module) to match the feature scale of the subsequent backbone network, and is fed into the multi-scale reconstruction path as the contrast prompt

[0122] DCM: Located at the end of each reconstruction stage, the DCM module applies frequency domain consistency constraints to the network output image, ensuring a strict match between its sampling location and the original frequency domain observation data. Based on undersampled mask information, this module physically maintains the reliability and clinical usability of the reconstructed image.

[0123] In summary, the Prompt MMR-Net proposed in this invention achieves a joint reconstruction framework that balances structural representation enhancement, contrast interaction optimization, and physical constraints through CIPP guidance, cross-stage information connection, CD-MIM fusion, and DCM correction. It is suitable for high-fidelity image restoration tasks in scenarios such as multi-contrast magnetic resonance imaging reconstruction.

[0124] The output of this invention is a high-quality magnetic resonance image at various contrast levels. Through joint modeling and training of the above-mentioned multiple modules, the output image exhibits excellent performance in terms of visual detail preservation, structural restoration, and noise suppression, and can meet the needs of various medical applications such as clinical image diagnosis and anatomical structure recognition.

[0125] S4: Perform end-to-end training on the reconstruction network Prompt MMR-Net, using normalized L1 norm error as the loss function for network training:

[0126] ;

[0127] in, The number of contrast elements. To rebuild the network for the first Output with contrast ratio The corresponding fully sampled image is used. The gradient of the loss function with respect to the network parameters is calculated using the backpropagation algorithm, and the Adam optimizer is used to optimize the network parameters.

[0128] S5: Apply the trained reconstruction network Prompt MMR-Net to reconstruct MRI images: the input is undersampled data in k-space, and the output is the reconstructed MRI image.

[0129] Example 2

[0130] In the numerical experiment, the volunteer brain data set and the hospital clinical brain data set are used for experiment. The volunteer brain data includes FLAIR, T1, T2 three contrasts, the training data amount is 7661, the test data amount is 1860, and the 10x2D Cartesian sampling mode is used. The hospital clinical brain data set also includes FLAIR, T1, T2 three contrasts, the training data amount is 2020, the test data amount is 324, and the 8x1D Cartesian sampling mode is used, the center sampling rate is 8%, and the high frequency is randomly selected. Since the original data sizes of different contrasts collected by the data set are different, the image domain and K space are filled with 0 to align the sizes.

[0131] In the experiments of the volunteer brain data set (Table 1) and the hospital clinical brain data set (Table 2), the Prompt MMR-Net method proposed in the application shows significant performance advantages under different contrasts (T1, T2, FLAIR) and different sampling rates.

[0132] Table 1: Comparison results of different methods in the volunteer brain data

[0133]

[0134] Table 2: Comparison results of different methods in the hospital clinical brain data

[0135]

[0136] Zero-Filling refers to a reconstruction method of directly filling the unsampled region with zero in the undersampled k-space data and performing inverse Fourier transform to obtain the image, which is the most basic prior-free fast reconstruction baseline. Restormer can capture long-distance pixel interaction and is an efficient Transformer model that can be applied to the task of restoring high-resolution images. MC-CDic is a multi-scale convolution dictionary model based on deep unfolding, which explicitly decomposes the common and difference features of multi-contrast MRI and realizes end-to-end optimization with a learnable proximity operator, can fully fuse multi-contrast complementary information in reconstruction and super-resolution tasks, and realize high-precision and interpretable MRI reconstruction. PromptMR is a full-scene MRI reconstruction method based on prompt learning, which fills the missing data in k-space and fuses features in the image domain to realize unified high-precision reconstruction of multi-contrast, dynamic and multi-view MRI data.

[0137] The specific performance is as follows:

[0138] Peak Signal-to-Noise Ratio (PSNR): PSNR is a commonly used metric to measure the error between the reconstructed image and the ground truth image. A higher value indicates better reconstruction quality. In Table 1, the PSNR values of Prompt MMR-Net for T1, T2, and FLAIR contrasts are 34.69, 34.75, and 32.04, respectively, which are higher than other comparison methods. In Table 2, the PSNR values are 35.58, 33.05, and 33.45, respectively, which are also superior to other methods.

[0139] Structural Similarity Index Measure (SSIM): SSIM is used to measure the structural similarity between the reconstructed image and the ground truth image. A value closer to 1 indicates that the structural information is closer. In Table 1, the SSIM values of Prompt MMR-Net are 0.9443, 0.9430, and 0.9066, respectively; in Table 2, the SSIM values are 0.9450, 0.9063, and 0.9149, respectively, which are higher than other methods, indicating that this method performs well in preserving image structural information.

[0140] Normalized Root Mean Square Error (NRMSE): NRMSE is used to measure the error between the reconstructed image and the ground truth image. A lower value indicates better reconstruction quality. In Table 1, the NRMSE values of Prompt MMR-Net are 0.0815, 0.0902, and 0.1167, respectively; in Table 2, the NRMSE values are 0.0747, 0.1155, and 0.1112, respectively, which are lower than other methods, indicating that this method has a significant advantage in reducing reconstruction error.

[0141] Reference Figures 7-12 As can be seen from the figure:

[0142] Detail preservation: The images reconstructed by Prompt MMR-Net can clearly preserve the detailed information of the tissue structure, especially in the anatomical structure of the brain and the pathological area, which is highly consistent with the full-sampling image.

[0143] Noise suppression: There are no obvious noise artifacts in the reconstructed image, indicating that this method can effectively suppress noise when processing undersampled data and improve image quality.

[0144] Contrast consistency: In the reconstruction of multi-contrast data (T1, T2, FLAIR), Prompt MMR-Net can fully utilize the complementary information between different contrasts to generate high-quality joint reconstruction images, verifying its effectiveness and superiority in multi-contrast MRI reconstruction.

[0145] Based on the quantitative analysis of Table 1 and Table 2 and the qualitative analysis of Table 3, Figures 7-12 It can be clearly seen that the Prompt MMR-Net method proposed in the present application has a significant advantage in the multi-contrast MRI joint reconstruction task. This method not only outperforms existing algorithms in numerical indicators, but also performs well in detail preservation and noise suppression of images, and can provide high-quality reconstructed images for clinical diagnosis. In addition, this method performs stably under different contrasts, and has wide applicability and practicality.

[0146] Embodiment 3

[0147] Based on the same concept, the present application also provides a multi-contrast MRI joint reconstruction system, comprising a generation module, a decomposition module and a solving module.

[0148] The generation module is used to collect under-sampled MRI image data of different contrasts, and generate an optimization objective function based on multiple contrasts; wherein the data consistency term in the original objective function of the compressed sensing algorithm is optimized to a multi-contrast data consistency term, the regularization term is optimized to a contrast-enhanced correlation regularization term, and a corresponding prompt function is introduced based on each contrast to obtain the optimization objective function.

[0149] The decomposition module is used to introduce auxiliary variables to the optimization objective function, and decompose the optimization objective function into a first sub-problem about the auxiliary variables and a second sub-problem about the target variables, the target variables being MRI reconstruction images corresponding to the under-sampled MRI image data of different contrasts.

[0150] The solving module is used to sequentially solve the first sub-problem and the second sub-problem to obtain multiple optimal MRI reconstruction images; in the solving process of the first sub-problem, the under-sampled MRI image data is encoded layer by layer to obtain multiple semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain multiple semantic enhanced feature maps; in each decoding process, a corresponding contrast prompt, a frequency domain enhanced feature and a spatial enhanced feature are generated for the current semantic enhanced feature map based on different contrasts; the contrast prompt, the frequency domain enhanced feature and the spatial enhanced feature of the current layer are fused to obtain the semantic enhanced feature map of the next layer.

[0151] Embodiment 4

[0152] The present application also provides a computer device, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the multi-contrast MRI joint reconstruction method described above when executing the program.

[0153] Embodiment 5

[0154] The application further provides a computer readable storage medium, the storage medium storing a computer program, and the computer program is executed by a processor to realize the multi-contrast MRI joint reconstruction method.

[0155] Although preferred embodiments of the application have been described herein, additional changes and modifications can be made by those skilled in the art once they learn of the basic inventive concepts. Therefore, the appended claims are intended to cover all such changes and modifications that fall within the scope of the application.

[0156] Obviously, various modifications and changes can be made to the present application by those skilled in the art without departing from the spirit and scope of the application. Accordingly, it is intended that the application embrace all such modifications and changes as fall within the scope of the appended claims and their equivalents.

Claims

1. A multi-contrast MRI joint reconstruction method, characterized in that, The method comprises the following steps: Collecting under-sampling MRI image data of different contrasts, generating an optimization objective function based on multiple contrasts; wherein, the data consistency term in the original objective function of the compressed sensing algorithm is optimized into a multi-contrast data consistency term, the regularization term is optimized into a contrast-enhanced correlation regularization term, and a corresponding hint function is introduced based on each contrast to obtain the optimization objective function; Introducing an auxiliary variable to the optimization objective function, decomposing the optimization objective function into a first sub-problem about the auxiliary variable and a second sub-problem about the target variable, the target variable being the MRI reconstruction image corresponding to the under-sampling MRI image data of different contrasts; Solving the first sub-problem and the second sub-problem alternately to obtain multiple optimal MRI reconstruction images; in the solving process of the first sub-problem, the under-sampling MRI image data is encoded layer by layer to obtain multiple semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain multiple semantic enhanced feature maps; in each decoding process, a corresponding contrast hint, a frequency domain enhanced feature and a spatial enhanced feature are generated for the current semantic enhanced feature map based on different contrasts; the contrast hint, the frequency domain enhanced feature and the spatial enhanced feature of the current layer are fused to obtain the semantic enhanced feature map of the next layer.

2. The multi-contrast MRI joint reconstruction method of claim 1, wherein, The optimization objective function is specifically as follows: ； wherein x i denotes the full-sampled image of the i contrast, N is the number of contrasts, is the k spatially under-sampled MRI image data, M i denotes the mask operator of the i contrast, is the Fourier transform operator, is the balancing parameter, is the contrast-enhanced correlation regularization term, is the indicator function.

3. The multi-contrast MRI joint reconstruction method of claim 1, wherein, The first sub-problem and the second sub-problem are solved alternately through an unfolding network, the unfolding network comprising multiple unfolding modules, each unfolding module comprising a reconstruction network, a hint module and a data consistency module, the reconstruction network comprising an encoder, a bottleneck layer and a decoder, the encoder comprising multiple encoding layers, the decoder comprising multiple decoding layers, the multiple encoding layers being connected to the multiple decoding layers in a skip connection manner, each decoding layer comprising a multi-contrast interaction module and an ASSF module; The multiple encoding layers encode the under-sampling MRI image data layer by layer to obtain multiple semantic feature maps with different resolutions; The bottleneck layer converges the semantic feature maps with different resolutions; The multiple decoding layers decode the semantic feature map with the smallest resolution layer by layer to obtain multiple semantic enhanced feature maps; The data consistency module is used to impose a frequency domain consistency constraint on the final output semantic enhanced feature map to obtain the reconstruction image of the current unfolding module.

4. The multi-contrast MRI joint reconstruction method of claim 3, wherein, Each multi-contrast interaction module is used to generate corresponding frequency domain enhanced features and spatial enhanced features based on different contrasts, and each multi-contrast interaction module comprises a spatial dynamic fusion module and a frequency domain selective fusion module; The spatial dynamic fusion module comprises a splicing module, a first CAB module, a down-sampling module, a first mapping module, a second CAB module, a second mapping module and a weighting module; In the spatial dynamic fusion module, one contrast is specified as a target contrast and the remaining contrasts are auxiliary contrasts; the splicing module splices the target contrast feature and the auxiliary contrast feature to obtain a splicing result; The first CAB module extracts features from the splicing result to obtain spatial structure information; The down-sampling module matches the spatial structure information to the size of the target contrast to obtain a matching result; The first mapping module maps the matching result to obtain a weight map; The second CAB module performs deep feature extraction on the target contrast feature to obtain structure representation information; The second mapping module maps the structure representation information to obtain a spatial structure attention map; The weighting module dynamically weights the target contrast feature, the weight map and the spatial structure attention map to obtain a spatial enhanced feature.

5. The multi-contrast MRI joint reconstruction method of claim 4, wherein, The frequency domain selective fusion module includes a global average pooling layer, a fully connected layer, a batch normalization layer, a third mapping module and a Softmax module; In the frequency domain selective fusion module, a contrast is specified as a target contrast, and the remaining contrasts are specified as reference contrasts, and frequency domain feature maps of the target contrast and the reference contrasts are obtained; The frequency domain feature maps of the target contrast and the reference contrasts are fused to obtain a joint frequency spectrum; The global average pooling layer extracts channels from the joint frequency spectrum to obtain statistical features; The fully connected layer, the batch normalization layer, the third mapping module and the Softmax module classify, normalize, map and output the statistical features to obtain channel attention weights of the target contrast and the reference contrasts; The channel attention weights are fused with the corresponding frequency domain feature maps to obtain frequency domain enhanced features; The ASSF module fuses the frequency domain enhanced features and the spatial enhanced features to obtain fused features.

6. The multi-contrast MRI joint reconstruction method of claim 5, wherein, The prompt module includes a prompt generation unit and a prompt fusion unit; the prompt generation unit includes a one-hot encoding module, an interpolation module and a convolution module, and the prompt fusion unit includes a concatenation module, a CAB module and a down-sampling module; One-hot encoding module x One-hot encoding is performed to obtain a weight vector w , the weight vector w for selecting the corresponding prompt vector p ; The interpolation module selects the prompt vector p The interpolation processing is performed, and is extended to the current semantic enhancement feature map x The size is consistent; the convolution module convolves the extended prompt vector p to obtain prompt features The splicing module splices the prompt feature and the x obtains a splicing result. The CAB module performs feature weighting fusion on the concatenation result to obtain a weighted fusion result; The downsampling module performs downsampling processing on the weighted result to obtain a contrast prompt corresponding to the current contrast P ; contrast prompt P is added to the fusion feature.

7. A multi-contrast MRI joint reconstruction system, characterized by, The method comprises the following steps: The generating module is configured to collect under-sampled MRI image data of different contrasts, and generate an optimized objective function based on multiple contrasts; wherein, a data consistency term in an original objective function of a compressed sensing algorithm is optimized as a multi-contrast data consistency term, a regularization term is optimized as a contrast-enhanced correlation regularization term, and a corresponding prompt function is introduced based on each contrast to obtain the optimized objective function; The decomposition module is configured to introduce an auxiliary variable into the optimized objective function, and decompose the optimized objective function into a first sub-problem about the auxiliary variable and a second sub-problem about a target variable, wherein the target variable is an MRI reconstruction image corresponding to the under-sampled MRI image data of different contrasts; The solving module is configured to sequentially solve the first sub-problem and the second sub-problem to obtain multiple optimal MRI reconstruction images; in the process of solving the first sub-problem, the under-sampled MRI image data is encoded layer by layer to obtain multiple semantic feature maps; the semantic feature map with the smallest resolution is decoded layer by layer to obtain multiple semantic enhanced feature maps; in each decoding process, a corresponding contrast prompt, a frequency domain enhanced feature and a spatial enhanced feature are generated for the current semantic enhanced feature map based on different contrasts; the contrast prompt, the frequency domain enhanced feature and the spatial enhanced feature of the current layer are fused to obtain a semantic enhanced feature map of the next layer.

8. A computer device, comprising: The application also discloses a computer readable storage medium.

9. A computer-readable storage medium, characterized in that, The application also discloses a computer readable storage medium.

Images