Method for constructing MRI image reconstruction model based on residual denoising diffusion and application thereof

Abstract

The application provides a construction method and application of an MRI image reconstruction model based on residual denoising diffusion, including the following steps: acquiring a training data set, taking a noisy modality in a multi-modality MRI image in the training data set as a target modality and a non-noisy modality as a prior modality, acquiring a prior tumor mask based on the prior modality, taking each multi-modality MRI image in the training data set and the corresponding prior tumor mask as a training sample, inputting the training sample into a constructed MRI image reconstruction architecture to obtain a reconstructed MRI image, and saving the parameters of the MRI image reconstruction architecture to obtain an MRI image reconstruction model when a loss function meets a set condition or reaches a preset iteration number. Each encoding level of the encoding module and each decoding level of the decoding module are provided with a double-path attention layer after the encoding level and the decoding level, so as to guide the model to focus on the tumor area and improve the feature extraction capability for complex lesions and atypical anatomical structures.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of medical image processing, in particular to a method for constructing a residual denoising diffusion-based MRI image reconstruction model and application thereof. BACKGROUND

[0002] According to the histopathological classification of the World Health Organization (WHO), glioma can be divided into grade I, grade II low-grade glioma (LGG) and grade III, grade IV high-grade glioma (HGG). As a core means for glioma diagnosis and treatment planning, magnetic resonance imaging (MRI) can provide key information such as tumor grading, positioning, morphology and size, which directly affects the selection of treatment plan and the survival expectation of patients.

[0003] In clinical practice, doctors usually rely on multi-sequence MRI for reliable diagnosis, because a single sequence often cannot fully describe the characteristics of the lesion. However, due to various factors such as patient psychological state, individual differences, scanning noise, etc., MRI images often have artifacts (such as Gibbs artifact, motion artifact), sequence missing or interpretation difficulty, etc., which seriously affect the accuracy of diagnosis.

[0004] In recent years, the development of deep learning technology has brought revolutionary breakthroughs in the field of medical image reconstruction. The generative adversarial network (GAN) significantly enhances the image structure preservation ability through the adversarial training mechanism, and its advanced variants have realized multi-to-multi modal synthesis, breaking the limitations of traditional single-modal conversion. However, the inherent mode collapse, training instability and super parameter sensitivity of GAN greatly hinder its clinical transformation.

[0005] As a robust alternative, denoising diffusion probability model (DDPM) has shown significant advantages in MRI image reconstruction with stable training dynamics and high-fidelity generation capabilities. Compared with traditional methods, this iterative optimization model performs outstandingly in maintaining anatomical structure consistency and reducing artifacts. Its application has expanded from image super-resolution, cross-modal synthesis and other generative tasks to lesion segmentation, tumor classification and other discriminative tasks, and has significantly reduced the mode collapse risk of the adversarial framework. On this basis, the residual denoising diffusion model (RDDM) realizes precise control of the denoising process by introducing a dual framework of residual diffusion and noise diffusion, effectively maintaining anatomical structure consistency and ensuring accurate restoration of generated images in details and structures.

[0006] Despite the continuous progress of technology, clinical transformation still faces two major challenges:

[0007] 1. Limitations of existing U-net denoising modules: The U-net standard architecture relies on skip connections for feature fusion, but when dealing with complex lesions such as glioma MRI images, it is difficult to effectively preserve high-frequency details, leading to blurred tumor boundaries and inaccurate reproduction of complex shapes. Especially in scenarios where the tumor boundary is blurred or the lesion shape is complex, the loss of fine-grained features is prominent, affecting the accuracy of lesion identification and localization.

[0008] 2. Insufficient generalization ability of RDDM: When faced with atypical anatomical manifestations (such as irregular tumor boundaries and low contrast with surrounding tissues), existing models struggle to effectively extract detailed features, leading to decreased image reconstruction accuracy and affecting the effectiveness of clinical diagnosis.

[0009] Therefore, there is an urgent need for a technical solution that can optimize feature extraction and fusion strategies and improve the reconstruction accuracy of complex lesions and atypical anatomical structures to address the shortcomings of existing technology in glioma MRI image reconstruction and provide a more reliable auxiliary tool for clinical diagnosis. SUMMARY

[0010] The embodiments of the present application provide a method for constructing an MRI image reconstruction model based on residual denoising diffusion and its application. By replacing traditional skip connections with deep separable gated networks, combining deep separable convolutions to effectively capture spatially adjacent pixel information and improve computational efficiency, and precisely regulating inter-layer information flow with gating mechanisms, fine control of features is achieved, enhancing cross-layer information fusion, effectively promoting image detail recovery while maintaining global context consistency, thereby improving the accuracy of glioma MRI image reconstruction.

[0011] In a first aspect, the embodiments of the present application provide a method for constructing an MRI image reconstruction model based on residual denoising diffusion, which comprises:

[0012] Obtaining a first data set storing multiple modal MRI images, step-by-step adding noise to any modality of each multi-modal MRI image in the first data set to obtain a second data set, and using the second data set as a training data set;

[0013] The noise-added modality in the multi-modality MRI image in the training data set is taken as a target modality, and the non-noise-added modality is taken as a prior modality, a prior tumor mask is obtained based on the prior modality, each multi-modality MRI image in the training data set and the corresponding prior tumor mask are taken as training samples, and the training samples are input into the constructed MRI image reconstruction architecture to obtain a reconstructed MRI image, the MRI image reconstruction architecture takes a U-net network as a basic network, includes an encoding module, a decoding module and a residual module, and a double-path attention layer is arranged after each encoding level of the encoding module and each decoding level of the decoding module, wherein the encoding level / decoding level processes the multi-modality MRI image in the training sample, and the double-path attention layer performs attention calculation on the output feature map of the encoding level / decoding level based on the prior tumor mask in the training sample.

[0014] The reconstructed MRI image is verified based on the multi-modality MRI image in the first data set, and a loss function is calculated, the parameters of the MRI image reconstruction architecture are adjusted based on the loss function, and when the loss function meets a set condition or reaches a preset iteration number, the parameters of the MRI image reconstruction architecture are saved to obtain an MRI image reconstruction model.

[0015] In a second aspect, the embodiments of the present application provide an MRI image reconstruction method, including:

[0016] An MRI image to be reconstructed is obtained, the MRI image to be reconstructed is a multi-modality MRI image with a missing single modality or an unusable modality, a missing modality or an unusable modality in the MRI image to be reconstructed is marked as a target modality and input into a constructed MRI image reconstruction model, and the MRI image reconstruction model reconstructs the target modality in the MRI image to be reconstructed to obtain a full-modality MRI image.

[0017] In a third aspect, the embodiments of the present application provide an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute a construction method of an MRI image reconstruction model based on residual denoising diffusion.

[0018] In a fourth aspect, the embodiments of the present application provide a readable storage medium, the readable storage medium stores a computer program, the computer program includes program code for controlling a process to execute a process, and the process includes a construction method of an MRI image reconstruction model based on residual denoising diffusion.

[0019] The main contributions and innovations of the present application are as follows:

[0020] The embodiment of the present application adopts a deep separable gating network to replace the traditional skip connection, combines the deep separable convolution and the gating mechanism, realizes fine regulation of features, enhances cross-layer information fusion, effectively promotes image detail recovery and maintains global context consistency; the embodiment of the present application sets a double-path attention layer after each coding level of the coding module and each decoding level of the decoding module, performs attention calculation based on the prior tumor mask, guides the model to focus on the tumor area, and improves the feature extraction capability for complex lesions and atypical anatomical structures; the embodiment of the present application takes the U-net network as the basis, the first three coding levels of the coding module are residual down-sampling layers, and the last two are coding overlapping block embedding networks; the first two decoding levels of the decoding module are decoding overlapping block embedding networks, and the last three are residual up-sampling layers; through reasonable hierarchical structure design, the feature extraction and processing process is optimized, which helps to improve the brain glioma MRI image reconstruction accuracy; the residual module of the embodiment of the present application adopts a double-path deep separable convolution to process the output feature map and combines the gating mechanism, further enhances cross-layer information fusion, promotes image detail recovery, maintains global context consistency, and improves reconstruction accuracy.

[0021] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0022] The drawings described herein are intended to provide further understanding of the present application, form a part of the present application, and are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0023] Figure 1 is a whole structure diagram of an MRI image reconstruction model based on residual denoising diffusion according to an embodiment of the present application;

[0024] Figure 2 is a structure diagram of a coding overlapping embedding network according to an embodiment of the present application;

[0025] Figure 3 is a structure diagram of a bottleneck module according to an embodiment of the present application;

[0026] Figure 4 is a structure diagram of a residual module according to an embodiment of the present application;

[0027] Figure 5 is a structure diagram of a double-path attention layer according to an embodiment of the present application;

[0028] Figure 6 is a hardware structure schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description herein relates to the drawings, in which the same numbers represent the same or similar elements, unless otherwise represented. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of the description. Instead, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the description, as detailed in the appended claims.

[0030] It should be noted that the steps of the methods in other embodiments are not necessarily performed in the order shown and described in the description. In some other embodiments, the steps included in the methods can be more or less than described in the description. Furthermore, a single step described in the description can be broken down into multiple steps in other embodiments; and multiple steps described in the description can be combined into a single step in other embodiments.

[0031] Embodiment One

[0032] The embodiment of the present application provides a construction method of an MRI image reconstruction model based on residual denoising diffusion, by adopting a depth separable gate network to replace a traditional skip connection, combining effective capture of spatial adjacent pixel information and improvement of calculation efficiency of the depth separable convolution, and precise adjustment of interlayer information flow of the gate mechanism, fine regulation of features is realized, cross-layer information fusion is enhanced, image detail recovery is effectively promoted and global context consistency is maintained, so that the reconstruction precision of the brain glioma MRI image is improved. Specifically, referring to Figure 1 , the method comprises:

[0033] acquire a first data set in which multi-modal MRI images are stored, step by step add noise to any modality of each multi-modal MRI image in the first data set to obtain a second data set, and use the second data set as a training data set;

[0034] The noisy mode in the multimodal MRI image in the training data set is used as the target mode, and the non-noisy mode is used as the prior mode. A prior tumor mask is obtained based on the prior mode. Each multimodal MRI image in the training data set and the corresponding prior tumor mask are used as training samples. The training samples are input into the constructed MRI image reconstruction architecture to obtain a reconstructed MRI image. The MRI image reconstruction architecture is based on a U-net network, including an encoding module, a decoding module, and a residual module, and a dual-path attention layer is set after each encoding level of the encoding module and each decoding level of the decoding module, wherein the encoding level / decoding level processes the multimodal MRI image in the training sample, and the dual-path attention layer performs attention calculation on the output feature map of the encoding level / decoding level based on the prior tumor mask in the training sample;

[0035] The reconstructed MRI image is verified based on the multimodal MRI image in the first data set and the loss function is calculated. The parameters of the MRI image reconstruction architecture are adjusted based on the loss function. When the loss function meets the set conditions or reaches the preset number of iterations, the parameters of the MRI image reconstruction architecture are saved to obtain the MRI image reconstruction model.

[0036] In some specific embodiments, the first data set described in this solution can be multimodal MRI images directly acquired in various hospital systems, or an open source data set can be directly acquired, such as BraTS2021.

[0037] In some embodiments, the present scheme gradually adds noise to any modality of each multimodal MRI image in the first data set at a preset time step. In the present scheme, the multimodal MRI images include T1 modality, T1ce modality, T2 modality and FLAIR modality. In the process of gradually adding noise to the multimodal MRI images in the first data set, the number of images added with noise in T1 modality, T1ce modality, T2 modality and FLAIR modality is made basically the same, thereby ensuring that the trained MRI image reconstruction model can reconstruct MRI images of any modality.

[0038] In some specific embodiments, when obtaining a prior tumor mask, this solution uses a segmentation algorithm to process any one or more non-noise-added modes in a multimodal MRI image to obtain a prior tumor mask. For example, when the T1 mode is a noisy mode, the T1ce mode, T2 mode and FLAIR mode are non-noise-added modes, and a segmentation algorithm is used to segment the T1ce mode or T2 mode or FLAIR mode to obtain a prior tumor mask, or a weighted average of the segmentation results of the T1ce mode, T2 mode and FLAIR mode is performed to obtain a prior tumor mask.

[0039] Specifically, taking the prior tumor mask as the model input can guide the model to pay more attention to the tumor area, thereby better reconstructing the missing modalities.

[0040] In some embodiments, the encoding module includes 5 encoding levels, the input of each encoding level is an input feature map, and the output is an output feature map. The first three encoding levels are residual down-sampling layers, and the last two encoding levels are encoding overlapping block embedding networks. The decoding module includes 5 decoding levels, the first two decoding levels are decoding overlapping block embedding networks, and the last three decoding levels are residual up-sampling layers.

[0041] Specifically, before processing the input feature map using the encoding level, the input feature map is mapped to an initial feature dimension. For example, the input feature map is represented as where B is the batch size, C is the number of channels, H is the width of the input feature map, and W is the height of the input feature map. A 7x7 convolutional layer is used to process the input feature map, thereby providing an appropriate feature representation for subsequent processing, which is represented by the formula:

[0042]

[0043] Further, a time vector of a preset time step is obtained, and the time vector is processed through two linear transformation units in succession to obtain a time embedding vector. The time embedding vector is used to weight each encoding level and decoding level.

[0044] Specifically, a sinusoidal and cosine vector of a preset time step is obtained using SinusoidalPosEmb, which is represented by the formula:

[0045]

[0046] where time i is the preset time step, and is the corresponding frequency coefficient.

[0047] Specifically, the time vector is processed through two linear transformation units in succession to obtain a time embedding vector, which is represented by the formula:

[0048]

[0049]

[0050] where is the weight of the first linear transformation, is the bias of the first linear transformation, is the weight of the second linear transformation, is the bias of the second linear transformation, is an activation function.

[0051] Specifically, by using time embedding to weight each encoding level and decoding level, the features can be modulated and optimized in the time dimension.

[0052] Specifically, in the encoding level, the down-sampled feature map is combined with the time embedding and further processed through a residual connection, and the down-sampling formula is as follows:

[0053]

[0054] where X is the input feature map, Downsample represents down-sampling, is the down-sampling result.

[0055] In some embodiments, the structure of the encoding overlapping block embedding network is as shown in Figure 2 The encoding overlapping block embedding network is composed of an overlapping patch embedding layer, a transformation layer, a KAN network, and a normalization layer. The overlapping patch embedding layer converts the input feature map into an embedding feature map. The transformation layer shifts the embedding feature map to obtain a transformed feature map. The KAN network processes the transformed feature map and performs a residual connection with the input feature map, and the output is obtained by the normalization layer. The decoding overlapping block embedding network has the same structure as the encoding overlapping block embedding network.

[0056] Specifically, the overlapping patch embedding layer realizes the function of overlapping patch embedding, converts the input image into embedding features through a convolution layer, and adjusts the shape of the features in the forward propagation, so as to facilitate the subsequent KAN network processing.

[0057] Specifically, the transformation layer shifts the embedding feature map left and right and up and down through the Shift operation, helps the model to capture more rich context information and spatial safety relationship, better understands the local and global features of the input data, and thus improves the overall performance.

[0058] Specifically, the KAN network is constructed by a series of KAN layers connected in series. Each KAN layer contains a set of learnable one-dimensional activation functions. Through the stacking of multiple KAN layers, the model can learn and extract complex features of high-dimensional data layer by layer. Each KAN layer performs non-linear transformation on the input through the learnable activation function, thereby enhancing the expression ability of the model. Finally, these layers effectively capture the complex relationships and potential patterns in the input data through deep processing, thereby improving the performance of the model. The specific calculation formula of the KAN network is as follows:

[0059]

[0060] where, represents the i-th layer of the entire KAN network, each KAN layer has n inInput and n out Output, including n in *n out learnable activation function.

[0061] In some other embodiments, the decoding overlapping block embedding network uses D_Single convolution for decoding, which enhances the feature representation through the processing of convolutional layers and temporal embeddings. The design of this module allows complex transformations in the feature space while preserving temporal information, thereby improving the performance of the model in image reconstruction. By fusing temporal embeddings, the model can better understand the dynamic changes of the input data, enhancing the generation effect.

[0062] In some embodiments, a bottleneck module is arranged between the encoding module and the decoding module, and the structure of the bottleneck module is as shown in Figure 3 The bottleneck module inputs the output feature map of the encoding module after self-attention calculation.

[0063] Specifically, the self-attention mechanism added in the bottleneck part can help the model capture global information and overcome the limitations of local regions by calculating the relationship between features and weighting input features. The self-attention mechanism expression is as follows:

[0064]

[0065] where d k represents the feature dimension, and Q, K, and V represent query, key, and value, respectively.

[0066] In some embodiments, the structure of the residual module is as shown in Figure 4 The residual module adopts a double-path depth separable convolution to process the output feature map to obtain a first depth separable feature and a second depth separable feature, respectively. After smoothing the first depth separable feature, the second depth separable feature is calculated by element-wise multiplication to obtain a gating feature. The gating feature and the output feature map are connected in residual to output.

[0067] Specifically, in the U-net structure, it is known that the residual module is used to connect the output feature map of each encoding level in the encoding module with the input of the corresponding size of the decoding level in residual, thereby further enhancing the fusion of cross-layer information, promoting the recovery of image details, and maintaining the consistency of the global context, thereby improving the accuracy of image reconstruction.

[0068] Specifically, for the depth separable convolution, a convolution kernel is independently applied to each channel, and the formula of the depth separable convolution is represented as:

[0069]

[0070] wherein, is the result of separable convolution, X is the input feature map, K is the convolution kernel, i, j are spatial positions, k is the channel index, m is the number of input channels, and n is the number of output channels, the deep separable convolution can effectively capture the information from the spatial adjacent pixels while greatly improving the computational efficiency, and the gating mechanism adjusts the information flow between layers, so that the network can highlight or suppress specific features as needed.

[0071] Specifically, the gating mechanism controls the feature flow through element-wise multiplication, which is expressed as:

[0072]

[0073] wherein, is the gated feature, , is a learnable weight, is the first deep separable feature, is the second deep separable feature, and SiLU is an activation function.

[0074] Specifically, in the gating mechanism, the SiLU activation function provides smooth gradient flow in the gating mechanism, avoids the problem of dead neurons, and helps better convergence of the model.

[0075] Further, after the deep separable convolution and the gating mechanism, a 1x1 point convolution is used to adjust the number of channels.

[0076] In some embodiments, the structure of the dual-path attention layer is as shown in Figure 5 The dual-path attention layer takes the output feature map output by the encoding level / decoding level and the prior tumor mask of the corresponding size as input, and includes an implicit tensor attention path and a spatial attention path. The implicit tensor attention path maps the output feature map to a non-negative space after tensor segmentation to obtain an implicit tensor attention result. The spatial attention path takes the output feature map and the prior tumor mask of the corresponding size as input, and performs attention distribution on the output feature map based on the prior tumor mask to obtain a spatial attention result. The spatial attention result, the implicit tensor attention result, and the multi-modal MRI image feature map are added to obtain the output of the dual-path attention layer. It is worth mentioning that the output of the dual-path attention layer is taken as the input feature map of the next encoding level / decoding level.

[0077] Further, in the implicit tensor attention path, first, the input feature map is subjected to layer normalization operation, and then the result of the layer normalization is converted into query, key and value using 1*1 convolution operation. The result after the convolution operation is sliced into three tensors, representing Q, K and V respectively. The formula is as follows:

[0078]

[0079] wherein chunk represents the tensor slicing operation, chunk3, dim=1 represents slicing into 3 segments along the direction of dimension 1, and the shapes of Q, K and V are (b, h, c, h*w).

[0080] Then, the query Q and the key K are subjected to Softmax processing respectively, and the value V is normalized. The formula is as follows:

[0081]

[0082]

[0083]

[0084] After obtaining Q, K and V, Q and K are mapped to the non-negative space for attention calculation. The formula is as follows:

[0085]

[0086] wherein, is the attention calculation result, and softmax is a Softmax feature mapping function, which is used to map Q and K to the non-negative space.

[0087] Specifically, when performing attention calculation, first, and are calculated, and then multiplied by , so as to avoid explicit calculation of , thereby reducing the complexity of calculation.

[0088] Specifically, after obtaining the attention calculation result, the channel dimension of the attention calculation result is unified with the channel dimension of the input feature map, and then the implicit tensor attention result is obtained after layer normalization. That is, first, the attention result is subjected to feature reconstruction, and the multi-head output is spliced back to the input feature map format, that is, the dimension of [B, heads, dim_head, N] is converted to [B, C, H, W], the multi-head pathological features are integrated, and finally the channel dimension is adjusted through 1*1 convolution to match the input feature map. Finally, the implicit tensor attention result is output after the feature distribution is stabilized by using the LayerNorm layer normalization.

[0089] Further, the global context features of the input feature map are calculated by average pooling and max pooling in the spatial attention path, and the global context features are processed using a convolution layer to obtain a spatial attention map, the spatial attention map is multiplied with the prior tumor mask to obtain a spatial feature map, and the spatial feature map is multiplied with the input feature map element by element to obtain a spatial attention result.

[0090] Specifically, when processing the input feature map, the global context representation is calculated by average pooling and max pooling, and the mean and maximum values of each channel in the spatial dimension are obtained, then the two are concatenated along the channel dimension to obtain the global context features, which are more expressive feature representations, and then the concatenated features are processed by a convolution layer to generate a spatial attention map, which focuses on capturing key regions in the feature map.

[0091] Further, the size of the prior tumor mask is adjusted to be the same as the corresponding input feature map using bilinear interpolation, and the adjusted prior tumor mask is further processed by convolution to generate a mask with a size of (3, H, W). When calculating the spatial attention, the convolutional spatial attention map is normalized by Sigmoid activation, and the obtained spatial attention map is multiplied with the prior tumor mask, thereby introducing anatomical prior information to guide the allocation of attention. Finally, the obtained spatial feature map is multiplied with the input feature map element by element, thereby adjusting the attention area of the feature map and ensuring that the model focuses on the most discriminative anatomical region.

[0092] Specifically, the dual-path attention layer can effectively combine anatomical prior information and spatial attention mechanisms to improve the performance of the model in medical image segmentation tasks, especially in cases where the boundaries are not clear and the tissue structure is complex, and significant advantages are exhibited.

[0093] In some specific embodiments, the present scheme can use any loss function as a condition for terminating training.

[0094] Table 1 shows the evaluation results of MRI modality conversion performance, including the average and standard deviation of peak signal-to-noise ratio (PSNR (dB)), structural similarity index (SSIM), and learning perceptual image block similarity (LPIPS (*10-2)).

[0095] Table 1, evaluation of glioma MRI image modality conversion indicators

[0096]

[0097] As shown in Table 1, when T1 is the target modality, the one-to-one modality conversion result with T2 as the source modality achieves the optimal performance in the three key evaluation indicators of PSNR, SSIM and LPIPS, and the PSNR, SSIM and LPIPS indicators reach 26.954±2.010 (dB), 0.928±0.026 and 6.400±2.439 (*10-2), respectively. The effect of generating T1 with T2 as the source modality is slightly better than that of T1ce as the source modality, which may be because the enhanced information interferes with the generation of T1. When T1ce is the target modality, the effect of generating T1 as the source modality is the best, and the PSNR, SSIM and LPIPS indicators reach 28.668±1.866 (dB), 0.940±0.018 and 4.844±1.708 (*10-2), respectively. The conversion stability of T1 modality to T1ce modality is the best, which is consistent with the characteristics of T1ce modality, because they share the same physical imaging mechanism, and only the influence of the contrast agent needs to be considered in the conversion process, without considering the differences of other imaging mechanisms. When T2 is the target modality, the effect of generating T1ce as the source modality is the best, and the PSNR, SSIM and LPIPS indicators reach 28.486±1.803 (dB), 0.932±0.025 and 5.615±1.810 (*10-2), respectively. The active area of tumor in T1ce image has certain overlap or similarity in space with the high signal area (such as edema area or tumor area) in T2 image. Such spatial correlation makes it possible to more accurately map the regional features during modality conversion, thereby improving the conversion quality. When FLAIR is the target modality, the effect of generating T1 as the source modality is the best, and the PSNR, SSIM and LPIPS indicators reach 27.244±2.253 (dB), 0.903±0.033 and 7.481±1.750 (*10-2), respectively. When FLAIR is the target modality, the performance of each modality conversion is generally poor, which may be due to the characteristics of FLAIR that suppresses water signal, making it difficult for other modalities to accurately generate its features. In summary, the best source modality for generating T1 modality is T2, the best source modality for generating T2 modality is T1ce, and the best source modality for generating T1ce modality and FLAIR modality is T1.

[0098] In the experiment of comparing the modal completion effect with other models, AttKAN-RDDM shows excellent performance on the BraTS2021 dataset. As shown in Table 2, among Pix2Pix, MMGAN, ResViT, Restormer, FgC2F-UDiff, RDDM and other models, AttKAN-RDDM stands out in the key indicators of PSNR, SSIM and LPIPS. Specifically, when T1 and FLAIR are the target modalities, the proposed method is slightly higher than the existing SOTA model in the PSNR, SSIM and LPIPS indicators. In the PSNR indicator, it is 26.954±2.010 (dB) higher than FgC2F-UDiff's 26.588±1.942 (dB) and 27.244±2.253 (dB) higher than Restormer's 26.847±2.272 (dB); in the SSIM indicator, it is 0.928±0.026 higher than FgC2F-UDiff's 0.916±0.026 and 0.903±0.033 higher than Restormer's 0.899±0.029; in the LPIPS indicator, it is 6.400±2.439 (*10-2) lower than Restormer's 7.291±2.529 (*10-2) and 7.481±1.750 (*10-2) higher than Restormer's 7.511±1.758 (*10-2). When T1ce and T2 are the target modalities, the proposed method is higher than the existing SOTA model in the PSNR, SSIM and LPIPS indicators. Moreover, the standard deviation of the indicators of the proposed method is generally lower than that of the comparison model, indicating that it is more robust to different samples. In summary, the experimental results of AttKAN-RDDM on the BraTS2021 dataset fully verify its effectiveness and advancement in the brain glioma MRI modal completion task, especially in the tumor lesion area mapping and model stability, which is superior to existing mainstream modal completion methods.

[0099] Table 2, quantitative comparison of sequence completion with SOTA methods on the BraTS2021 dataset

[0100]

[0101] Table 3, quantitative comparison of Gibbs artifact elimination with SOTA methods on the BraTS2021 dataset

[0102]

[0103] As shown in Table 3, in the experiment of comparing the Gibbs artifact elimination effect of other models, AttKAN-RDDM exhibits excellent performance on the Gibbs artifact dataset. Our AttKAN-RDDM is higher than Pix2Pix, MMGAN, ResViT, Restormer, FgC2F-UDiff, RDDM and other models in PSNR, SSIM and LPIPS key indicators. Specifically, when eliminating Gibbs artifacts on the T2 modality, our proposed method achieves the best effect in PSNR, SSIM and LPIPS indicators, and is much higher than the existing SOTA model in PSNR and LPIPS indicators. When eliminating Gibbs artifacts on T1, T1ce and FLAIR modalities, AttKAN-RDDM is also higher than the existing SOTA model in PSNR, SSIM and LPIPS indicators.

[0104] Embodiment Two

[0105] Based on the same concept, the present application also proposes an MRI image reconstruction method based on an MRI image reconstruction model, comprising:

[0106] Obtain the to-be-reconstructed MRI image, the to-be-reconstructed MRI image is a single modality missing or a multi-modality MRI image that cannot be used, mark the missing modality or the modality that cannot be used in the to-be-reconstructed MRI image as a target modality and input it into the constructed MRI image reconstruction model, the MRI image reconstruction model reconstructs the target modality in the to-be-reconstructed MRI image to obtain a full modality MRI image.

[0107] Embodiment Three

[0108] The embodiment also provides an electronic device, referring to Figure 6 , comprising a memory 404 and a processor 402, the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0109] Specifically, the above processor 402 can include a central processing unit (CPU), or a specific integrated circuit (Application Specific Integrated Circuit, abbreviated as ASIC), or can be configured as one or more integrated circuits that implement one or more embodiments of the present application.

[0110] The memory 404 can include a mass storage that stores data or instructions. For example, and without limitation, the memory 404 can include a Hard Disk Drive (HDD), a floppy disk drive, a Solid State Drive (SSD), a flash drive, a Compact Disc Read Only Memory (CD-ROM), a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 404 can be removable and / or non-removable (or fixed) as appropriate. The memory 404 can be internal or external as appropriate. In particular embodiments, the memory 404 is a Non-Volatile memory. In particular embodiments, the memory 404 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random Access Memory (FPMDRAM), an Extended Data Output Dynamic Random Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0111] The memory 404 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 402.

[0112] The processor 402 implements the construction method of the MRI image reconstruction model based on residual denoising diffusion in any of the above embodiments by reading and executing the computer program instructions stored in the memory 404.

[0113] Optionally, the electronic device described above can further include a transmission device 406 connected to the processor 402 and an input / output device 408 connected to the processor 402.

[0114] The transmission device 406 can be used to receive or send data via a network. Specific examples of the network can include wired or wireless networks provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (RF) module for communicating with the Internet in a wireless manner.

[0115] The input / output device 408 is used to input or output information. In the present embodiment, the input information can be multi-modal MRI images, prior tumor masks, etc., and the output information can be parameter information of the MRI image reconstruction model, etc.

[0116] Optionally, in the present embodiment, the processor 402 can be configured to perform the following steps by computer program:

[0117] obtain a first data set in which multi-modal MRI images are stored, step-by-step add noise to any modality of each multi-modal MRI image in the first data set to obtain a second data set, and use the second data set as a training data set;

[0118] The noise-added modality in the multi-modality MRI image in the training data set is taken as the target modality, and the non-noise-added modality is taken as the prior modality, a prior tumor mask is obtained based on the prior modality, each multi-modality MRI image in the training data set and the corresponding prior tumor mask are taken as training samples, and the training samples are input into the constructed MRI image reconstruction architecture to obtain a reconstructed MRI image, the MRI image reconstruction architecture takes a U-net network as a basic network, includes an encoding module, a decoding module and a residual module, and a double-path attention layer is arranged after each encoding level of the encoding module and each decoding level of the decoding module, wherein the encoding level / decoding level processes the multi-modality MRI image in the training sample, and the double-path attention layer performs attention calculation on the output feature map of the encoding level / decoding level based on the prior tumor mask in the training sample.

[0119] The reconstructed MRI image is verified based on the multi-modality MRI image in the first data set, and a loss function is calculated, parameters of the MRI image reconstruction architecture are adjusted based on the loss function, and when the loss function meets a set condition or reaches a preset iteration number, the parameters of the MRI image reconstruction architecture are saved to obtain an MRI image reconstruction model.

[0120] It should be noted that the specific examples in the present embodiment can refer to the examples described in the above embodiments and optional implementation manners, and the present embodiment will not be described here.

[0121] Generally, various embodiments can be implemented in hardware or special-purpose circuitry, software, logic or any combination thereof. Some aspects of the application can be implemented in hardware, while other aspects can be implemented by firmware or software executed by a controller, microprocessor or other computing device, but the application is not limited thereto. Although various aspects of the application can be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein can be implemented in hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0122] Embodiments of the application can be implemented by computer software executable by a data processor of the mobile device such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs, also called program products when executed, including software routines, applets, and / or macros, can be stored in any apparatus-readable data storage medium and they include program instructions to implement specific tasks. The program (which can be a component of a larger program) can be implemented in a high level procedural or object-oriented programming and / or scripting languages, and / or in assembly or machine language. It will be appreciated that the application can be implemented on a variety of different general purpose platforms, with a variety of architectures and / or operating systems.Figure 6 Any block in the logical flow of the above embodiments can represent a program step, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software can be stored on such physical media as memory chips, or memory blocks implemented in the processor, magnetic media such as hard disk or floppy disks, and optical media such as for example DVD and the data variants thereof, CD and the like. The physical media are non-transitory media.

[0123] Those skilled in the art should understand that each technical feature of the above embodiments can be combined arbitrarily, and for the sake of brevity, each technical feature of the above embodiments is not described in all possible combinations, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the description.

[0124] The above embodiments only express several implementation manners of the present application, the description is relatively specific and detailed, but it should not be understood as the limitation on the scope of the present application. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for constructing an MRI image reconstruction model based on residual denoising diffusion, characterized in that: The following steps are involved: Acquire a first data set storing multimodal MRI images, gradually add noise to any modality of each multimodal MRI image in the first data set to obtain a second data set, and use the second data set as a training data set; The noisy mode in the multimodal MRI image in the training data set is used as the target mode, and the non-noisy mode is used as the prior mode. A prior tumor mask is obtained based on the prior mode. Each multimodal MRI image in the training data set and the corresponding prior tumor mask are used as training samples. The training samples are input into the constructed MRI image reconstruction architecture to obtain a reconstructed MRI image. The MRI image reconstruction architecture is based on a U-net network, including an encoding module, a decoding module, and a residual module, and a dual-path attention layer is set after each encoding level of the encoding module and each decoding level of the decoding module, wherein the encoding level / decoding level processes the multimodal MRI image in the training sample, and the dual-path attention layer performs attention calculation on the output feature map of the encoding level / decoding level based on the prior tumor mask in the training sample; The reconstructed MRI image is verified based on the multimodal MRI image in the first data set and the loss function is calculated. The parameters of the MRI image reconstruction architecture are adjusted based on the loss function. When the loss function meets the set conditions or reaches the preset number of iterations, the parameters of the MRI image reconstruction architecture are saved to obtain the MRI image reconstruction model.

2. The method for constructing an MRI image reconstruction model based on residual denoising diffusion according to claim 1, characterized in that: The encoding module includes 5 encoding levels, the first 3 encoding levels are residual downsampling layers, and the last two encoding levels are encoding overlapping block embedding networks. The decoding module includes 5 decoding levels, the first two decoding levels are decoding overlapping block embedding networks, and the last 3 decoding levels are residual upsampling layers.

3. The method for constructing an MRI image reconstruction model based on residual denoising diffusion according to claim 2, characterized in that: The encoding overlapping block embedding network consists of an overlapping patch embedding layer, a transformation layer, a KAN network and a normalization layer. The overlapping patch embedding layer converts the input feature map into an embedded feature map. The transformation layer shifts the embedded feature map to obtain a transformed feature map. The KAN network processes the transformed feature map and performs a residual connection with the input feature map, and outputs it by a normalization layer. The decoding overlapping block embedding network has the same structure as the encoding overlapping block embedding network.

4. The method for constructing an MRI image reconstruction model based on residual denoising diffusion according to claim 1, characterized in that: A time vector of a preset time step is obtained, and the time vector is processed continuously through two linear transformation units to obtain a time embedding vector, and each encoding level and decoding level is weighted by the time embedding vector.

5. The method for constructing an MRI image reconstruction model based on residual denoising diffusion according to claim 1, characterized in that: The residual module uses a two-way depth-wise separable convolution to process the output feature map to obtain a first depth-wise separable feature and a second depth-wise separable feature respectively. After smoothing the first depth-wise separable feature, the gating mechanism is calculated with the second depth-wise separable feature by element-by-element multiplication to obtain a gated feature. The gated feature is residually connected with the output feature map and then output.

6. The method for constructing an MRI image reconstruction model based on residual denoising diffusion according to claim 1, characterized in that: The dual-path attention layer takes the output feature map output by the encoding layer / decoding layer and the prior tumor mask of the corresponding size as input. The dual-path attention layer includes an implicit tensor attention path and a spatial attention path. The implicit tensor attention path performs tensor splitting on the output feature map and maps it to the non-negative space for attention calculation to obtain an implicit tensor attention result. The spatial attention path takes the output feature map and the prior tumor mask of the corresponding size as input, and allocates attention to the output feature map based on the prior tumor mask to obtain a spatial attention result. The spatial attention result, the implicit tensor attention result and the multimodal MRI image feature map are added as the output of the dual-path attention layer.

7. The method for constructing an MRI image reconstruction model based on residual denoising diffusion according to claim 6, characterized in that: The prior tumor mask is resized to the same size as the corresponding input feature map using bilinear interpolation.

8. A MRI image reconstruction method, characterized in that: include: Obtain an MRI image to be reconstructed, where the MRI image to be reconstructed is a multimodal MRI image in which a single modality is missing or unusable, mark the missing modality or unusable modality in the MRI image to be reconstructed as a target modality and input the mark into an MRI image reconstruction model constructed by any one of the methods of claims 1 to 7, and reconstruct the target modality in the MRI image to be reconstructed by the MRI image reconstruction model to obtain a full-modality MRI image.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for constructing an MRI image reconstruction model based on residual denoising and diffusion according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes a method for constructing an MRI image reconstruction model based on residual denoising diffusion according to any one of claims 1-7.

Images