A Hybrid Domain Multicontrast MRI Super-Resolution Method and System Based on Variational Networks

Abstract

This application belongs to the field of medical image processing technology, specifically disclosing a hybrid domain multi-contrast MRI super-resolution method and system based on variational networks. The method includes: acquiring and preprocessing high-resolution auxiliary images, high-resolution target images, and masks to obtain multi-channel high-resolution auxiliary images, low-resolution target images, and masks; iteratively updating the low-resolution target image of each channel using a hybrid domain multi-contrast variational network, and weightedly fusing the super-resolution target images of all channels to obtain the final super-resolution target image; measuring a loss function to train an MRI super-resolution model; and inputting the actual low-resolution MRI image into the MRI super-resolution model to obtain the actual super-resolution image. This application maintains the good interpretability of model-based methods, increasing the credibility in clinical practice, while leveraging the powerful feature representation capabilities of deep neural networks to effectively improve image reconstruction results, providing more accurate pathological information for precision medicine.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, specifically to a hybrid domain multi-contrast MRI super-resolution method and system based on variational networks. Background Technology

[0002] With the continuous development of medical imaging technology, various medical imaging devices have become important tools for clinical diagnosis. Among them, magnetic resonance imaging (MRI) has been widely used in medical research and clinical applications due to its non-invasiveness, high soft tissue contrast, and multimodal characteristics. In particular, MRI images of different modalities can provide complementary information to highlight specific tissue characteristics, thereby assisting in precise clinical diagnosis and surgical planning. However, due to limitations in scan time, signal-to-noise ratio (SNR), and hardware conditions, high-resolution (HR) MRI imaging often faces many challenges, often accompanied by problems such as image blurring and loss of detail, which inconveniences doctors in diagnosis and subsequent treatment. To address these issues, medical image super-resolution technology has emerged, with its core objective being to maintain or enhance the structural and textural information of images while improving spatial resolution. Through efficient super-resolution reconstruction methods, more details can be recovered from low-resolution images, thereby obtaining high-quality MRI images, which can further improve the accuracy of clinical judgment and provide strong support for precision medicine.

[0003] Research on medical image super-resolution began with traditional interpolation methods, such as bilinear interpolation and bicubic interpolation, which enlarge images through simple mathematical interpolation. However, due to their lack of effective recovery of high-frequency details, these methods often result in blurred images and fail to significantly improve actual resolution. With the rapid development of deep learning, research on medical image super-resolution has gradually shifted towards solutions based on deep learning models. Methods such as convolutional neural networks (CNNs), generative adversarial networks (GANs), and transformer models learn mapping relationships from a large number of low-resolution images through end-to-end training, effectively reconstructing high-resolution images. These methods excel in recovering edge information and high-frequency details, significantly improving the quality of super-resolution reconstruction and providing more accurate image data support for medical image analysis and clinical applications.

[0004] Deep learning, with its powerful feature representation capabilities, has made significant progress in medical image super-resolution tasks, mainly divided into single-contrast super-resolution and multi-contrast super-resolution methods. Single-contrast super-resolution methods rely solely on MRI images from a single modality for reconstruction, failing to fully utilize auxiliary information from other modalities. Consequently, when restoring high-resolution images, some texture details are often lost. Further analysis shows that, compared to pixel-level information, MRI images from different modalities exhibit higher consistency in texture features. This is because, although images from different modalities were acquired under different scanning parameters, resulting in contrast differences, they all reflect the same anatomical structures and therefore possess similar texture features. In contrast to single-contrast super-resolution methods, multi-contrast methods, through the design of fusion strategies, aggregate complementary information from reference images from different modalities to improve the quality of the target image. However, existing methods still have certain limitations. First, most existing methods only perform feature extraction and fusion in the image domain, ignoring the original data information in the k-space. However, each data point in the k-space carries global frequency information, which is crucial for the consistency of image spectral features and the restoration of high-frequency details. Secondly, current fusion strategies, whether based on simple connections or feature matching, often lack physical interpretability and are difficult to accurately utilize the common features and complementary information of images from different modalities.

[0005] In summary, there is an urgent need to provide an MRI super-resolution method that can fully utilize prior information in the k-space domain, is highly interpretable, and can effectively model the potential relationships between images of different modalities, thereby significantly improving image quality. Summary of the Invention

[0006] In view of this, the present invention proposes a hybrid domain multi-contrast MRI super-resolution method and system based on variational networks to address the technical deficiencies of existing technologies. The method and system can fully utilize k-space frequency domain information to explicitly model the potential relationships between different modal images, improve the quality of super-resolution MRI images, enhance their reliability in processing medical images in clinical practice, further improve doctors' ability to diagnose diseases, and provide patients with better treatment options.

[0007] In a first aspect, embodiments of this application provide a hybrid domain multi-contrast MRI super-resolution method based on variational networks, including:

[0008] S1, acquire high-resolution auxiliary image, high-resolution target image and mask;

[0009] S2, preprocesses the high-resolution auxiliary image, high-resolution target image and mask to obtain multi-channel high-resolution auxiliary image, low-resolution target image and mask;

[0010] S3, the hybrid domain multi-contrast variational network iteratively updates the low-resolution target image of each channel, and then weights and fuses the super-resolution target images of all channels to obtain the final super-resolution target image.

[0011] S4 is a loss function that measures the difference between the final super-resolution target image and the high-resolution target image. It is used to train the hybrid domain multi-contrast variational network into an MRI super-resolution model.

[0012] S5. Input the actual low-resolution MRI image into the MRI super-resolution model to obtain the actual super-resolution image.

[0013] In one possible implementation, the preprocessing specifically involves:

[0014] Generating low-resolution target images based on high-resolution target images and masks;

[0015] By employing a channel expansion strategy, the high-resolution auxiliary image, low-resolution target image, and mask are copied along the channel dimension, thereby expanding the high-resolution auxiliary image, low-resolution target image, and mask from a single channel to a multi-channel representation.

[0016] In one possible implementation, S3 is specifically:

[0017] Initialize the hybrid domain multi-contrast variational network;

[0018] The super-resolution target image of all channels is obtained by iteratively calculating the low-resolution target image and auxiliary variables for each channel.

[0019] The super-resolution target images from all channels are weighted and fused to obtain the final super-resolution target image.

[0020] In one possible implementation, the hybrid domain multi-contrast variational network is initialized as follows:

[0021] The low-resolution target image and the high-resolution auxiliary image are initialized using the denoising network U-Net to obtain the initial low-resolution target image. and denoising high-resolution auxiliary images ;

[0022] The low-resolution target image is subjected to Fourier transform to obtain undersampled k-space data. ;

[0023] Texture information was extracted from a high-resolution auxiliary image by applying the Laplacian operator. G ;

[0024] Initial auxiliary variables Set as .

[0025] In one possible implementation, the calculation formula for iteratively calculating the low-resolution target image and auxiliary variables for each channel is as follows:

[0026] ,

[0027] ,

[0028] in, , They represent the first Second and third The reconstructed image from the next iteration; Indicates the inverse Fourier transform; Indicates the reconstruction of the image during the iteration process. The corresponding k-space data; , , , , and All indicate the first The trade-off parameters for the next iteration; Indicates a mask; This represents undersampled k-space data; , , , These represent convolutional dictionary layers; and These represent convolutional dictionary layers. and Transposed convolution; Represents a denoised high-resolution auxiliary image; G Represents texture information; and They represent the first Second and third Auxiliary variables for the next iteration; This refers to the U-Net network.

[0029] In one possible implementation, the formula for weighted fusion of the super-resolution target images across all channels is as follows:

[0030] ,

[0031] in, Represents a super-resolution target image; Indicates the total number of channels; Indicates the channel number; Indicates the first The learnable weights of each channel; Indicates the first A multi-channel super-resolution target image.

[0032] Secondly, embodiments of this application provide a hybrid domain multi-contrast MRI super-resolution system based on variational networks, including:

[0033] The data acquisition module is used to acquire high-resolution auxiliary images, high-resolution target images, and masks;

[0034] The preprocessing module is used to preprocess the high-resolution auxiliary image, high-resolution target image, and mask to obtain a multi-channel high-resolution auxiliary image, low-resolution target image, and mask.

[0035] The hybrid domain multi-contrast variational network module is used to iteratively update the low-resolution target image of each channel, and then weightedly fuse the super-resolution target images of all channels to obtain the final super-resolution target image.

[0036] The deep supervision module is used to measure the loss function between the final super-resolution target image and the high-resolution target image, and trains the hybrid domain multi-contrast variational network into an MRI super-resolution model.

[0037] The image generation module is used to input actual low-resolution MRI images into the MRI super-resolution model to obtain actual super-resolution images.

[0038] In one possible implementation, the hybrid domain multi-contrast variational network module includes an initialization module, an iterative update module, and a reconstruction module;

[0039] The iterative update module includes T iterative update sub-modules connected in series, and each iterative update sub-module includes a target image update module and an auxiliary variable update module;

[0040] The target image update module includes a parallel numerical fidelity layer and a structural texture refinement layer.

[0041] In one possible implementation, the numerical fidelity layer includes a Fourier transform module, a first formula calculation module, and an inverse Fourier transform module, wherein...

[0042] The Fourier transform module is used to perform Fourier transform on the low-resolution target image and the reconstructed image to obtain undersampled k-space data and iteratively updated k-space data;

[0043] The first formula calculation module is used for iterative expansion in the k-space domain. Operational connections;

[0044] The inverse Fourier transform module is used to inversely transform the iteratively expanded k-space domain data back to the image domain, ensuring that the reconstructed image and the target modality are consistent in spatial structure.

[0045] In one possible implementation, the structure texture refinement layer includes a convolutional dictionary module and a second formula calculation module, wherein,

[0046] The convolution dictionary module is used to implement the transformation matrix in the iterative unfolding of the image domain;

[0047] The second formula calculation module includes an auxiliary image calculation module, a texture calculation module, an auxiliary variable calculation module, and an integration module; the auxiliary image calculation module is used for denoising high-resolution auxiliary images. The system includes: a related image domain iterative expansion module; a texture calculation module for image domain iterative expansion related to texture information; an auxiliary variable calculation module for image domain iterative expansion related to auxiliary variables; and an integration module for integrating the calculation results of the auxiliary image calculation module, texture calculation module, and auxiliary variable calculation module to complete the image domain iterative expansion.

[0048] This invention proposes a hybrid-domain multi-contrast MRI super-resolution method and system based on variational networks. It constructs a hybrid-domain multi-contrast variational network to solve the problem of medical image super-resolution reconstruction. In the model building stage, this invention explicitly models the relationship matrix between images of different modalities, fully utilizing the complementary information between them. In the model solving stage, this invention employs an optimization method based on a semi-quadratic splitting algorithm, mapping the optimization process of the variational model to the iterative expansion of a deep neural network. In the neural network optimization stage, this invention designs channel expansion and hybrid-domain optimization strategies, combining the advantages of the image domain and the k-space domain, effectively improving the image reconstruction effect. Compared with existing technologies, this invention achieves the following beneficial effects:

[0049] (1) This invention proposes a model-driven hybrid domain multi-contrast variational network to solve the problem of medical image super-resolution reconstruction. This method maintains the good interpretability of model-based methods, increases the credibility required for processing medical images in clinical practice, and improves the accuracy of the super-resolution process by leveraging the powerful feature representation capabilities of deep neural networks.

[0050] (2) The present invention designs a hybrid domain optimization strategy that combines the advantages of the image domain and the k-space domain, effectively improving the image reconstruction effect. In the k-space domain, the data fidelity layer applies consistency constraints to ensure global consistency and reduce reconstruction errors; while in the image domain, the spatial features are optimized through the structure and texture refinement layer to reduce artifacts and enhance image structure and texture details.

[0051] (3) In the structural texture refinement layer of the image domain, this invention utilizes different modalities to assist the image in providing stable macroscopic structural guidance, ensuring overall anatomical consistency; and utilizes extracted texture information to provide rich detail compensation capabilities, refining the reconstruction of local regions. With the synergistic effect of the two, the model can balance structural accuracy and detail realism, thereby significantly improving the overall reconstruction quality of the image.

[0052] (4) This invention introduces texture loss on the basis of traditional reconstruction loss and constructs a new hybrid texture loss function to supervise the reconstruction of image content and edge details. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating the hybrid domain multi-contrast MRI super-resolution method based on variational networks provided in an embodiment of the present invention.

[0055] Figure 2 This is a structural block diagram of a hybrid domain multi-contrast MRI super-resolution system based on variational networks provided in an embodiment of the present invention;

[0056] Figure 3 This is a network structure diagram of the hybrid domain multi-contrast variational network module provided in an embodiment of this application;

[0057] Figure 4 The network structure diagrams of each module in the iterative update module provided in the embodiments of the present invention are shown in Figure (a), which is the network structure diagram of the numerical fidelity layer, Figure (b) is the network structure diagram of the structural texture refinement layer, and Figure (c) is the network structure diagram of the auxiliary variable update module.

[0058] Figure 5 A comparison image of a high-resolution target image, a high-resolution auxiliary image, a low-resolution target image, and a super-resolution target image provided in an embodiment of the present invention.

[0059] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0060] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0061] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0062] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0063] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0064] See Figure 1 This is a flowchart illustrating the hybrid domain multi-contrast MRI super-resolution method based on variational networks provided in an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] Step S1: Obtain the high-resolution auxiliary image, the high-resolution target image, and the mask.

[0066] Specifically, the high-resolution auxiliary image and the high-resolution target image are different modal images of the same part.

[0067] Preferably, the high-resolution target image is a T2 modality image (acquired over a long time); the high-resolution auxiliary image is a T1 modality image (acquired over a short time), providing complementary information for the target image as it transitions from low resolution to high resolution.

[0068] Specifically, the mask is a center downsampling mask, which is generated and obtained through program code.

[0069] Step S2 involves preprocessing the high-resolution auxiliary image, high-resolution target image, and mask to obtain a multi-channel high-resolution auxiliary image, low-resolution target image, and mask. Specifically:

[0070] Step S21: Generate a low-resolution target image based on the high-resolution target image and the mask. Specifically, slice the 3D NII format image of the high-resolution target image to obtain a 2D MRI image; multiply the 2D MRI image in k-space with the mask for downsampling, and then transform it back to the image domain to obtain the low-resolution target image.

[0071] Step S22: Using a channel expansion strategy, the high-resolution auxiliary image, low-resolution target image, and mask are copied along the channel dimension, expanding the high-resolution auxiliary image, low-resolution target image, and mask from a single channel to a multi-channel representation.

[0072] It is worth noting that, to overcome the limitations of single-channel representation of MRI images, this embodiment employs a channel expansion strategy to enhance the model's feature representation capabilities and reconstruction quality. Specifically, the input high-resolution auxiliary image, low-resolution target image, and their corresponding masks are copied along the channel dimension, expanding them from a single-channel to a multi-channel representation. This not only provides richer feature information during network training but also leverages the complementarity of different channels in subsequent iterations to more comprehensively recover structural and texture details.

[0073] Step S3: The hybrid domain multi-contrast variational network iteratively updates the low-resolution target image for each channel, and then weights and fuses the resulting super-resolution target images from all channels to obtain the final super-resolution target image. Specifically:

[0074] Step S31: Initialize the hybrid domain multi-contrast variational network. Specifically:

[0075] The low-resolution target image and the high-resolution auxiliary image are initialized using the denoising network U-Net to obtain the initial low-resolution target image. and denoising high-resolution auxiliary images ;

[0076] The low-resolution target image is subjected to Fourier transform to obtain undersampled k-space data for numerical fidelity constraints. ;

[0077] Texture information was extracted from a high-resolution auxiliary image by applying the Laplacian operator. G ;

[0078] Initial auxiliary variables Set as It is used to guide the learning of prior information during the optimization process.

[0079] It should be noted that in subsequent iterative optimization processes, only the low-resolution target image and auxiliary variables participate in the iterative update, while other variables remain fixed to reduce computational overhead and improve optimization stability.

[0080] Step S32: Perform T iterations of calculation on the low-resolution target image and auxiliary variables for each channel to obtain the super-resolution target image for all channels. The iterative calculation formula for any channel is:

[0081] ,

[0082] ,

[0083] in, , They represent the first Second and third The reconstructed image from the last iteration is the super-resolution target image for that channel. Indicates the inverse Fourier transform; Indicates the reconstruction of the image during the iteration process. The corresponding k-space data, By means of Obtained by performing a Fourier transform; , , , , and All indicate the first The trade-off parameters for each iteration are all learnable. , , , , and The settings can be customized according to actual needs. In this embodiment, , , , , and All were initially set to 1.0; Indicates a mask; This represents undersampled k-space data; , , , These represent convolutional dictionary layers; and These represent convolutional dictionary layers. and Transposed convolution; Represents a denoised high-resolution auxiliary image; G Represents texture information; and They represent the first Second and third Auxiliary variables for the next iteration; This refers to the U-Net network.

[0084] Furthermore, U-Net network It consists of three encoder modules, three decoder modules, and skip connections. The encoder modules contain multiple convolutional layers and ReLU activation function layers, extracting multi-scale features of the image through layer-by-layer downsampling. A skip connection mechanism is introduced between the encoder and decoder to fuse low-level features from shallow layers with high-level features from deeper layers, thereby improving reconstruction performance while effectively preserving image details. The decoder first performs upsampling and then concatenates the corresponding encoder features through skip connections. Subsequently, it gradually reconstructs the high-resolution target image through multiple convolutional layers and ReLU layers.

[0085] Preferably, in this embodiment, six rounds of iterative calculations are performed on the low-resolution target image and auxiliary variables respectively. That is, the iterative update module of the hybrid domain multi-contrast variational network includes six iterative update sub-modules. The iterative process is as follows: the undersampled k-space data is processed... Texture information G Denoising high-resolution auxiliary images Initial low-resolution target image and initial auxiliary variables Input is fed into the first iterative update submodule to obtain the reconstructed image of the first iteration. and the auxiliary variables of the first iteration The reconstructed image from the first iteration Auxiliary variables in the first iteration Other input variables that remain unchanged are fed into the second iteration update submodule to obtain the reconstructed image of the second iteration. and auxiliary variables in the second iteration This process continues until six iterations are completed.

[0086] It is important to note that iterative computation, as a key component of this embodiment, progressively optimizes the super-resolution reconstruction result of the low-resolution target image through multiple iterations. Each iteration primarily involves two core update processes: iteration of the low-resolution target image and iteration of auxiliary variables. Specifically, this stage first receives inputs from the initialization stage and the previous stage, including the reconstructed image and auxiliary variables from the previous iteration, as well as input variables that remain unchanged: undersampled k-space data for numerical fidelity constraints, a high-resolution auxiliary image, and texture information extracted from the high-resolution auxiliary image. In this stage, the reconstructed image is first processed by a numerical fidelity layer (DFL) and a structure-texture refinement layer (STRL). DFL applies global consistency constraints based on the undersampled k-space data to ensure the reconstruction result conforms to physical priors, while STRL utilizes the high-resolution auxiliary image and texture information to enhance the structural and textural representation of the target image, thereby generating a new iteratively reconstructed image. Simultaneously, the auxiliary variables are updated according to the changes in the reconstructed image, generating new auxiliary variables to provide additional prior information for subsequent optimization processes. Finally, the updated reconstructed image and auxiliary variables are passed to the next stage as output to continuously optimize the quality of super-resolution reconstruction. The reconstructed image obtained from the last iteration is the super-resolution target image for that channel.

[0087] In this embodiment, the iterative calculation formula for the low-resolution target image divides the iteration of the low-resolution target image into two parts to fully leverage the advantages of both the k-space and the image domain: one part is the Data Fidelity Layer (DFL) formula. Mapped to k-space via Fourier transform The data is iteratively updated in the k-space domain, fully utilizing the global frequency information carried by the original k-space data. Since k-space data directly reflects the frequency components acquired during MRI image sampling, updating in k-space can more effectively maintain data consistency, thereby ensuring data reliability and physical feasibility, and reducing reconstruction errors. After updating, the image is converted back to the image domain using inverse Fourier transform to ensure that the reconstructed image and the target modality are consistent in spatial structure, facilitating further optimization in the image domain. The second part is the formula for the Structure-Texture Refinement Layer (STRL). This embodiment performs iterative updates in the image domain, introducing auxiliary images as structural priors, significantly improving the model's ability to reconstruct the overall structure of the target image. Simultaneously, it utilizes texture information to enhance high-frequency feature recovery, strengthening local detail representation and texture restoration capabilities. Furthermore, it replaces matrix transformation operations with convolution-based sparse representations, i.e. .

[0088] in, , , , These represent convolutional dictionary layers, with a window size of 3×3. and These represent convolutional dictionary layers. and The transposed convolution is used. In this way, k-space consistency ensures physical realism and global constraints, auxiliary images provide structural guidance, and texture information supplements high-frequency details. The three form a global-structure-texture collaborative mechanism, which jointly promotes the model to restore undersampled images with high quality from multiple perspectives, thereby achieving medical image reconstruction results with better structural consistency and perceptual quality.

[0089] In the iteration of the auxiliary variables in this embodiment, the gradient operator of the prior information relative to the auxiliary variables is transformed into a prior learning task of the deep neural network, which is regarded as a denoising process for low-resolution target images. The prior distribution is automatically learned by the neural network, and the update strategy of the auxiliary variables is adaptively adjusted during the optimization process to improve the reconstruction quality and enhance the generalization ability of the model.

[0090] Step S33: Weighted fusion of the super-resolution target images from all channels is performed to obtain the final super-resolution target image. The calculation formula is:

[0091] ,

[0092] in, Represents a super-resolution target image; Indicates the total number of channels; Indicates the channel number; Indicates the first The learnable weights of each channel; Indicates the first A multi-channel super-resolution target image.

[0093] After T iterations, this embodiment employs a learnable channel-weighted fusion mechanism to integrate the multi-channel super-resolution target images generated in the initialization phase to synthesize a new super-resolution target image. Specifically, each channel represents a different intermediate reconstruction result, which is progressively optimized in different iterations, capturing multi-level information about the image's structure and texture. To fully integrate these reconstruction results, this embodiment introduces a channel-weighted fusion operation, assigning a learnable weight to each channel. The weighted fusion operation is performed linearly at each spatial location. In implementation, this weighted fusion operation is completed by a 1×1 convolutional layer whose kernel linearly weights each intermediate result along the channel direction, thereby improving efficiency and trainability while maintaining expressive power. Through this fusion mechanism, the model can adaptively integrate the feature extraction results of the final stage, enhancing the detail representation and structural consistency of the target image. Furthermore, this method can effectively suppress noise or bias that may be introduced by a single reconstruction, making the final generated super-resolution target image more accurate and stable, thus improving the overall reconstruction quality.

[0094] Step S4: Measure the loss function between the final super-resolution target image and the high-resolution target image to train the hybrid domain multi-contrast variational network as an MRI super-resolution model.

[0095] Specifically, the loss function between the final super-resolution target image and the high-resolution target image is calculated. If the loss function is acceptable, the hybrid domain multi-contrast variational network is the MRI super-resolution model. If the loss function is unacceptable, step S3 is repeated to train the hybrid domain multi-contrast variational network repeatedly until the loss function is acceptable. At this point, the hybrid domain multi-contrast variational network is the MRI super-resolution model.

[0096] The loss function The calculation formula is:

[0097] ,

[0098] in, Indicates the image reconstruction loss; Indicates texture loss; and These represent the weighting coefficients, which are set to [default value]. , .

[0099] It is worth noting that, to improve the structural clarity and texture fidelity of the final super-resolution target image, this embodiment introduces texture loss on top of the traditional reconstruction loss, constructing a hybrid texture loss function to jointly optimize the image content and edge details. Specifically, the loss function includes two parts: first, a pixel-level L1 reconstruction loss, used to ensure the overall consistency between the final super-resolution target image and the high-resolution auxiliary image; second, a texture loss extracted based on a Laplacian filter, used to enhance the reconstruction capability of high-frequency details such as edges and textures. The texture loss is achieved by calculating the L1 difference between the final super-resolution target image and the high-resolution target image in the texture domain, effectively guiding the network to focus on the accuracy of structural regions. This scheme can effectively enhance the texture and edge clarity of the image while maintaining overall reconstruction accuracy, especially showing a significant improvement in the restoration of tissue boundaries and lesion details in medical images.

[0100] Step S5: Input the actual low-resolution MRI image into the MRI super-resolution model to obtain the actual super-resolution image.

[0101] Building upon the above embodiments, this embodiment further provides a hybrid domain multi-contrast MRI super-resolution system based on variational networks. See also... Figure 2 This is a structural block diagram of a hybrid domain multi-contrast MRI super-resolution system based on variational networks provided in an embodiment of the present invention. Figure 2 As shown, the system includes:

[0102] The data acquisition module is used to acquire high-resolution auxiliary images, high-resolution target images, and masks;

[0103] The preprocessing module is used to preprocess the high-resolution auxiliary image, high-resolution target image, and mask to obtain a multi-channel high-resolution auxiliary image, low-resolution target image, and mask.

[0104] The hybrid domain multi-contrast variational network module is used to iteratively update the low-resolution target image of each channel, and then weightedly fuse the super-resolution target images of all channels to obtain the final super-resolution target image.

[0105] The deep supervision module is used to measure the loss function between the final super-resolution target image and the high-resolution target image, and trains the hybrid domain multi-contrast variational network into an MRI super-resolution model.

[0106] The image generation module is used to input actual low-resolution MRI images into the MRI super-resolution model to obtain actual super-resolution images.

[0107] See Figure 3This is a network structure diagram of the hybrid domain multi-contrast variational network module provided in an embodiment of this application. Figure 3 As shown, the hybrid domain multi-contrast variational network module includes an initialization module, an iterative update module, and a reconstruction module, wherein...

[0108] The initialization module is used to initialize the hybrid domain multi-contrast variational network.

[0109] The iterative update module is used to iteratively update the low-resolution target image of each channel to obtain a multi-channel super-resolution target image.

[0110] The reconstruction module is used to perform weighted fusion of multi-channel super-resolution target images to obtain the final super-resolution target image.

[0111] Furthermore, the iterative update module includes T iterative update sub-modules connected in series. Each iterative update sub-module includes a target image update module and an auxiliary variable update module. The target image update module includes a numerical fidelity layer (DFL) and a structural texture refinement layer (STRL) connected in parallel. The outputs of the numerical fidelity layer and the structural texture refinement layer of the previous iterative update sub-module are superimposed and fused as the input of the next iterative update sub-module. The outputs of the numerical fidelity layer and the structural texture refinement layer of the last iterative update sub-module are superimposed and fused as the input of the reconstruction module.

[0112] See Figure 4 Figure 1 shows the network structure diagrams of each module in the iterative update module provided in this embodiment of the invention; wherein, Figure (a) is the network structure diagram of the numerical fidelity layer, Figure (b) is the network structure diagram of the structural texture refinement layer, and Figure (c) is the network structure diagram of the auxiliary variable update module. Figure 4 As shown, the numerical fidelity layer is used for k-space domain iterative expansion, making full use of the global frequency information carried by the original data in k-space, and performing iterative updates in the k-space domain. Since the k-space data directly reflects the frequency components obtained during the MRI image sampling process, updating in k-space can more effectively maintain data consistency and reduce reconstruction errors.

[0113] The numerical fidelity layer includes a Fourier transform module, a first formula calculation module, and an inverse Fourier transform module.

[0114] The Fourier transform module is used to perform Fourier transform on the low-resolution target image and the reconstructed image to obtain undersampled k-space data and iteratively updated k-space data.

[0115] The first formula calculation module is used for k-space domain iterative expansion. The operation connection.

[0116] The inverse Fourier transform module is used to transform the iteratively expanded k-space domain data back to the image domain, ensuring that the reconstructed image and the target mode are consistent in spatial structure, so that further optimization can be performed in the image domain.

[0117] The output characteristics of the numerical fidelity layer (DFL) are as follows: .

[0118] The structure texture refinement layer is used for iterative unfolding of the image domain, explicitly utilizing information from the high-resolution auxiliary image and similar texture features between different modal images to improve the overall structure and local details of the reconstructed image.

[0119] The structural texture refinement layer includes a convolution dictionary module and a second formula calculation module, wherein,

[0120] The convolutional dictionary module is used to implement the transformation matrix in the iterative unfolding of the image domain. The convolutional dictionary module includes four convolutional dictionary layers. , , , and two corresponding transposed convolutional dictionary layers and .

[0121] The convolutional dictionary layer Used to implement the transformation matrix in the iterative expansion of the image domain. A It includes two convolutional layers and one ReLU activation layer, with a 3×3 kernel size. The convolutional dictionary layer... Used to implement the transformation matrix in the iterative expansion of the image domain. B It includes two convolutional layers and one ReLU activation layer, with a 3×3 kernel size. The convolutional dictionary layer... Used to implement the transformation matrix in the iterative expansion of the image domain. C It includes two convolutional layers and one ReLU activation layer, with a 3×3 kernel size. The convolutional dictionary layer... Used to implement the transformation matrix in the iterative expansion of the image domain. D It includes two convolutional layers and one ReLU activation layer, with a convolutional kernel size of 3×3.

[0122] The transposed convolution dictionary layer Used to implement the transpose matrix in the iterative expansion of the image domain. It includes two transposed convolutional layers and one ReLU activation layer, with a kernel size of 3×3. The transposed convolutional dictionary layer... Used to implement the transformation matrix in the iterative expansion of the image domain. It includes two transposed convolutional layers and one ReLU activation layer, with a kernel size of 3×3.

[0123] The second formula calculation module is used for iterative expansion in the image domain. The operation connection.

[0124] The second formula calculation module includes an auxiliary image calculation module, a texture calculation module, an auxiliary variable calculation module, and an integration module. The auxiliary image calculation module is used for denoising high-resolution auxiliary images. Related image domain iterative expansion, i.e. The texture calculation module is used for iterative expansion of the image domain related to texture information, i.e. The auxiliary variable calculation module is used for iterative expansion of the image domain related to the auxiliary variables, i.e. The integration module is used to integrate the calculation results of the auxiliary image calculation module, the texture calculation module, and the auxiliary variable calculation module, that is... The image domain iterative expansion is completed.

[0125] The output features of the Structure Texture Refinement Layer (STRL) are: .

[0126] The auxiliary variable update module includes a prior information module and a third formula calculation module.

[0127] The prior information module is used to automatically learn the prior distribution through the neural network U-Net.

[0128] The third formula calculation module is used for auxiliary variables. W Iterative expansion .

[0129] The output characteristic of the auxiliary variable update module is: .

[0130] See Figure 5 This is a comparison image of a high-resolution target image, a high-resolution auxiliary image, a low-resolution target image, and a super-resolution target image provided in an embodiment of the present invention. Figure 5 As shown, it is evident that the super-resolution target image generated by the method and system provided in this disclosure is very similar to the high-resolution target image, proving the effectiveness of the method and system provided in this disclosure.

[0131] Corresponding to the above embodiments, the present invention also provides an electronic device.

[0132] See Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Figure 6As shown, the electronic device 600 may include a processor 601, a memory 602, and a communication unit 603. These components communicate via one or more buses. Those skilled in the art will understand that the electronic device structure shown in the figure does not constitute a limitation on the embodiments of the present invention. It may be a bus-type structure or a star-type structure, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0133] The communication unit 603 is used to establish a communication channel, thereby enabling the electronic device to communicate with other devices.

[0134] Processor 601 serves as the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in memory 602, and calls data stored in memory to perform various functions and / or process data. The processor may be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, processor 601 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0135] Memory 602 is used to store the execution instructions of processor 601. Memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0136] When the execution instructions in memory 602 are executed by processor 601, the electronic device 600 is able to perform some or all of the steps in the above method embodiments.

[0137] Corresponding to the above embodiments, this invention also provides a computer-readable storage medium, wherein the computer-readable storage medium may store a program, wherein, when the program is executed, it can control the device where the computer-readable storage medium is located to execute some or all of the steps in the above method embodiments. Specifically, the computer-readable storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0138] Corresponding to the above embodiments, the present invention also provides a computer program product containing executable instructions that, when executed on a computer, cause the computer to perform some or all of the steps in the above method embodiments.

[0139] In this embodiment of the invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0140] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0141] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the several embodiments provided by this invention, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0143] The above description is merely a specific embodiment of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this invention should be included within the protection scope of this invention. The protection scope of this invention should be determined by the scope of the claims.

Claims

1. A hybrid domain multi-contrast MRI super-resolution method based on variational networks, characterized in that, include: S1, acquire high-resolution auxiliary image, high-resolution target image and mask; S2, preprocesses the high-resolution auxiliary image, high-resolution target image and mask to obtain multi-channel high-resolution auxiliary image, low-resolution target image and mask; S3, the hybrid domain multi-contrast variational network iteratively updates the low-resolution target image of each channel, and then weights and fuses the super-resolution target images of all channels to obtain the final super-resolution target image. S4 is a loss function that measures the difference between the final super-resolution target image and the high-resolution target image. It is used to train the hybrid domain multi-contrast variational network into an MRI super-resolution model. S5, input the actual low-resolution MRI image into the MRI super-resolution model to obtain the actual super-resolution image; The hybrid domain multi-contrast variational network includes an initialization module, an iterative update module, and a reconstruction module; S3 specifically comprises: The initialization module is used to initialize the hybrid domain multi-contrast variational network; The iterative update module is used to iteratively calculate the low-resolution target image and auxiliary variables of each channel based on the high-resolution auxiliary image and mask of the corresponding channel to obtain the super-resolution target image of all channels. The reconstruction module is used to perform weighted fusion of the super-resolution target images of all channels to obtain the final super-resolution target image; The calculation formula for iterative calculation of the low-resolution target image and auxiliary variables for each channel is as follows: in, F represents the reconstructed images at iteration (t+1) and iteration (t), respectively; -1 (·) indicates the inverse Fourier transform; Indicates the reconstruction of the image during the iteration process. The corresponding k-space data; μ t γ t ε t θ t η t and λ t Both represent the trade-off parameters for the t-th iteration; M represents the mask; E represents the undersampled k-space data obtained by Fourier transforming a low-resolution target image; a E b E c E d These represent convolutional dictionary layers; and These represent the convolutional dictionary layer E respectively. a and E c The transposed convolution of ; x2 represents the denoised high-resolution auxiliary image; G represents the texture information extracted from the high-resolution auxiliary image by applying the Laplacian operator; W t+1 and W t Let represent the auxiliary variables for the (t+1)th and tth iterations, respectively; This refers to the U-Net network, which consists of three encoder modules, three decoder modules, and skip connections.

2. The hybrid domain multi-contrast MRI super-resolution method based on variational networks according to claim 1, characterized in that, The preprocessing is as follows: Generating low-resolution target images based on high-resolution target images and masks; By employing a channel expansion strategy, the high-resolution auxiliary image, low-resolution target image, and mask are copied along the channel dimension, thereby expanding the high-resolution auxiliary image, low-resolution target image, and mask from a single channel to a multi-channel representation.

3. The hybrid domain multi-contrast MRI super-resolution method based on variational networks according to claim 1, characterized in that, The hybrid domain multi-contrast variational network is initialized as follows: The low-resolution target image and the high-resolution auxiliary image are initialized using the denoising network U-Net to obtain the initial low-resolution target image. and two denoised high-resolution auxiliary images; The low-resolution target image is subjected to Fourier transform to obtain undersampled k-space data. Texture information G is extracted from a high-resolution auxiliary image by applying the Laplacian operator. The initial auxiliary variable W 0 Set as 4. The hybrid domain multi-contrast MRI super-resolution method based on variational networks according to claim 1, characterized in that, The formula for weighted fusion of super-resolution target images across all channels is as follows: Where X1 represents the super-resolution target image; c i Indicates the total number of channels; i represents the channel number; w i This represents the learnable weight of the i-th channel; Let i represent the super-resolution target image of the i-th channel.

5. A system for the hybrid domain multi-contrast MRI super-resolution method based on variational networks as described in claim 1, characterized in that, include: The data acquisition module is used to acquire high-resolution auxiliary images, high-resolution target images, and masks; The preprocessing module is used to preprocess the high-resolution auxiliary image, high-resolution target image, and mask to obtain a multi-channel high-resolution auxiliary image, low-resolution target image, and mask. A hybrid domain multi-contrast variational network is used to iteratively update the low-resolution target image of each channel separately, and then weightedly fuse the super-resolution target images of all channels to obtain the final super-resolution target image. The deep supervision module is used to measure the loss function between the final super-resolution target image and the high-resolution target image, and trains the hybrid domain multi-contrast variational network into an MRI super-resolution model. The image generation module is used to input actual low-resolution MRI images into the MRI super-resolution model to obtain actual super-resolution images.

6. The system according to claim 5, characterized in that, The hybrid domain multi-contrast variational network includes an initialization module, an iterative update module, and a reconstruction module; The iterative update module includes T iterative update sub-modules connected in series, and each iterative update sub-module includes a target image update module and an auxiliary variable update module; The target image update module includes a parallel numerical fidelity layer and a structural texture refinement layer.

7. The system according to claim 6, characterized in that, The numerical fidelity layer includes a Fourier transform module, a first formula calculation module, and an inverse Fourier transform module, wherein... The Fourier transform module is used to perform Fourier transform on the low-resolution target image and the reconstructed image to obtain undersampled k-space data and iteratively updated k-space data; The first formula calculation module is used for iterative expansion in the k-space domain. Operational connections; The inverse Fourier transform module is used to inversely transform the iteratively expanded k-space data back to the image domain, ensuring that the reconstructed image and the target modality are consistent in spatial structure.

8. The system according to claim 6, characterized in that, The structural texture refinement layer includes a convolution dictionary module and a second formula calculation module, wherein, The convolution dictionary module is used to implement the transformation matrix in the iterative unfolding of the image domain; The second formula calculation module includes an auxiliary image calculation module, a texture calculation module, an auxiliary variable calculation module, and an integration module. The auxiliary image calculation module is used for iterative expansion of the image domain related to the x2 of the denoised high-resolution auxiliary image. The texture calculation module is used for iterative expansion of the image domain related to texture information. The auxiliary variable calculation module is used for iterative expansion of the image domain related to auxiliary variables. The integration module is used to integrate the calculation results of the auxiliary image calculation module, the texture calculation module, and the auxiliary variable calculation module to complete the iterative expansion of the image domain.

Images