MRI (Magnetic Resonance Imaging) super-resolution reconstruction method and device based on adaptive frequency domain loss

Abstract

The invention discloses an MRI (Magnetic Resonance Imaging) super-resolution reconstruction method and device based on adaptive frequency domain loss, and the method comprises the steps: obtaining an initial magnetic resonance image comprising at least one modal magnetic resonance image, inputting the initial magnetic resonance image into a super-resolution reconstruction model, and carrying out the spatial domain and frequency domain transformation of the initial magnetic resonance image, and the super-resolution reconstruction model is constructed by carrying out self-adaptive frequency domain loss training on the training input images of the multiple modalities and the target magnetic resonance image, adjusting on the frequency domain, reconstructing the resolution of the initial magnetic resonance image, and obtaining a target reconstructed image. The super-resolution reconstruction model is constructed by carrying out self-adaptive frequency domain loss training on the training input images of the multiple modalities and the target magnetic resonance image. According to the method, the initial magnetic resonance image is processed by using the pre-constructed super-resolution reconstruction model to obtain the target reconstruction image, super-resolution reconstruction of the initial magnetic resonance image is completed, and the definition of details in the target reconstruction image is improved through the construction mode of the super-resolution reconstruction model; and a good data basis is provided for application of the initial magnetic resonance image in a complex scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to an MRI super-resolution reconstruction method and device based on adaptive frequency-domain loss. Background Art

[0002] Magnetic resonance imaging (MRI) utilizes the principle of nuclear magnetic resonance (NMR). Based on the different attenuations of the released energy in different structural environments within a substance, by applying an external gradient magnetic field to detect the emitted electromagnetic waves, the position and type of the atomic nuclei constituting this object can be known, and accordingly, an internal structural image of the object can be drawn. Because of the advantages of no biological damage and high imaging resolution, this technology has become an important detection means in clinical medicine. However, the imaging time of MRI is slow, which requires the patient to remain stationary for a long time. During this period, the patient's movement will also cause imaging blurring, resulting in a decrease in image quality. In addition, the proton density, T1, T2, etc. characteristics of the measured object will affect the intensity and distribution of MRI signals, and artifacts such as motion artifacts and susceptibility artifacts will also reduce the clarity and resolution of the image. Therefore, how to improve the clarity and resolution of magnetic resonance images is a problem that needs to be solved currently.

[0003] Patent CN107993204A discloses an MRI image reconstruction method based on enhanced sparse representation of image patches, which is a method that uses pixel sorting within image patches and non-convex norm constraints to improve the sparsity of coefficients and estimation performance. The method includes extracting target image patches in the MRI image, then establishing a sorting training model based on the image patches, and combining coefficient non-convex constraints to establish a reconstruction model of the MRI image. Then, the alternating direction method is used to iteratively solve the sorting matrix and sparse coefficients in the model, and the final MRI image is reconstructed using the estimated sparse coefficients. By sorting the pixels within the image patches, the performance of the sparse transform is improved, and non-convex norm minimization constraints are imposed on the coefficients, making the estimated coefficients closer to the true coefficients, resulting in a better overall effect of the reconstructed image, richer detail information, and higher reconstruction accuracy.

[0004] Although the current technology has improved the resolution of magnetic resonance imaging to some extent, the research is limited to the analysis of magnetic resonance images with a fixed size, and the presentation of details in magnetic resonance imaging is still limited. How to improve the clarity of details in magnetic resonance imaging is a problem that needs to be solved currently. Summary of the Invention

[0005] Aiming at the defects existing in the above-mentioned prior art, the present invention provides an MRI super-resolution reconstruction method based on adaptive frequency-domain loss, including:

[0006] Obtain an initial magnetic resonance image, where the initial magnetic resonance image includes magnetic resonance images of any scale of at least one modality;

[0007] Input the initial magnetic resonance image into a pre - constructed super - resolution reconstruction model, based on the spatial - domain and frequency - domain transformation of the initial magnetic resonance image, and make adjustments in the frequency domain to reconstruct the resolution of the initial magnetic resonance image to obtain a target reconstructed image;

[0008] Among them, the super - resolution reconstruction model is constructed by training multiple - modality training input images and target magnetic resonance images with an adaptive frequency - domain loss.

[0009] Furthermore, the super - resolution reconstruction model is obtained by obtaining multiple - modality training input images and target magnetic resonance images, combining the high - dimensional features of the training input images to obtain training output images, analyzing the gap between the training output images and the target magnetic resonance images in the frequency domain and spatial domain, and training the super - resolution reconstruction model with an adaptive frequency - domain loss.

[0010] Furthermore, the initial magnetic resonance image includes magnetic resonance images of any scale of multiple modalities. The specific steps for obtaining the initial magnetic resonance image include:

[0011] Obtain a magnetic resonance image of a single modality;

[0012] Convert the magnetic resonance image of a single modality from the spatial domain to the frequency domain, and give the amplitude component and phase component of the magnetic resonance image of a single modality;

[0013] Based on a pre - constructed multi - modality generation model and the amplitude component and phase component of the magnetic resonance image of a single modality, obtain the amplitude components and phase components of multiple modalities corresponding to the single modality;

[0014] Convert the amplitude components and phase components of multiple modalities corresponding to the single modality from the frequency domain to the spatial domain to obtain magnetic resonance images of multiple modalities.

[0015] Furthermore, the construction of the multi - modality generation model specifically includes:

[0016] Obtain a first training data set, where the first training data set includes multiple training data groups, and the training data groups include magnetic resonance training images of multiple modalities;

[0017] Respectively convert the magnetic resonance training images of multiple modalities in each training data group from the spatial domain to the frequency domain to obtain the amplitude components and phase components of the magnetic resonance training images of multiple modalities;

[0018] Based on the amplitude components and phase components of the magnetic resonance training images of multiple modalities, train a preset multi - modality initial model to construct a multi - modality generation model;

[0019] Train a preset multi-modal initial model, specifically including: iterating the adaptive learning masks corresponding to the amplitude component and the phase component in the multi-modal initial model until convergence, obtaining the adaptive learning masks between the amplitude component and the phase component of the magnetic resonance training images of each modality, and determining the multi-modal generation model.

[0020] Furthermore, the amplitude component of the magnetic resonance image is specifically expressed as:

[0021]

[0022] where A(u, v) is the amplitude component of the initial magnetic resonance image, F(u, v) is the expression of the initial magnetic resonance image in the frequency domain, real() is the real part function of a complex number, imag() is the imaginary part function of a complex number, real(F(u, v)) is the real part of F(u, v), and imag(F(u, v)) is the imaginary part of F(u, v).

[0023] Furthermore, the phase component of the magnetic resonance image is specifically expressed as:

[0024]

[0025] where Φ(u, v) is the phase component of the initial magnetic resonance image, F(u, v) is the expression of the initial magnetic resonance image in the frequency domain, arctan() is the arctangent function, real() is the real part function of a complex number, imag() is the imaginary part function of a complex number, real(F(u, v)) is the real part of F(u, v), and imag(F(u, v)) is the imaginary part of F(u, v).

[0026] Furthermore, the construction of the super-resolution reconstruction model specifically includes:

[0027] Obtain a second training dataset, input the training input images of multiple modalities in the second training dataset into the feature encoder for high-dimensional feature extraction to obtain implicit neural representations, where the second training dataset includes training input images of multiple modalities and target magnetic resonance images, and the training input images are obtained by downsampling the target magnetic resonance images;

[0028] Input the implicit neural representations into the feature decoder for feature reduction to obtain multiple first output pixel blocks;

[0029] Upsample the training input images to obtain multiple second output pixel blocks;

[0030] Analyze the positions of the first output pixel block and the second output pixel block in the training input image, perform matching processing on the first output pixel block and the second output pixel block, and add them point by point to form a composite pixel block;

[0031] Collect the composite pixel blocks to obtain a training output image;

[0032] Based on the gap between the training output image and the target magnetic resonance image in the frequency domain and the spatial domain, train the super-resolution reconstruction model with an adaptive frequency domain loss to determine the super-resolution reconstruction model.

[0033] Furthermore, the training input image is determined in the following manner:

[0034] Obtain at least one downsampling magnification;

[0035] Based on a preset downsampling interpolation function, obtain multiple pixel blocks of the target magnetic resonance image;

[0036] Process the initial pixel values of each pixel point in the pixel block of the target magnetic resonance image to obtain the target pixel value corresponding to the pixel block, where the number of pixel points in the pixel block is determined according to the downsampling magnification;

[0037] Based on the positional relationship of each pixel block in the target magnetic resonance image, combine the target pixel values corresponding to the pixel blocks to obtain the training input image;

[0038] The training input image is specifically expressed as:

[0039] I LR =Bicubic(I HR ,s)

[0040] where I LR is the training input image, I HR is the target magnetic resonance image, Bicubic() is the downsampling interpolation function, and s is the downsampling magnification.

[0041] Furthermore, input the training input images of multiple modalities in the second training dataset into the feature encoder for high-dimensional feature extraction to obtain implicit neural representations, specifically including:

[0042] Perform pixel embedding on the training input image to obtain a training feature map;

[0043] Based on a reduction factor, reduce the dimensions of the query channel, key channel, and value channel;

[0044] Perform normalization processing and linear mapping on the training feature map, and split it along the reduced query channel, key channel, and value channel;

[0045] The training feature map is reshaped and matrix-permuted along the reduced query channels, key channels, and value channels to obtain channel matrices corresponding to each channel;

[0046] The channel matrices are successively subjected to matrix multiplication, reshaping, and linear mapping to obtain implicit neural representations.

[0047] Furthermore, based on the differences between the training output image and the target magnetic resonance image in the frequency domain and the spatial domain, the super-resolution reconstruction model is trained with an adaptive frequency-domain loss to determine the super-resolution reconstruction model, specifically including:

[0048] The training output image and the target magnetic resonance image are respectively subjected to frequency-domain transformation and spatial-domain transformation to obtain the differences between the training output image and the target magnetic resonance image in the frequency domain;

[0049] Based on the differences between the training output image and the target magnetic resonance image in the frequency domain, an adaptive frequency-domain weight matrix is constructed;

[0050] Based on the differences between the training output image and the target magnetic resonance image in the frequency domain, combined with the adaptive frequency-domain weight matrix, an adaptive frequency-domain loss is constructed;

[0051] The adaptive frequency-domain loss is iterated, and combined with the differences between the training output image and the target magnetic resonance image in the spatial domain, the training of the super-resolution reconstruction model is completed to determine the super-resolution reconstruction model.

[0052] Furthermore, based on the differences between the training output image and the target magnetic resonance image in the frequency domain, an adaptive frequency-domain weight matrix is constructed, specifically including:

[0053] The output expression of the training output image in the frequency domain and the target expression of the target magnetic resonance image in the frequency domain are obtained;

[0054] Based on the differences between the output expression and the target expression, the first norm of the differences between the output expression and the target expression is given;

[0055] Based on the first norm, combined with the weights corresponding to the frequency domain, the adaptive frequency-domain weight matrix is determined.

[0056] Furthermore, the adaptive frequency-domain weight matrix is specifically expressed as:

[0057] W(u,v) = α|F GT -F HR | 1

[0058] where W(u,v) is the adaptive frequency-domain weight matrix, F GT is the target expression of the target magnetic resonance image in the frequency domain, F HRis the output expression of the training output image in the frequency domain, |·| 1 is the first norm, and ɑ is the weight corresponding to the frequency domain.

[0059] Furthermore, based on the gap between the training output image and the target magnetic resonance image in the frequency domain, combined with the adaptive frequency domain weight matrix, an adaptive frequency domain loss is constructed, specifically including:

[0060] Obtain the output expression of the training output image in the frequency domain and the target expression of the target magnetic resonance image in the frequency domain;

[0061] Based on the difference between the output expression and the target expression, give the corresponding first norm;

[0062] Perform matrix multiplication on the first norm and the adaptive frequency domain weight matrix to determine the adaptive frequency domain loss.

[0063] Furthermore, the adaptive frequency domain loss is specifically expressed as:

[0064] L freq = W(u, v)·|F GT - F HR | 1

[0065] where L freq is the adaptive frequency domain loss, W(u, v) is the adaptive frequency domain weight matrix, F GT is the target expression of the target magnetic resonance image in the frequency domain, F HR is the output expression of the training output image in the frequency domain, |·| 1 is the first norm.

[0066] The present invention also discloses an MRI super-resolution reconstruction device based on adaptive frequency domain loss, including:

[0067] A data acquisition module for acquiring an initial magnetic resonance image, where the initial magnetic resonance image includes arbitrary-scale magnetic resonance images of at least one modality;

[0068] A super-resolution reconstruction module for inputting the initial magnetic resonance image into a pre-constructed super-resolution reconstruction model, based on the spatial domain and frequency domain transformation of the initial magnetic resonance image, and adjusting in the frequency domain to reconstruct the resolution of the initial magnetic resonance image to obtain a target reconstruction image, where the super-resolution reconstruction model is constructed by training the training input images and target magnetic resonance images of multiple modalities with adaptive frequency domain loss.

[0069] The MRI super-resolution reconstruction method and device provided by the present invention have at least the following beneficial effects:

[0070] (1) By using a pre - constructed super - resolution reconstruction model, the initial magnetic resonance image is processed to obtain a target reconstructed image, completing the super - resolution reconstruction of the initial magnetic resonance image. Among them, the super - resolution reconstruction model is constructed by training multiple - modality training input images and target magnetic resonance images with an adaptive frequency - domain loss, improving the clarity of details in the target reconstructed image and providing a good data basis for the application of the initial magnetic resonance image in complex scenarios.

[0071] (2) In the process of constructing the super - resolution reconstruction model, the second training dataset is obtained by using the down - sampling method. By using different down - sampling ratios, image data of different scales are obtained, enabling the super - resolution reconstruction model to perform super - resolution reconstruction on magnetic resonance images of any scale.

[0072] (3) When obtaining the initial magnetic resonance image, by processing the magnetic resonance image of a single modality in the frequency domain, multiple - modality magnetic resonance images corresponding to the single modality are obtained, enriching the data processed by the super - resolution reconstruction model and further improving the reconstruction effect of the super - resolution reconstruction model. Brief Description of the Drawings

[0073] Figure 1 It is a flowchart of the MRI super - resolution reconstruction method based on adaptive frequency - domain loss provided by an embodiment of the present invention;

[0074] Figure 2 It is a flowchart of constructing a multi - modality generation model provided by an embodiment of the present invention;

[0075] Figure 3 It is a schematic structural diagram of the multi - modality generation model provided by an embodiment of the present invention;

[0076] Figure 4 It is a flowchart of constructing the super - resolution reconstruction model provided by an embodiment of the present invention;

[0077] Figure 5 It is a flowchart of determining the training input image provided by an embodiment of the present invention;

[0078] Figure 6 It is a schematic structural diagram of the super - resolution reconstruction model provided by an embodiment of the present invention;

[0079] Figure 7 It is a flowchart of training the super - resolution reconstruction model based on adaptive frequency - domain loss provided by an embodiment of the present invention.

[0080] Figure 8 It is a schematic structural diagram of the feature encoder in the super - resolution reconstruction model provided by an embodiment of the present invention.

[0081] Figure 9Flowchart of the MRI super-resolution reconstruction method based on adaptive frequency-domain loss provided by the embodiments of the present invention.

[0082] Figure 10 Structural block diagram of the MRI super-resolution reconstruction device based on adaptive frequency-domain loss provided by the embodiments of the present invention.

[0083] Among them, 201 is the data acquisition module; 202 is the super-resolution reconstruction module. Detailed implementation manners

[0084] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0085] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms of "a", "the" and "said" used in the embodiments of the present invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. "Plural" generally includes at least two.

[0086] It should also be noted that the term "comprises", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such commodity or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the commodity or device comprising said element.

[0087] Super-resolution image reconstruction technology (SRIR) is a process of converting a low-resolution (LR) image into a high-resolution (HR) image through software algorithms. The super-resolution image reconstruction technology utilizes the existing low-resolution image information and reconstructs a high-resolution image with higher pixel density and richer details through signal processing and image processing methods.

[0088] Applying SRIR to magnetic resonance imaging in the medical field has a profound impact on the medical imaging field. For example, in the early stages of neurodegenerative diseases such as Alzheimer's disease, subtle but crucial changes may occur in the brain structure, which are usually difficult to capture by standard-resolution MRI. At this time, SRIR can be used to obtain higher-resolution brain magnetic resonance images, facilitating the accurate identification of these subtle changes. For magnetic resonance images of tumors, SRIR can clearly show the subtle differences between the tumor and the surrounding healthy brain tissue, thereby accurately determining the boundary, size, and invasion range of the tumor. In magnetic resonance images of cardiovascular diseases, SRIR can provide detailed images, clearly showing the minor lesions and blood flow conditions of the coronary arteries. In magnetic resonance images in the orthopedic field, especially in magnetic resonance images of joint injuries, SRIR can provide detailed images of the joint and surrounding soft tissues, helping to identify subtle injuries of the soft tissues and changes in joint structure. On magnetic resonance images of liver diseases, especially liver cancer, SRIR can provide higher image resolution to help identify minor liver lesions.

[0089] It can be seen that the extensive application potential of SRIR provides a new direction for the development of future medical imaging technologies and is of great significance in medical applications.

[0090] Early SRIR mainly relied on image interpolation techniques such as bilinear interpolation and cubic interpolation. Although these methods can improve image resolution, they are limited in restoring high-frequency details. With the improvement of computing power, deep learning-based methods have gradually become mainstream. Among them, convolutional neural network (CNN) and generative adversarial network (GAN) can more effectively restore details when generating high-resolution images by learning from a large amount of data. For example, super resolution convolutional neural network (SRCNN), as an early application of deep learning technology in super-resolution tasks, has significantly improved the clarity of images.

[0091] The existing applications of RIR in MRI often require retraining the model for specific scales and tasks, which has become a major technical challenge in the research of existing methods. Specifically, in magnetic resonance images in different fields or at different positions, there are differences in the structural features of details on magnetic resonance images, posing challenges to the generality of the model.

[0092] Generally speaking, there are currently problems in the application of SRIR in MRI, such as the lack of a unified and efficient super-resolution model, the insufficient utilization of multi-modal information, limited computational complexity in application, and limited accuracy in high-frequency information recovery.

[0093] By performing a frequency-domain transformation on MRI to process high-frequency information, the effect of image reconstruction is improved. The frequency-domain based super-resolution method uses the Fourier transform to represent the image in the frequency domain and more specifically optimizes the high-frequency components of the image. In addition, the introduction of the frequency-domain loss function also provides a new solution for MRI super-resolution. The adaptive frequency-domain loss function can dynamically adjust the weights of the loss function and better retain high-frequency details during the reconstruction process.

[0094] The embodiment of the present invention provides a method for MRI super-resolution reconstruction based on an adaptive frequency-domain loss. By using the adaptive frequency-domain loss, combining the Fourier transform with an implicit neural network, through the frequency-domain fusion of multi-modal MRI and arbitrary-scale super-resolution reconstruction, the details of MRI are enhanced, and the accuracy of tiny parts is improved to cope with complex application scenarios. Among them, the adaptive frequency-domain loss is a loss function for image reconstruction, aiming to make up for the deficiency of the spatial-domain loss by optimizing the image quality in the frequency domain. Its core idea is to enable the model to adaptively focus on the frequency components that are difficult to synthesize. Adaptive means that this loss function is not fixed, and it can dynamically adjust parameters or weights according to the frequency-domain characteristics of the input data, so as to better adapt to different data distributions and task requirements. The frequency domain is the abbreviation of the frequency domain. The scale refers to the size of the image and is used to describe the resolution or the number of pixels of the image. The scale can be represented by the pixel size, that is, the number of pixels of the image. For example, for an image with a width of 1920 pixels and a height of 1080 pixels, its resolution is 1920×1080, then 1920×1080 is the scale of this image.

[0095] The present invention proposes a method for MRI super-resolution reconstruction based on an adaptive frequency-domain loss. By integrating information from different modalities (such as T1 and T2), the detail quality and authenticity of the reconstructed image are improved, providing more accurate image information for subsequent image use. The present invention uses the Fourier transform to achieve effective fusion of the brightness and structural information between different modalities of MRI in the frequency domain during the construction of the multi-modal generation model, especially showing obvious advantages when the modal information of MRI is missing. The present invention uses implicit neural representation during the construction of the super-resolution reconstruction model to achieve super-resolution reconstruction at any scale, and improves the high-frequency details and visual performance of the image through the adaptive frequency-domain loss, meeting more complex application scenarios.

[0096] As Figure 1 shown, the embodiment of the present invention provides a method for MRI super-resolution reconstruction based on an adaptive frequency-domain loss, and the specific steps are as follows:

[0097] S101: Obtain an initial magnetic resonance image.

[0098] Specifically, the initial magnetic resonance image includes magnetic resonance images of at least one modality. The modality of a magnetic resonance image refers to different types of magnetic resonance imaging techniques, which reveal different aspects of the internal structure and function of the human body through different imaging parameters and sequences. Magnetic resonance imaging modalities include T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), diffusion weighted imaging (DWI), perfusion weighted imaging (PWI), susceptibility weighted imaging (SWI), magnetic resonance spectroscopy (MRS), fluid attenuated inversion recovery (FLAIR), etc. Among them, T1WI shows anatomical structures more clearly. T2WI is more sensitive to tissues with a higher water content. In the embodiments provided by the present invention, T1WI and T2WI are mainly used as examples for description. In other embodiments, magnetic resonance images of other modalities can be used for analysis according to actual needs, and this is not limited.

[0099] The initial magnetic resonance image can be a magnetic resonance image of a single modality or a magnetic resonance image of multiple modalities. The magnetic resonance image of multiple modalities can be obtained through image acquisition or through the analysis and processing of a magnetic resonance image of a single modality.

[0100] Furthermore, the specific steps for obtaining magnetic resonance images of multiple modalities include:

[0101] Obtain a magnetic resonance image of a single modality, convert the magnetic resonance image of the single modality from the spatial domain to the frequency domain, and give the amplitude component and phase component of the magnetic resonance image of the single modality.

[0102] Among them, the amplitude component of the magnetic resonance image is specifically expressed as:

[0103]

[0104] Among them, A(u, v) is the amplitude component of the initial magnetic resonance image, F(u, v) is the expression of the initial magnetic resonance image in the frequency domain, real() is the real part function of a complex number, imag() is the imaginary part function of a complex number, real(F(u, v)) is the real part of F(u, v), and imag(F(u, v)) is the imaginary part of F(u, v).

[0105] Among them, the phase component of the magnetic resonance image is specifically expressed as:

[0106]

[0107] Among them, Φ(u, v) is the phase component of the initial magnetic resonance image, F(u, v) is the expression of the initial magnetic resonance image in the frequency domain, arctan() is the arctangent function, real() is the real part function of a complex number, imag() is the imaginary part function of a complex number, real(F(u, v)) is the real part of F(u, v), and imag(F(u, v)) is the imaginary part of F(u, v).

[0108] Based on the pre-constructed multi-modal generation model and the amplitude component and phase component of the magnetic resonance image of a single modality, the amplitude component and phase component of multiple modalities corresponding to the single modality are obtained. The amplitude component and phase component of multiple modalities corresponding to the single modality are transformed from the frequency domain to the spatial domain to obtain magnetic resonance images of multiple modalities.

[0109] Refer to Figure 2 , further, the construction of the multi-modal generation model specifically includes:

[0110] Obtain the first training data set, where the first training data set includes multiple training data groups, and each training data group includes magnetic resonance training images of multiple modalities at any scale. The magnetic resonance training images of multiple modalities in each training data group are respectively transformed from the spatial domain to the frequency domain to obtain the amplitude component and phase component of the magnetic resonance training images of multiple modalities. Based on the amplitude component and phase component of the magnetic resonance training images of multiple modalities, a preset multi-modal initial model is trained to construct a multi-modal generation model.

[0111] Training the preset multi-modal initial model specifically includes: iterating the adaptive learning masks corresponding to the amplitude component and phase component in the multi-modal initial model until convergence to obtain the adaptive learning masks between the amplitude component and phase component of the magnetic resonance training images of each modality, and determining the multi-modal generation model, where the magnetic resonance training images of multiple modalities have the same dimension size as the amplitude component and phase component of the magnetic resonance training images of each modality, and the adaptive learning masks have the same dimension size as the magnetic resonance training images of each modality in the magnetic resonance training images of multiple modalities.

[0112] In a specific implementation, the first training dataset includes multiple training data groups, and each training data group includes magnetic resonance training images of two modalities (I T_T1WI , I T_T2WI ), that is, each training data group includes magnetic resonance training images of two modalities, namely TIWI and the corresponding T2WI of TIWI. In this example, T1WI and T2WI in MRI are used for research. The T1 image and the T2 image in each training data group are the results of the same object in the same scan examination. The combination of the two modalities can provide structural information and pathological information simultaneously, improving the recognition accuracy.

[0113] First, in the model training stage, magnetic resonance training images of two modalities, TIWI and the corresponding T2WI of TIWI, are input. Both T1WI and T2WI are grayscale images, and the image dimension is H×W×1, where H is the image height and W is the image width.

[0114] To enhance the robustness of the model and improve the quality of the synthesized multi-modal images, the first training dataset in this example includes multiple datasets, such as the BraTS (brain tumor segmentation) dataset, the IXI dataset (IXI brain development dataset), etc. To enrich the diversity of the data, operations such as flipping and cropping are performed on the first training dataset to obtain the dataset required for training the multi-modal generation model M T1WI→T2WI The dataset required for training. The first training dataset is specifically represented as:

[0115] I Train_T2WI = M T1WI→T2WI (I Train_T1WI ), I ∈ R H×W×1

[0116] where I Train_T2WI is the magnetic resonance training image corresponding to T2WI, I Train_T1WI is the magnetic resonance training image corresponding to T1WI, M T1WI→T2WI () is the multi-modal generation model that generates T2WI based on T1WI, I is the magnetic resonance training image, and R H×W×1 is the dimension of the magnetic resonance training image.

[0117] Perform Fourier transform on the magnetic resonance training images in each training data group. In the Fourier space, the magnetic resonance training images are decomposed into two parts: the amplitude part and the phase part. Among them, the amplitude part contains the brightness information of the image, which can be understood as the cell metabolism situation in pathology. The phase part contains the structural and edge spatial position information of the image, which can be understood as the anatomical structure in physiology.

[0118] Perform a discrete Fourier transform on the magnetic resonance training images in each training data group and decompose them into an amplitude part A Train_T1WI 、A Train_T2WI and a phase part φ Train_T1WI 、φ Train_T2WI . Among them, both the amplitude component A and the phase component φ have the same dimension H×W×1 as the input magnetic resonance training image I. For the magnetic resonance training image I Train_T1WI 、I Train_T2WI perform a two-dimensional Fourier transform to obtain F Train (u,v), which is specifically expressed as:

[0119]

[0120] where, I Train (x,y) is the expression of the magnetic resonance training image in the spatial domain, x and y are the coordinates of the magnetic resonance training image in the spatial domain, F Train (u,v) is the expression of the magnetic resonance training image in the frequency domain, u and v are the spatial frequencies of the magnetic resonance training image in the horizontal and vertical directions respectively, M is the maximum value of x, N is the maximum value of y, and e is the natural constant.

[0121] The decomposed amplitude component is specifically expressed as:

[0122]

[0123] The decomposed phase component is specifically expressed as:

[0124]

[0125] where, A Train (u,v) is the amplitude component of the magnetic resonance training image, Φ Train (u,v) is the phase component of the magnetic resonance training image, F Train (u,v) is the expression of the magnetic resonance training image in the frequency domain, arctan() is the arctangent function, real() is the real part function of a complex number, imag() is the imaginary part function of a complex number, real(F Train (u,v)) is the real part of F T (u,v), imag(F Train (u,v)) is the imaginary part of F Train (u,v), arg(F Train (u,v)) is the argument of the complex number F Train (u,v).

[0126] Input the frequency-domain components of the magnetic resonance training images into a multi-modal generation model. Through training, the multi-modal generation model adaptively synthesizes the frequency-domain components of the corresponding multiple-modal magnetic resonance training images from the input frequency-domain components of the magnetic resonance training images. During model training, use the distributions of the amplitude components and phase components of the T1 decomposition as the objective function, input the amplitude components and phase components of the T2 image decomposition, and enable the model to learn and fit the corresponding distribution of T1 according to an adaptive ratio.

[0127] According to the obtained amplitude component A and phase component φ, set corresponding adaptive learnable masks τ am 、τ ph respectively to control the contributions of the amplitude component and the phase component. After training, we get:

[0128] A Train_T2WI =A Train_T1WI ×τ am

[0129]

[0130] Where A Train_T2WI is the amplitude component of the T2WI output by the multi-modal generation model, A Train_T1WI is the amplitude component of the T1WI input to the multi-modal generation model, τ am is the amplitude mask obtained by training the multi-modal generation model, is the phase component of the T2WI output by the multi-modal generation model, is the phase component of the T1WI input to the multi-modal generation model, τ ph is the phase mask obtained by training the multi-modal generation model. The dimension of the amplitude mask is the same as that of the amplitude component of the magnetic resonance training image, and the dimension of the phase mask is the same as that of the phase component of the magnetic resonance training image, that is, A ∈ R H×W×1 , then τ am ∈ R H×W×1 , φ ∈ R H×W×1 , then τ ph ∈ R H×W×1 .

[0131] In a specific example provided by the present invention, use the Adam optimizer with a momentum of 0.9 for training, and set the total number of training to epoch = 2.0×10 -5 , and initialize the learning rate to 4.0×10 -4 . In other embodiments, different training parameters can be set according to actual needs or different models can be adopted, which are not limited herein.

[0132] The amplitude component A Train_T2WI and the phase component output by the multi-modal generation model Perform an inverse discrete Fourier transform to obtain the spatial domain image I Train_T2WI . The inverse discrete Fourier transform is specifically expressed as:

[0133]

[0134] where I Train (x, y) is the expression of the magnetic resonance training image in the spatial domain, x and y are the coordinates of the magnetic resonance training image in the spatial domain, F Train (u, v) is the expression of the magnetic resonance training image in the frequency domain, u and v are the spatial frequencies of the magnetic resonance training image in the horizontal and vertical directions respectively, M is the maximum value of x, N is the maximum value of y, and e is the natural constant. M and N are the sizes of the magnetic resonance training image, corresponding to the image height H and image width W.

[0135] In a specific example, use the trained multi-modal generation model to process a single-modal magnetic resonance image to obtain multiple-modal magnetic resonance images. Refer to Figure 3 , input the single-modal magnetic resonance image I T1WI into the multi-modal generation model, perform a Fourier transform on I T1WI to obtain the amplitude component A T1WI corresponding to I T1WI and the phase component According to the amplitude mask τ am and the phase mask τ ph in the multi-modal generation model, obtain the amplitude component A T1WI corresponding to I T2WI and the phase component T2WI of I Perform an inverse Fourier transform on the amplitude component A T2WI and the phase component T2WI of I to obtain the magnetic resonance image I T1WI corresponding to the single-modal magnetic resonance image I T2WI , that is, two-modal magnetic resonance images I T1WI and I T2WI are obtained. In other examples, magnetic resonance images of other modalities can be generated according to actual needs, which is not limited herein.

[0136] S102: Input the initial magnetic resonance image into the pre-constructed super-resolution reconstruction model to obtain the target reconstructed image.

[0137] Specifically, the initial magnetic resonance image is input into a pre-constructed super-resolution reconstruction model. Based on the spatial domain and frequency domain transformation of the initial magnetic resonance image and adjustment in the frequency domain, the resolution of the initial magnetic resonance image is reconstructed to obtain the target reconstructed image. The super-resolution reconstruction model is constructed by obtaining training input images and target magnetic resonance images of multiple modalities, combining the high-dimensional features of the training input images to obtain training output images, and analyzing the gaps between the training output images and the target magnetic resonance images in the frequency domain and spatial domain to perform adaptive frequency domain loss training on the super-resolution reconstruction model.

[0138] Further, the construction of the super-resolution reconstruction model specifically includes:

[0139] Referring to Figure 4 , the training input images of multiple modalities in the second training dataset are input into the feature encoder for high-dimensional feature extraction to obtain implicit neural expressions. The second training dataset includes training input images of multiple modalities and target magnetic resonance images, and the training input images are obtained by downsampling the target magnetic resonance images. The implicit neural expressions are input into the feature decoder for feature restoration to obtain multiple first output pixel blocks. The training input images are upsampled to obtain multiple second output pixel blocks. The positions of the first output pixel blocks and the second output pixel blocks in the training input images are analyzed, and the first output pixel blocks and the second output pixel blocks are matched and added point by point to form composite pixel blocks. The composite pixel blocks are pooled to obtain the training output image. Based on the gaps between the training output image and the target magnetic resonance image in the frequency domain and spatial domain, the super-resolution reconstruction model is trained with adaptive frequency domain loss to determine the super-resolution reconstruction model. In the example provided by the present invention, the second training dataset and the first training dataset can be the same dataset. Figure 4 The upsampling method used in is bilinear interpolation. In other embodiments, the upsampling method can also use other methods such as nearest neighbor interpolation and bicubic interpolation, which are not limited herein. In the example provided in this application, based on the corresponding relationship between each first output pixel block and each second pixel block and their positions in the training input image, the first output pixel blocks and the second output pixel blocks are matched. That is, when the position corresponding to the first output pixel block in the training input image is the same as the position corresponding to the second output pixel block in the training input image, the first output pixel block and the second output pixel block in this group are added point by point to obtain the corresponding composite pixel block. Then, according to the relative position relationship of each composite pixel block in the training input image, each composite pixel block is pooled to obtain the target output image. The above-mentioned addition of the first output pixel block and the second output pixel block point by point means that after the first output pixel block and the second output pixel block are aligned, the pixel value of each pixel point in the first output pixel block is added to the pixel value of the corresponding pixel point in the second output pixel block to obtain the composite pixel block.

[0140] Among them, high-dimensional features refer to high-dimensional information or attributes extracted from data in machine learning. These features are usually used to describe certain aspects of the data and are used for tasks such as classification, clustering, and recognition. High-dimensional features are usually extracted from the original data through deep learning models, such as convolutional neural networks. Convolutional neural networks gradually extract more abstract and higher-level features by stacking multiple convolutional layers and pooling layers, and finally obtain a high-dimensional vector, where each dimension corresponds to a different feature. High-dimensional features can provide richer information, which helps to improve the accuracy and robustness of the model, and they can capture more details and patterns in the data.

[0141] Implicit neural representation is a method of parameterizing various signals as continuous functions through neural networks. Different from traditional discrete signal representations (such as the pixel grid of an image, the amplitude samples of audio, etc.), implicit neural representation maps the domain of the signal to the attribute value at that coordinate. For example, for an image, implicit neural representation can map two-dimensional coordinates to RGB color values.

[0142] Furthermore, the training input image is determined in the following way:

[0143] Referring to Figure 5 , at least one downsampling ratio is obtained. Based on a preset downsampling interpolation function, multiple pixel blocks of the target magnetic resonance image are obtained. The initial pixel values of each pixel point in the pixel block of the target magnetic resonance image are processed to obtain the target pixel value corresponding to the pixel block, where the number of pixel points in the pixel block is determined according to the downsampling ratio. Based on the positional relationship of each pixel block in the target magnetic resonance image, the target pixel values corresponding to the pixel blocks are combined to obtain the training input image.

[0144] Among them, the training input image is specifically represented as:

[0145] I LR = Bicubic(I HR , s)

[0146] Among them, I LR is the training input image, I HR is the target magnetic resonance image, Bicubic() is the downsampling interpolation function, and s is the downsampling ratio.

[0147] For example, when the downsampling interpolation function is a bilinear interpolation function and the downsampling ratio is 2, the number of pixel points in the target pixel block is 4. For each new pixel position in the training input image, according to its position in the target magnetic resonance image, the weighted average of the surrounding four pixels is calculated.

[0148] Specifically, the above-mentioned second training dataset is obtained in the following way:

[0149] The images in the above first training dataset can be used as the target magnetic resonance images, or new images can be re-acquired as the target magnetic resonance images. Then, the target magnetic resonance images are downsampled to obtain the training input images corresponding to the target magnetic resonance images.

[0150] By setting different magnification factors for the above downsampling, super-resolution reconstruction models of any scale can be constructed. The training input images and target magnetic resonance images in the second training dataset are used as control samples. In the example provided by the present invention, they are constructed from the given target magnetic resonance images according to the in-distribution scales (for example, ×1 - ×4 scale magnification factors).

[0151] In a specific example, first, the given T1WI and the corresponding T2WI are input as the target magnetic resonance image I in the super-resolution reconstruction GT_T1WI , I GT_T2WI . Assuming the downsampling magnification factor is s, the corresponding training input image I LR_T1WI , I LR_T2WI is generated using bicubic interpolation, specifically expressed as:

[0152] I LR = Bicubic(I HR , s)

[0153] where, I LR is the training input image, I HR is the target magnetic resonance image, Bicubic() is the downsampling interpolation function, and s is the downsampling magnification factor.

[0154] For example, I GT is the input target magnetic resonance image I GT_T1WI , I GT_T2WI . The dimension size is I HR ∈ R H×W×1 . Through bicubic downsampling interpolation, the corresponding training input images I LR , I LR_T1WI , I LR_T2WI are output. The dimension size is s is the downsampling magnification factor, and the value range is 1 ≤ s ≤ 4. In the example provided by the present invention, bicubic downsampling interpolation is used to downsample the target magnetic resonance images. In other embodiments, methods such as nearest neighbor interpolation, bilinear interpolation, average pooling, and max pooling can also be used to implement downsampling, which is not limited herein. Through downsampling, the second training dataset required for the super-resolution reconstruction model is determined.

[0155] Input the second data set into the initial model of super-resolution reconstruction to perform implicit feature extraction on the training input images. Arbitrary-scale super-resolution reconstruction is achieved by adopting the method of implicit neural representation, which aims to obtain local scale-invariant high-dimensional features and corresponding local target coordinates in the high-resolution from the input low-resolution images, and then map the RGB values of the high-resolution images through a neural network, thereby achieving arbitrary-scale super-resolution independent of pixel size.

[0156] In a specific example, refer to Figure 6 , and input the obtained training input images I LR_T1WI 、I LR_T2WI into the feature encoder to extract the implicit neural representations z LR_T1WI 、z LR_T2WI of the training input images. In the example provided by the present invention, the feature encoder can be obtained by deleting the last upsampling layer in EDSR-baseline, RDN, and SwinIR. Subsequently, the extracted high-dimensional features pass through the feature decoder f θ , and are mapped to the RGB values at the corresponding coordinates of the training output images according to the corresponding local coordinates. Through the local high-dimensional feature invariance, the feature decoder realizes the mapping from continuous coordinate signals to discrete RGB values, thereby achieving arbitrary-scale super-resolution independent of pixel size.

[0157] According to the extracted implicit neural representations z LR_T1WI 、z LR_T2WI , and combined with the corresponding local coordinates xy, for the convenience of expression at this time, the dimension of the training input image is represented as Therefore, the range of the local coordinates is expressed as: -H ≤ x ≤ H, -W ≤ y ≤ W. According to the local high-dimensional feature invariance, the feature decoder f θ maps z LR_T1WI 、z LR_T2WI to the corresponding RGB values. The present invention is implemented by using a multilayer perceptron (MLP), and together with the bicubic interpolation upsampling for image generation to obtain the training output images I HR_T1WI 、I HR_T2WI , which is specifically expressed as:

[0158] I HR =f θ (z LR ,xy)

[0159]

[0160] where, I HR is the generated training output image I HR_T1WI, I HR_T2WI , with dimension I HR ∈R H×W×1 , f θ is a feature decoder, is a feature encoder, z LR is the implicit neural representation z LR_T1WI , z LR_T2WI . In this example, the encoding dimension is 256, that is

[0161] After obtaining the training output image, based on the gap between the training output image and the target magnetic resonance image in the frequency domain and the spatial domain, train the super-resolution reconstruction model with an adaptive frequency domain loss to determine the super-resolution reconstruction model. In the training stage of the super-resolution reconstruction model, constrain the reconstruction of the high-resolution image by the super-resolution reconstruction model according to the adaptive frequency domain loss. By reducing the frequency gap between the generated training output image and the target magnetic resonance image, and adaptively weighting the high-frequency details to enhance the robustness of the super-resolution reconstruction model, as well as generating the high-frequency details and authenticity of the high-resolution image.

[0162] Furthermore, referring to Figure 7 , perform frequency domain transformation and spatial domain transformation on the training output image and the target magnetic resonance image respectively to obtain the gap between the training output image and the target magnetic resonance image in the frequency domain and the spatial domain. Based on the gap between the training output image and the target magnetic resonance image in the frequency domain, construct an adaptive frequency domain weight matrix. Based on the gap between the training output image and the target magnetic resonance image in the frequency domain, combined with the adaptive frequency domain weight matrix, construct an adaptive frequency domain loss. Iterate on the adaptive frequency domain loss, and combined with the gap between the training output image and the target magnetic resonance image in the spatial domain, complete the training of the super-resolution reconstruction model to determine the super-resolution reconstruction model.

[0163] Furthermore, obtain the output expression of the training output image in the frequency domain and the target expression of the target magnetic resonance image in the frequency domain. Based on the difference between the output expression and the target expression, give the first norm of the difference between the output expression and the target expression. Based on the first norm, combined with the corresponding weights in the frequency domain, determine the adaptive frequency domain weight matrix.

[0164] Among them, the adaptive frequency domain weight matrix is specifically expressed as:

[0165] W(u, v) = α|F GT - F HR | 1

[0166] Among them, W(u, v) is the adaptive frequency domain weight matrix, F GT is the target expression of the target magnetic resonance image in the frequency domain, FHR is the output expression of the training output image in the frequency domain, |·| 1 is the first norm, and ɑ is the weight corresponding to the frequency domain.

[0167] Among them, the first norm is also called the Manhattan distance. The first norm is the sum of the absolute values of the elements of the vector.

[0168] Furthermore, obtain the output expression of the training output image in the frequency domain and the target expression of the target magnetic resonance image in the frequency domain. Based on the difference between the output expression and the target expression, give the corresponding first norm. Multiply the first norm and the adaptive frequency domain weight matrix to determine the adaptive frequency domain loss.

[0169] Among them, the adaptive frequency domain loss is specifically expressed as:

[0170] L freq = W(u, v)·|F GT -F HR | 1

[0171] Among them, L freq is the adaptive frequency domain loss, W(u, v) is the adaptive frequency domain weight matrix, F GT is the target expression of the target magnetic resonance image in the frequency domain, F HR is the output expression of the training output image in the frequency domain, |·| 1 is the first norm.

[0172] In a specific example, during the training phase of the super-resolution reconstruction model, by constraining the training output images I HR_T1WI 、I HR_T2WI and the target magnetic resonance images I GT_T1WI 、I GT_T2WI in the spatial domain and the frequency domain, and using them as the objective function for training the super-resolution reconstruction model. For the spatial domain, the Manhattan distance is adopted, and it is specifically expressed as:

[0173] L Spatial = |I GT -I HR | 1

[0174] Among them, L Spatial is the gap between the training output image and the target magnetic resonance image in the spatial domain, I GT is the target magnetic resonance image, I HR is the training output image.

[0175] For the frequency domain, the present invention adopts an adaptive discrete cosine transform loss. Different from the spatial coordinates, in the frequency domain space, the value of each coordinate reflects the information of a certain frequency band in the image. First, perform a discrete cosine transform on the training output image generated by the super-resolution reconstruction model, and then calculate the frequency domain gap between the target magnetic resonance image and the training output image, which is specifically expressed as:

[0176] D freq =|F GT -F HR | 1

[0177] Where D freq is the gap between the training output image and the target magnetic resonance image in the frequency domain, F GT is the target expression of the target magnetic resonance image in the frequency domain, and F HR is the output expression of the training output image in the frequency domain.

[0178] In super-resolution reconstruction, high-frequency details are often difficult to accurately restore, and as the resolution scale increases, the high-frequency information becomes more difficult to restore, so the distance in the frequency domain space is larger. Therefore, for super-resolution reconstructions of different scales, the high-frequency parts that are difficult to synthesize with dynamic adaptive weighting can force the super-resolution reconstruction model to learn high-frequency details, thereby restoring a more accurate super-resolution reconstruction image. Therefore, the present invention defines a weight matrix from the frequency domain distance to dynamically constrain the super-resolution reconstruction model to fit the distribution in the frequency domain. The adaptive frequency domain weight matrix is specifically expressed as:

[0179] W(u,v)=α|F GT -F HR | 1

[0180] Where W(u,v) is the adaptive frequency domain weight matrix, the dimension of W(u,v) is W∈R H×W×1 , F GT is the target expression of the target magnetic resonance image in the frequency domain, F HR is the output expression of the training output image in the frequency domain, |·| 1 is the first norm, and ɑ is the weight corresponding to the frequency domain, and ɑ is adjustable.

[0181] Thus, the loss in the frequency domain is defined as:

[0182] L freq =W(u,v)·|F GT -F HR | 1

[0183] Where L freqis the adaptive frequency domain loss, W(u, v) is the adaptive frequency domain weight matrix, and F GT is the target expression of the target magnetic resonance image in the frequency domain, and F HR is the output expression of the training output image in the frequency domain, and |·| 1 is the first norm.

[0184] In summary, the objective function of the super-resolution reconstruction model in the training stage is specifically expressed as:

[0185] L total = L spatial + γ·L freq

[0186] where L total is the objective function, γ is the proportion weight of the adaptive frequency domain loss, and γ is adjustable. In the example provided by the present invention, ɑ = 1, γ = 100. In other embodiments, the values of ɑ and γ can be adjusted according to actual situations, and no limitation is made thereto.

[0187] Furthermore, pixel embedding is performed on the training input image to obtain a training feature map. Based on the reduction factor, the dimensions of the query channel, key channel, and value channel are reduced. Normalization processing and linear mapping are performed on the training feature map, and splitting is performed along the reduced query channel, key channel, and value channel. The training feature map is transformed in shape and matrix permuted along the reduced query channel, key channel, and value channel to obtain channel matrices corresponding to each channel. Matrix multiplication, shape transformation, and linear mapping are sequentially performed on the channel matrices to obtain implicit neural expressions.

[0188] Pixel embedding is a technique that maps each pixel in an image to a low-dimensional vector space. The core idea is to map each pixel (or a group of pixels) in the image to a low-dimensional vector space, which is called the embedding space. In this embedding space, the vector representation (embedding) of the pixel contains the feature information of the pixel, such as comprehensive semantic information in many aspects such as color, texture, and positional relationship.

[0189] In addition to being obtained by deleting the last upsampling layer in EDSR-baseline, RDN, and SwinIR, in order to mitigate the aliasing effect, the present invention provides a new feature encoder that reduces each channel in the feature encoder, improves the image processing speed during the training process of the super-resolution reconstruction model, and at the same time reduces the memory space occupied during the training process of the super-resolution reconstruction model, improving the training efficiency of the super-resolution reconstruction model while ensuring the training effect of the super-resolution reconstruction model. In a specific example, refer to Figure 8, after pixel embedding of the training input image to obtain a training feature map, it enters the feature extraction module, which is mainly a permuted self-attention (PSA) module. Compared with the traditional permuted self-attention module, in the present invention, based on a reduction factor, the dimensions of the query channel, key channel, and value channel are reduced, which can improve the processing rate of the training feature map.

[0190] Specifically, the key channel key and value channel value are compressed with the reduction factor r, and the channel dimension is reduced to Therefore, when calculating self-attention, the input key dimension is The dimension of value is In the example provided by the present invention, the dimension of the extracted feature is C = 256, and the magnification r of the dimension channel reduction 2 is 4. Then, after rearranging the key channel key and value channel value and multiplying the matrices to evaluate, the implicit neural representation z LR_T1WI , z LR_T2WI , the dimension of the implicit neural representation is z LR ∈R H×W×C .

[0191] Referring to Figure 9 , the embodiment of the present invention provides an MRI super-resolution reconstruction method based on an adaptive frequency domain loss. First, a single-modal magnetic resonance image is input into a multi-modal generation model. In the multi-modal generation model, it sequentially undergoes Fourier transform, corresponding phase components, corresponding amplitude components, and inverse Fourier transform to obtain multiple-modal magnetic resonance images corresponding to a single modal, forming an initial magnetic resonance image, enriching the input data of the super-resolution reconstruction model, and enabling the super-resolution reconstruction model to more clearly restore the details in the initial magnetic resonance image. Inputting the initial magnetic resonance image into the super-resolution reconstruction model, a target reconstruction image with more details can be obtained. During the construction of the super-resolution reconstruction model, it is necessary to go through downsampling, feature extraction, implicit neural representation, and feature restoration. By calculating the frequency domain loss in the image analysis process, the super-resolution reconstruction model is trained with an adaptive frequency domain loss to obtain the final super-resolution reconstruction model. Among them, the downsampling step enriches the training data of different scales in the super-resolution reconstruction model, enabling the final super-resolution reconstruction model to support super-resolution reconstruction of initial magnetic resonance images of different scales to obtain corresponding target reconstruction images.

[0192] Referring to Figure 10 , the embodiment of the present invention provides an MRI super-resolution reconstruction device based on an adaptive frequency domain loss, including:

[0193] A data acquisition module 201, configured to acquire an initial magnetic resonance image, where the initial magnetic resonance image includes arbitrary-scale magnetic resonance images of at least one modality;

[0194] A super-resolution reconstruction module 202, configured to input the initial magnetic resonance image into a pre-constructed super-resolution reconstruction model, based on the spatial-domain and frequency-domain transforms of the initial magnetic resonance image, and perform adjustment in the frequency domain to reconstruct the resolution of the initial magnetic resonance image to obtain a target reconstructed image, where the super-resolution reconstruction model is constructed by training multiple-modal training input images and target magnetic resonance images with an adaptive frequency-domain loss.

[0195] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the described modules can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0196] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the present invention. Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An MRI super-resolution reconstruction method based on adaptive frequency domain loss, characterized in that: The specific steps include: Acquiring an initial magnetic resonance image, wherein the initial magnetic resonance image comprises an arbitrary-scale magnetic resonance image of at least one modality; Inputting the initial magnetic resonance image into a pre-built super-resolution reconstruction model, transforming the initial magnetic resonance image in the spatial domain and the frequency domain, and adjusting the image in the frequency domain, reconstructing the resolution of the initial magnetic resonance image, and obtaining a target reconstructed image; Among them, the super-resolution reconstruction model is constructed by training the training input images of multiple modalities and the target magnetic resonance images with adaptive frequency domain loss.

2. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 1, characterized in that: The initial magnetic resonance image includes magnetic resonance images of multiple modalities and at any scale; obtaining the initial magnetic resonance image specifically includes the following steps: Acquiring single modality magnetic resonance images; Converting a single modality magnetic resonance image from a spatial domain to a frequency domain to obtain an amplitude component and a phase component of the single modality magnetic resonance image; Based on the pre-built multi-modality generation model and the amplitude component and phase component of the magnetic resonance image of a single modality, the amplitude components and phase components of multiple modalities corresponding to the single modality are obtained; The amplitude components and phase components of multiple modes corresponding to a single mode are converted from the frequency domain to the spatial domain to obtain magnetic resonance images of multiple modes.

3. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 2, characterized in that: The construction of the multimodal generation model specifically includes: Acquire a first training data set, wherein the first training data set includes multiple training data groups, and the training data groups include magnetic resonance training images of multiple modalities; Converting the magnetic resonance training images of multiple modes in each training data group from the spatial domain to the frequency domain to obtain amplitude components and phase components of the magnetic resonance training images of multiple modes; Based on the amplitude components and phase components of the magnetic resonance training images of multiple modalities, a preset multimodal initial model is trained to construct a multimodal generation model; The preset multimodal initial model is trained, specifically including: iterating the adaptive learning masks corresponding to the amplitude component and the phase component in the multimodal initial model until convergence, obtaining the adaptive learning masks between the amplitude component and the phase component of the magnetic resonance training images of each modality, and determining the multimodal generation model.

4. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 1, characterized in that: The construction of the super-resolution reconstruction model includes: Obtain a second training data set, input the training input images of multiple modalities in the second training data set into a feature encoder, perform high-dimensional feature extraction, and obtain implicit neural expression, wherein the second training data set includes the training input images of multiple modalities and a target magnetic resonance image, and the training input image is obtained by downsampling the target magnetic resonance image; Inputting the implicit neural expression into a feature decoder to perform feature restoration to obtain a plurality of first output pixel blocks; Upsampling the training input image to obtain a plurality of second output pixel blocks; Analyzing the positions of the first output pixel block and the second output pixel block in the training input image, performing matching processing on the first output pixel block and the second output pixel block, and adding them point by point to form a composite pixel block; Gather all composite pixel blocks to obtain a training output image; Based on the gap between the training output image and the target magnetic resonance image in the frequency domain and the spatial domain, the super-resolution reconstruction model is trained with an adaptive frequency domain loss to determine the super-resolution reconstruction model.

5. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 4, characterized in that: The acquisition of training input images includes: Obtain at least one downsampling ratio; Based on a preset downsampling interpolation function, a plurality of pixel blocks of a target magnetic resonance image are obtained; Processing the initial pixel value of each pixel point of the pixel block in the target magnetic resonance image to obtain a target pixel value corresponding to the pixel block, wherein the number of pixel points in the pixel block is determined according to the downsampling ratio; Based on the positional relationship of each pixel block in the target magnetic resonance image, the target pixel values ​​corresponding to the pixel blocks are combined to obtain a training input image; Training input image, specifically expressed as: I LR =Bicubic(I HR ,s) Among them, I LR is the training input image, I HR is the target magnetic resonance image, Bicubic() is the downsampling interpolation function, and s is the downsampling ratio.

6. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 4, characterized in that: Input the training input images of multiple modalities in the second training data set into the feature encoder to extract high-dimensional features and obtain implicit neural expressions, including: Perform pixel embedding on the training input image to obtain the training feature map; Based on the reduction factor, the dimensions of the query channel, key channel, and value channel are reduced; The training feature map is normalized and linearly mapped, and split along the reduced query channel, key channel, and value channel; The training feature graph is transformed and matrix-permuted along the reduced query channel, key channel, and value channel to obtain the channel matrix corresponding to each channel; The channel matrix is ​​subjected to matrix multiplication, shape transformation, and linear mapping in sequence to obtain the implicit neural expression.

7. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 4, characterized in that: Based on the gap between the training output image and the target magnetic resonance image in the frequency domain and the spatial domain, the super-resolution reconstruction model is trained with adaptive frequency domain loss to determine the super-resolution reconstruction model, which specifically includes: Performing frequency domain conversion and space domain conversion on the training output image and the target magnetic resonance image respectively, and obtaining the difference between the training output image and the target magnetic resonance image in the frequency domain; Based on the gap between the training output image and the target magnetic resonance image in the frequency domain, an adaptive frequency domain weight matrix is ​​constructed; Based on the gap between the training output image and the target MRI image in the frequency domain, an adaptive frequency domain loss is constructed in combination with an adaptive frequency domain weight matrix; The adaptive frequency domain loss is iterated, and the gap between the training output image and the target magnetic resonance image in the spatial domain is combined to complete the training of the super-resolution reconstruction model and determine the super-resolution reconstruction model.

8. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 7, characterized in that: Based on the difference between the training output image and the target magnetic resonance image in the frequency domain, an adaptive frequency domain weight matrix is ​​constructed, which includes: Obtaining an output expression of a training output image in the frequency domain and a target expression of a target magnetic resonance image in the frequency domain; Based on the difference between the output expression and the target expression, a first norm of the difference between the output expression and the target expression is given; Based on the first norm and in combination with the weight corresponding to the frequency domain, an adaptive frequency domain weight matrix is ​​determined.

9. The MRI super-resolution reconstruction method based on adaptive frequency domain loss according to claim 7, characterized in that: Based on the gap between the training output image and the target magnetic resonance image in the frequency domain, combined with the adaptive frequency domain weight matrix, an adaptive frequency domain loss is constructed, which includes: Obtaining an output expression of a training output image in the frequency domain and a target expression of a target magnetic resonance image in the frequency domain; Based on the difference between the output expression and the target expression, the corresponding first norm is given; The first norm and the adaptive frequency domain weight matrix are matrix multiplied to determine the adaptive frequency domain loss.

10. An MRI super-resolution reconstruction device based on adaptive frequency domain loss, characterized in that: The method for MRI super-resolution reconstruction based on adaptive frequency domain loss as claimed in any one of claims 1 to 9 comprises: A data acquisition module, configured to acquire an initial magnetic resonance image, wherein the initial magnetic resonance image comprises an arbitrary-scale magnetic resonance image of at least one modality; The super-resolution reconstruction module is used to input the initial magnetic resonance image into a pre-constructed super-resolution reconstruction model, based on the spatial domain and frequency domain transformation of the initial magnetic resonance image, and adjust it in the frequency domain, to reconstruct the resolution of the initial magnetic resonance image and obtain a target reconstructed image, wherein the super-resolution reconstruction model is constructed by training the training input images of multiple modalities and the target magnetic resonance image with adaptive frequency domain loss.

Images