MRI image super-resolution reconstruction method and system based on adaptive fourier operator

Abstract

The application relates to the technical field of image reconstruction, in particular to an MRI image super-resolution reconstruction method and system based on an adaptive Fourier operator, wherein the method comprises the following steps: obtaining target shallow features of at least one to-be-reconstructed MRI image and inputting the target shallow features into a Transformer set; performing iterative processing based on a space mixing area and a channel mixing area of each Transformer extraction block in the Transformer set, respectively, to obtain target deep features of the at least one to-be-reconstructed MRI image; and performing up-sampling on the target deep features and the target shallow features based on a transpose convolution, to obtain a target super-resolution MRI image. According to the embodiment of the application, the adaptive Fourier operator and the convolutional neural network are used to process the features in the frequency domain and the spatial domain in the space mixing area of the Transformer extraction block, so that the frequency domain information of the MRI image can be effectively extracted and integrated, the combination of the frequency domain information and the spatial domain information is further realized, the richness of the feature extraction result and the resolution of the reconstructed image are improved, and the actual requirements of medical applications are better met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image reconstruction, and particularly relates to an MRI (Magnetic Resonance Imaging) image super-resolution reconstruction method and system based on an adaptive Fourier operator. BACKGROUND

[0002] Super-resolution (SR) reconstruction technology can recover a high-resolution (HR) image from one or more low-resolution (LR) images. An MRI image is a medical image generated by using a magnetic resonance imaging technique, and is used to study the structure and function of biological tissues.

[0003] In related technologies, a convolutional neural network can be used to implement super-resolution reconstruction of MRI images and other image types. The convolution kernel is used as the main feature extraction method, the relationship between pixels in the spatial domain is learned, and the indicators of super-resolution reconstruction are improved by continuously deepening the network depth and increasing the network receptive field.

[0004] However, in related technologies, the work focus of the convolutional neural network in the super-resolution reconstruction process is mainly in the spatial domain, and the feature extraction of the frequency domain information in the LR image is insufficient. Moreover, the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, which leads to the difficulty of meeting the special needs of clinical applications for the image reconstruction result, and affects the accuracy and practicality of the super-resolution reconstruction, which needs to be solved urgently. SUMMARY

[0005] The present application provides an MRI image super-resolution reconstruction method and system based on an adaptive Fourier operator, to solve the problems in related technologies, such as the work focus of the convolutional neural network in the super-resolution reconstruction process being mainly in the spatial domain, the feature extraction of the frequency domain information in the LR image being insufficient, and the existing super-resolution reconstruction method not considering the medical characteristics of the MRI image, which leads to the difficulty of meeting the special needs of clinical applications for the image reconstruction result, and affects the accuracy and practicality of the super-resolution reconstruction.

[0006] The first aspect embodiment of the present application provides an MRI image super-resolution reconstruction method based on an adaptive Fourier operator, including the following steps: obtaining at least one to-be-reconstructed MRI image, and extracting target shallow features of the at least one to-be-reconstructed MRI image; inputting the target shallow features into a Transformer set, and performing iterative processing based on a spatial mixing area and a channel mixing area of each Transformer extraction block in the Transformer set respectively to obtain target deep features of the at least one to-be-reconstructed MRI image; performing up-sampling on the target deep features and the target shallow features based on transposed convolution to obtain a reconstruction result of the at least one to-be-reconstructed MRI image, and outputting a target super-resolution MRI image.

[0007] Optionally, in an embodiment of the present application, the iterative processing based on the spatial mixing area and the channel mixing area of each Transformer extraction block in the Transformer set respectively to obtain the target deep features of the at least one to-be-reconstructed MRI image includes: performing feature processing in a frequency domain and a spatial domain based on a spatial mixing area of a current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; performing channel processing on the multi-scale deep features by using a channel mixing area of the current Transformer extraction block to obtain target deep features under the current Transformer extraction block; and traversing each Transformer extraction block in the Transformer set to obtain the target deep features of the at least one to-be-reconstructed MRI image.

[0008] Optionally, in an embodiment of the present application, the feature processing in the frequency domain and the spatial domain based on the spatial mixing area of the current Transformer extraction block to obtain the multi-scale deep features under the current Transformer extraction block includes: performing two-dimensional discrete Fourier transform based on input features of the current Transformer extraction block to obtain features in the frequency domain; processing the features in the frequency domain by using a pre-constructed adaptive Fourier neural operator to obtain processed frequency domain features, and performing two-dimensional inverse discrete Fourier transform on the processed frequency domain features to obtain features in the spatial domain; and processing the features in the spatial domain by using a preset convolutional neural network to obtain the multi-scale deep features under the current Transformer extraction block.

[0009] Optionally, in an embodiment of the present application, the upsampling the target deep feature and the target shallow feature based on the transposed convolution to obtain the target super-resolution MRI image comprises: obtaining a target feature sum based on the target deep feature and the target shallow feature; and performing convolution on the target feature sum by using the transposed convolution kernel to obtain the target super-resolution MRI image.

[0010] The second aspect embodiment of the present application provides an MRI image super-resolution reconstruction system based on an adaptive Fourier operator, comprising: an acquisition module configured to acquire at least one to-be-reconstructed MRI image and extract a target shallow feature of the at least one to-be-reconstructed MRI image; an iteration module configured to input the target shallow feature into a Transformer set, perform iterative processing based on a spatial mixing area and a channel mixing area of each Transformer extraction block in the Transformer set, and obtain a target deep feature of the at least one to-be-reconstructed MRI image; and a reconstruction module configured to perform upsampling on the target deep feature and the target shallow feature based on transposed convolution to obtain a reconstruction result of the at least one to-be-reconstructed MRI image, and output a target super-resolution MRI image.

[0011] Optionally, in an embodiment of the present application, the iteration module comprises: a first processing unit configured to perform feature processing in a frequency domain and a spatial domain based on a spatial mixing area of a current Transformer extraction block to obtain a multi-scale deep feature under the current Transformer extraction block; a second processing unit configured to perform channel processing on the multi-scale deep feature by using a channel mixing area of the current Transformer extraction block to obtain a target deep feature under the current Transformer extraction block; and a traversal unit configured to traverse each Transformer extraction block in the Transformer set to obtain the target deep feature of the at least one to-be-reconstructed MRI image.

[0012] Optionally, in an embodiment of the present application, the first processing unit is specifically configured to: perform two-dimensional discrete Fourier transform on an input feature of the current Transformer extraction block to obtain a feature in a frequency domain; perform processing on the feature in the frequency domain by using a pre-constructed adaptive Fourier neural operator to obtain a processed feature in the frequency domain, and perform two-dimensional inverse discrete Fourier transform on the processed feature in the frequency domain to obtain a feature in a spatial domain; and perform processing on the feature in the spatial domain by using a preset convolutional neural network to obtain the multi-scale deep feature under the current Transformer extraction block.

[0013] Optionally, in an embodiment of the present application, the reconstruction module comprises: a summing unit configured to obtain a target feature sum based on the target deep feature and the target shallow feature; and a convolution unit configured to perform convolution on the target feature sum using the transposed convolution kernel to obtain the target super-resolution MRI image.

[0014] The third aspect of the embodiments of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier algorithm as described in the above embodiments.

[0015] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program executable by a processor to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier algorithm as described above.

[0016] The fifth aspect of the embodiments of the present application provides a computer program executable to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier algorithm as described above.

[0017] The embodiments of the present application can utilize the adaptive Fourier algorithm and the convolutional neural network to process the features in the frequency domain and the spatial domain in the spatial mixing area of the Transformer extraction block, thereby effectively extracting and integrating the frequency domain information of the MRI image, further realizing the combination of the frequency domain information and the spatial domain information, improving the richness of the feature extraction result and the resolution of the reconstructed image, and more meeting the actual needs of medical applications. Thus, the problems in the related art that the working focus of the convolutional neural network in the super-resolution reconstruction process is mainly the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, resulting in that the image reconstruction result is difficult to meet the special needs of clinical applications, and affecting the accuracy and practicality of the super-resolution reconstruction are solved.

[0018] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, wherein:

[0020] Figure 1 A flowchart of an MRI image super-resolution reconstruction method based on an adaptive Fourier algorithm according to an embodiment of the present application is provided.

[0021] Figure 2 Structure diagram of a deep feature extraction block according to an embodiment of the present application;

[0022] Figure 3 Working logic diagram of a spatial mixing area according to an embodiment of the present application;

[0023] Figure 4 Logic diagram of MRI image super-resolution reconstruction according to an embodiment of the present application;

[0024] Figure 5 Structure diagram of an MRI image super-resolution reconstruction system based on an adaptive Fourier operator according to an embodiment of the present application;

[0025] Figure 6 Structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Embodiments of the present application are described in detail below with reference to the accompanying drawings, in which the same or similar elements or elements having the same or similar functions are denoted by the same or similar reference numerals throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0027] The MRI image super-resolution reconstruction method and system based on an adaptive Fourier operator according to an embodiment of the present application are described below with reference to the accompanying drawings. In view of the related art mentioned in the background art, the working focus of the convolutional neural network in the super-resolution reconstruction process is mainly in the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, which leads to the problem that the image reconstruction result is difficult to meet the special needs of clinical application, affecting the accuracy and practicality of the super-resolution reconstruction. The present application provides an MRI image super-resolution reconstruction method based on an adaptive Fourier operator. In this method, the adaptive Fourier operator and the convolutional neural network can be used for feature processing in the frequency domain and the spatial domain in the spatial mixing area of the Transformer extraction block, thereby effectively extracting and integrating the frequency domain information of the MRI image, further realizing the combination of the frequency domain information and the spatial domain information, improving the richness of the feature extraction result and the resolution of the reconstructed image, and more meeting the actual needs of medical applications. Thus, the problems such as the working focus of the convolutional neural network in the super-resolution reconstruction process being mainly in the spatial domain, the feature extraction of the frequency domain information in the LR image being insufficient, and the existing super-resolution reconstruction method not considering the medical characteristics of the MRI image, which leads to the problem that the image reconstruction result is difficult to meet the special needs of clinical application, affecting the accuracy and practicality of the super-resolution reconstruction, etc. in the related art are solved.

[0028] Specifically, Figure 1A flowchart of an MRI image super-resolution reconstruction method based on an adaptive Fourier operator provided by an embodiment of the present application is shown.

[0029] As shown in the figure, the MRI image super-resolution reconstruction method based on the adaptive Fourier operator includes the following steps: Figure 1

[0030] In step S101, at least one MRI image to be reconstructed is obtained, and target shallow features of the at least one MRI image to be reconstructed are extracted.

[0031] It can be understood that in the embodiment of the present application, the MRI image to be reconstructed can be one or more low-resolution magnetic resonance medical images. In the process of clinical application, MRI images need to clearly distinguish different types of soft tissues, and need to obtain images of multiple angles such as transverse, sagittal and coronal planes in diagnosis. MRI images have the characteristics of high contrast and multi-planar imaging. Compared with the super-resolution reconstruction of general images, MRI images need to retain their high contrast, and have higher requirements for the accuracy and quality of reconstruction.

[0032] Further, the low-resolution image can be input into a shallow feature extraction block to obtain target shallow features. The shallow feature extraction block in the above process includes one convolution layer for preliminary processing. The one convolution layer of the shallow feature extraction block uses a 3x3 convolution kernel, the step is 1, the input channel is 3, i.e. the 3 channels of a normal RGB image, and the output channel is 64. The extraction of shallow features before the extraction of complex deep features can preliminarily process the input image, extract shallow image information, thereby stabilizing subsequent training and accelerating the convergence speed.

[0033] In step S102, the target shallow features are input into a Transformer set, and are iteratively processed based on the spatial mixing area and the channel mixing area of each Transformer extraction block in the Transformer set to obtain target deep features of the at least one MRI image to be reconstructed.

[0034] It can be understood that in the embodiment of the present application, the target shallow features obtained in the above steps can be input into a deep feature extraction block to obtain deep features. The deep feature extraction block is composed of repeatedly stacked Transformer blocks, i.e. a Transformer set. Each Transformer block includes a spatial mixing area and a channel mixing area. After iterative processing by each Transformer block, target deep features of the at least one MRI image to be reconstructed are obtained.

[0035] ​Optionally, in one embodiment of this application, iterative processing is performed based on the spatial blending region and channel blending region of each Transformer extraction block in the Transformer set to obtain at least one target deep feature of the MRI image to be reconstructed. This includes: performing feature processing in the frequency domain and spatial domain based on the spatial blending region of the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; performing channel processing on the multi-scale deep features using the channel blending region of the current Transformer extraction block to obtain the target deep feature under the current Transformer extraction block; and traversing each Transformer extraction block in the Transformer set to obtain at least one target deep feature of the MRI image to be reconstructed.

[0036] In actual execution, the Transformer module includes two parts: a spatial blending region and a channel blending region. The spatial blending region is composed of global convolutions based on Fourier transform, which process features in the frequency domain and spatial domain to obtain multi-scale deep features. The channel blending region is used to process the relationship between channels in the feature map.

[0037] For example, such as Figure 2 The diagram shown is a structural schematic of a deep feature extraction block according to an embodiment of this application. The current Transformer extraction block obtains the target deep features output by the previous Transformer extraction block as input features (the input of the initial Transformer extraction block is the target shallow features). After passing through the spatial blending region and the channel blending region, the target deep features under the current Transformer extraction block are output. After traversing each Transformer extraction block, the final target deep features are obtained.

[0038] Optionally, in one embodiment of this application, feature processing in the frequency and spatial domains is performed based on the spatial mixing region of the current Transformer extraction block to obtain multi-scale deep features of the current Transformer extraction block. This includes: performing a two-dimensional discrete Fourier transform on the input features of the current Transformer extraction block to obtain features in the frequency domain; processing the features in the frequency domain using a pre-constructed adaptive Fourier neural operator to obtain processed frequency domain features, and performing a two-dimensional inverse discrete Fourier transform on the processed frequency domain features to obtain features in the spatial domain; and processing the features in the spatial domain using a pre-defined convolutional neural network to obtain multi-scale deep features of the current Transformer extraction block.

[0039] It should be noted that the preset convolutional neural network can be set by those skilled in the art according to the actual situation, and no specific limitations are made here.

[0040] In actual implementation, the spatial mixing area uses two processing methods of adaptive Fourier neural operator (AFNO) and convolutional neural network (CNN) to process the features in the frequency domain and the spatial domain, respectively.

[0041] For example Figure 3 As shown in the figure, it is a working logic diagram of the spatial mixing area of an embodiment of the present application. First, the input features are transformed into the frequency domain through two-dimensional discrete Fourier transform, and the features in the frequency domain are obtained. The features in the frequency domain are processed using adaptive Fourier neural operator, and then the processed frequency domain features are converted to the spatial domain using two-dimensional discrete inverse Fourier transform. The features in the spatial domain are processed by convolutional neural network, so as to simultaneously consider the extraction of the features in the frequency domain and the spatial domain.

[0042] Specifically, for the calculation process of AFNO, it is assumed that the current features are N = hω is the sequence length, is the (n, m) token, and the calculation process of self-attention (SA) can be represented by formula (1):

[0043]

[0044] wherein, are query, key and value matrices, respectively, and K: = Softmax(<XW q , XW k > / d is regarded as an attention matrix, then the self-attention can be written in the form of a kernel function, which is represented by formula (2):

[0045]

[0046] wherein, SA(X)[s] represents the s-th row of the matrix, which is an N-dimensional vector, is a weighted sum of a series of N-dimensional vectors X[t], wherein the weight corresponding to X[t] is The kernel function can be further represented by formula (3):

[0047]

[0048] Then the Fourier operator can be represented by formula (4):

[0049]

[0050] wherein, “·” represents matrix multiplication, represents continuous Fourier transform and its inverse transform. Discretization, we get equation (5):

[0051] Z m,n = [DFT(X)] m，n (5)

[0052] Discretization, we get equation (6):

[0053] Discretization, we get equation (7):

[0054]

[0055]

[0056] Thus, we get the discrete Fourier neural operator expression, but since W m，n parameters are fixed, it cannot perform step-by-step scanning of the input feature map by the convolution operator to get the output feature map. To solve this problem, the adaptive Fourier neural operator decomposes the complex weight into multiple shared weights, and the dimension of each weight (6) can be rewritten as (8):

[0057]

[0058] where a double-layer MLP is used to implement it, which can be represented by equation (9):

[0059]

[0060] All tokens share parameters W1, W2, and b. Finally, AFNO uses the Soft-Thresholding and Shrinkage operation to make the relationship between tokens as sparse as possible, which can be represented by equation (10):

[0061]

[0062] where S λ = Sign(x) Max{|x|-λ, 0}, λ controls the sparsity of the token.

[0063] ​​Furthermore, the pre-defined Convolutional Neural Network (CNN) has one convolutional layer, using three convolutional kernels of different sizes: 3×3, 5×5, and 7×7. By concatenating or fusing the outputs of different convolutional kernels, richer multi-scale feature information can be obtained, thereby better capturing information at multiple scales in MRI images and improving the accuracy of the reconstructed image. At the same time, compared to using multiple independent convolutional layers, using multiple convolutional kernels can reduce the number of parameters in the model, thus reducing the computational complexity and improving the model's operating efficiency.

[0064] In step S103, the deep and shallow features of the target are upsampled based on transposed convolution to obtain the reconstruction result of at least one MRI image to be reconstructed, and the target super-resolution MRI image is output.

[0065] It is understood that, in the embodiments of this application, transposed convolution can be used to upsample the feature summation, outputting a high-resolution image, and obtaining the target super-resolution MRI image as the reconstruction result of the MRI image to be reconstructed. Upsampling, as a technique to increase image size, makes the image clearer by inserting extra pixels. For the super-resolution reconstruction process, it is divided into two parts: feature extraction and reconstruction. The purpose of feature extraction is to extract the rich information contained in the LR image, while the reconstruction process generally directly enlarges the LR image through upsampling to restore it to the size of the original HR image.

[0066] Among them, such as Figure 4 The diagram shown is a schematic diagram of the MRI image super-resolution reconstruction logic according to an embodiment of this application. A long-distance cross-layer connection is made between the shallow feature extraction block and the upsampling block, so that the shallow features are directly input into the upsampling block, ensuring that the deep feature extraction block focuses on learning the deep feature representation of the network.

[0067] Optionally, in one embodiment of this application, upsampling of the target's deep and shallow features based on transposed convolution to obtain a target super-resolution MRI image includes: obtaining a sum of target features based on the target's deep and shallow features; and performing convolution using the transposed convolution kernel and the sum of target features to obtain the target super-resolution MRI image.

[0068] In actual execution, the target feature sum is the sum of the target deep feature and the target shallow feature, and then the target feature sum is input into the up-sampling block, and the convolution kernel is transposed by using the transpose convolution method, that is, the rows of the convolution kernel are changed into columns and the columns are changed into rows, and then the transposed convolution kernel is convolved with the input feature map to obtain the up-sampled feature map. The method of transpose convolution can reduce the checkerboard effect that is prone to occur in sub-pixel convolution and the like, and at the same time, the number of parameters of the transpose convolution is the same as that of ordinary convolution, which is less than that of deconvolution, and is more easy to implement.

[0069] The MRI image super-resolution reconstruction method based on the adaptive Fourier operator according to the embodiments of the present application can utilize the adaptive Fourier operator and the convolutional neural network in the spatial mixing area of the Transformer extraction block to process the features in the frequency domain and the spatial domain, thereby effectively extracting and integrating the frequency domain information of the MRI image, to further realize the combination of the frequency domain information and the spatial domain information, improve the richness of the feature extraction result and the resolution of the reconstructed image, and better meet the actual needs of medical applications. Therefore, the problems in the related art that the working focus of the convolutional neural network in the super-resolution reconstruction process is mainly the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, resulting in that the image reconstruction result is difficult to meet the special needs of clinical applications, and affecting the accuracy and practicality of the super-resolution reconstruction are solved.

[0070] Secondly, the MRI image super-resolution reconstruction system based on the adaptive Fourier operator according to the embodiments of the present application is described with reference to the accompanying drawings.

[0071] Figure 5 is a structural schematic diagram of the MRI image super-resolution reconstruction system based on the adaptive Fourier operator according to the embodiments of the present application.

[0072] As Figure 5 shown, the MRI image super-resolution reconstruction system 10 based on the adaptive Fourier operator includes an acquisition module 100, an iteration module 200 and a reconstruction module 300.

[0073] The acquisition module 100 is configured to acquire at least one to-be-reconstructed MRI image and extract a target shallow feature of the at least one to-be-reconstructed MRI image.

[0074] The iteration module 200 is configured to input the target shallow feature into a Transformer set, and perform iterative processing based on a spatial mixing area and a channel mixing area of each Transformer extraction block in the Transformer set, to obtain a target deep feature of the at least one to-be-reconstructed MRI image.

[0075] The reconstruction module 300 is configured to up-sample the target deep features and the target shallow features based on the transposed convolution, to obtain a reconstruction result of the at least one to-be-reconstructed MRI image, and to output a target super-resolution MRI image.

[0076] Optionally, in an embodiment of the present application, the iteration module 200 comprises a first processing unit, a second processing unit and a traversal unit.

[0077] The first processing unit is configured to perform feature processing in the frequency domain and the spatial domain based on the spatial mixing area of the current Transformer extraction block, to obtain multi-scale deep features under the current Transformer extraction block.

[0078] The second processing unit is configured to perform channel processing on the multi-scale deep features by using the channel mixing area of the current Transformer extraction block, to obtain target deep features under the current Transformer extraction block.

[0079] The traversal unit is configured to traverse each Transformer extraction block in the Transformer set, to obtain target deep features of the at least one to-be-reconstructed MRI image.

[0080] Optionally, in an embodiment of the present application, the first processing unit is specifically configured to perform two-dimensional discrete Fourier transform on the input features of the current Transformer extraction block based on the input features, to obtain features in the frequency domain; perform processing on the features in the frequency domain by using a pre-constructed adaptive Fourier neural operator, to obtain processed features in the frequency domain, and perform two-dimensional inverse discrete Fourier transform on the processed features in the frequency domain, to obtain features in the spatial domain; and perform processing on the features in the spatial domain by using a preset convolutional neural network, to obtain the multi-scale deep features under the current Transformer extraction block.

[0081] Optionally, in an embodiment of the present application, the reconstruction module 300 comprises an adding unit and a convolution unit.

[0082] The adding unit is configured to obtain target feature addition based on the target deep features and the target shallow features.

[0083] The convolution unit is configured to perform convolution on the target feature addition and the transposed convolution kernel, to obtain the target super-resolution MRI image.

[0084] It should be noted that the foregoing explanation and description of the MRI image super-resolution reconstruction method embodiment based on the adaptive Fourier algorithm also applies to the MRI image super-resolution reconstruction system embodiment based on the adaptive Fourier algorithm, and thus will not be described herein again.

[0085] The MRI image super-resolution reconstruction system based on the adaptive Fourier operator proposed in the embodiment of the application can utilize the adaptive Fourier operator and the convolutional neural network to process the features in the frequency domain and the spatial domain in the spatial mixing area of the Transformer extraction block, so as to effectively extract and integrate the frequency domain information of the MRI image, to further realize the combination of the frequency domain information and the spatial domain information, improve the richness of the feature extraction result and the resolution of the reconstructed image, and be more in line with the actual needs of medical applications. Therefore, the problems in the related art that the work focus of the convolutional neural network in the super-resolution reconstruction process is mainly the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, so that the image reconstruction result is difficult to meet the special needs of clinical application, and the accuracy and practicality of the super-resolution reconstruction are affected, are solved.

[0086] Figure 6 The structure schematic diagram of the electronic device provided by the embodiment of the application is provided. The electronic device can include:

[0087] The memory 601, the processor 602 and the computer program stored in the memory 601 and executable on the processor 602.

[0088] The processor 602 implements the MRI image super-resolution reconstruction method based on the adaptive Fourier operator provided in the above embodiment when executing the program.

[0089] Further, the electronic device further includes:

[0090] The communication interface 603 is used for communication between the memory 601 and the processor 602.

[0091] The memory 601 is used to store the computer program executable on the processor 602.

[0092] The memory 601 can include a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0093] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 6 Only one thick line is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0094] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can complete communication between each other through an internal interface.

[0095] The processor 602 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0096] The embodiment further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the MRI image super-resolution reconstruction method based on an adaptive Fourier operator.

[0097] The embodiment further provides a computer program, and the computer program is executed to implement the MRI image super-resolution reconstruction method based on an adaptive Fourier operator.

[0098] In the description of the application, reference to "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that a particular feature, structure, material, or characteristic being described is included in at least one embodiment or example of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples. In addition, the usage of "N" means at least two, for example, two, three or the like, unless explicitly stated otherwise.

[0099] Furthermore, the terms "first", "second", or the like, are used merely as a designation of certain elements or features, and do not imply or connote relative importance or a specific order of categorization of the indicated features. Accordingly, features described as "first" or "second" can be explicitly or implicitly included in at least one of the features. In the description of the application, the meaning of "N" is at least two, for example, two, three, etc., unless explicitly specified otherwise.

[0100] Any process or method descriptions or blocks in flow charts or otherwise described herein represent embodiments which can be managed as one or more modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions or steps, and alternate implementations are possible. In some embodiments, the processes and methods described can be executably encoded on a machine- readable medium in a data signal embodied in an electromagnetic signal, a wireless signal, or a propagated signal.

[0101] The logic and / or steps represented in the flowcharts and / or described herein, for example, can be considered as a sequence of executable instructions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, system, or device, such as a computer-based system, processor- containing system, or other system that can fetch the instructions from the instruction execution system, system, or device and execute the instructions. For purposes of this specification, a "computer-readable medium" can be any medium that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, system, or device. The computer-readable medium can be a system of one or more wires, a portable computer diskette, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an optical fiber system, and a portable compact disc read-only memory (CDROM), just to name a few examples. Additionally, the computer-readable medium can be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.

[0102] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. As such, if implemented in hardware and in another embodiment, any of the following technologies, known in the art, or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and the like.

[0103] Those of skill in the art could readily implement the above described example methods with all or a subset of the recited steps carried out with a program of instructions directed to the relevant hardware, the program of instructions being stored in a computer readable storage medium which, when executed, includes one or a combination of the steps of the example methods.

[0104] In addition, each of the functional units in the various embodiments of the present application can be integrated in one processing module, or each of the units can be physically present separately, or two or more units can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.

[0105] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. An adaptive Fourier operator based An image super-resolution reconstruction method characterized by, The method comprises the following steps: obtaining at least one to be reconstructed MRI image, and extracting target shallow features of the at least one to be reconstructed MRI image; inputting the target shallow features into a Transformer set, and performing iterative processing based on a spatial mixing area and a channel mixing area of each Transformer extraction block in the Transformer set respectively to obtain target deep features of the at least one to be reconstructed MRI image; transposed convolution is performed on the target deep layer feature and the target shallow layer feature to obtain a reconstruction result of the at least one to-be-reconstructed MRI image, and a target super-resolution is output image; the iterative processing based on the spatial mixing area and the channel mixing area of each Transformer extraction block in the Transformer set respectively to obtain the target deep features of the at least one to be reconstructed MRI image comprises: performing feature processing in a frequency domain and a spatial domain based on a spatial mixing area of a current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; performing channel processing on the multi-scale deep features by using a channel mixing area of the current Transformer extraction block to obtain target deep features under the current Transformer extraction block; iterating through each Transformer extraction block in the Transformer set to obtain the target deep features of the at least one to be reconstructed MRI image; the feature processing in the frequency domain and the spatial domain based on the spatial mixing area of the current Transformer extraction block to obtain the multi-scale deep features under the current Transformer extraction block comprises: performing two-dimensional discrete Fourier transform on input features of the current Transformer extraction block to obtain features in a frequency domain; performing processing on the features in the frequency domain by using a pre-constructed adaptive Fourier neural operator to obtain processed frequency domain features, and performing two-dimensional inverse discrete Fourier transform on the processed frequency domain features to obtain features in a spatial domain; performing processing on the features in the spatial domain by using a preset convolutional neural network to obtain the multi-scale deep features under the current Transformer extraction block.

2. The method of claim 1, wherein, The transpose convolution is used for up-sampling the target deep layer feature and the target shallow layer feature to obtain a target super-resolution An image, comprising: obtaining target feature sums based on the target deep features and the target shallow features; Convolving using the transposed convolution kernel and the target feature sum to obtain the target super-resolution image.

3. An adaptive Fourier operator based image super-resolution system comprising: An image super-resolution reconstruction system characterized by comprising: comprises: an obtaining module configured to obtain at least one to be reconstructed MRI image, and extract target shallow features of the at least one to be reconstructed MRI image; an iterative module configured to input the target shallow features into a Transformer set, and perform iterative processing based on a spatial mixing area and a channel mixing area of each Transformer extraction block in the Transformer set respectively to obtain target deep features of the at least one to be reconstructed MRI image; The reconstruction module is configured to upsample the target deep layer feature and the target shallow layer feature based on a transpose convolution to obtain a reconstruction result of the at least one to-be-reconstructed MRI image, and output a target super-resolution image; the iterative module comprises: a first processing unit configured to perform feature processing in a frequency domain and a spatial domain based on a spatial mixing area of a current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; a second processing unit, configured to perform channel processing on the multi-scale deep features by using a channel mixing area of the current Transformer extraction block, to obtain target deep features under the current Transformer extraction block; a traversal unit, configured to traverse each Transformer extraction block in the set of Transformers, to obtain target deep features of the at least one to-be-reconstructed MRI image; the first processing unit is specifically configured to: perform two-dimensional discrete Fourier transform on the input features of the current Transformer extraction block, to obtain features in a frequency domain; perform processing on the features in the frequency domain by using a pre-constructed adaptive Fourier neural operator, to obtain processed features in the frequency domain, and perform two-dimensional inverse discrete Fourier transform on the processed features in the frequency domain, to obtain features in a spatial domain; perform processing on the features in the spatial domain by using a preset convolutional neural network, to obtain multi-scale deep features under the current Transformer extraction block.

4. The system of claim 3, wherein, the reconstruction module comprises: an adding unit, configured to obtain target feature addition based on the target deep features and the target shallow features. a convolution unit configured to perform convolution on the target feature and the transposed convolution kernel to obtain the target super-resolution image image.

5. An electronic device, comprising: comprise: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier algorithm according to any one of claims 1-2.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier algorithm according to any one of claims 1-2.

7. A computer program product comprising a computer program, characterized in that, The computer program is executed to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier algorithm according to any one of claims 1-2.

Images