MRI (Magnetic Resonance Imaging) image super-resolution reconstruction method and system based on adaptive Fourier operator

Abstract

The invention relates to the technical field of image reconstruction, in particular to an MRI (Magnetic Resonance Imaging) image super-resolution reconstruction method and system based on an adaptive Fourier operator, and the method comprises the steps: obtaining target shallow layer features of at least one to-be-reconstructed MRI image and inputting the target shallow layer features into a Transform set, and performing iterative processing on the basis of the space mixing region and the channel mixing region of each Transform extraction block in the Transform set to obtain a target deep feature of at least one to-be-reconstructed MRI image, and performing up-sampling on the target deep feature and the target shallow feature based on transposition convolution to obtain a target super-resolution MRI image. According to the embodiment of the invention, the feature processing in the frequency domain and the spatial domain can be carried out in the spatial mixing region of the Transform extraction block by using the adaptive Fourier operator and the convolutional neural network, so that the frequency domain information of the MRI image is effectively extracted and integrated, and the combination of the frequency domain information and the spatial domain information is further realized. Richness of feature extraction results and resolution of reconstructed images are improved, and actual requirements of medical application are better met.

Description

Technical Field

[0001] The present application relates to the technical field of image reconstruction, and in particular to a method and system for super-resolution reconstruction of MRI (Magnetic Resonance Imaging) images based on an adaptive Fourier operator. Background Art

[0002] Super-resolution (SR) reconstruction technology can restore high-resolution (HR) images through one or more low-resolution (LR) images. MRI images are a type of medical image produced using magnetic resonance imaging technology and are used to study the structure and function of biological tissues.

[0003] In related technologies, convolutional neural networks can be used to achieve super-resolution reconstruction of image types such as MRI. The convolution kernel is used as the main feature extraction method to learn the relationship between pixels in the spatial domain. The super-resolution reconstruction indicators are improved by continuously deepening the network depth and increasing the network receptive field.

[0004] However, in the related technology, the focus of the convolutional neural network in the super-resolution reconstruction process is mainly on the spatial domain, and the feature extraction of the frequency domain information in the LR image is insufficient. In addition, the existing super-resolution reconstruction method does not consider the medical characteristics of MRI images, resulting in image reconstruction results that are difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of super-resolution reconstruction, which needs to be urgently resolved. Summary of the invention

[0005] The present application provides a method and system for super-resolution reconstruction of MRI images based on an adaptive Fourier operator to solve the problems in the related art that, during the super-resolution reconstruction process, the working focus of the convolutional neural network is mainly in the spatial domain, and the feature extraction of the frequency domain information in the LR image is insufficient. In addition, the existing super-resolution reconstruction method does not take the medical characteristics of MRI images into consideration, resulting in that the image reconstruction results are difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of super-resolution reconstruction.

[0006] The first aspect of the present application provides an MRI image super-resolution reconstruction method based on an adaptive Fourier operator, comprising the following steps: acquiring at least one MRI image to be reconstructed, and extracting target shallow features of the at least one MRI image to be reconstructed; inputting the target shallow features into a Transformer set, and performing iterative processing based on the spatial mixing area and the channel mixing area of ​​each Transformer extraction block in the Transformer set, respectively, to obtain target deep features of the at least one MRI image to be reconstructed; upsampling the target deep features and the target shallow features based on transposed convolution to obtain a reconstruction result of the at least one MRI image to be reconstructed, and outputting a target super-resolution MRI image.

[0007] Optionally, in one embodiment of the present application, the iterative processing is performed based on the spatial mixing area and the channel mixing area of ​​each Transformer extraction block in the Transformer set respectively to obtain the target deep features of the at least one MRI image to be reconstructed, including: performing feature processing in the frequency domain and the spatial domain based on the spatial mixing area of ​​the current Transformer extraction block to obtain the multi-scale deep features of the current Transformer extraction block; performing channel processing on the multi-scale deep features using the channel mixing area of ​​the current Transformer extraction block to obtain the target deep features of the current Transformer extraction block; and traversing each Transformer extraction block in the Transformer set to obtain the target deep features of the at least one MRI image to be reconstructed.

[0008] Optionally, in one embodiment of the present application, the feature processing in the frequency domain and the spatial domain is performed based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block, including: performing a two-dimensional discrete Fourier transform based on the input features of the current Transformer extraction block to obtain features in the frequency domain; using a pre-built adaptive Fourier neural operator to process the features in the frequency domain to obtain processed frequency domain features, and performing a two-dimensional discrete Fourier inverse transform on the processed frequency domain features to obtain features in the spatial domain; using a preset convolutional neural network to process the features in the spatial domain to obtain multi-scale deep features under the current Transformer extraction block.

[0009] Optionally, in one embodiment of the present application, the upsampling of the target deep features and the target shallow features based on the transposed convolution to obtain a target super-resolution MRI image includes: obtaining the sum of the target features based on the target deep features and the target shallow features; and convolving the transposed convolution kernel and the sum of the target features to obtain the target super-resolution MRI image.

[0010] The second aspect of the present application provides an MRI image super-resolution reconstruction system based on an adaptive Fourier operator, including: an acquisition module, used to acquire at least one MRI image to be reconstructed, and extract target shallow features of the at least one MRI image to be reconstructed; an iteration module, used to input the target shallow features into a Transformer set, and perform iterative processing based on the spatial mixing area and the channel mixing area of ​​each Transformer extraction block in the Transformer set, respectively, to obtain target deep features of the at least one MRI image to be reconstructed; a reconstruction module, used to upsample the target deep features and the target shallow features based on transposed convolution, obtain the reconstruction result of the at least one MRI image to be reconstructed, and output the target super-resolution MRI image.

[0011] Optionally, in one embodiment of the present application, the iteration module includes: a first processing unit, used to perform feature processing in the frequency domain and the spatial domain based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; a second processing unit, used to perform channel processing on the multi-scale deep features using the channel mixing area of ​​the current Transformer extraction block to obtain target deep features under the current Transformer extraction block; a traversal unit, used to traverse each Transformer extraction block in the Transformer set to obtain the target deep features of the at least one MRI image to be reconstructed.

[0012] Optionally, in one embodiment of the present application, the first processing unit is specifically used to: perform a two-dimensional discrete Fourier transform based on the input features of the current Transformer extraction block to obtain features in the frequency domain; use a pre-built adaptive Fourier neural operator to process the features in the frequency domain to obtain processed frequency domain features, and perform a two-dimensional discrete Fourier inverse transform on the processed frequency domain features to obtain features in the spatial domain; use a preset convolutional neural network to process the features in the spatial domain to obtain multi-scale deep features under the current Transformer extraction block.

[0013] Optionally, in one embodiment of the present application, the reconstruction module includes: a summation unit, used to obtain the sum of target features based on the target deep features and the target shallow features; a convolution unit, used to perform convolution using the transposed convolution kernel and the sum of target features to obtain the target super-resolution MRI image.

[0014] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier operator as described in the above embodiment.

[0015] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned MRI image super-resolution reconstruction method based on the adaptive Fourier operator.

[0016] The fifth aspect of the present application provides a computer program, which, when executed, implements the above-mentioned MRI image super-resolution reconstruction method based on the adaptive Fourier operator.

[0017] The embodiment of the present application can use the adaptive Fourier operator and the convolutional neural network in the spatial mixing area of ​​the Transformer extraction block to perform feature processing in the frequency domain and the spatial domain, thereby effectively extracting and integrating the frequency domain information of the MRI image, so as to further realize the combination of frequency domain information and spatial domain information, improve the richness of the feature extraction results and the resolution of the reconstructed image, and better meet the actual needs of medical applications. Thus, the problem in the related technology that the focus of the convolutional neural network in the super-resolution reconstruction process is mainly in the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, resulting in the image reconstruction results being difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of super-resolution reconstruction.

[0018] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0020] Figure 1 A flowchart of a method for super-resolution reconstruction of MRI images based on an adaptive Fourier operator according to an embodiment of the present application;

[0021] Figure 2 A schematic diagram of the structure of a deep feature extraction block according to an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the working logic of the spatial mixing zone of one embodiment of the present application;

[0023] Figure 4 A logical diagram of super-resolution reconstruction of MRI images according to an embodiment of the present application;

[0024] Figure 5 Schematic diagram of the structure of an MRI image super-resolution reconstruction system based on an adaptive Fourier operator according to an embodiment of the present application;

[0025] Figure 6 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0027] The following describes the MRI image super-resolution reconstruction method and system based on the adaptive Fourier operator of the embodiment of the present application with reference to the accompanying drawings. In view of the related technologies mentioned in the above background technology, in the super-resolution reconstruction process, the focus of the convolutional neural network is mainly on the spatial domain, and the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not take into account the medical characteristics of the MRI image, resulting in the image reconstruction results being difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of the super-resolution reconstruction. The present application provides an MRI image super-resolution reconstruction method based on an adaptive Fourier operator, in which the adaptive Fourier operator and the convolutional neural network can be used in the spatial mixing area of ​​the Transformer extraction block to perform feature processing in the frequency domain and the spatial domain, thereby effectively extracting and integrating the frequency domain information of the MRI image, so as to further realize the combination of the frequency domain information and the spatial domain information, improve the richness of the feature extraction results and the resolution of the reconstructed image, and better meet the actual needs of medical applications. As a result, the problems in related technologies such as the convolutional neural network's focus on the spatial domain during super-resolution reconstruction and insufficient feature extraction of frequency domain information in LR images are solved, and the existing super-resolution reconstruction methods do not consider the medical characteristics of MRI images, resulting in image reconstruction results that are difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of super-resolution reconstruction.

[0028] Specifically, Figure 1A flowchart of a method for super-resolution reconstruction of MRI images based on an adaptive Fourier operator is provided in an embodiment of the present application.

[0029] like Figure 1 As shown, the MRI image super-resolution reconstruction method based on the adaptive Fourier operator includes the following steps:

[0030] In step S101, at least one MRI image to be reconstructed is acquired, and target shallow features of at least one MRI image to be reconstructed are extracted.

[0031] It is understandable that in the embodiment of the present application, the MRI image to be reconstructed may be one or more low-resolution magnetic resonance medical images. In clinical applications, MRI images need to clearly distinguish different types of soft tissues, and images of multiple angles such as cross-sections, sagittal planes, and coronal planes need to be obtained during diagnosis. MRI images have the characteristics of high contrast and multi-plane imaging. Compared with the super-resolution reconstruction of general images, MRI images need to retain their high contrast and have higher reconstruction accuracy and quality requirements.

[0032] Furthermore, the low-resolution image can be input into the shallow feature extraction block to obtain the target shallow features. The shallow feature extraction block in the above process includes a convolution layer for preliminary processing. Among them, the convolution layer of the shallow feature extraction block uses a 3×3 convolution kernel, a step size of 1, and an input channel of 3, that is, 3 channels of a normal RGB image, and an output channel of 64. Extracting shallow features before extracting complex deep features can perform preliminary processing on the input image and extract shallow image information, thereby stabilizing subsequent training and accelerating convergence.

[0033] In step S102, the target shallow features are input into a Transformer set, and iterative processing is performed based on the spatial mixing region and the channel mixing region of each Transformer extraction block in the Transformer set to obtain at least one target deep feature of the MRI image to be reconstructed.

[0034] It is understandable that in the embodiment of the present application, the target shallow features obtained in the above steps can be input into the deep feature extraction block to obtain the deep features. The deep feature extraction block is composed of repeatedly stacked Transformer blocks, that is, a Transformer set, each Transformer block includes a spatial mixing area and a channel mixing area, and after each Transformer block is iteratively processed, at least one target deep feature of the MRI image to be reconstructed is obtained.

[0035] Optionally, in one embodiment of the present application, iterative processing is performed based on the spatial mixing area and the channel mixing area of ​​each Transformer extraction block in the Transformer set to obtain the target deep features of at least one MRI image to be reconstructed, including: performing feature processing in the frequency domain and the spatial domain based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; performing channel processing on the multi-scale deep features using the channel mixing area of ​​the current Transformer extraction block to obtain the target deep features under the current Transformer extraction block; traversing each Transformer extraction block in the Transformer set to obtain at least one target deep feature of the MRI image to be reconstructed.

[0036] In the actual execution process, the Transformer module consists of two parts: the spatial mixing area and the channel mixing area. The spatial mixing area is composed of global convolution based on Fourier transform. After processing the features in the frequency domain and the spatial domain, multi-scale deep features are obtained. The channel mixing area is used to process the relationship between channels in the feature map.

[0037] For example, Figure 2 As shown, it is a structural diagram of the deep feature extraction block of an embodiment of the present application. The current Transformer extraction block obtains the target deep features output by the previous Transformer extraction block as input features (the input of the initial Transformer extraction block is the target shallow features), and outputs the target deep features under the current Transformer extraction block after passing through the spatial mixing area and the channel mixing area. After traversing each Transformer extraction block, the final target deep features are obtained.

[0038] Optionally, in one embodiment of the present application, feature processing in the frequency domain and the spatial domain is performed based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block, including: performing a two-dimensional discrete Fourier transform based on the input features of the current Transformer extraction block to obtain features in the frequency domain; using a pre-built adaptive Fourier neural operator to process the features in the frequency domain to obtain processed frequency domain features, and performing a two-dimensional discrete Fourier inverse transform on the processed frequency domain features to obtain features in the spatial domain; using a preset convolutional neural network to process the features in the spatial domain to obtain multi-scale deep features under the current Transformer extraction block.

[0039] It should be noted that the preset convolutional neural network can be set by technical personnel in this field according to actual conditions and is not specifically limited here.

[0040] In the actual implementation process, the spatial mixing area uses two processing methods, Adaptive Fourier Neural Operator (AFNO) and Convolutional Neural Network (CNN), to process the features in the frequency domain and the spatial domain respectively.

[0041] For example Figure 3 The figure shows a schematic diagram of the working logic of the spatial mixing zone of an embodiment of the present application. First, the input features are transformed into the frequency domain by two-dimensional discrete Fourier transform to obtain the features in the frequency domain, and the features in the frequency domain are processed by an adaptive Fourier neural operator in the frequency domain, and then the processed frequency domain features are transferred to the spatial domain by two-dimensional discrete Fourier inverse transform, and the features in the spatial domain are processed by a convolutional neural network, so as to take into account the extraction of both frequency domain and spatial domain features.

[0042] Specifically, for the calculation process of AFNO, it can be assumed that the current characteristics are N = hω is the sequence length, For the (n, m)th token, the calculation process of self-attention (SA) can be expressed by formula (1):

[0043]

[0044] in, They are query, key and value matrices respectively, and K: = Softmax( <XW q , XW k > / d is regarded as an attention matrix, then self-attention can be written in the form of a kernel function, expressed by formula (2):

[0045]

[0046] Where SA(X)[s] represents The sth row of the matrix is ​​an N-dimensional vector, which is the weighted sum of a series of N-dimensional vectors X[t], where the weight corresponding to X[t] is The kernel function can be further expressed by formula (3):

[0047]

[0048] Then the Fourier operator can be expressed by formula (4):

[0049]

[0050] Here, “·” represents matrix multiplication, Represents the continuous Fourier transform and its inverse transform. Discretization can give formula (5):

[0051] Z m,n =[DFT(X)] m，n (5)

[0052] Will Discretization can give formula (6):

[0053]

[0054] Will Discretization can give formula (7):

[0055]

[0056] This gives the discrete Fourier neural operator expression, but due to W m，n The parameters are fixed, and it is impossible to perform a convolution operator to gradually scan the input feature map to obtain the output feature map. To solve this problem, the adaptive Fourier neural operator converts the complex weights Decomposed into multiple shared weights, each weight has the dimension Formula (6) can be rewritten as (8):

[0057]

[0058] Among them, a two-layer MLP is used to implement it, which can be expressed by formula (9):

[0059]

[0060] All tokens share parameters W1, W2, and b. Finally, AFNO uses soft-thresholding and shrinkage operations to make the relationship between tokens as sparse as possible, which can be expressed by formula (10):

[0061]

[0062] Among them, S λ =Sign(x)Max{|x|-λ,0}, where λ controls the sparsity of the token.

[0063] Furthermore, the preset convolutional neural network (CNN) has one convolution layer, using three different sizes of convolution kernels: 3×3, 5×5, and 7×7. By splicing or fusing the outputs of different convolution kernels, richer multi-scale feature information can be obtained, thereby better capturing information on multiple scales of MRI images and improving the accuracy of reconstructed images. At the same time, compared with using multiple independent convolution layers, using multiple convolution kernels can reduce the number of model parameters, thereby reducing the computational complexity of the model and improving the operating efficiency of the model.

[0064] In step S103, the target deep features and the target shallow features are upsampled based on the transposed convolution to obtain a reconstruction result of at least one MRI image to be reconstructed, and a target super-resolution MRI image is output.

[0065] It can be understood that in the embodiment of the present application, the feature sum can be upsampled by transposed convolution, and a high-resolution image can be output to obtain a target super-resolution MRI image as a reconstruction result of the MRI image to be reconstructed. Upsampling is a technology to increase the image size. It makes the image clearer by inserting additional pixels. For the super-resolution reconstruction process, it is divided into two parts: feature extraction and reconstruction. The purpose of feature extraction is to extract the rich information contained in the LR image, and the reconstruction process generally directly enlarges the LR image by upsampling to restore it to the size of the original HR image.

[0066] Among them, Figure 4 As shown, it is a logical schematic diagram of MRI image super-resolution reconstruction of an embodiment of the present application. A long-distance cross-layer connection is made between the shallow feature extraction block and the upsampling block, and the shallow features are directly input into the upsampling block, ensuring that the deep feature extraction block focuses on the deep feature representation of the learning network.

[0067] Optionally, in one embodiment of the present application, the target deep features and the target shallow features are upsampled based on transposed convolution to obtain a target super-resolution MRI image, including: obtaining the target feature sum based on the target deep features and the target shallow features; and convolving the transposed convolution kernel and the target feature sum to obtain the target super-resolution MRI image.

[0068] In the actual implementation process, the target feature sum is the sum of the target deep feature and the target shallow feature, and then the target feature sum is input into the upsampling block, and the convolution kernel is transposed by transposing the convolution kernel, that is, the rows of the convolution kernel are changed into columns, and the columns are changed into rows, and then the transposed convolution kernel is convolved with the input feature map to obtain the upsampled feature map. The transposed convolution method can reduce the chessboard effect that is prone to occur in methods such as sub-pixel convolution. At the same time, the number of parameters of the transposed convolution is the same as that of the ordinary convolution, which is less than the number of parameters of the deconvolution and is easier to implement.

[0069] According to the MRI image super-resolution reconstruction method based on the adaptive Fourier operator proposed in the embodiment of the present application, the adaptive Fourier operator and the convolutional neural network can be used in the spatial mixing area of ​​the Transformer extraction block to perform feature processing in the frequency domain and the spatial domain, so as to effectively extract and integrate the frequency domain information of the MRI image, so as to further realize the combination of frequency domain information and spatial domain information, improve the richness of the feature extraction results and the resolution of the reconstructed image, and better meet the actual needs of medical applications. Thus, the problem in the related technology that the focus of the convolutional neural network in the super-resolution reconstruction process is mainly in the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, resulting in the image reconstruction results being difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of super-resolution reconstruction.

[0070] Next, an MRI image super-resolution reconstruction system based on an adaptive Fourier operator proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0071] Figure 5 It is a structural schematic diagram of an MRI image super-resolution reconstruction system based on an adaptive Fourier operator according to an embodiment of the present application.

[0072] like Figure 5 As shown, the MRI image super-resolution reconstruction system 10 based on the adaptive Fourier operator includes: an acquisition module 100 , an iteration module 200 and a reconstruction module 300 .

[0073] The acquisition module 100 is used to acquire at least one MRI image to be reconstructed, and extract target shallow features of at least one MRI image to be reconstructed.

[0074] The iterative module 200 is used to input the target shallow features into the Transformer set, and perform iterative processing based on the spatial mixing area and channel mixing area of ​​each Transformer extraction block in the Transformer set to obtain at least one target deep feature of the MRI image to be reconstructed.

[0075] The reconstruction module 300 is used to upsample the target deep features and the target shallow features based on transposed convolution, obtain a reconstruction result of at least one MRI image to be reconstructed, and output a target super-resolution MRI image.

[0076] Optionally, in one embodiment of the present application, the iteration module 200 includes: a first processing unit, a second processing unit and a traversal unit.

[0077] Among them, the first processing unit is used to perform feature processing in the frequency domain and the spatial domain based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block.

[0078] The second processing unit is used to perform channel processing on the multi-scale deep features by using the channel mixing area of ​​the current Transformer extraction block to obtain the target deep features under the current Transformer extraction block.

[0079] The traversal unit is used to traverse each Transformer extraction block in the Transformer set to obtain at least one target deep feature of the MRI image to be reconstructed.

[0080] Optionally, in one embodiment of the present application, the first processing unit is specifically used to: perform a two-dimensional discrete Fourier transform based on the input features of the current Transformer extraction block to obtain features in the frequency domain; use a pre-built adaptive Fourier neural operator to process the features in the frequency domain to obtain processed frequency domain features, and perform a two-dimensional discrete Fourier inverse transform on the processed frequency domain features to obtain features in the spatial domain; use a preset convolutional neural network to process the features in the spatial domain to obtain multi-scale deep features under the current Transformer extraction block.

[0081] Optionally, in one embodiment of the present application, the reconstruction module 300 includes: a summing unit and a convolution unit.

[0082] The summing unit is used to obtain the sum of target features based on the target deep features and the target shallow features.

[0083] The convolution unit is used to perform convolution using the transposed convolution kernel and the target feature sum to obtain a target super-resolution MRI image.

[0084] It should be noted that the aforementioned explanation of the embodiment of the MRI image super-resolution reconstruction method based on the adaptive Fourier operator is also applicable to the MRI image super-resolution reconstruction system based on the adaptive Fourier operator of this embodiment, which will not be repeated here.

[0085] According to the MRI image super-resolution reconstruction system based on the adaptive Fourier operator proposed in the embodiment of the present application, the adaptive Fourier operator and the convolutional neural network can be used in the spatial mixing area of ​​the Transformer extraction block to perform feature processing in the frequency domain and the spatial domain, so as to effectively extract and integrate the frequency domain information of the MRI image, so as to further realize the combination of frequency domain information and spatial domain information, improve the richness of the feature extraction results and the resolution of the reconstructed image, and better meet the actual needs of medical applications. Thus, the problem in the related technology that the focus of the convolutional neural network in the super-resolution reconstruction process is mainly in the spatial domain, the feature extraction of the frequency domain information in the LR image is insufficient, and the existing super-resolution reconstruction method does not consider the medical characteristics of the MRI image, resulting in the image reconstruction results being difficult to meet the special needs of clinical applications, affecting the accuracy and practicality of super-resolution reconstruction.

[0086] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0087] A memory 601 , a processor 602 , and a computer program stored in the memory 601 and executable on the processor 602 .

[0088] When the processor 602 executes the program, the MRI image super-resolution reconstruction method based on the adaptive Fourier operator provided in the above embodiment is implemented.

[0089] Furthermore, the electronic device further comprises:

[0090] The communication interface 603 is used for communication between the memory 601 and the processor 602 .

[0091] The memory 601 is used to store computer programs that can be executed on the processor 602 .

[0092] The memory 601 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0093] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0094] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0095] The processor 602 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0096] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the above-mentioned MRI image super-resolution reconstruction method based on the adaptive Fourier operator is implemented.

[0097] This embodiment also provides a computer program, which, when executed, implements the above-mentioned MRI image super-resolution reconstruction method based on the adaptive Fourier operator.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0099] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0100] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0101] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by or in conjunction with an instruction execution system, system or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, system or device and execute the instructions). For the purposes of this specification, a "computer-readable medium" can be any system that can contain, store, communicate, propagate or transport a program for use by or in conjunction with an instruction execution system, system or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wirings (electronic systems), a portable computer disk case (magnetic systems), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic system, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0102] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0103] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0104] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0105] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for super-resolution reconstruction of MRI images based on adaptive Fourier operator, characterized in that: The following steps are involved: Acquire at least one MRI image to be reconstructed, and extract target shallow features of the at least one MRI image to be reconstructed; Inputting the target shallow features into a Transformer set, and performing iterative processing based on the spatial mixing region and the channel mixing region of each Transformer extraction block in the Transformer set, respectively, to obtain the target deep features of the at least one MRI image to be reconstructed; The target deep features and the target shallow features are upsampled based on transposed convolution to obtain a reconstruction result of the at least one MRI image to be reconstructed, and a target super-resolution MRI image is output.

2. The method according to claim 1, characterized in that The iterative processing is performed based on the spatial mixing area and the channel mixing area of ​​each Transformer extraction block in the Transformer set to obtain the target deep features of the at least one MRI image to be reconstructed, including: Performing feature processing in the frequency domain and the spatial domain based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; Performing channel processing on the multi-scale deep features using the channel mixing area of ​​the current Transformer extraction block to obtain target deep features under the current Transformer extraction block; Each Transformer extraction block in the Transformer set is traversed to obtain target deep features of the at least one MRI image to be reconstructed.

3. The method according to claim 2, characterized in that The feature processing in the frequency domain and the spatial domain is performed based on the spatial mixing area of ​​the current Transformer extraction block to obtain the multi-scale deep features under the current Transformer extraction block, including: Perform a two-dimensional discrete Fourier transform based on the input features of the current Transformer extraction block to obtain features in the frequency domain; Using a pre-constructed adaptive Fourier neural operator to process the features in the frequency domain to obtain processed frequency domain features, and performing a two-dimensional discrete Fourier inverse transform on the processed frequency domain features to obtain features in the spatial domain; The features in the spatial domain are processed using a preset convolutional neural network to obtain multi-scale deep features under the current Transformer extraction block.

4. The method according to claim 1, characterized in that The upsampling of the target deep features and the target shallow features based on transposed convolution to obtain a target super-resolution MRI image includes: Obtaining a target feature sum based on the target deep feature and the target shallow feature; The target super-resolution MRI image is obtained by performing convolution using the transposed convolution kernel and the target feature.

5. An MRI image super-resolution reconstruction system based on an adaptive Fourier operator, characterized in that: include: An acquisition module, used to acquire at least one MRI image to be reconstructed, and extract target shallow features of the at least one MRI image to be reconstructed; An iterative module, configured to input the target shallow features into a Transformer set, and perform iterative processing based on the spatial mixing region and the channel mixing region of each Transformer extraction block in the Transformer set, respectively, to obtain the target deep features of the at least one MRI image to be reconstructed; A reconstruction module is used to upsample the target deep features and the target shallow features based on transposed convolution to obtain a reconstruction result of the at least one MRI image to be reconstructed, and output a target super-resolution MRI image.

6. The system according to claim 5, characterized in that The iteration module includes: A first processing unit is used to perform feature processing in the frequency domain and the spatial domain based on the spatial mixing area of ​​the current Transformer extraction block to obtain multi-scale deep features under the current Transformer extraction block; A second processing unit is used to perform channel processing on the multi-scale deep features by using the channel mixing area of ​​the current Transformer extraction block to obtain the target deep features under the current Transformer extraction block; The traversal unit is used to traverse each Transformer extraction block in the Transformer set to obtain the target deep features of the at least one MRI image to be reconstructed.

7. The system according to claim 6, characterized in that The first processing unit is specifically configured to: Perform a two-dimensional discrete Fourier transform based on the input features of the current Transformer extraction block to obtain features in the frequency domain; Using a pre-constructed adaptive Fourier neural operator to process the features in the frequency domain to obtain processed frequency domain features, and performing a two-dimensional discrete Fourier inverse transform on the processed frequency domain features to obtain features in the spatial domain; The features in the spatial domain are processed using a preset convolutional neural network to obtain multi-scale deep features under the current Transformer extraction block.

8. The system according to claim 5, characterized in that The reconstruction module comprises: A summing unit, used for obtaining a sum of target features based on the target deep features and the target shallow features; The convolution unit is used to perform convolution using the transposed convolution kernel and the target feature to obtain the target super-resolution MRI image.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier operator as described in any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier operator as described in any one of claims 1 to 4.

11. A computer program product, comprising a computer program, characterized in that The computer program is executed to implement the MRI image super-resolution reconstruction method based on the adaptive Fourier operator as described in any one of claims 1 to 4.

Images