Magnetic resonance image reconstruction method and device based on Fourier convolution

Abstract

The present disclosure relates to a magnetic resonance image reconstruction method and device based on Fourier convolution. The method uses an image reconstruction model to reconstruct the magnetic resonance image. The image reconstruction includes extracting features from the magnetic resonance image to be reconstructed to obtain multiple target feature maps; converting each target feature map from the spatial domain to the frequency domain and then performing Fourier convolution calculation to obtain a feature map to be converted corresponding to each target feature map; converting each feature map to be converted from the frequency domain to the spatial domain to obtain each Fourier convolution feature map; performing point-by-point convolution on all Fourier convolution feature maps to obtain a point-by-point convolution feature map; and merging features on the point-by-point convolution feature maps to obtain a reconstructed magnetic resonance image. The magnetic resonance image reconstruction method and device based on Fourier convolution provided by the embodiment of the present disclosure can well remove aliasing artifacts in undersampled magnetic resonance scanning, improve the accuracy of the reconstructed magnetic resonance image, and have a global receptive field and low computational complexity.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a magnetic resonance image reconstruction method and device based on Fourier convolution. Background Art

[0002] Magnetic resonance imaging (MRI) is a technology that uses strong magnetic fields and radio frequency pulses to image the inside of the human body. MRI is widely used in clinical diagnosis because of its high-resolution imaging capability for soft tissues. However, MRI images obtained in clinical diagnosis have inevitable aliasing artifacts, which reduces the quality of MRI images and affects the accurate diagnosis of diseases.

[0003] In recent years, with the rapid development of artificial intelligence technology, deep learning has made remarkable progress in the field of image processing. Models based on deep learning have performed well in the reconstruction of magnetic resonance images. These models use neural networks to restore images from undersampled data, surpassing traditional perception methods in terms of reconstruction quality and speed, but still have problems such as small receptive field and excessive computational requirements.

[0004] Therefore, how to remove aliasing artifacts in undersampled MRI scans is a challenging problem. Summary of the invention

[0005] In view of this, the present invention proposes a magnetic resonance image reconstruction method and device based on Fourier convolution, which can effectively remove aliasing artifacts in under-sampled magnetic resonance scanning and improve the accuracy of reconstructed magnetic resonance images, while having a global receptive field and low computational complexity.

[0006] According to one aspect of the present disclosure, a magnetic resonance image reconstruction method based on Fourier convolution is provided, wherein the method uses an image reconstruction model to perform image reconstruction of a magnetic resonance image, and the image reconstruction comprises the following steps: a feature extraction step: performing feature extraction on the magnetic resonance image to be reconstructed to obtain a plurality of target feature maps of the magnetic resonance image to be reconstructed; a Fourier convolution step: performing corresponding Fourier convolution calculations on each of the target feature maps after converting the spatial domain to the frequency domain to obtain a feature map to be converted corresponding to each of the target feature maps; performing a frequency domain to spatial domain conversion on each of the feature maps to be converted to obtain a Fourier convolution feature map corresponding to each of the target feature maps; a point-by-point convolution step: performing point-by-point convolution on all of the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map; a feature merging step: performing feature merging on the at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image.

[0007] In this way, through the feature extraction step, the Fourier convolution step, the point-by-point convolution step, and the feature merging step, the magnetic resonance image can be reconstructed quickly and accurately, which has significant advantages in improving image quality, reducing noise and artifact interference, and improving reconstruction efficiency, providing more reliable and efficient image support for medical diagnosis.

[0008] In a possible implementation, when the image reconstruction model includes an iterative reconstruction model, the image reconstruction further includes the following steps: an iterative determination step: after the feature merging step, when it is determined that the current iteration does not satisfy the stopping condition, the reconstructed magnetic resonance image is determined as a new magnetic resonance image to be reconstructed and returns to execute the feature extraction step and the steps thereafter; when it is determined that the current iteration satisfies the stopping condition, the reconstructed magnetic resonance image is output.

[0009] In this way, the accuracy of magnetic resonance image reconstruction is significantly improved by introducing the iterative determination step. Each iteration uses the reconstruction result of the previous step as a new input to further extract and integrate image features. Multiple iterations can gradually approach the real image information, thereby obtaining a more accurate and detailed reconstructed image. In addition, the iterative determination step makes the method more robust when processing complex magnetic resonance images. For images containing noise, artifacts or other interference factors, through multiple iterations, the method can gradually eliminate these unfavorable factors and improve the clarity and contrast of the image, which helps doctors to more accurately identify and analyze lesions and improve the accuracy and reliability of diagnosis. Although the iterative process may increase a certain amount of calculation, the efficiency and speed of the iterative process can be ensured through reasonable algorithm optimization and computing resource allocation. At the same time, since the iteration will be terminated after a certain stop condition is met, meaningless excessive iterations can be avoided, ensuring the efficiency and stability of the entire reconstruction process.

[0010] In a possible implementation, each of the target feature maps is converted from the spatial domain to the frequency domain and then a corresponding Fourier convolution calculation is performed to obtain a feature map to be converted corresponding to each of the target feature maps, including: using a Fourier convolution kernel for each of the target feature maps to process each of the converted target feature maps to obtain a feature map to be converted corresponding to each of the target feature maps.

[0011] In a possible implementation, the size of the Fourier convolution kernel for each target feature map is the same, and the size of each Fourier convolution kernel is the same as the size of the target feature map. In this way, a corresponding Fourier convolution kernel is set for each target feature map to independently perform Fourier convolution calculations, which can fully utilize the advantages of frequency domain analysis to perform feature extraction in the frequency domain, help extract deeper image features, and at the same time, have strong robustness in frequency domain processing.

[0012] In a possible implementation, the image reconstruction model is a trained image reconstruction model, and the method also includes a training process of the image reconstruction model, and the training process includes the following steps: a sample construction step: constructing a sample set, the sample set including multiple training images, and each of the training images is obtained based on undersampling; a first training step: using the sample set to train the original model to obtain a first model, wherein the original model uses channel-by-channel convolution and point-by-point convolution to achieve image reconstruction; a parameter adjustment step: adjusting the parameters of the channel-by-channel convolution in the first model to the parameters of Fourier convolution to obtain a second model, so that the second model uses Fourier convolution and point-by-point convolution to achieve image reconstruction; a second training step: using the sample set to train the second model to obtain the trained image reconstruction model.

[0013] In this way, through the above training strategy, the trained image reconstruction model can better integrate the local information and global information of the magnetic resonance image to be reconstructed, which is conducive to the convergence of the magnetic resonance reconstruction neural network.

[0014] According to another aspect of the present disclosure, a magnetic resonance image reconstruction device based on Fourier convolution is provided, wherein the device uses an image reconstruction model to reconstruct the magnetic resonance image, and the device includes: a first module, used to extract features of the magnetic resonance image to be reconstructed, and obtain multiple target feature maps of the magnetic resonance image to be reconstructed; a second module, used to convert each of the target feature maps from the spatial domain to the frequency domain and then perform corresponding Fourier convolution calculations respectively to obtain a feature map to be converted corresponding to each of the target feature maps; convert each of the feature maps to be converted from the frequency domain to the spatial domain to obtain a Fourier convolution feature map corresponding to each of the target feature maps; a third module, used to perform point-by-point convolution on all of the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map; and a fourth module, used to perform feature merging on the at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image.

[0015] In this way, the first module, the second module, the third module, and the fourth module can realize rapid and accurate reconstruction of magnetic resonance images, which has significant advantages in improving image quality, reducing noise and artifact interference, and improving reconstruction efficiency, providing more reliable and efficient image support for medical diagnosis.

[0016] In a possible implementation, when the image reconstruction model includes an iterative reconstruction model, the device also includes: a fifth module, which is used to, after performing feature extraction on the magnetic resonance image to be reconstructed to obtain multiple target feature maps of the magnetic resonance image to be reconstructed, determine the reconstructed magnetic resonance image as a new magnetic resonance image to be reconstructed and return it to the first module to perform corresponding operations and subsequent operations when it is determined that the current iteration does not meet the stopping condition; and output the reconstructed magnetic resonance image when it is determined that the current iteration meets the stopping condition.

[0017] In this way, the accuracy of magnetic resonance image reconstruction is significantly improved by introducing the fifth module. Each iteration uses the reconstruction result of the previous step as a new input to further extract and integrate image features. Multiple iterations can gradually approach the real image information, thereby obtaining a more accurate and detailed reconstructed image. In addition, the fifth module makes the device more robust when processing complex magnetic resonance images. For images containing noise, artifacts or other interference factors, these unfavorable factors can be gradually eliminated through multiple iterations, and the clarity and contrast of the image can be improved, which helps doctors to more accurately identify and analyze lesions and improve the accuracy and reliability of diagnosis. Although the iterative process may increase a certain amount of calculation, the efficiency and speed of the iterative process can be ensured through reasonable algorithm optimization and computing resource allocation. At the same time, since the iteration will be terminated after a certain stop condition is met, meaningless excessive iterations can be avoided, ensuring the efficiency and stability of the entire reconstruction process.

[0018] In a possible implementation, each of the target feature maps is converted from the spatial domain to the frequency domain and then a corresponding Fourier convolution calculation is performed to obtain a feature map to be converted corresponding to each of the target feature maps, including: using a Fourier convolution kernel for each of the target feature maps to process each of the converted target feature maps to obtain a feature map to be converted corresponding to each of the target feature maps.

[0019] In a possible implementation, the size of the Fourier convolution kernel for each of the target feature maps is the same, and the size of each of the Fourier convolution kernels is the same as the size of the target feature map.

[0020] In this way, a corresponding Fourier convolution kernel is set for each target feature map to perform Fourier convolution calculation independently, which can make full use of the advantages of frequency domain analysis to extract features in the frequency domain, help to extract deeper image features, and at the same time, it is more robust when processing in the frequency domain.

[0021] In a possible implementation, the image reconstruction model is a trained image reconstruction model, and the device also includes a training module, which is used to execute a training process of the image reconstruction model, and the training process includes: constructing a sample set, the sample set including multiple training images, each of the training images is obtained based on undersampling; using the sample set to train the original model to obtain a first model, wherein the original model uses channel-by-channel convolution and point-by-point convolution to achieve image reconstruction; adjusting the parameters of the channel-by-channel convolution in the first model to the parameters of the Fourier convolution to obtain a second model, so that the second model uses Fourier convolution and point-by-point convolution to achieve image reconstruction; using the sample set to train the second model to obtain the trained image reconstruction model.

[0022] In this way, through the above training strategy, the trained image reconstruction model can better integrate the local information and global information of the magnetic resonance image to be reconstructed, which is conducive to the convergence of the magnetic resonance reconstruction neural network.

[0023] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0024] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0027] Figure 1 A flowchart of a magnetic resonance image reconstruction method based on Fourier convolution provided according to an embodiment of the present disclosure is shown.

[0028] Figure 2 A schematic diagram of Fourier convolution provided according to an embodiment of the present disclosure is shown.

[0029] Figure 3 A schematic diagram of a common convolution provided according to an embodiment of the present disclosure is shown.

[0030] Figure 4A schematic diagram of point-by-point convolution provided according to an embodiment of the present disclosure is shown.

[0031] Figure 5 A schematic diagram of an image reconstruction process using an end-to-end reconstruction model provided according to an embodiment of the present disclosure is shown.

[0032] Figure 6 A schematic diagram of a first convolution module of an end-to-end reconstruction model provided according to an embodiment of the present disclosure is shown.

[0033] Figure 7 A schematic diagram showing the receptive field of an end-to-end model based on ordinary convolution provided according to an embodiment of the present disclosure.

[0034] Figure 8 A schematic diagram showing the receptive field of an end-to-end model based on Fourier convolution provided according to an embodiment of the present disclosure.

[0035] Fig. 9 A schematic diagram showing an image reconstruction process using an iterative reconstruction model provided according to an embodiment of the present disclosure.

[0036] Fig.10 A schematic diagram of a second convolution module of an iterative reconstruction model provided according to an embodiment of the present disclosure is shown.

[0037] Fig.11 A schematic diagram showing reconstruction results obtained using different magnetic resonance image reconstruction methods in a knee reconstruction scenario.

[0038] Fig.12 and Fig.13 A block diagram of a magnetic resonance image reconstruction device based on Fourier convolution according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0039] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0040] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0041] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0042] In order to facilitate those skilled in the art to understand the technical solution provided by the embodiments of the present disclosure, the technical environment in which the technical solution is implemented is described below.

[0043] Magnetic resonance imaging plays an irreplaceable role in the medical field, especially in the diagnosis of diseases in the brain, heart, joints and other parts. The acquisition of magnetic resonance images depends on repeated scanning in the encoding space. Due to the slow acquisition speed of magnetic resonance, it will not only introduce more motion artifacts, but also cause discomfort to patients due to the long examination time. Therefore, in order to shorten the acquisition time, the common method adopted is to reduce the amount of acquired data. However, acquiring fewer data points in the acquisition space of magnetic resonance violates Shannon's sampling law, which will lead to inevitable aliasing artifacts in the under-sampled MRI images, which will reduce the quality of magnetic resonance images and affect the accurate diagnosis of diseases.

[0044] Traditional magnetic resonance image reconstruction mainly uses compressed sensing methods to solve the aliasing artifact problem. The compressed sensing method uses predefined transformations and sparsity priors in iterative optimization algorithms to solve the aliasing artifact problem. However, this method lacks universal optimal parameter selection for sparse types and usually exhibits poor performance at higher undersampling rates.

[0045] In recent years, with the rapid development of artificial intelligence technology, deep learning has made remarkable progress in the field of image processing. Models based on deep learning have performed well in the reconstruction of magnetic resonance images. These models use neural networks to restore images from undersampled data, surpassing traditional perception methods in terms of reconstruction quality and speed.

[0046] Convolutional neural networks play a central role in deep learning-based MRI reconstruction models. But most convolutional neural networks today use small-sized convolutions (such as 3x3 or 5x5) and use sliding window methods, and stack multiple convolutional layers to achieve a larger receptive field. However, convolutional neural networks using stacked 3x3 convolutions are still not enough to capture global information, and also bring about the problem of excessive computational requirements. There are some studies in the related art that explore large-sized convolutions, such as Vision Transformers (ViT). Although these studies have increased the receptive field to a certain extent, they still face problems such as excessive computational requirements and training difficulties.

[0047] The disclosed embodiment provides a magnetic resonance image reconstruction method based on Fourier convolution, wherein the method uses an image reconstruction model to perform image reconstruction of a magnetic resonance image, wherein the image reconstruction comprises a feature extraction step, a Fourier convolution step, a point-by-point convolution step, and a feature merging step, wherein the feature extraction step comprises extracting features from a magnetic resonance image to be reconstructed to obtain a plurality of target feature maps of the magnetic resonance image to be reconstructed; the Fourier convolution step comprises converting each of the target feature maps from a spatial domain to a frequency domain and then performing corresponding Fourier convolution calculations respectively to obtain a feature map to be converted corresponding to each of the target feature maps; converting each of the feature maps to be converted from a frequency domain to a spatial domain to obtain a Fourier convolution feature map corresponding to each of the target feature maps; the point-by-point convolution step comprises performing point-by-point convolution on all of the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map; the feature merging step comprises performing feature merging on the at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image. In this way, Fourier convolution and point-by-point convolution can better remove aliasing artifacts in undersampled magnetic resonance scans and improve the accuracy of reconstructed magnetic resonance images, while having a global receptive field and low computational complexity.

[0048] The magnetic resonance image reconstruction method provided by the embodiment of the present disclosure uses an image reconstruction model to reconstruct a magnetic resonance image. The magnetic resonance image reconstruction method can reconstruct a magnetic resonance image to be reconstructed to obtain a reconstructed magnetic resonance image. The magnetic resonance image to be reconstructed is obtained based on undersampling.

[0049] Figure 1 FIG. 2 is a flowchart of a magnetic resonance image reconstruction method based on Fourier convolution according to an embodiment of the present disclosure. Figure 1 As shown, the image reconstruction involved in the image reconstruction method may include the following steps S101 to S104.

[0050] S101, feature extraction step: extracting features of the magnetic resonance image to be reconstructed to obtain a plurality of target feature maps of the magnetic resonance image to be reconstructed.

[0051] The feature extraction step in step S101 can be implemented by convolution calculation, which can be a common convolution operation, that is, convolution is performed with each input channel using a convolution kernel to obtain the convolution results of all channels, and the convolution results at the same position on different channels are summed to obtain the target feature map of the output channel corresponding to the convolution kernel. Among them, each input channel is determined according to the magnetic resonance image to be reconstructed, and an input channel represents the color distribution of the image in the channel. One output channel corresponds to one target feature map, the number of output channels is equal to the number of convolution kernels, and the number of target feature maps is determined according to the number of convolution kernels. In fact, the number of input channels and the number of convolution kernels (or output channels) can be set according to actual needs to obtain a target feature map that meets the expected number, and the embodiments of the present disclosure are not limited to this.

[0052] In this way, through step S101, corresponding multiple target feature maps can be extracted from the magnetic resonance image to be reconstructed, which not only reduces the complexity of data processing, but also retains the key information of the image, thereby improving the efficiency and accuracy of image reconstruction.

[0053] S102, Fourier convolution step: After converting each target feature map from the spatial domain to the frequency domain, corresponding Fourier convolution calculations are performed to obtain feature maps to be converted corresponding to each target feature map; each feature map to be converted is converted from the frequency domain to the spatial domain to obtain Fourier convolution feature maps corresponding to each target feature map.

[0054] Figure 2 FIG. 2 is a schematic diagram of Fourier convolution according to an embodiment of the present disclosure. Taking one of the target feature maps outputted from step S101 as an example, step S102 is described. Figure 2 As shown, in the Fourier convolution step (or Fourier convolution calculation process) provided in the embodiment of the present disclosure, firstly, a fast Fourier transform (FFT) is performed on the input target feature map, that is, the target feature map is converted from the spatial domain to the frequency domain (or Fourier domain) to obtain a Fourier domain feature map, and then in the Fourier domain, the Fourier domain feature map (that is, the target feature map converted to the frequency domain) is element-by-element multiplied with the Fourier domain convolution kernel to obtain a frequency spectrum (that is, the feature map to be converted corresponding to the target feature map), and then the inverse fast Fourier transform (IFFT) is used to convert the feature map to be converted from the frequency domain back to the spatial domain to obtain the Fourier convolution feature map corresponding to the target feature map. The same is true for the remaining target feature maps.

[0055] Both the target feature map and the Fourier domain feature map represent the image features of the magnetic resonance image to be reconstructed. The image features represented by the two are essentially the same. The difference is that the target feature map represents the image features in the spatial domain, and the Fourier domain feature represents the image features in the Fourier domain.

[0056] Since Fourier domain convolution can use spatial convolution kernels of any size between 1 and the size of the input image, Fourier domain convolution can cover the global receptive field. The receptive field refers to the area of ​​the original input image that a point on the feature map can see, that is, the points on the feature map are calculated based on the receptive field size area in the original image. The larger the receptive field value, the larger the range of the original image it can access, which also means that the features it may contain are more global and have a higher semantic level.

[0057] In this way, by introducing the Fourier convolution step, the original spatial domain of the feature map is converted to the frequency domain and then the convolution calculation is performed, so that feature extraction is performed in the frequency domain, which fully utilizes the advantages of Fourier transform in signal processing. It can not only effectively extract the deep features of the image, but also reduce the interference of noise and artifacts at the same time, which helps to further improve the signal-to-noise ratio and contrast of the image and improve the image quality.

[0058] In step S102 provided in the embodiment of the present disclosure, the corresponding Fourier convolution calculation is performed after the spatial domain to frequency domain conversion of each target feature map, and the feature map to be converted corresponding to each target feature map can be obtained by: using the Fourier convolution kernel for each target feature map to process each target feature map after conversion, and obtaining the feature map to be converted corresponding to each target feature map. In other words, the Fourier convolution calculation in step S102 is an independent convolution calculation for each target feature map and the convolution is a Fourier convolution. The size of each target feature map is the same, the size of the Fourier convolution kernel corresponding to each target feature map is the same, and the size of each Fourier convolution kernel is equal to the size of the target feature map. The user can set the size of the Fourier convolution kernel according to the size of the target feature map, and the embodiment of the present disclosure does not limit this. In order to better extract features, the weights of the Fourier convolution kernels corresponding to each target feature map may be different, and the embodiment of the present disclosure does not limit this. The Fourier convolution kernel is a learnable convolution kernel, and its related parameters can be learned during the training process of the image reconstruction model. In this way, a corresponding Fourier convolution kernel is set for each target feature map to perform Fourier convolution calculation independently, which can make full use of the advantages of frequency domain analysis to extract features in the frequency domain, help to extract deeper image features, and at the same time, it is more robust when processing in the frequency domain.

[0059] Since the Fourier convolution calculation in step S102 is an independent convolution calculation for each target feature map, step S102 implies the concept of grouping. In each group, a corresponding target feature map can be processed using a Fourier convolution kernel, so that the result of each group is the feature map to be converted corresponding to the target feature map in the current group. Step S102 in this article is based on this. In some embodiments, in each group, a corresponding plurality of target feature maps can be processed using a Fourier convolution kernel, so that the result of each group is determined based on the feature maps to be converted corresponding to all target feature maps in the current group. It should be noted that the number of corresponding target feature maps to be processed in each group can be set according to actual needs, and the embodiments of the present disclosure do not limit this.

[0060] Figure 3 FIG. 2 shows a schematic diagram of a common convolution according to an embodiment of the present disclosure. Figure 3 As shown in , the parameters used for ordinary convolution include batch size s, the number of input channels f, the number of output channels f', height h, width w, and the size of the convolution kernel k×k. Figure 2 As shown, the parameters used for Fourier convolution include batch size s, the number of input channels f, the number of output channels f', height h, and width w.

[0061] Table 1 shows the number of operations and parameters of Fourier convolution and ordinary convolution. Figure 2 , Figure 3 As shown in Table 1, the input shape of ordinary convolution is (s, f, h, w), the size of spatial convolution is (f', f, k, k), the speed of ordinary convolution kernel is limited by the size of convolution kernel, and Fourier convolution can achieve a larger receptive field compared to large kernel convolution. At the same time, due to its inherent product property and efficient fast Fourier transform algorithm, it has low computational complexity, can reduce the consumption of computing resources and achieve faster computing speed, thereby improving the speed of image reconstruction.

[0062] Table 1 Number of operations and parameters of Fourier convolution and ordinary convolution

[0063] Ordinary convolution Fourier Convolution operand s·f·f′·h·w·k2 s·f·h·w(f′+logh+logw) Parameter quantity f·f′·k2 f·f′·h·w

[0064] S103, point-by-point convolution step: performing point-by-point convolution on all Fourier convolution feature maps to obtain at least one point-by-point convolution feature map.

[0065] The size of the convolution kernel of the point-by-point convolution used in step S103 is 1×1×M, where M is the number of channels of the previous layer, i.e., the number of Fourier convolution feature maps. The point-by-point convolution step (or point-by-point convolution calculation process) refers to weighted combination of all Fourier convolution feature maps in the depth direction to generate a new feature map, i.e., a point-by-point convolution feature map. The number of point-by-point convolution feature maps is equal to the number of point-by-point convolution kernels. Users can set the number of point-by-point convolution kernels according to actual needs to obtain a desired number of point-by-point convolution feature maps.

[0066] Figure 4 A schematic diagram of point-by-point convolution according to an embodiment of the present disclosure is shown. Taking M=3 and the number of point-by-point convolution kernels as 1 as an example, the calculation process of point-by-point convolution is described. The calculation process may include two steps. The first step may be to perform a Fourier convolution feature map obtained in step S102 according to Figure 4 The corresponding relationship shown is weightedly calculated. Taking the first Fourier convolution feature map as an example, the values ​​at all positions of the feature map are multiplied by the corresponding weight values ​​in the point-by-point convolution kernel to obtain a product feature map. The remaining Fourier convolution feature maps are similar to the first Fourier convolution feature map. Through the first step, three product feature maps can be obtained. The second step can be to combine and calculate all the product feature maps obtained in the first step according to the position relationship. Taking the position with the upper left corner coordinate (1,1) as an example, the values ​​of the three product feature maps at the position with the upper left corner coordinate (1,1) are summed, and the sum is used as the value of the position with the upper left corner coordinate (1,1) in the point-by-point convolution feature map corresponding to the point-by-point convolution kernel. The values ​​at the remaining positions are calculated in the same way, thereby outputting the point-by-point convolution feature map.

[0067] In this way, all Fourier convolution feature maps can be fused through the point-by-point convolution step, so that the output point-by-point convolution feature map has a stronger image representation ability, which helps to improve the accuracy and robustness of image reconstruction.

[0068] S104, feature merging step: merging features of at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image.

[0069] The feature merging step in step S104 can be a deconvolution calculation. The principle of deconvolution is to reverse the convolution operation, that is, to rotate the convolution kernel 180 degrees, and fill and convolve the feature map to achieve the effect of feature merging. In some embodiments, deconvolution fills a certain number of zeros around each pixel of the low-resolution feature map, and uses the rotated convolution kernel to perform a convolution operation on the feature map to generate a high-resolution feature map. Among them, the convolution operation in the feature merging can also be a normal convolution operation. For details, please refer to the previous text. For the sake of brevity, this article will not repeat it.

[0070] In this way, the final feature merging step can realize the conversion process from the point-by-point convolution feature map to the reconstructed magnetic resonance image. In this process, not only the detailed information of the image is retained, but also the reconstructed image has a higher resolution and better visual effect.

[0071] In summary, the magnetic resonance image reconstruction method based on Fourier convolution provided in the embodiment of the present disclosure can realize fast and accurate reconstruction of magnetic resonance images through steps S101 to S104, and has significant advantages in improving image quality, reducing noise and artifact interference, and improving reconstruction efficiency, providing more reliable and efficient image support for medical diagnosis.

[0072] Different reconstruction methods may be used based on different selected models. The image reconstruction model may include an end-to-end reconstruction model or an iterative reconstruction model, so the embodiment of the present disclosure provides two reconstruction methods, which are schematically described below.

[0073] The first reconstruction method is to reconstruct the magnetic resonance image to be reconstructed by using an end-to-end reconstruction model to obtain a first reconstructed magnetic resonance image, and the first reconstructed magnetic resonance image is the reconstructed magnetic resonance image.

[0074] The end-to-end reconstruction model may include a first feature extraction module, a first convolution module, and a first feature merging module connected in sequence. The first feature extraction module can be used to implement the above-mentioned feature extraction step, and the first feature merging module can be used to implement the above-mentioned feature merging step. The specific implementation process is described above, and this article will not be repeated here for the sake of brevity. The first convolution module may include a first convolution unit provided with a Fourier convolution kernel and a point-by-point convolution kernel. The first convolution unit can be used to implement the above-mentioned Fourier convolution step and the above-mentioned point-by-point convolution step in sequence. The specific implementation process is described above, and this article will not be repeated here for the sake of brevity. It should be noted that the number of the first convolution module and the first convolution unit can be set according to actual needs to obtain a first reconstructed magnetic resonance image that meets the expectations, and the embodiments of the present disclosure do not limit this.

[0075] In one example, the number of the first convolution module and the number of the first convolution unit are both 1. At this time, the image reconstruction process of the magnetic resonance image performed by the end-to-end reconstruction model is: executing the above step S101 through the first feature extraction module to obtain multiple target feature maps; then executing the above steps S102 and step S103 in sequence through the first convolution unit in the first convolution module to obtain at least one point-by-point convolution feature map; finally, executing the above S104 through the first feature merging module to obtain the first reconstructed magnetic resonance image.

[0076] In some embodiments, the first convolution module may further include a first activation unit. The first activation unit may be used to process the output of the first convolution unit, and the activation function used for processing may be a linear rectified function (Rectified Linear Unit, ReLU) or other functions to improve the expressive power of the model. It should be noted that the number of first activation units may also be set according to actual needs, and the embodiments of the present disclosure do not limit this.

[0077] In another example, there are multiple first convolution modules and multiple first convolution units in each first convolution module. In this case, the end-to-end reconstruction model needs to execute the above steps S102 and S103 multiple times during the image reconstruction of the magnetic resonance image. Figure 5 A schematic diagram of an image reconstruction process using an end-to-end reconstruction model provided according to an embodiment of the present disclosure is shown. Figure 6 FIG. 1 is a schematic diagram of a first convolution module of an end-to-end reconstruction model provided according to an embodiment of the present disclosure. Figure 1 , Figure 5 , Figure 6 A schematic description is given.

[0078] like Figure 5 As shown, the end-to-end reconstruction model may include a first feature extraction module (Conv1), 8 first convolution modules (ConvBlock n1 、ConvBlock n2 ...ConvBlock n8 ), 1 first feature merging module (Conv2). The 8 first convolution modules are all used to perform convolution processing on the input feature map. The convolution processing process includes Fourier convolution process and point-by-point convolution process. It should be noted that although the inputs of each first convolution module are different, the convolution processing processes performed by each of them are essentially the Fourier convolution calculation process corresponding to the above step S102 and the point-by-point convolution calculation process corresponding to the above step S103. For details, please refer to the previous text and will not be repeated here.

[0079] like Figure 6As shown, each first convolution module may include two first convolution units and one first activation unit, the first first convolution unit includes an FC1 subunit and a PW1 subunit, and the second first convolution unit includes an FC2 subunit and a PW2 subunit, wherein the inputs of the FC1 subunit and the FC2 subunit are different, but the calculation processes performed by each subunit are essentially the Fourier convolution calculation process corresponding to the above step S102, and the inputs of the PW1 subunit and the PW2 subunit are different, but the calculation processes performed by each subunit are essentially the point-by-point convolution calculation process corresponding to the above step S103. In order to better illustrate the image reconstruction process, they are referred to as Fourier convolution and point-by-point convolution hereinafter. In this example, the number of point-by-point convolution kernels is the same as the number of target feature maps.

[0080] Figure 5 The image reconstruction process of the end-to-end reconstruction model shown is as follows: feature extraction (i.e., feature extraction) is performed on the input magnetic resonance image to be reconstructed through Conv1, and a corresponding plurality of target feature maps are output; n1 Perform convolution on multiple target feature maps output by Conv1 and output ConvBlock n1 Corresponding multiple point-by-point convolution feature maps; through ConvBlock n2 ConvBlock n1 The output multiple point-by-point convolution feature maps are convolved and output ConvBlock n2 Corresponding multiple point-by-point convolution feature maps; ConvBlock n3 To ConvBlock n8 The same goes for this until ConvBlock n8 Output the corresponding multiple point-by-point convolution feature maps; ConvBlock through Conv2 n8 The output multiple point-by-point convolution feature maps are deconvolved (ie, feature merged) to output a first reconstructed magnetic resonance image.

[0081] ConvBlock n1 Take ConvBlock as an example to illustrate the process of convolution: n1 The FC1 subunit in the ConvBlock performs Fourier convolution on multiple target feature maps output by Conv1, and outputs the Fourier convolution feature map corresponding to each target feature map; through ConvBlock n1 The PW1 subunit in the FC1 subunit performs point-by-point convolution on all Fourier convolution feature maps output by the FC1 subunit, and outputs multiple point-by-point convolution feature maps; through ConvBlock n1 The first activation unit in processes the multiple point-by-point convolution feature maps output by the PW1 subunit to obtain multiple point-by-point convolution feature maps after activation; then, continue through ConvBlockn1 The FC2 subunit in performs Fourier convolution on these new multiple target feature maps, and outputs the Fourier convolution feature maps corresponding to each new target feature map; through ConvBlock n1 The PW2 subunit in the FC2 subunit performs point-by-point convolution on all new Fourier convolution feature maps output by the FC2 subunit to obtain multiple new point-by-point convolution feature maps, and these new multiple point-by-point convolution feature maps are used as ConvBlock n1 The output of to be input to ConvBlock n2 ConvBlock n2 To ConvBlock n8 The convolution processing is similar to ConvBlock n1 , I will not go into details here.

[0082] In the first reconstruction method, the end-to-end reconstruction model is obtained by applying Fourier convolution to the end-to-end convolutional neural network architecture. In the specific implementation, not only the ordinary convolution (or conventional convolution) used in the original convolution block in the end-to-end convolutional neural network architecture is modified, but also the residual link method (e.g. Figure 6 As shown), to integrate Fourier convolution. Similar to neural networks with large kernel convolution, Fourier convolution introduces additional parameters (i.e., h, w in Table 1 above). To address this shortcoming, in some embodiments, point-by-point convolution in depthwise separable convolution can be used as a solution, by setting point-by-point convolution after Fourier convolution, and since most of the parameters of depthwise separable convolution are concentrated in point-by-point convolution, this allows us to scale depthwise convolution at a relatively low cost. In this way, by replacing conventional convolution with Fourier convolution, other image reconstruction models such as end-to-end models have a global receptive field and low computational complexity. In addition, the use of Fourier convolution kernels in convolutional neural networks is limited by a large number of parameters. The use of point-by-point convolution in depthwise separable convolution helps to keep the parameters of Fourier convolution within a practical range.

[0083] Figure 7 FIG. 1 is a schematic diagram showing the receptive field of an end-to-end model based on ordinary convolution according to an embodiment of the present disclosure. Figure 7 As shown in Figure 2, the receptive field of the end-to-end network based on ordinary convolution presents a distribution similar to a normal distribution, with its mean located at the center of the image. Figure 8 FIG. 1 is a schematic diagram showing the receptive field of an end-to-end model based on Fourier convolution according to an embodiment of the present disclosure. Figure 8 As shown in the figure, the receptive field of the end-to-end model based on Fourier convolution is not concentrated in the center of the image, but is spread over a larger area. Its wider coverage is conducive to the end-to-end model (network) to utilize long-distance information to better remove undersampling artifacts.

[0084] In this way, when the image reconstruction model is an end-to-end reconstruction model, the reconstructed magnetic resonance image is the first reconstructed magnetic resonance image obtained by reconstructing the image using the end-to-end reconstruction model. Generally, for situations where reconstruction results need to be obtained quickly, an end-to-end reconstruction model can be selected to achieve efficient image reconstruction.

[0085] The second reconstruction method is to reconstruct the magnetic resonance image to be reconstructed using an iterative reconstruction model to obtain a second reconstructed magnetic resonance image, and the second reconstructed magnetic resonance image is the reconstructed magnetic resonance image. In the case where the image reconstruction model includes an iterative reconstruction model, the image reconstruction may also include an iterative determination step. The iterative determination step may include: after the feature merging step, when it is determined that the current iteration does not meet the stop condition, the reconstructed magnetic resonance image is determined as a new magnetic resonance image to be reconstructed and the feature extraction step and subsequent steps are returned; when it is determined that the current iteration meets the stop condition, the reconstructed magnetic resonance image is output. Among them, the iterative stop condition can be flexibly set according to actual needs, and the embodiments of the present disclosure do not limit this.

[0086] In this way, the accuracy of magnetic resonance image reconstruction is significantly improved by introducing the iterative determination step. Each iteration uses the reconstruction result of the previous step as a new input to further extract and integrate image features. Multiple iterations can gradually approach the real image information, thereby obtaining a more accurate and detailed reconstructed image. In addition, the iterative determination step makes the method more robust when processing complex magnetic resonance images. For images containing noise, artifacts or other interference factors, through multiple iterations, the method can gradually eliminate these unfavorable factors and improve the clarity and contrast of the image, which helps doctors to more accurately identify and analyze lesions and improve the accuracy and reliability of diagnosis. Although the iterative process may increase a certain amount of calculation, the efficiency and speed of the iterative process can be ensured through reasonable algorithm optimization and computing resource allocation. At the same time, since the iteration will be terminated after a certain stop condition is met, meaningless excessive iterations can be avoided, ensuring the efficiency and stability of the entire reconstruction process.

[0087] The iterative reconstruction model may include a second feature extraction module, a second convolution module, a second feature merging module, and an iterative confirmation module connected in sequence. The second feature extraction module can be used to implement the above-mentioned feature extraction step, and the second feature merging module can be used to implement the above-mentioned feature merging step. For the specific implementation process, please refer to the previous text. For the sake of brevity, this article will not repeat it here. The second convolution module may include a second convolution unit provided with a Fourier convolution kernel and a point-by-point convolution kernel. The second convolution unit can be used to sequentially implement the above-mentioned Fourier convolution step and the above-mentioned point-by-point convolution step. For the specific implementation process, please refer to the previous text. For the sake of brevity, this article will not repeat it here. The iterative confirmation module can be used to implement the above-mentioned iterative determination step. For the specific implementation process, please refer to the previous text. For the sake of brevity, this article will not repeat it here. It should be noted that the number of the second convolution module and the second convolution unit can be set according to actual needs to obtain a second reconstructed magnetic resonance image that meets the expectations, and the embodiments of the present disclosure do not limit this.

[0088] In some embodiments, the iterative reconstruction model may further include a data processing module. The data processing module may be used to process the data consistency problem of the input data. The processing method used is set according to actual needs, and the embodiments of the present disclosure do not limit this.

[0089] Fig. 9 A schematic diagram showing an image reconstruction process using an iterative reconstruction model provided according to an embodiment of the present disclosure. Fig.10 FIG. 2 is a schematic diagram of a second convolution module of an iterative reconstruction model provided according to an embodiment of the present disclosure. Figure 1 , Fig. 9 , Fig.10 The image reconstruction process is schematically illustrated.

[0090] like Fig. 9 As shown, the iterative reconstruction model may include a second feature extraction module (Conv1), three second convolution modules (ConvBlock n1 、ConvBlock n2 、ConvBlock n3 ), a second feature merging module (Conv2), a data processing module (DC1), and an iterative confirmation module (DC2). The three second convolution modules are all used to perform convolution processing on the input feature map. The convolution processing process includes a Fourier convolution process and a point-by-point convolution process. It should be noted that although the inputs of each second convolution module are different, the convolution processing processes performed by each of them are essentially the Fourier convolution calculation process corresponding to the above step S102 and the point-by-point convolution calculation process corresponding to the above step S103. For details, please refer to the previous text and will not be repeated here.

[0091] like Fig.10As shown, each second convolution module may include 1 second convolution unit, 1 normalization unit, and 1 second activation unit. The second convolution unit includes an FC subunit and a PW subunit. The calculation process performed by the FC subunit is essentially the Fourier convolution calculation process corresponding to the above step S102. The calculation process performed by the PW subunit is essentially the point-by-point convolution calculation process corresponding to the above step S103. In order to better illustrate the image reconstruction process, it is referred to as Fourier convolution and point-by-point convolution in the following text. The normalization unit can be used for normalization of data, so that the accuracy can be improved while ensuring that subsequent data processing is more convenient and faster. The second activation unit can be used to process the output of the normalization unit, and the activation function used for processing can be a linear rectified function (Rectified Linear Unit, ReLU) or other functions to improve the expression ability of the model. In this example, the number of point-by-point convolution kernels is the same as the number of target feature maps, and the preset stop condition is to iterate 20 times.

[0092] Fig. 9 The image reconstruction process of the iterative reconstruction model shown is as follows: in the first iteration (n=1), Conv1 is used to extract features of the input magnetic resonance image to be reconstructed (i.e., x0), and a plurality of corresponding target feature maps are output; ConvBlock n1 Perform convolution on multiple target feature maps output by Conv1 and output ConvBlock n1 Corresponding multiple point-by-point convolution feature maps; through ConvBlock n2 ConvBlock n1 The output multiple point-by-point convolution feature maps are convolved and output ConvBlock n2 Corresponding multiple point-by-point convolution feature maps; through ConvBlock n3 ConvBlock n2 The output multiple point-by-point convolution feature maps are convolved and output ConvBlock n3 Corresponding multiple point-by-point convolution feature maps; ConvBlock through Conv2 n3 The output multiple point-by-point convolution feature maps are deconvolved to output the reconstructed magnetic resonance image; ConvBlock is reconstructed by DC1 n3 The output reconstructed magnetic resonance image is processed to obtain a processed reconstructed magnetic resonance image (i.e., x1); DC2 is then used to determine whether the current iteration meets the preset stop condition, and it is determined that the current iteration does not meet the stop condition. In this case, the processed reconstructed magnetic resonance image is determined as a new magnetic resonance image to be reconstructed and returned to Conv1 for processing and continues to pass through ConvBlock in sequence. n1 、ConvBlockn2 、ConvBlock n3 , DC1 processes until DC2 outputs the final reconstructed magnetic resonance image, that is, the second reconstructed magnetic resonance image, when it determines that the current iteration satisfies the stopping condition.

[0093] ConvBlock n1 Take ConvBlock as an example to illustrate the process of convolution: n1 The FC subunit in performs Fourier convolution on multiple target feature maps output by Conv1, and outputs the Fourier convolution feature map corresponding to each target feature map; through ConvBlock n1 The PW subunit in the FC subunit performs point-by-point convolution on all Fourier convolution feature maps output by the FC subunit, and outputs multiple point-by-point convolution feature maps; then the multiple point-by-point convolution feature maps output by the PW subunit are normalized by the normalization unit to obtain multiple normalized point-by-point convolution feature maps; finally, the ConvBlock n1 The second activation unit in the process processes the normalized multiple point-by-point convolution feature maps output by the normalization unit to obtain multiple activated point-by-point convolution feature maps, and uses these activated multiple point-by-point convolution feature maps as ConvBlock n1 The output of to be input to ConvBlock n2 ConvBlock n2 、ConvBlock n3 The convolution processing is similar to ConvBlock n1 , I will not go into details here.

[0094] In the second reconstruction method, the above-mentioned iterative reconstruction model is obtained by applying Fourier convolution to the model-based convolutional neural network architecture. The specific implementation method can refer to the first reconstruction method and will not be repeated here.

[0095] In this way, when the image reconstruction model is an iterative reconstruction model, the reconstructed magnetic resonance image is a second reconstructed magnetic resonance image obtained by reconstructing the image using the iterative reconstruction model. Generally, for situations where higher accuracy is required, an iterative reconstruction model can be selected to optimize the reconstruction results through multiple iterations.

[0096] The disclosed embodiment evaluates the proposed image reconstruction method. The data used for evaluation comes from the FastMRI public dataset. 200 cases of brain data were used for training and 50 cases for validation. 100 cases of knee data were used for training and 18 cases for validation. In order to obtain a better signal-to-noise ratio, the training and validation datasets only include proton density PD ((proton density)) weighted images. The evaluation uses the peak signal ratio (PSNR) and structural similarity measure (SSIM) on the validation set to calculate the comparison between the amplitude map of the reconstruction result and the full-sample image.

[0097] Table 2 shows the quantitative effects of the reconstruction method proposed on the brain and knee datasets at 8× and 12× undersampling rates. Among them, the 8× undersampling rate means that the acquired data of the magnetic resonance image to be reconstructed is one eighth of the data of the full-sampled image, and the same is true for 12×. As shown in Table 2, the Fourier convolution proposed in the embodiment of the present disclosure effectively improves the reconstruction performance of the baseline model. Under different undersampling rates, the model improved by Fourier convolution (the improved end-to-end network in Table 2, i.e., the above-mentioned end-to-end reconstruction model, and the improved model-based network in Table 2, i.e., the above-mentioned iterative reconstruction model) achieved excellent quantitative indicators in the two datasets.

[0098] Table 2 Quantitative results of the proposed reconstruction methods on the brain and knee datasets at 8× and 12× undersampling rates

[0099]

[0100] Fig.11 Schematic diagram showing reconstruction results obtained using different magnetic resonance image reconstruction methods in a knee reconstruction scenario. Fig.11 As shown, the end-to-end network and ViT provide smoother results, the improved end-to-end network (i.e., the end-to-end reconstruction model mentioned above) and the improved model-based network (i.e., the iterative reconstruction model mentioned above) reconstruct detailed textures on the bones, the model-based network shows horizontal aliasing artifacts, while the iterative reconstruction model still suppresses artifacts well. The enlarged area and error map show that the end-to-end reconstruction model and the iterative reconstruction model perform better in reconstructing detailed cartilage and edge information, among which the end-to-end reconstruction model has the smallest residual error. Fig.11 R=8 in the above is the 8× undersampling rate, and the same is true for R=12.

[0101] The image reconstruction model (end-to-end reconstruction model or iterative reconstruction model) can be a trained image reconstruction model. The image reconstruction model provided by the disclosed embodiment has a large receptive field, which makes its training process challenging because local and global information needs to be considered. To this end, the magnetic resonance image reconstruction method may also include a training process for the image reconstruction model, and the training process may include a sample construction step, a first training step, a parameter adjustment step, and a second training step that are performed in sequence.

[0102] The sample construction step may include constructing a sample set. The sample set may include multiple training images, each of which is obtained based on an undersampling method.

[0103] The first training step may include training the original model using the sample set to obtain a first model. The original model uses channel-by-channel convolution and point-by-point convolution to achieve image reconstruction, and the original model may adopt a pre-trained model based on a convolutional network. The training stop condition in the first training step is set according to actual needs, and the embodiment of the present disclosure does not limit this. In the first training step (training stage), a 3×3 convolution kernel can be used to train the original model to capture local patterns, and the parameter adjustment step is performed after the model converges.

[0104] The parameter adjustment step may include adjusting the parameters of the channel-by-channel convolution in the first model to the parameters of the Fourier convolution to obtain the second model, so that the second model realizes image reconstruction using Fourier convolution and point-by-point convolution.

[0105] The second training step may include training the second model using the sample set to obtain a trained image reconstruction model. The training stop condition in the second training step is set according to actual needs, and the embodiment of the present disclosure does not limit this. In the second training step (training stage), the equivalence of Fourier convolution can be used to re-parameterize the deep convolution in the convolution module so that it is replaced by Fourier convolution, and then fine-tune the model to adapt to the global receptive field.

[0106] In some embodiments, the full-sampled images can also be used to verify whether the image reconstruction effect performed by the trained image reconstruction model meets expectations. If the actual expectations are not met, further training can be performed until an image reconstruction model that meets expectations is obtained.

[0107] In this way, through the above training strategy, the trained image reconstruction model can better integrate the local information and global information of the magnetic resonance image to be reconstructed, which is conducive to the convergence of the magnetic resonance reconstruction neural network.

[0108] In some embodiments, the trained image reconstruction model can be integrated into a small device, such as a portable USB flash drive, so that the doctor can simply insert the USB flash drive into the computer to reconstruct the magnetic resonance image to be reconstructed. It is plug-and-play and more convenient.

[0109] The magnetic resonance image reconstruction method based on Fourier convolution provided in the embodiment of the present disclosure expands the local receptive field of the existing convolutional neural network and provides a global receptive field through Fourier convolution. It not only has low computational complexity, but also can restore the edges and textures of the magnetic resonance image in more detail, and can widely improve the performance of the existing magnetic resonance image reconstruction neural network to achieve more accurate and effective image reconstruction. In addition, it can be plug-and-play, making it easy for doctors to obtain reconstructed magnetic resonance images.

[0110] The disclosed embodiment also provides a magnetic resonance image reconstruction device based on Fourier convolution. Fig.12 FIG. 1 is a block diagram of a magnetic resonance image reconstruction device based on Fourier convolution according to an embodiment of the present disclosure. The magnetic resonance image reconstruction device uses an image reconstruction model to perform image reconstruction of a magnetic resonance image, such as Fig.12 As shown, the magnetic resonance image reconstruction device 120 based on Fourier convolution may include a first module 121 , a second module 122 , a third module 123 , and a fourth module 124 .

[0111] The first module 121 is used to extract features of the magnetic resonance image to be reconstructed to obtain a plurality of target feature maps of the magnetic resonance image to be reconstructed.

[0112] The second module 122 is used to convert each of the target feature maps from the spatial domain to the frequency domain and then perform corresponding Fourier convolution calculations to obtain feature maps to be converted corresponding to each of the target feature maps; convert each of the feature maps to be converted from the frequency domain to the spatial domain to obtain Fourier convolution feature maps corresponding to each of the target feature maps.

[0113] The third module 123 is used to perform point-by-point convolution on all the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map.

[0114] The fourth module 124 is used to merge the features of the at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image.

[0115] In this way, the first module, the second module, the third module, and the fourth module can realize rapid and accurate reconstruction of magnetic resonance images, which has significant advantages in improving image quality, reducing noise and artifact interference, and improving reconstruction efficiency, providing more reliable and efficient image support for medical diagnosis.

[0116] In a possible implementation, when the image reconstruction model includes an iterative reconstruction model, the device also includes: a fifth module, which is used to, after performing feature extraction on the magnetic resonance image to be reconstructed to obtain multiple target feature maps of the magnetic resonance image to be reconstructed, if it is determined that the current iteration does not meet the stopping condition, determine the reconstructed magnetic resonance image as a new magnetic resonance image to be reconstructed and return it to the first module to perform the corresponding feature extraction operation, that is, perform feature extraction on the magnetic resonance image to be reconstructed to obtain multiple target feature maps of the magnetic resonance image to be reconstructed and the corresponding operations of the second module, the third module, and the fourth module thereafter; and output the reconstructed magnetic resonance image if it is determined that the current iteration meets the stopping condition.

[0117] In this way, the accuracy of magnetic resonance image reconstruction is significantly improved by introducing the fifth module. Each iteration uses the reconstruction result of the previous step as a new input to further extract and integrate image features. Multiple iterations can gradually approach the real image information, thereby obtaining a more accurate and detailed reconstructed image. In addition, the fifth module makes the device more robust when processing complex magnetic resonance images. For images containing noise, artifacts or other interference factors, these unfavorable factors can be gradually eliminated through multiple iterations, and the clarity and contrast of the image can be improved, which helps doctors to more accurately identify and analyze lesions and improve the accuracy and reliability of diagnosis. Although the iterative process may increase a certain amount of calculation, the efficiency and speed of the iterative process can be ensured through reasonable algorithm optimization and computing resource allocation. At the same time, since the iteration will be terminated after a certain stop condition is met, meaningless excessive iterations can be avoided, ensuring the efficiency and stability of the entire reconstruction process.

[0118] In a possible implementation, each of the target feature maps is converted from the spatial domain to the frequency domain and then a corresponding Fourier convolution calculation is performed to obtain a feature map to be converted corresponding to each of the target feature maps, including: using a Fourier convolution kernel for each of the target feature maps to process each of the converted target feature maps to obtain a feature map to be converted corresponding to each of the target feature maps.

[0119] In a possible implementation, the size of the Fourier convolution kernel for each of the target feature maps is the same, and the size of each of the Fourier convolution kernels is the same as the size of the target feature map.

[0120] In this way, a corresponding Fourier convolution kernel is set for each target feature map to perform Fourier convolution calculation independently, which can make full use of the advantages of frequency domain analysis to extract features in the frequency domain, help to extract deeper image features, and at the same time, it is more robust when processing in the frequency domain.

[0121] In a possible implementation, the image reconstruction model is a trained image reconstruction model, and the device also includes a training module, which is used to execute a training process of the image reconstruction model, and the training process includes: constructing a sample set, the sample set including multiple training images, each of the training images is obtained based on undersampling; using the sample set to train the original model to obtain a first model, wherein the original model uses channel-by-channel convolution and point-by-point convolution to achieve image reconstruction; adjusting the parameters of the channel-by-channel convolution in the first model to the parameters of the Fourier convolution to obtain a second model, so that the second model uses Fourier convolution and point-by-point convolution to achieve image reconstruction; using the sample set to train the second model to obtain the trained image reconstruction model.

[0122] In this way, through the above training strategy, the trained image reconstruction model can better integrate the local information and global information of the magnetic resonance image to be reconstructed, which is conducive to the convergence of the magnetic resonance reconstruction neural network.

[0123] In some embodiments, the functions or modules included in the magnetic resonance image reconstruction device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiment. Its specific implementation can refer to the description of the above magnetic resonance image reconstruction method embodiment, and for the sake of brevity, it will not be repeated here.

[0124] The present disclosure also provides an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above-mentioned magnetic resonance image reconstruction method when executing the instructions stored in the memory. In some embodiments, the functions or modules included in the electronic device provided by the present disclosure can be used to execute the method described in the above method embodiment, and its specific implementation can refer to the description of the above magnetic resonance image reconstruction method embodiment, and for the sake of brevity, it will not be repeated here.

[0125] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above-mentioned magnetic resonance image reconstruction method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium. In some embodiments, the functions or modules included in the computer-readable storage medium provided by the present disclosure can be used to execute the method described in the above method embodiment, and its specific implementation can refer to the description of the above magnetic resonance image reconstruction method embodiment, and for the sake of brevity, it will not be repeated here.

[0126] The present disclosure also provides a computer program product, including a computer readable code, or a non-volatile computer readable storage medium carrying the computer readable code, and when the computer readable code is executed in a processor of an electronic device, the processor in the electronic device executes the above-mentioned magnetic resonance image reconstruction method. In some embodiments, the functions or modules included in the computer program product provided by the present disclosure can be used to execute the method described in the above method embodiment, and its specific implementation can refer to the description of the above magnetic resonance image reconstruction method embodiment, and for the sake of brevity, it will not be repeated here.

[0127] Fig.13 1900 is a block diagram of a magnetic resonance image reconstruction device based on Fourier convolution according to an embodiment of the present disclosure. For example, the device 1900 may be provided as a server or a terminal device. Fig.13 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0128] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0129] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0130] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0131] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0132] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0133] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0134] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0135] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0136] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0137] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0138] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A magnetic resonance image reconstruction method based on Fourier convolution, characterized in that: The method uses an image reconstruction model to perform image reconstruction of a magnetic resonance image. The following steps are involved: Feature extraction step: extracting features from the magnetic resonance image to be reconstructed to obtain a plurality of target feature maps of the magnetic resonance image to be reconstructed; Fourier convolution step: after converting each of the target feature graphs from the spatial domain to the frequency domain, respectively perform corresponding Fourier convolution calculations to obtain feature graphs to be converted corresponding to each of the target feature graphs; convert each of the feature graphs to be converted from the frequency domain to the spatial domain to obtain Fourier convolution feature graphs corresponding to each of the target feature graphs; Point-by-point convolution step: performing point-by-point convolution on all the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map, wherein the point-by-point convolution refers to a process of weighted combination of all the Fourier convolution feature maps in the depth direction; Feature merging step: merging the at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image; Among them, performing point-by-point convolution on all the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map includes: multiplying the values ​​at all positions in each of the Fourier convolution feature maps by the corresponding weight values ​​in the point-by-point convolution kernel to obtain the product feature map corresponding to each of the Fourier convolution feature maps; and combining and calculating all the product feature maps according to the position relationship to obtain the point-by-point convolution feature map.

2. The method according to claim 1, characterized in that In the case where the image reconstruction model comprises an iterative reconstruction model, the image reconstruction further comprises the following steps: Iterative determination step: after the feature merging step, when it is determined that the current iteration does not satisfy the stopping condition, the reconstructed magnetic resonance image is determined as a new magnetic resonance image to be reconstructed and returns to execute the feature extraction step and the steps thereafter; when it is determined that the current iteration satisfies the stopping condition, the reconstructed magnetic resonance image is output.

3. The method according to claim 1, characterized in that After converting each of the target feature graphs from the spatial domain to the frequency domain, corresponding Fourier convolution calculations are performed to obtain feature graphs to be converted corresponding to each of the target feature graphs, including: The converted target feature maps are processed using the Fourier convolution kernel corresponding to each target feature map to obtain the feature maps to be converted corresponding to each target feature map.

4. The method according to claim 3, characterized in that: The size of the Fourier convolution kernel for each of the target feature maps is the same, and the size of each of the Fourier convolution kernels is the same as the size of the target feature map.

5. The method according to claim 1, characterized in that The image reconstruction model is a trained image reconstruction model, and the method further includes a training process of the image reconstruction model, wherein the training process includes the following steps: Sample construction step: constructing a sample set, wherein the sample set includes a plurality of training images, each of which is obtained based on undersampling; The first training step: using the sample set to train the original model to obtain a first model, wherein the original model uses channel-by-channel convolution and point-by-point convolution to achieve image reconstruction; Parameter adjustment step: adjusting the parameters of the channel-by-channel convolution in the first model to the parameters of the Fourier convolution to obtain the second model, so that the second model realizes image reconstruction by using the Fourier convolution and the point-by-point convolution; The second training step: using the sample set to train the second model to obtain the trained image reconstruction model.

6. A magnetic resonance image reconstruction device based on Fourier convolution, characterized in that: The device uses an image reconstruction model to perform image reconstruction of a magnetic resonance image, and the device comprises: The first module is used to extract features of the magnetic resonance image to be reconstructed to obtain a plurality of target feature maps of the magnetic resonance image to be reconstructed; The second module is used to convert each of the target feature graphs from the spatial domain to the frequency domain and then perform corresponding Fourier convolution calculations to obtain feature graphs to be converted corresponding to each of the target feature graphs; convert each of the feature graphs to be converted from the frequency domain to the spatial domain to obtain Fourier convolution feature graphs corresponding to each of the target feature graphs; A third module is used to perform point-by-point convolution on all the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map, wherein the point-by-point convolution refers to a process of weighted combination of all the Fourier convolution feature maps in a depth direction; A fourth module is used to merge the features of the at least one point-by-point convolution feature map to obtain a reconstructed magnetic resonance image; Among them, performing point-by-point convolution on all the Fourier convolution feature maps to obtain at least one point-by-point convolution feature map includes: multiplying the values ​​at all positions in each of the Fourier convolution feature maps by the corresponding weight values ​​in the point-by-point convolution kernel to obtain the product feature map corresponding to each of the Fourier convolution feature maps; and combining and calculating all the product feature maps according to the position relationship to obtain the point-by-point convolution feature map.

7. The device according to claim 6, characterized in that After converting each of the target feature maps from the spatial domain to the frequency domain, corresponding Fourier convolution calculations are performed respectively to obtain feature maps to be converted corresponding to each of the target feature maps, including: The converted target feature maps are processed using the Fourier convolution kernel corresponding to each target feature map to obtain the feature maps to be converted corresponding to each target feature map.

8. The device according to claim 7, characterized in that The size of the Fourier convolution kernel for each of the target feature maps is the same, and the size of each of the Fourier convolution kernels is the same as the size of the target feature map.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 5 when executing the instructions stored in the memory.

10. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images