Magnetic resonance image super-resolution reconstruction method, device and terminal

Abstract

The invention provides a magnetic resonance image super-resolution reconstruction method and device and a terminal, and the method comprises the steps: inputting an obtained to-be-reconstructed low-resolution magnetic resonance image into a trained magnetic resonance image super-resolution reconstruction model, so as to reconstruct a corresponding high-resolution magnetic resonance image; the magnetic resonance image super-resolution reconstruction model comprises a shallow feature extraction module, a deep feature extraction module, a residual error and feature fusion module and a reconstruction module, wherein a high-low frequency feature fusion unit and a high-frequency feature guide unit are introduced into the deep feature extraction module. According to the method, the corresponding high-resolution magnetic resonance image can be obtained only by scanning the low-resolution magnetic resonance image, so that the scanning time is shortened, and the scanning efficiency is improved; the parameter quantity is effectively reduced, so that the network is lighter, and equipment with limited hardware resources is more friendly; the reconstruction model can better utilize high and low frequency features, thereby obtaining a high-resolution image with clear details and real organization structure features.

Description

Technical Field

[0001] The present application relates to the technical field of magnetic resonance image reconstruction, and in particular to a method, device and terminal for super-resolution reconstruction of magnetic resonance images. Background Art

[0002] High-resolution MRI images have clearer tissue textures and richer details, which helps doctors to make more accurate lesion analysis and early clinical diagnosis. Image super-resolution reconstruction (SR) technology can restore the corresponding high-resolution (HR) image from a given low-resolution (LR) image, and is an effective technology to assist in generating high-resolution MRI images.

[0003] Super-resolution reconstruction comes from the concept of breaking through the diffraction limit and restoring a single HR image. At present, this technology mainly includes interpolation-based methods, reconstruction-based methods and learning-based methods. Interpolation-based methods, such as nearest neighbor interpolation and bilinear interpolation, are prone to aliasing and ringing effects when reconstructing images; reconstruction-based methods, such as iterative back projection (IBP) and projection onto convex set (POCS), are difficult to select based on the image prior information, which restricts the reconstruction effect; learning-based methods, which are mainly instance learning and dictionary learning, have insufficient learning ability, which will affect the reconstruction effect.

[0004] In recent years, with the rapid development of deep learning technology, various advanced super-resolution reconstruction methods have emerged. For example, SRCNN (Super-Resolution Convolutional Neural Network) uses a fully convolutional network to learn the mapping between LR and HR images; FSRCNN (Fast Super-Resolution Convolutional Neural Network) is proposed by accelerating the training and testing of SRCNN and extracting features from the original LR input; SRResNet / SRGAN is proposed by using generative adversarial networks for high-rate super-resolution using perceptual loss and adversarial loss; VDSR, DRCN, DRRN, and RDN are proposed by increasing the depth and width of the network, which improves the reconstruction effect. With the increase of the number of network layers, the network complexity and the number of parameters also increase greatly, and the requirements for hardware devices are increased, which limits its use on devices with limited hardware resources. For example, EDSR and RCAN, although they have good reconstruction effects, they have a large number of convolutional layers, a large number of parameters, and a long iteration time, which limits their use. To address the problem of limited memory and computing resources, a lightweight network can be used. For example, DRCN and DRRN reduce the number of network parameters by sharing recursive network parameters. Although the number of parameters has been reduced, the network depth is still deep, and the reconstruction speed has not been greatly improved. RFDN is used to improve channel splitting and feature distillation; SwinIR is used. This network is further improved based on Transformer, and Swin Transformer is proposed for processing, which has achieved excellent results.

[0005] However, the existing methods still have the following problems: (1) Most existing networks extract and calculate nonlinear features, ignoring linear features, resulting in over-sharpening of images and poor reconstruction of tissue structures;

[0006] (2) The existing convolutional neural network super-resolution reconstruction technology has the problems of large number of parameters and high computational complexity, which limits its use on devices with limited hardware resources;

[0007] (3) MRI images are single-channel images, and tissues have regions with similar structures and information. The degradation of MRI training samples is also more likely to lead to unstable model training.

[0008] (4) Common loss functions used in network training, such as MSE and L1 norm, mainly compare the difference between the reconstructed image and the target image, but do not evaluate the amount of high-frequency feature injection. High-frequency feature injection will tend to saturation when it reaches a certain level, making it difficult to obtain higher-quality image details.

[0009] (5) Since magnetic resonance images lack color information, detailed tissue texture, and relatively little image information, using existing networks for reconstruction will make problems such as unclear structure, blurred images, and poor detail reconstruction more prominent. Therefore, how to effectively reduce network complexity while achieving high-quality reconstruction remains a difficult problem. Summary of the invention

[0010] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a method, device and terminal for super-resolution reconstruction of magnetic resonance images, so as to solve at least one problem in the above-mentioned prior art.

[0011] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a magnetic resonance image super-resolution reconstruction method, which obtains a low-resolution magnetic resonance image to be reconstructed; inputs the low-resolution magnetic resonance image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct a high-resolution magnetic resonance image corresponding to the low-resolution magnetic resonance image to be reconstructed; wherein the magnetic resonance image super-resolution reconstruction model includes: a shallow feature extraction module, a deep feature extraction module that introduces a high- and low-frequency feature fusion unit and a high-frequency feature guiding unit, a residual and feature fusion module, and a reconstruction module.

[0012] In some embodiments of the first aspect of the present application, the training method of the magnetic resonance image super-resolution reconstruction model includes: acquiring multiple types of low-resolution and high-resolution magnetic resonance image pairs to establish a magnetic resonance image data set; preprocessing the magnetic resonance image data set to obtain a model data training set; wherein the preprocessing process includes: rotating, translating and randomly cropping the magnetic resonance image data set; inputting the model training data set into an untrained magnetic resonance image super-resolution reconstruction model for model training and evaluation; the magnetic resonance image super-resolution reconstruction model that passes the evaluation is used to perform high-resolution image reconstruction on the low-resolution magnetic resonance image to be reconstructed.

[0013] In some embodiments of the first aspect of the present application, the shallow feature extraction module is used to extract shallow features from the low-resolution magnetic resonance image.

[0014] In some embodiments of the first aspect of the present application, the deep feature extraction module specifically includes: a plurality of high- and low-frequency feature fusion units connected in sequence; wherein the extracted shallow features are used as input of the first high- and low-frequency feature fusion unit; each high- and low-frequency feature fusion unit is used to extract, fuse and enhance high- and low-frequency features based on its input; and a high-frequency feature guiding unit is used to segment the extracted shallow features to obtain the high-frequency features of the guiding unit.

[0015] In some embodiments of the first aspect of the present application, each high- and low-frequency feature fusion unit specifically includes: a plurality of high- and low-frequency feature sub-units based on depthwise separable convolution connected in sequence; wherein each high- and low-frequency feature sub-unit based on depthwise separable convolution is used to extract and fuse high and low-frequency features on its input in sequence; each high- and low-frequency feature sub-unit based on depthwise separable convolution includes: a nonlinear part and a linear part; a frequency enhancement sub-unit, used to perform frequency enhancement processing on the output of the last high- and low-frequency feature sub-unit based on depthwise separable convolution; wherein the frequency enhancement sub-unit adopts an inverted bottleneck structure; a channel attention sub-unit, used to process the enhanced features output by the frequency enhancement sub-unit; wherein the specific structure of the channel attention sub-unit includes: a global pooling layer, a first convolution layer, a ReLU layer and a second convolution layer connected in sequence.

[0016] In some embodiments of the first aspect of the present application, the processing process of the high-frequency feature guiding unit includes: performing convolution downsampling and bilinear interpolation upsampling on the shallow features in turn to obtain upsampled image features; through the self-attention mechanism, the high-frequency features obtained by subtracting the shallow features from the upsampled image features are enhanced to obtain the high-frequency features of the guiding unit.

[0017] In some embodiments of the first aspect of the present application, the processing of the residual and feature fusion module includes: splicing the output features of the multiple high- and low-frequency feature fusion units and performing a first fusion to obtain a first fusion result; performing a second fusion of the first fusion result, the high-frequency features of the guiding unit extracted by the high-frequency feature guiding unit, and the shallow features, and using a deep separable convolution to enhance the result obtained after the second fusion to obtain a fusion feature of a low-resolution magnetic resonance image.

[0018] In some embodiments of the first aspect of the present application, the processing process of the reconstruction module includes: performing residual connection between the fusion features processed by the upsampling module and the low-resolution magnetic resonance image processed by interpolation upsampling to reconstruct a high-resolution magnetic resonance image.

[0019] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a magnetic resonance image super-resolution reconstruction device, comprising: an acquisition module, used to acquire a low-resolution magnetic resonance image to be reconstructed; a reconstruction module, connected to the acquisition module, used to input the low-resolution image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct a high-resolution image corresponding to the low-resolution magnetic resonance image to be reconstructed; wherein the magnetic resonance image super-resolution reconstruction model comprises: a shallow feature guidance module, a deep feature extraction module, a residual and feature fusion module and a reconstruction module.

[0020] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides an electronic terminal, including a memory, a processor and a computer program stored in the memory; the processor executes the computer program to implement the method.

[0021] As described above, the magnetic resonance image super-resolution reconstruction method, device and terminal of the present application have the following beneficial effects:

[0022] (1) Users only need to scan low-resolution magnetic resonance images to obtain high-resolution images through the reconstruction model, which reduces scanning time and improves scanning efficiency;

[0023] (2) It effectively reduces the number of model parameters, making the network more lightweight and more user-friendly for devices with limited hardware resources;

[0024] (3) The reconstructed model can make better use of high- and low-frequency features to obtain high-resolution images with clear details and realistic tissue structure features. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Shown is a flow chart of a magnetic resonance image super-resolution reconstruction method according to an embodiment of the present application.

[0026] Figure 2 Shown is a schematic block diagram of a magnetic resonance image super-resolution reconstruction model in one embodiment of the present application.

[0027] Figure 3 Shown is a schematic block diagram of a high- and low-frequency feature fusion unit in one embodiment of the present application.

[0028] Figure 4 Shown is a schematic block diagram of nine high- and low-frequency feature sub-units based on depthwise separable convolution in one embodiment of the present application.

[0029] Figure 5 Shown is a schematic block diagram of a high and low frequency feature subunit based on standard convolution in one embodiment of the present application.

[0030] Figure 6Shown is a schematic block diagram of a frequency enhancement subunit in an embodiment of the present application.

[0031] Figure 7 Shown is a schematic block diagram of a high-frequency feature guiding unit in one embodiment of the present application.

[0032] Figure 8 Shown is a schematic block diagram of a magnetic resonance image super-resolution reconstruction device in one embodiment of the present application.

[0033] Fig. 9 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION

[0034] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0035] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference.

[0036] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0037] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0038] To facilitate understanding of the embodiments of the present application, first Figure 1 Detailed description. Figure 1 The following is a flow chart of a method for super-resolution reconstruction of magnetic resonance images in an embodiment of the present invention. The method for super-resolution reconstruction of magnetic resonance images in this embodiment mainly includes the following steps:

[0039] Step S11: Acquire a low-resolution magnetic resonance image to be reconstructed.

[0040] It should be understood that low-resolution magnetic resonance images are generally images with a small number of pixels and less detailed information. Low-resolution magnetic resonance images are generally blurry and lack details.

[0041] In one embodiment, the acquired low-resolution magnetic resonance images may be magnetic resonance images of various parts, including but not limited to magnetic resonance images of the head, magnetic resonance images of the abdomen, magnetic resonance images of the spine, etc.

[0042] In one embodiment, the acquired low-resolution magnetic resonance images may be magnetic resonance images under various sequences, including but not limited to T1 images, T2 images, PD images, TOF images, and the like.

[0043] In one embodiment, the acquired low-resolution magnetic resonance image may be a 2D image or a 3D image. It should be noted that the 3D image needs to be cropped according to the direction of the layer to generate a 2D image and then reconstructed.

[0044] In one embodiment, the acquired low-resolution magnetic resonance image may be a normal magnetic resonance image without disease or a magnetic resonance image with lesions.

[0045] In one embodiment, the acquired low-resolution magnetic resonance images may be magnetic resonance images of people of any age group; or may be magnetic resonance images of people of any gender.

[0046] In one embodiment, the low-resolution magnetic resonance image can be obtained by scanning with a magnetic resonance image scanning device, or by an acceleration algorithm, such as CS (Compressed Sensing), SENSE (Sensitivity Encoding), and GRAPPA (Generalized Auto calibrating Partial Parallel Acquisition).

[0047] Step S12: inputting the low-resolution magnetic resonance image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct a high-resolution magnetic resonance image corresponding to the low-resolution magnetic resonance image to be reconstructed.

[0048] It should be understood that high-resolution magnetic resonance images are generally images with a large number of pixels and rich detail information. High-resolution magnetic resonance images are clear, rich in detail, and can capture subtle changes in texture and lines.

[0049] In one embodiment, the magnetic resonance image super-resolution reconstruction model includes: a shallow feature extraction module, a deep feature extraction module that introduces a high- and low-frequency feature fusion unit and a high-frequency feature guidance unit, a residual and feature fusion module, and a reconstruction module.

[0050] In one embodiment, the training method of the magnetic resonance image super-resolution reconstruction model includes: acquiring multiple types of low-resolution and high-resolution magnetic resonance image pairs to establish a magnetic resonance image data set; preprocessing the magnetic resonance image data set to obtain a model data training set; wherein the preprocessing process includes: rotating, translating and randomly cropping the input magnetic resonance image pairs; inputting the model training data set into an untrained magnetic resonance image super-resolution reconstruction model for model training and evaluation; the magnetic resonance image super-resolution reconstruction model that passes the evaluation is used to perform high-resolution image reconstruction on the low-resolution magnetic resonance image to be reconstructed.

[0051] In one embodiment, the magnetic resonance image data set includes magnetic resonance images of various parts, and the types of parts include but are not limited to the head, abdomen, spine, etc.

[0052] In one embodiment, the magnetic resonance image data set includes magnetic resonance images under various magnetic resonance sequences, and the types of magnetic resonance sequences include but are not limited to T1, T2, PD and TOF.

[0053] In one embodiment, the magnetic resonance images in the magnetic resonance image data set are all 2D images. If the acquired magnetic resonance image pair is a 3D image, the 3D image needs to be cropped according to the direction of the layer to generate a 2D image.

[0054] In one embodiment, the magnetic resonance image data set includes normal magnetic resonance images without disease and magnetic resonance images with lesions.

[0055] In one embodiment, the magnetic resonance images in the magnetic resonance image dataset cover people of all age groups, including adults, children, and the elderly; the magnetic resonance images in the magnetic resonance image dataset also cover males and females, and the ratio of the two is close.

[0056] In one embodiment, multiple types of low-resolution and high-resolution magnetic resonance image pairs may be acquired in any of the following three ways:

[0057] The first method: low-resolution magnetic resonance image acquisition and high-resolution magnetic resonance image acquisition are performed on the same part of the same patient in sequence. Except for the resolution-related parameters and acceleration-related parameters, the other parameters of the two acquisitions are consistent, such as T1 images. The image acquisition time should not be too long to prevent the imaging part from being displaced and the high-resolution image from having serious artifacts due to the difficulty of the acquisition object to maintain a uniform posture for a long time. It should be noted that in the first method, a magnetic resonance imaging scanner can be used for scanning, and the types of magnetic resonance imaging scanners include but are not limited to superconducting magnetic resonance imaging systems, permanent magnet magnetic resonance imaging systems, etc.

[0058] The second method: Use acceleration algorithms, such as CS (Compressed Sensing), SENSE (Sensitivity Encoding), and GRAPPA (Generalized Auto calibrating Partial Parallel Acquisition) to acquire low-resolution MRI images of patients. And use the same acquisition method as the first method to acquire high-resolution MRI images of the same part of the same patient. It should be noted that the reason why the acceleration algorithm is not used in the acquisition of high-resolution MRI images is to ensure that the high-resolution MRI images in the model training data set have high imaging quality.

[0059] The third method: If only high-resolution MRI images are available, the high-resolution images can be downsampled, such as by using the nearest interpolation method or the linear interpolation method, to obtain the corresponding low-resolution MRI images. The corresponding low-resolution MRI images can also be obtained by using methods such as K-space truncation and deletion of high-frequency information.

[0060] It should be noted that the acquired low-resolution magnetic resonance images may contain appropriate noise, such as Gaussian noise, and may also contain appropriate artifacts, such as Gibbs artifacts, motion artifacts, etc. The impact of noise and artifacts on low-resolution magnetic resonance images should not be too severe.

[0061] It should also be noted that when the acceleration algorithm is used to acquire low-resolution magnetic resonance images, the K space is not fully acquired.

[0062] In a specific embodiment, the preprocessing process includes: rotating and translating each magnetic resonance image pair in the magnetic resonance image data set to obtain an initial model training data set; performing a random cropping operation on the magnetic resonance image pairs in the initial model training data set to obtain a final model training data set; wherein the random cropping operation includes: randomly selecting magnetic resonance image pairs and generating an image origin for cropping to obtain smaller training data, thereby reducing training time.

[0063] It should be understood that the random cropping operation must achieve a random effect, that is, cover all parts of the image, and perform random cropping operations only when the data set has a large amount of data to ensure data integrity. When the data set has a small amount of data, no cropping is required. Since the model in the embodiment of the present invention is mainly used for 2x, 3x, and 4x reconstruction, the image reconstruction multiple refers to the multiple of the low-resolution (LR) image to the high-resolution (HR) image during the image super-resolution reconstruction process. This multiple determines the size and resolution of the reconstructed image and is an important parameter in image super-resolution reconstruction. Therefore, the image data obtained after cropping is generally a common multiple of 2, 3, and 4, such as a 96×96 image.

[0064] In one embodiment, the magnetic resonance image super-resolution model is first evaluated using a Loss function. The Loss function is used to calculate the difference between the high-resolution magnetic resonance image reconstructed by the model and the real high-resolution magnetic resonance image. Loss functions include but are not limited to MSE, L1 norm, etc. When the MSE and L1 norm curves tend to be stable, that is, the numerical changes are very small, the frequency similarity is used to evaluate the magnetic resonance image super-resolution model to improve the reconstruction effect of the magnetic resonance image super-resolution model and obtain higher quality image details. Frequency similarity can be measured using frequency spectrum distribution similarity (FSDS). The frequency spectrum distribution similarity (FSDS) indicator integrates the spectrum power distribution diagrams of two images. The difference is then calculated to generate an error map, and the sum of its absolute values ​​is calculated.

[0065] It should be understood that when the frequency similarity meets the use requirements of the magnetic resonance image super-resolution model, the evaluation passes.

[0066] In one embodiment, if Figure 2 As shown, the shallow feature extraction module 1 is used to extract shallow features from the low-resolution magnetic resonance image.

[0067] Specifically, this process is expressed by the following formula 1:

[0068] x f =F f (x); (Formula 1)

[0069] Among them, x f represents shallow features, x represents low-resolution magnetic resonance images, and F f (·) indicates a shallow feature extraction module.

[0070] In a specific embodiment, the shallow feature extraction module uses 9×9 convolution (large kernel convolution) or 3×3 convolution and 5×5 depthwise separable convolution connected in sequence.

[0071] Specifically, the low-resolution magnetic resonance image is input into a 9×9 convolution to obtain shallow features; or, the low-resolution magnetic resonance image is processed by 3×3 convolution and 5×5 depthwise separable convolution in sequence to obtain shallow features.

[0072] In one embodiment, if Figure 2 As shown, the shallow features extracted by the shallow feature guidance module 1 will be input into the deep feature extraction module 2 for processing, and then the residual and feature fusion module 3 will fuse the output of the deep feature extraction module to obtain the fused features.

[0073] In one embodiment, if Figure 2 As shown, the deep feature extraction module includes:

[0074] Multiple high- and low-frequency feature fusion units connected in sequence; wherein the shallow features extracted by the shallow feature extraction module 1 are used as the input of the first high- and low-frequency feature fusion unit; each high- and low-frequency feature fusion unit is used to extract, fuse and enhance the high- and low-frequency features based on its input;

[0075] The high-frequency feature guiding unit is used to segment the shallow features extracted by the shallow feature extraction module to obtain the guiding unit high-frequency features.

[0076] Specifically, it is assumed that the deep feature extraction module 1 includes n high- and low-frequency feature fusion units. It should be noted that the present invention does not limit the number of high- and low-frequency feature fusion units, and those skilled in the art can set them according to actual needs. f As the input x0 of the first high- and low-frequency feature fusion unit, that is, x0 = x f The processing of the i-th high- and low-frequency feature fusion unit is expressed as The output of the i-th high- and low-frequency feature fusion unit is used as the input of the i+1-th high- and low-frequency feature fusion unit. The process is shown in the following formula 2:

[0077]

[0078] Among them, x i Represented as the output of the i-th high- and low-frequency feature fusion unit, x i-1 Represented as the input of the i-th high- and low-frequency feature fusion unit (x i-1 It also represents the output of the i-1th high- and low-frequency feature fusion unit).

[0079] Furthermore, the output of the last high- and low-frequency feature fusion unit (the nth high- and low-frequency feature fusion unit) is obtained using the following formula 3:

[0080]

[0081] In one embodiment, if Figure 3 As shown, each high- and low-frequency feature fusion unit specifically includes:

[0082] A plurality of high- and low-frequency feature subunits based on depthwise separable convolutions are sequentially connected; wherein each high- and low-frequency feature subunit based on depthwise separable convolutions is used to sequentially extract and fuse high- and low-frequency features of its input; each high- and low-frequency feature subunit based on depthwise separable convolutions includes: a nonlinear part and a linear part;

[0083] A frequency enhancement subunit, used for performing frequency enhancement processing on the output of the last high- and low-frequency feature subunit based on depthwise separable convolution; wherein the frequency enhancement subunit adopts an inverted bottleneck structure;

[0084] The channel attention subunit is used to process the enhanced features output by the frequency enhancement subunit; wherein the specific structure of the channel attention subunit includes: a global pooling layer, a first convolutional layer, a ReLU layer, and a second convolutional layer connected in sequence. It should be noted that the specific structure of the channel attention subunit can refer to the channel attention mechanism in the RCAN network. The first convolutional layer and the second convolutional layer are generally 1×1 convolutions, which will not be repeated here.

[0085] Specifically, for the first high- and low-frequency feature fusion unit, the input of the first high- and low-frequency feature subunit based on depthwise separable convolution in the first high- and low-frequency feature fusion unit is the shallow feature; for the i-th high- and low-frequency feature fusion unit. The output of the i-1-th high- and low-frequency feature fusion unit is used as the input of the first high- and low-frequency feature subunit based on depthwise separable convolution. Finally, the output of the first high- and low-frequency feature subunit based on depthwise separable convolution is used as the input of the frequency enhancement subunit. The output of the frequency enhancement subunit is used as the input of the channel attention subunit.

[0086] Furthermore, each high- and low-frequency feature subunit based on depthwise separable convolution includes a nonlinear part and a linear part. The nonlinear part is used for calculating high-frequency features. The nonlinear part adopts one or more of standard convolution and depthwise separable convolution. The nonlinear part also adopts a nonlinear activation function. The linear part is used for calculating low-frequency features. The linear part adopts one or more of standard convolution and depthwise separable convolution. The output of the nonlinear part and the output of the linear part are concatenated and fused to generate new output features. Among them, feature concatenation can be performed by adding tensors element by element. The new tensor increases the amount of information of the image features, while the dimension describing the image does not increase. Only the amount of information in each dimension is increased, which is more beneficial to the description of the final features. Figure 4 As shown in Figure 1, the high- and low-frequency feature subunits based on depthwise separable convolution can adopt any of the following nine structures:

[0087] The first structure (C3-D5): The nonlinear part is a 3×3 convolution and ReLU module connected in sequence; the linear part uses a 5×5 depthwise separable convolution; the high-frequency features output by the ReLU module and the low-frequency features output by the 5×5 depthwise separable convolution are superimposed and input into the 3×3 convolution for processing, obtaining the output of the high- and low-frequency feature subunits based on the depthwise separable convolution;

[0088] The second structure (2D3-2C3): The nonlinear part is a 3×3 depth-wise separable convolution, a ReLU module, and a 3×3 depth-wise separable convolution connected in sequence; the linear part is two 3×3 convolutions connected in sequence; the high-frequency features of the last 3×3 depth-wise separable convolution output in the nonlinear part and the low-frequency features of the last 3×3 convolution output in the linear part are superimposed to obtain the output of the high- and low-frequency feature sub-units based on the depth-wise separable convolution;

[0089] The third structure (2C3-2D3): The nonlinear part is a 3×3 convolution, a ReLU module, and a 3×3 convolution connected in sequence; the linear part is two 3×3 depthwise separable convolutions connected in sequence; the high-frequency features of the last 3×3 depthwise separable convolution output in the nonlinear part and the low-frequency features of the last 3×3 depthwise separable convolution output in the linear part are superimposed to obtain the output of the high- and low-frequency feature sub-units based on the depthwise separable convolution;

[0090] The fourth structure (2D3-2D3): The nonlinear part is a 3×3 depth-wise separable convolution, a ReLU module, and a 3×3 depth-wise separable convolution connected in sequence; the linear part is two 3×3 depth-wise separable convolutions connected in sequence; the high-frequency features of the last 3×3 depth-wise separable convolution output in the nonlinear part and the low-frequency features of the last 3×3 depth-wise separable convolution output in the linear part are superimposed to obtain the output of the high- and low-frequency feature sub-units based on the depth-wise separable convolution;

[0091] The fifth structure (2C3-2D5): The nonlinear part is a 3×3 convolution, a ReLU module, and a 3×3 convolution connected in sequence; the linear part is two 5×5 depthwise separable convolutions connected in sequence; the high-frequency features of the last 3×3 depthwise separable convolution output in the nonlinear part and the low-frequency features of the last 5×5 depthwise separable convolution output in the linear part are superimposed to obtain the output of the high- and low-frequency feature sub-units based on the depthwise separable convolution;

[0092] The sixth structure (2C3R-2D5): The processing process of the nonlinear part includes: the features obtained by processing the input of the high- and low-frequency feature subunits based on the depth-separable convolution through a 3×3 convolution are input into the ReLU module together with the input of the high- and low-frequency feature subunits based on the depth-separable convolution, and then a 3×3 convolution is used to process the output of the ReLU module to obtain high-frequency features; the linear part is two 5×5 depth-separable convolutions connected in sequence; the high-frequency features of the last 3×3 convolution output in the nonlinear part and the low-frequency features of the last 5×5 depth-separable convolution output in the linear part are superimposed to obtain the output of the high- and low-frequency feature subunits based on the depth-separable convolution;

[0093] The seventh structure (2C3-2D5R): The nonlinear part is a 3×3 convolution, a ReLU module and a 3×3 convolution connected in sequence; the processing process of the linear part includes: the input of the high- and low-frequency feature subunit structure based on the depthwise separable convolution is processed by a 5×5 depthwise separable convolution to obtain the features, and the input of the high- and low-frequency feature subunit based on the depthwise separable convolution is input into a 5×5 depthwise separable convolution together; the high-frequency features of the last 3×3 convolution output in the nonlinear part and the low-frequency features of the last 5×5 depthwise separable convolution output in the linear part are superimposed to obtain the output of the high- and low-frequency feature subunit based on the depthwise separable convolution;

[0094] The eighth structure (2C3R-2D5R): The processing process of the nonlinear part includes: the features obtained by processing the input of the high and low frequency feature subunits based on the depthwise separable convolution through a 3×3 convolution are input into the ReLU module together with the input of the high and low frequency feature subunits based on the depthwise separable convolution, and then a 3×3 convolution is used to process the output of the ReLU module to obtain high frequency features; The processing process of the linear part includes: the features obtained by processing the input of the high and low frequency feature subunits based on the depthwise separable convolution through a 5×5 depthwise separable convolution are input into a 5×5 depthwise separable convolution together with the input of the high and low frequency feature subunits based on the depthwise separable convolution; The high frequency features of the last 3×3 convolution output in the nonlinear part and the low frequency features of the last 5×5 depthwise separable convolution output in the linear part are superimposed to obtain the output of the high and low frequency feature subunits based on the depthwise separable convolution;

[0095] The ninth structure (NDC3R-2D5): The nonlinear part includes: N convolutional ReLU structures; each convolutional ReLU structure is composed of a 3×3 convolution and a ReLU module connected in sequence; the N convolutional ReLU structures are densely connected.

[0096] The linear part is two 5×5 depthwise separable convolutions connected in sequence;

[0097] The output of the linear part is superimposed with the output of the nonlinear part to obtain the output of the high and low frequency feature sub-units based on depthwise separable convolution.

[0098] It should be noted that the ninth structure is densely connected (the method of dense connection can refer to DenseNet (densely connected network)), and the ninth structure has a larger amount of calculation than other structures. The number N of convolutional ReLU structures in the ninth structure can be set according to training requirements, and the present invention does not limit this.

[0099] It should also be noted that the use of depthwise separable convolution in the above structures reduces the number of network parameters and the computational complexity; the residual structure in the sixth structure, the seventh structure, and the eighth structure can improve the problems of gradient vanishing, overfitting, and high consumption of computing resources that occur as the number of network layers increases. In one embodiment, the depthwise separable convolution (DSC) consists of two parts: depthwise convolution and pointwise convolution. It reduces the number of parameters required for convolution calculations and improves the efficiency of convolution kernel parameter usage by splitting the correlation between spatial dimensions and channel dimensions.

[0100] Specifically, the depthwise separable convolution first uses depthwise convolution to convolve each channel in the input feature with the corresponding single-channel convolution kernel, and the convolved feature map is integrated through pointwise convolution to obtain the output feature map. The size of the convolution kernel is k×k, and the number of input channels is C. in , the number of output channels is C out , the size of the output feature map is F×F, the standard convolution P con , deep convolution P d , point-wise convolution P p , Depthwise Separable Convolution P DSCon The parameter quantities are calculated using the following formulas 4 to 7:

[0101] P con = k × k × C in ×C out ; (Formula 4)P d = k × k × C in ; (Formula 5)

[0102] P p =C in ×C out ; (Formula 6)

[0103] P DSCon =P d +P p = k × k × C in +C in ×C out ; (Formula 7)

[0104] The ratio γ of the number of parameters of the depthwise separable convolution and the standard convolution is calculated using the following formula 8:

[0105]

[0106] As can be seen from Formula 8, depthwise separable convolution can effectively reduce the number of parameters in the calculation process, making the network more lightweight.

[0107] In a specific embodiment, the magnetic resonance image super-resolution reconstruction model using the first structure, the second structure, the third structure, the fourth structure and the fifth structure is used to reconstruct the image. The magnetic resonance image super-resolution reconstruction model based on the high and low frequency feature subunit (2C3-2C3) of the standard convolution is also used to reconstruct the image. It should be noted that the specific structure of the high and low frequency feature subunit (2C3-2C3) based on the standard convolution is as follows: Figure 5 The figure includes: a nonlinear part and a linear part; the nonlinear part is a 3×3 convolution, a ReLU module and a 3×3 convolution connected in sequence; the linear part is two 3×3 convolutions connected in sequence; the high-frequency features of the last 3×3 convolution output in the nonlinear part and the low-frequency features of the last 3×3 convolution output in the linear part are superimposed to obtain the output of the high and low frequency feature sub-units based on the depth-separable convolution.

[0108] The deep feature extraction module of each magnetic resonance image super-resolution reconstruction model uses four high- and low-frequency feature fusion units, and each high- and low-frequency feature fusion unit uses one high- and low-frequency feature subunit based on deep separable convolution. The shallow feature extraction module and reconstruction module are the same. Peak Signal to Noise Ratio (PSNR) and Structural Similarity (SSIM) are used as evaluation indicators. The simulation results of 4x super-resolution reconstruction of proton density (PD) images are shown in Table 1 below:

[0109] Table 1. 4x reconstruction results of PD images based on different convolution combinations of high and low frequency feature sub-units with deep separable convolution

[0110] C3-D5 2C3-2C3 2D3-2C3 2C3-2D3 2D3-2D3 2C3-2D5 Parameter quantity 685319 957447 700423 700423 443399 708615 PSNR 30.143 30.167 29.626 29.968 29.535 30.229 SSIM 0.918 0.919 0.909 0.916 0.909 0.92

[0111] As can be seen from Table 1, by using depthwise separable convolution to replace the standard convolution in the linear part, and increasing the convolution kernel from 3 to 5, the PSNR and SSIM are higher than the standard convolution effect, and the number of parameters is reduced by about 200,000. While achieving good reconstruction effect, the network is more lightweight.

[0112] It should be noted that the magnetic resonance image super-resolution reconstruction model can also adopt high and low frequency feature sub-units (2C3-2C3) based on standard convolution, and those skilled in the art can choose according to reconstruction requirements.

[0113] In one embodiment, if Figure 6As shown in the figure, the frequency enhancement subunit adopts an inverted bottleneck structure, including: a 1×1 standard convolution, a 7×7 depth-separable convolution, and a 1×1 standard convolution connected in sequence; the input of the first 1×1 standard convolution is the input of the frequency enhancement subunit, and the output of the last 1×1 standard convolution is nonlinearly activated by the Silu (Sigmoid Linear Unit) activation function that combines the characteristics of the Sigmoid function and the ReLU function to obtain enhanced features.

[0114] Specifically, the processing process of the frequency enhancement subunit is referred to the following formula 9:

[0115] x E =Silu(C 1×1 (D 7×7 (C 1×1 (x H )))); (Formula 9)

[0116] Among them, x H is the input of the frequency enhancement subunit (i.e., the output of the last high- and low-frequency feature subunit based on depthwise separable convolution connected to the frequency enhancement subunit), C 1×1 (·) is a 1×1 standard convolution, D 7×7 (·) is a 7×7 depthwise separable convolution, x E To enhance the features, Silu(·) is the Silu activation function.

[0117] It should be noted that the frequency enhancement subunit uses a depthwise separable convolution, which reduces the amount of computation, and the depthwise separable convolution uses a 7×7 to increase the receptive field, so a large receptive field can be used to capture wider contextual information in the image, which can better assemble the details of high-resolution information. The frequency enhancement subunit increases the nonlinear expressiveness of the network while allowing the network to maintain the feature dimension.

[0118] In one embodiment, the processing process of the high-frequency feature guiding unit includes: performing convolution downsampling and bilinear interpolation upsampling on shallow features in turn to obtain upsampled image features; through a self-attention mechanism, enhancing the high-frequency features obtained by subtracting the shallow features from the upsampled image features to obtain the high-frequency features of the guiding unit.

[0119] Specifically, Figure 7 As shown, the high-frequency feature guiding unit includes: a HFS subunit and a SA subunit connected to the HFS subunit;

[0120] The HFS subunit is used to perform convolution downsampling and bilinear interpolation upsampling on the shallow features in sequence to obtain upsampled image features; the shallow features are subtracted from the upsampled image features to obtain high-frequency features;

[0121] The SA subunit is used to enhance the high-frequency features output by the HFS subunit through the self-attention mechanism to obtain the final high-frequency features; the SA subunit adopts the Self-Attention mechanism in the Transformer model. It should be understood that the Transformer model is a deep learning model based on the self-attention mechanism. The Self-Attention Mechanism is one of its core components, which allows the model to capture the dependencies within the sequence when processing sequence data without relying on the absolute position of the elements in the sequence.

[0122] Furthermore, the processing process of the HFS subunit is referred to the following formula 10:

[0123] x HC =FH F (x0)=x0-Bilinear(AvgPool(x0)); (Formula 10)

[0124] Among them, x HC is the high-frequency feature output by the HFS subunit, x0 is the shallow feature, AvgPool(·) is the convolution downsampling, and Bilinear(·) is the bilinear interpolation upsampling.

[0125] It should be noted that the high-frequency features extracted by the high-frequency feature guidance unit are combined with the features extracted by the high- and low-frequency feature fusion unit as feature guidance, thereby enriching the texture information of the image.

[0126] In one embodiment, if Figure 2 As shown, the processing of the residual and feature fusion module includes: splicing the output features of the multiple high- and low-frequency feature fusion units and performing the first fusion to obtain the first fusion result; performing the second fusion of the first fusion result, the high-frequency features of the guide unit extracted by the high-frequency feature guide unit, and the shallow features, and enhancing the result obtained after the second fusion by using the depth-separable convolution to obtain the fusion feature of the low-resolution magnetic resonance image. Specifically, the outputs of multiple high- and low-frequency feature fusion units are spliced ​​with reference to the following formula 11:

[0127] x M =[x1, x2, …, x n ]; (Formula 11)

[0128] Among them, x1, x2, …, x nare the outputs of each high-frequency and low-frequency feature fusion unit, [·] is the Concatenation operation, x M The features after splicing.

[0129] Furthermore, the first fusion is performed by sequentially connecting 1×1 standard convolution and 3×3 standard convolution. Specifically, the concatenated features are sequentially processed by 1×1 standard convolution and 3×3 standard convolution to obtain the first fusion result. The process of the first fusion is referred to the following formula 12:

[0130] x C =F M (x M ); (Formula 12)

[0131] Among them, F M (·) is the first fusion, x C is the result after fusion.

[0132] Furthermore, the first fusion result and the high-frequency feature guide unit extracted by the guide unit are HF The shallow feature x0 is fused for the second time, and the result after the second fusion is enhanced by 3×3 depthwise separable convolution to obtain the fusion feature of the low-resolution magnetic resonance image.

[0133] Among them, the second fusion adopts the fusion method of skip connection;

[0134] Refer to the following formula 13 to obtain the fusion feature x R :

[0135] x R =F D (F M (x M )+x0+x HF )=F D (F M ([x1, x2, …, x n ])+x0+x HF ); (Formula 13)

[0136] Among them, F D Represented as a 3×3 depthwise separable convolution, F M (·) is the first fusion.

[0137] It should be noted that the “+” in Formula 13 represents the fusion method of skip connection, also known as residual connection. Residual connection is a technology in deep learning. It helps solve the problem of gradient disappearance or gradient explosion that occurs as the network depth increases by introducing the output of the previous layer as part of the input of the subsequent layer in a certain layer of the network.

[0138] In one embodiment, if Figure 2 As shown, the processing process of the reconstruction module includes: performing residual connection between the fusion features processed by the upsampling module and the low-resolution magnetic resonance image processed by interpolation upsampling to reconstruct a high-resolution magnetic resonance image. Specifically, the processing process of the reconstruction module refers to the following formula 14:

[0139] x H =F U (x R )+F B (x); (Formula 14)

[0140] Among them, x H is the reconstructed high-resolution magnetic resonance image, x is the low-resolution magnetic resonance image to be reconstructed, and x R is the fusion feature, F U (·) is the upsampling module, F B (·) is interpolation upsampling.

[0141] It should be understood that the upsampling module in this embodiment adopts Pixel-Shuffle upsampling, and Pixel-Shuffle (pixel reorganization / pixel shuffling) is an upsampling method that can effectively amplify the reduced feature map to generate a high-resolution image. In the interpolation upsampling in this embodiment, bicubic interpolation (Bicubic) or other interpolation can be selected, and the present invention does not limit this.

[0142] It should be noted that, as the number of network layers increases, although the reconstruction effect is improved to a certain extent, there will also be problems such as gradient disappearance, overfitting, and large consumption of computing resources. Residual calculation can improve the above problems to a certain extent. For magnetic resonance images, they are single-channel images, and there are areas with similar structures and information in tissues. The problem of degradation of magnetic resonance training samples is also more likely to cause unstable model training. Therefore, the addition of residuals is of great help to the super-resolution task of a single nuclear magnetic resonance image. The present invention adopts multi-layer residuals, and residual calculation is added to each high- and low-frequency feature subunit, residual and feature fusion module, and reconstruction module. Since low-resolution magnetic resonance images are mainly missing high-frequency features relative to high-resolution magnetic resonance images, and low-frequency features mainly determine the structure of the image, low-resolution magnetic resonance images and high-resolution magnetic resonance images with similar low-frequency features share the same information to a large extent, and there is a large part of similar structures. Therefore, it is beneficial to establish the relationship between low-resolution magnetic resonance images and high-resolution magnetic resonance images through residuals.

[0143] It should also be noted that the magnetic resonance image super-resolution reconstruction model in the embodiment of the present invention has a certain filtering function. The input low-resolution magnetic resonance image will have certain Gibbs artifacts, pulsation artifacts, etc., and the high-resolution magnetic resonance image used as a reference or standard has no artifacts and noise or is relatively mild compared to the low-resolution magnetic resonance image. Through the magnetic resonance image super-resolution reconstruction model, the problem of artifacts and noise can be improved to a certain extent.

[0144] Figure 8 is a schematic block diagram of a magnetic resonance image super-resolution reconstruction device provided in an embodiment of the present application. Figure 7 As shown, the magnetic resonance image super-resolution reconstruction device 8 includes:

[0145] An acquisition module 81, used for acquiring a low-resolution magnetic resonance image to be reconstructed;

[0146] A reconstruction module 82, connected to the acquisition module 81, is used to input the low-resolution image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct a high-resolution image corresponding to the low-resolution magnetic resonance image to be reconstructed;

[0147] The magnetic resonance image super-resolution reconstruction model includes: a shallow feature guidance module, a deep feature extraction module, a residual and feature fusion module and a reconstruction module.

[0148] It should be understood that the specific process of each module executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0149] It should also be understood that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0150] In one embodiment, the training method of the magnetic resonance image super-resolution reconstruction model includes: acquiring multiple types of low-resolution and high-resolution magnetic resonance image pairs to establish a magnetic resonance image data set; preprocessing the magnetic resonance image data set to obtain a model data training set; wherein the preprocessing process includes: rotating, translating and randomly cropping the magnetic resonance image data set; inputting the model training data set into an untrained magnetic resonance image super-resolution reconstruction model for model training and evaluation; the magnetic resonance image super-resolution reconstruction model that passes the evaluation is used to perform high-resolution image reconstruction on the low-resolution magnetic resonance image to be reconstructed.

[0151] In one embodiment, the shallow feature extraction module is used to extract shallow features from low-resolution magnetic resonance images.

[0152] In some embodiments of the first aspect of the present application, the deep feature extraction module specifically includes: a plurality of high- and low-frequency feature fusion units connected in sequence; wherein the extracted shallow features are used as input of the first high- and low-frequency feature fusion unit; each high- and low-frequency feature fusion unit is used to extract, fuse and enhance high- and low-frequency features based on its input; and a high-frequency feature guiding unit is used to segment the extracted shallow features to obtain the high-frequency features of the guiding unit.

[0153] In one embodiment, each high- and low-frequency feature fusion unit specifically includes: a plurality of high- and low-frequency feature sub-units based on depthwise separable convolutions connected in sequence; wherein each high- and low-frequency feature sub-unit based on depthwise separable convolutions is used to extract and fuse high and low-frequency features on its input in sequence; each high- and low-frequency feature sub-unit based on depthwise separable convolutions includes: a nonlinear part and a linear part; a frequency enhancement sub-unit, used to perform frequency enhancement processing on the output of the last high- and low-frequency feature sub-unit based on depthwise separable convolutions; wherein the frequency enhancement sub-unit adopts an inverted bottleneck structure; a channel attention sub-unit, used to process the enhanced features output by the frequency enhancement sub-unit; wherein the specific structure of the channel attention sub-unit includes: a global pooling layer, a first convolutional layer, a ReLU layer and a second convolutional layer connected in sequence.

[0154] In one embodiment, the processing process of the high-frequency feature guiding unit includes: performing convolution downsampling and bilinear interpolation upsampling on the shallow features in turn to obtain upsampled image features; through the self-attention mechanism, the high-frequency features obtained by subtracting the shallow features from the upsampled image features are enhanced to obtain the high-frequency features of the guiding unit.

[0155] In one embodiment, the processing of the residual and feature fusion module includes: performing a first fusion after splicing the output features of the multiple high- and low-frequency feature fusion units to obtain a first fusion result; performing a second fusion on the first fusion result, the high-frequency features of the guide unit extracted by the high-frequency feature guide unit, and the shallow features, and using a deep separable convolution to enhance the result obtained after the second fusion to obtain a fusion feature of a low-resolution magnetic resonance image.

[0156] In one embodiment, the processing process of the reconstruction module includes: performing residual connection between the fusion features processed by the upsampling module and the low-resolution magnetic resonance image processed by interpolation upsampling to reconstruct a high-resolution magnetic resonance image.

[0157] Fig. 9 is a schematic block diagram of an electronic terminal provided in an embodiment of the present application. Fig. 9 As shown, the electronic terminal 900 includes: at least one processor 901, a memory 902, at least one network interface 903 and a user interface 905. The various components in the device are coupled together through a bus system 904. It can be understood that the bus system 904 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Fig. 9 In the specification, various buses are labeled as bus systems.

[0158] The user interface 905 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.

[0159] It is understood that the memory 902 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.

[0160] The memory 902 in the embodiment of the present invention is used to store various categories of data to support the operation of the electronic terminal 900. Examples of these data include: any executable program for operating on the electronic terminal 900, such as an operating system 9021 and an application 9022; the operating system 9021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 9022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The method for super-resolution reconstruction of magnetic resonance images provided in the embodiment of the present invention can be included in the application 9022.

[0161] The method disclosed in the above embodiment of the present invention can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 901 or the instruction in the form of software. The above processor 901 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 801 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 901 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0162] In an exemplary embodiment, the electronic terminal 900 may be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned method.

[0163] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which includes: a computer program code, when the computer program code is run on a computer, the computer executes Figure 1 The magnetic resonance image super-resolution reconstruction method in the illustrated embodiment.

[0164] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable storage medium, which stores a program code, and when the program code is run on a computer, the computer executes Figure 1 The magnetic resonance image super-resolution reconstruction method in the illustrated embodiment.

[0165] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0166] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0167] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0168] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0171] In the above embodiments, the functions of each functional unit can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When loading and executing computer program instructions (programs) on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media integrations. Available media may be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., high-density digital video discs (DVDs), or semiconductor media (e.g., solid state disks (SSDs), etc.).

[0172] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0173] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0174] In summary, the present application provides a method, device and terminal for super-resolution reconstruction of magnetic resonance images, the method comprising: inputting the acquired low-resolution magnetic resonance image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct the corresponding high-resolution magnetic resonance image; wherein the magnetic resonance image super-resolution reconstruction model comprises: a shallow feature extraction module, a deep feature extraction module that introduces a high- and low-frequency feature fusion unit and a high-frequency feature guidance unit, a residual and feature fusion module and a reconstruction module. The present application can obtain the corresponding high-resolution magnetic resonance image only by scanning the low-resolution magnetic resonance image, which reduces the scanning time and improves the scanning efficiency; effectively reduces the number of parameters, making the network more lightweight and more friendly to devices with limited hardware resources; the reconstruction model can make better use of high- and low-frequency features, thereby obtaining high-resolution images with clear details and true tissue structure features. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0175] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for super-resolution reconstruction of magnetic resonance images, characterized in that: include: acquiring a low-resolution magnetic resonance image to be reconstructed; Inputting the low-resolution magnetic resonance image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct a high-resolution magnetic resonance image corresponding to the low-resolution magnetic resonance image to be reconstructed; The magnetic resonance image super-resolution reconstruction model includes: a shallow feature extraction module, a deep feature extraction module that introduces a high- and low-frequency feature fusion unit and a high-frequency feature guidance unit, a residual and feature fusion module, and a reconstruction module.

2. The magnetic resonance image super-resolution reconstruction method according to claim 1, characterized in that: The training method of the magnetic resonance image super-resolution reconstruction model includes: Acquire multiple types of low-resolution and high-resolution magnetic resonance image pairs to establish a magnetic resonance image data set; preprocess the magnetic resonance image data set to obtain a model data training set; wherein the preprocessing process includes: rotating, translating, and randomly cropping the magnetic resonance image data set; The model training data set is input into an untrained magnetic resonance image super-resolution reconstruction model for model training and evaluation; the magnetic resonance image super-resolution reconstruction model that passes the evaluation is used to perform high-resolution image reconstruction on the low-resolution magnetic resonance image to be reconstructed.

3. The magnetic resonance image super-resolution reconstruction method according to claim 1, characterized in that: The shallow feature extraction module is used to extract shallow features from low-resolution magnetic resonance images.

4. The magnetic resonance image super-resolution reconstruction method according to claim 3, characterized in that: The deep feature extraction module specifically includes: A plurality of high- and low-frequency feature fusion units connected in sequence; wherein the extracted shallow features are used as inputs of the first high- and low-frequency feature fusion unit; each high- and low-frequency feature fusion unit is used to extract, fuse and enhance high- and low-frequency features based on its input; The high-frequency feature guiding unit is used to segment the extracted shallow features to obtain the guiding unit high-frequency features.

5. The magnetic resonance image super-resolution reconstruction method according to claim 4, characterized in that: Each high- and low-frequency feature fusion unit specifically includes: A plurality of high- and low-frequency feature sub-units based on depthwise separable convolutions are sequentially connected; wherein each high- and low-frequency feature sub-unit based on depthwise separable convolutions is used to sequentially extract and fuse high- and low-frequency features of its input; Each high- and low-frequency feature subunit based on depthwise separable convolution includes: a nonlinear part and a linear part; A frequency enhancement subunit, used for performing frequency enhancement processing on the output of the last high- and low-frequency feature subunit based on depthwise separable convolution; wherein the frequency enhancement subunit adopts an inverted bottleneck structure; The channel attention subunit is used to process the enhanced features output by the frequency enhancement subunit; wherein the specific structure of the channel attention subunit includes: a global pooling layer, a first convolutional layer, a ReLU layer and a second convolutional layer connected in sequence.

6. The magnetic resonance image super-resolution reconstruction method according to claim 4, characterized in that: The processing process of the high-frequency feature guiding unit includes: The shallow features are sequentially downsampled by convolution and upsampled by bilinear interpolation to obtain upsampled image features; Through the self-attention mechanism, the high-frequency features obtained by subtracting the shallow features from the upsampled image features are enhanced to obtain the high-frequency features of the guidance unit.

7. The magnetic resonance image super-resolution reconstruction method according to claim 4, characterized in that: The processing of the residual and feature fusion module includes: The output features of the plurality of high- and low-frequency feature fusion units are spliced ​​and then fused for the first time to obtain a first fusion result; The first fusion result, the high-frequency features of the guide unit extracted by the high-frequency feature guide unit, and the shallow features are fused for the second time, and the result after the second fusion is enhanced by using deep separable convolution to obtain the fusion features of the low-resolution magnetic resonance image.

8. The magnetic resonance image super-resolution reconstruction method according to claim 7, characterized in that: The processing process of the reconstruction module includes: performing residual connection between the fusion features processed by the upsampling module and the low-resolution magnetic resonance image processed by interpolation upsampling, so as to reconstruct a high-resolution magnetic resonance image.

9. A magnetic resonance image super-resolution reconstruction device, characterized in that: include: An acquisition module, used for acquiring a low-resolution magnetic resonance image to be reconstructed; A reconstruction module, connected to the acquisition module, for inputting the low-resolution image to be reconstructed into a trained magnetic resonance image super-resolution reconstruction model to reconstruct a high-resolution image corresponding to the low-resolution magnetic resonance image to be reconstructed; The magnetic resonance image super-resolution reconstruction model includes: a shallow feature guidance module, a deep feature extraction module, a residual and feature fusion module and a reconstruction module.

10. An electronic terminal comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 8.

Images