A magnetic resonance imaging super-resolution image reconstruction method, system, terminal and readable storage medium

Abstract

The present invention relates to the field of image processing technology, and discloses a method, system, terminal, and readable storage medium for super-resolution magnetic resonance imaging image reconstruction. The method comprises: improving a semantic segmentation network and designing an effective semantic prior estimation branch; introducing a multi-category internal semantic prior estimated by the semantic segmentation branch into the super-resolution network; and utilizing a modulation method to fuse features between the super-resolution network and the semantic estimation branch, thereby achieving effective feature fusion for common upstream and downstream tasks in the target image. The present invention utilizes prior information within the target image to give the super-resolution model semantic perception capabilities, utilizes higher-resolution data to learn stable semantic priors, and constructs a bidirectional optimization framework to further improve the model's accuracy and robustness, thereby generating high-quality reconstructed images.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, terminal and computer-readable storage medium for super-resolution image reconstruction of magnetic resonance imaging. Background Art

[0002] Medical images such as Magnetic Resonance Imaging (MRI) and Ultrasound (US) are important sources of medical data. Doctors can use medical images to significantly improve the efficiency of clinical practice.

[0003] However, in the MR (Magnetic Resonance) image super-resolution task, it is extremely challenging to accurately extract high-quality semantic information from a single low-resolution image and effectively apply it to the reconstruction process. Inaccurate semantic information may even introduce artifacts or erroneous tissue details, thereby reducing the reliability of the reconstructed image.

[0004] Therefore, the existing technology still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method, system, terminal and computer-readable storage medium for super-resolution image reconstruction of magnetic resonance images, aiming to solve the problems in the prior art of difficulty in obtaining prior information from low-resolution images and the one-way effect of semantic prior information on super-resolution models, which may lead to misclassification or loss of extracted semantic information.

[0006] To achieve the above object, the present invention provides a method for reconstructing a magnetic resonance image with super-resolution, comprising the following steps:

[0007] Acquire an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features;

[0008] Inputting the target image into a semantic segmentation network for semantic segmentation, and outputting multiple semantic segmentation maps containing shallow features;

[0009] After encoding all the semantic segmentation maps, a feature extraction operation is performed to obtain a plurality of position embedding vectors, and a semantic prior in each of the position embedding vectors is extracted;

[0010] All the shallow features and all the semantic priors are input into a semantic fusion model for feature fusion, and the deep features of the initial image and a reconstructed image are output.

[0011] Optionally, the method for super-resolution image reconstruction of magnetic resonance imaging, wherein the steps of obtaining an initial image, inputting the initial image into a super-resolution network for feature extraction, outputting initial shallow features, and performing feature extraction on the initial shallow features to obtain a target image containing deep features, specifically include:

[0012] Obtaining an initial image input by a user, and inputting the initial image into a super-resolution network in a semantic fusion model;

[0013] The super-resolution network extracts features from the initial image and outputs initial shallow features of the initial image;

[0014] The initial shallow features are input into a semantic fusion model to extract deep features, and a target image containing deep features is output.

[0015] Optionally, the method for super-resolution image reconstruction of magnetic resonance imaging, wherein the inputting of the target image into a semantic segmentation network for semantic segmentation and outputting a plurality of semantic segmentation maps containing shallow features, specifically comprises:

[0016] Inputting the target image into the semantic segmentation network, wherein an encoder in the semantic segmentation network extracts features of the target image according to multiple semantic labels and outputs shallow features of different image regions;

[0017] The decoder in the semantic segmentation network performs feature recovery on all the image regions and outputs high-level semantic information of each image region;

[0018] All the shallow features and all the high-level semantic information in each image region are matched and aligned through an alignment segmentation network, and multiple semantic segmentation maps are output.

[0019] Optionally, the method for super-resolution image reconstruction of magnetic resonance imaging, wherein the encoding of all the semantic segmentation maps is followed by a feature extraction operation to obtain a plurality of position embedding vectors, and the extraction of a semantic prior in each of the position embedding vectors, specifically comprises:

[0020] Performing position encoding on all pixels in each of the semantic segmentation images to obtain a rotation angle of each pixel;

[0021] Calculating a rotation position vector of each pixel according to the rotation angle, and embedding all the rotation position vectors into the corresponding semantic segmentation map to obtain a position embedding vector corresponding to each semantic segmentation map;

[0022] Feature extraction is performed on each of the position embedding vectors to obtain a semantic prior corresponding to each of the semantic segmentation maps.

[0023] Optionally, the method for super-resolution image reconstruction of magnetic resonance imaging, wherein all the shallow features and all the semantic priors are input into a semantic fusion model for feature fusion, and deep features of the initial image and a reconstructed image are output, specifically includes:

[0024] Inputting all the shallow features and all the semantic priors corresponding to each semantic segmentation map into a semantic fusion model;

[0025] When the semantic fusion model performs shallow feature extraction on all the semantic priors, a first parameter and a second parameter are generated;

[0026] The first parameter and the second parameter are weightedly fused with all the shallow features to generate deep features of the initial image and a reconstructed image.

[0027] Optionally, in the magnetic resonance imaging super-resolution image reconstruction method, when the semantic fusion model performs shallow feature extraction on all the semantic priors, generating the first parameter and the second parameter specifically includes:

[0028] Performing shallow feature extraction on all the semantic priors through the convolutional layer of the semantic fusion model, and outputting corresponding semantic shallow features and a first modulation parameter corresponding to each of the semantic shallow features in the feature extraction process;

[0029] Generating a second modulation parameter corresponding to each of the semantic shallow features in the feature extraction process through a GELU activation function;

[0030] performing element-by-element multiplication on all the first modulation parameters to obtain a first parameter;

[0031] Perform element-by-element multiplication on all the second modulation parameters to obtain second parameters.

[0032] Optionally, the magnetic resonance imaging super-resolution image reconstruction method, wherein the weighted fusion processing is performed on all the parameters and all the shallow features to generate the deep features of the initial image and the reconstructed image, further comprises:

[0033] The reconstructed image is used as an iterative target image and input into the semantic segmentation network again for image segmentation processing to obtain multiple iterative semantic segmentation maps;

[0034] Using the deep-level features as iterative shallow-level features, and generating corresponding iterative semantic priors based on all the iterative semantic segmentation maps after segmentation;

[0035] All the iterative shallow features and all the iterative semantic priors are input into the semantic fusion model, and the deep features of the reconstructed image are output until a final reconstructed image that meets the standards is obtained.

[0036] In addition, to achieve the above-mentioned object, the present invention further provides an image reconstruction system for magnetic resonance imaging with super-resolution, wherein the image reconstruction system for magnetic resonance imaging with super-resolution comprises:

[0037] A feature extraction module is used to obtain an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features;

[0038] An image segmentation module is used to input the target image into a semantic segmentation network for semantic segmentation, and output a plurality of semantic segmentation maps containing shallow features;

[0039] A position encoding module is used to perform feature extraction on all the semantic segmentation maps after encoding them to obtain multiple position embedding vectors, and to extract semantic priors from each of the position embedding vectors;

[0040] The feature fusion module is used to input all the shallow features and all the semantic priors into a semantic fusion model for feature fusion, and output the deep features of the initial image and a reconstructed image.

[0041] In addition, to achieve the above-mentioned objectives, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a magnetic resonance imaging super-resolution image reconstruction program stored in the memory and executable on the processor, wherein the magnetic resonance imaging super-resolution image reconstruction program, when executed by the processor, implements the steps of the magnetic resonance imaging super-resolution image reconstruction method described above.

[0042] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image reconstruction program for magnetic resonance imaging super-resolution, and when the image reconstruction program for magnetic resonance imaging super-resolution is executed by a processor, the steps of the image reconstruction method for magnetic resonance imaging super-resolution as described above are implemented.

[0043] In the present invention, an initial image is obtained, the initial image is input into a super-resolution network for feature extraction, initial shallow features are output, and feature extraction is performed on the initial shallow features to obtain a target image containing deep features; the target image is input into a semantic segmentation network for semantic segmentation, and multiple semantic segmentation maps containing shallow features are output; all the semantic segmentation maps are encoded and then subjected to feature extraction operations to obtain multiple position embedding vectors, and the semantic prior in each position embedding vector is extracted; all the shallow features and all the semantic priors are input into a semantic fusion model for feature fusion, and the deep features of the initial image and a reconstructed image are output. The present invention utilizes the prior information within the target image to give the super-resolution model semantic perception capabilities, utilizes higher-resolution data to learn stable semantic priors, and constructs a bidirectional optimization framework to further improve the accuracy and robustness of the model, thereby generating high-quality reconstructed images. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0045] Figure 2 1 is a framework diagram of an overall model of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0046] Figure 3 1. It is a coding framework diagram of a semantic segmentation network of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0047] Figure 4 This is a decoding framework diagram of a semantic segmentation network of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0048] Figure 5 1 is a schematic diagram of a fusion module of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0049] Figure 6 is a first result-error map of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0050] Figure 7 is a second result-error map of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0051] Figure 8 This is a tumor segmentation result diagram of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction method of the present invention;

[0052] Figure 9 1 is a structural diagram of a preferred embodiment of the magnetic resonance imaging super-resolution image reconstruction system of the present invention;

[0053] Figure 10 FIG. 4 is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clear and distinct, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0055] Super-resolution (SR), a technique for restoring a high-resolution image from one or more low-resolution images, is a key research task in computer vision. While deep learning-based super-resolution algorithms have achieved significant breakthroughs in reconstruction accuracy and computational efficiency, there is still room for further improvement in the texture restoration and visual perception quality of the reconstructed images, and several challenges and deficiencies remain to be addressed.

[0056] Furthermore, there are two ways in which existing super-resolution reconstruction techniques are gradually combined with semantic segmentation tasks: (1) the semantic segmentation network first extracts semantic information and then uses the semantic prior to unidirectionally guide the super-resolution model; (2) the semantic segmentation network and the super-resolution network are jointly optimized to achieve shared feature representation between the two tasks through a parallel architecture. However, the above two methods have the problem of difficulty in obtaining priors from low-resolution images and the unidirectional effect of semantic priors on the super-resolution model, resulting in the possibility of misclassification or loss of the extracted semantic information, and the unidirectional effect mechanism can cause semantic information mismatch.

[0057] The method for super-resolution image reconstruction of magnetic resonance imaging according to a preferred embodiment of the present invention is as follows: Figure 1 As shown, the magnetic resonance imaging super-resolution image reconstruction method includes the following steps:

[0058] Step S10: Acquire an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features.

[0059] Therefore, the present invention proposes a collaborative learning model SCSR (SC stands for Semantic Composition; SR stands for Super-Resolution) consisting of an MR image super-resolution network and a semantic prior estimation branch, such as Figure 2 As shown, (STL represents the shifted Transformer layer, represents element-by-element addition), aiming to make full use of the prior information in MR images to enable the super-resolution model to have semantic perception capabilities, use higher-resolution data to learn stable semantic priors, and build a bidirectional optimization framework to further improve accuracy and robustness.

[0060] Specifically, an initial image input by a user is obtained, and the initial image is input into a super-resolution network in a semantic fusion model; the super-resolution network performs feature extraction on the initial image and outputs initial shallow features of the initial image; the initial shallow features are input into a semantic fusion model for deep feature extraction, and a target image containing deep features is output.

[0061] Compared to traditional super-resolution network architectures, the super-resolution reconstruction branch incorporates an additional prior fusion component. This component is divided into four stages: shallow feature extraction, semantic prior fusion, and deep feature extraction and image reconstruction. First, in the shallow feature extraction stage, a convolutional layer is used to extract features from the low-resolution input image. Then, in the semantic prior fusion stage, the shallow features obtained from the input low-resolution MR image in the previous stage are fused with the priors acquired by the semantic estimation branch through a semantic prior fusion module. In the deep feature extraction stage, a series of Residual Swin Transformer Blocks (RSTBs) are used to extract deep features to capture long-range dependencies during feature extraction and enhance the model's ability to globally model MR images. This improves feature utilization and enables the super-resolution model to perceive multi-level semantic information. Finally, in the image reconstruction stage, an upsampling module combined with sub-pixel convolution is used to complete feature extraction and upsampling.

[0062] Among them, a framework based on collaborative learning is used to achieve collaboration between super-resolution and semantic segmentation tasks through a cyclic framework, which solves the one-way effect of MR image super-resolution reconstruction on downstream tasks and provides a paradigm for the effective combination of MR image super-resolution and common downstream tasks such as semantic segmentation.

[0063] However, during the first coordination process, since the semantic segmentation branch has no input data and cannot complete the semantic estimation to obtain the prior, during the first collaborative cycle, the super-resolution reconstruction branch can only perform shallow feature extraction on the low-resolution image (that is, the initial image input by the user), and further extract deep features to achieve the initial reconstruction of the initial image, and obtain the input image for the next round of collaborative optimization, that is, the target image.

[0064] Step S20: Input the target image into a semantic segmentation network for semantic segmentation, and output a plurality of semantic segmentation maps containing shallow features.

[0065] In order to estimate accurate semantic priors from the MR images reconstructed by the super-resolution branch, the semantic segmentation network adopts a two-stage processing strategy.

[0066] Specifically, the target image is input into the semantic segmentation network, the encoder in the semantic segmentation network extracts features of the target image according to multiple semantic labels, and outputs shallow features of different image regions; the decoder in the semantic segmentation network restores features of all the image regions, and outputs high-level semantic information of each image region; all the shallow features and all the high-level semantic information in each image region are matched and aligned through the alignment segmentation network, and multiple semantic segmentation maps are output.

[0067] In the first stage, the reconstructed image obtained by the recurrent super-resolution network is passed through a segmentation network to generate a segmentation map containing multiple different semantic labels. These labels correspond to the semantic structure of different regions in the MR image. The segmentation process is based on the specific display type of the MR image. For example, the IXI (Information e Xtraction from Images) brain dataset, a public dataset focusing on brain magnetic resonance imaging, segments an MR image into four categories: white matter, gray matter, cerebrospinal fluid, and background. For example, the BraTS2020 dataset (Brain Tumor Segmentation Challenge 2020 Dataset) for brain tumors is divided into four categories: tumor core, entire tumor, tumor enhancement region, and background.

[0068] Among them, the encoder part (such as Figure 3 As shown, the downward dark arrow represents the 2×2 maximum pooling operation, and the upward light arrow represents the 2×2 inverted convolution. Represents the channel concatenation operation, ReLU represents the ReLU (Rectified Linear Unit) activation function) The features extracted are usually lower-level features, while the decoder part (such as Figure 4 As shown, Represents element-by-element addition) will try to restore the high-level semantic information of the image. If the features of the encoder and decoder have different semantic levels, it will be difficult for the decoder to integrate this information when restoring the image, which will affect the final segmentation result. In order to obtain more accurate semantic segmentation results, the QuickNat network based on the U-Net structure in the present invention (a neural network model designed for brain MRI image segmentation) is improved to implement a semantic alignment segmentation network DCNet (DilatedContext Network, medical image segmentation scenario). The network additionally introduces a semantic alignment path between the encoder and decoder to reduce the semantic differences of features at different levels, and solve the problem of feature mismatch caused by semantic differences between the encoder and decoder when performing jump connections. The semantic alignment path is implemented based on a residual block composed of dense connections of convolutional layers, batch normalization layers and ReLU activation functions.

[0069] Step S30: After encoding all the semantic segmentation maps, perform a feature extraction operation to obtain multiple position embedding vectors, and extract the semantic prior in each of the position embedding vectors.

[0070] Specifically, position encoding is performed on all pixels in each semantic segmentation map to obtain the rotation angle of each pixel; based on the rotation angle, the rotation position vector of each pixel is calculated, and all the rotation position vectors are embedded into the corresponding semantic segmentation map to obtain the position embedding vector corresponding to each semantic segmentation map; feature extraction is performed on each position embedding vector to obtain the semantic prior corresponding to each semantic segmentation map.

[0071] Among them, after completing the semantic segmentation of the first stage, the output semantic segmentation map is encoded and feature extracted.

[0072] First, Rotary Position Embedding (RoPE) is used to encode the structural position information of the semantic segmentation map to obtain a position embedding vector. For each pixel in the semantic segmentation map, the rotation angle must be calculated. This rotation angle is then used to calculate the two-dimensional rotation position vector. By introducing the rotation position encoding vector, the rotation position information is combined with the original feature map to obtain the rotationally encoded feature map.

[0073] Furthermore, a RSTB module, using multi-layer residual connections and a self-attention mechanism, extracts deep features from the position embedding vector, thereby learning more detailed texture structures, such as brain tissue boundary transition information, gray matter / white matter texture distribution, and edge features of small-scale structures such as sulci and gyri. The position encoding and feature extraction in this second stage further enhances the expressiveness of semantic information and strengthens the model's spatial perception of different semantic regions.

[0074] Step S40: input all the shallow features and all the semantic priors into a semantic fusion model for feature fusion, and output the deep features of the initial image and a reconstructed image.

[0075] Among them, such as Figure 5 As shown in the figure, the present invention designs a semantic prior fusion module based on modulation mode to achieve feature fusion. By performing nonlinear mapping on the semantic prior, a pair of stable and robust modulation parameters are generated. The modulation parameters are then used to modulate the super-resolution network to achieve feature fusion. Wherein, GELU represents Gaussian Error Linear Unit, Indicates channel direction splicing, represents pixel-by-pixel addition, Represents pixel-by-pixel multiplication.

[0076] Specifically, all the shallow features and all the semantic priors corresponding to each semantic segmentation map are input into the semantic fusion model; when the semantic fusion model performs shallow feature extraction on all the semantic priors, a first parameter and a second parameter are generated; the first parameter and the second parameter are combined with all the shallow features (i.e. Figure 5 The reconstructed intermediate features in the image are weightedly fused to generate the deep features of the initial image and the reconstructed image.

[0077] Among them, the semantic prior fusion module has a dual-branch structure, which receives super-resolution shallow features and semantic priors as input. Since the features of the super-resolution network and the semantic estimation branch have large feature differences due to different sources, the modulation parameters generated simply linearly are easily dominated by the strongly activated area, causing the fusion to be biased towards a certain branch. Therefore, the GELU activation function is introduced in the module design to construct a nonlinear mapping function. When performing prior fusion, a convolutional layer is first used to extract shallow features of the semantic prior, and then two modulation parameters are generated respectively through the GELU activation function and the convolutional layer. Finally, the calculated modulation parameters are used to perform weighted fusion with the intermediate layer features of the super-resolution network in an element-by-element multiplication manner to achieve dynamic adjustment of features and fusion of semantic information.

[0078] Among them, shallow feature extraction is performed on all the semantic priors through the convolutional layer of the semantic fusion model, and the corresponding semantic shallow features and the first modulation parameters corresponding to each of the semantic shallow features in the feature extraction process are output; the second modulation parameters corresponding to each of the semantic shallow features are generated in the feature extraction process through the GELU activation function; all the first modulation parameters are element-wise multiplied to obtain the first parameter; all the second modulation parameters are element-wise multiplied to obtain the second parameter.

[0079] Among them, when performing prior fusion, a convolutional layer is first used to perform shallow feature extraction on the semantic prior, and then two modulation parameters are generated through the GELU activation function and the convolutional layer respectively. One is used as a scaling factor to dynamically adjust the amplitude of the super-resolution network features to adapt to structural changes, and the other is used as a bias term to optimize the degree of alignment between the semantic prior and the super-resolution feature space.

[0080] Furthermore, the calculated modulation parameters are used to perform weighted fusion with the super-resolution network intermediate layer features in an element-by-element multiplication manner to achieve dynamic adjustment of features and fusion of semantic information.

[0081] Furthermore, the reconstructed image is used as the iterative target image and input into the semantic segmentation network again for image segmentation processing to obtain multiple iterative semantic segmentation maps; the deep-level features are used as iterative shallow-level features, and corresponding iterative semantic priors are generated based on all the iterative semantic segmentation maps after segmentation; all the iterative shallow-level features and all the iterative semantic priors are input into the semantic fusion model, and the deep-level features of the reconstructed image are output until a final reconstructed image that meets the standards is obtained.

[0082] Among them, the collaboration between super-resolution and semantic segmentation tasks is achieved through a loop framework, which solves the one-way effect of MR image super-resolution reconstruction on downstream tasks; and an effective semantic prior estimation branch is designed in combination with the improved semantic segmentation network, and the collaborative loop is used to achieve simultaneous improvement in the performance of MR image super-resolution and semantic segmentation tasks; providing a paradigm for the effective combination of MR image super-resolution and common downstream tasks such as semantic segmentation.

[0083] Furthermore, in another embodiment of the present invention, the semantic collaboration-based super-resolution model (SCSR) of the present invention is used to perform 2× reconstruction (2× represents the combination of MRI sequences of two modalities) and compare the results and errors. Figure 6As shown in the figure, sample visualization graphs of different MR image super-resolution methods in BraTS2020 and IXI datasets are shown. At the same time, the brightness distribution in the sample error graph reflects the reconstruction error distribution of each method in different areas. Among them, HR represents the original high-resolution image, Bicubic represents bicubic interpolation, EDSR represents Enhanced Deep Residual Networks, RCAN represents Residual Channel Attention Network, Segnet represents Segmentation Network, RDN represents Residual Dense Network, SwinIR represents Shifted Window Transformer for Image Restoration, and SAN represents Semantic Attention Network, which is the algorithm used in the method disclosed in the present invention.

[0084] Furthermore, in another embodiment of the present invention, the semantic collaboration-based super-resolution model (SCSR) of the present invention is used to perform 4× (4× represents a combination of MRI sequences of four input modalities) reconstruction results and errors compared with other mainstream super-resolution reconstruction methods. Figure 7 As shown in the figure, sample visualizations of different MR image super-resolution methods in the BraTS2020 and IXI datasets are shown. At the same time, the brightness distribution in the sample error map reflects the reconstruction error distribution of each method in different areas.

[0085] Furthermore, in another embodiment of the present invention, the tumor segmentation results of the semantic collaboration-based super-resolution model (SCSR) of the present invention are compared with those of other mainstream super-resolution reconstruction methods, specifically as follows: Figure 8 As shown, the super-resolution reconstruction and segmentation results of 4× magnification factor on the BraTS2020 dataset are shown. GT represents the true image. Different colors in the segmentation results represent different types of tumor areas. The red area represents the whole tumor (WT), the yellow area corresponds to the enhanced tumor (ET), and the green area represents the tumor core (TC).

[0086] The present invention utilizes the prior information in the target image to enable the super-resolution model to have semantic perception capabilities, uses higher-resolution data to learn stable semantic priors, and constructs a bidirectional optimization framework to further improve the accuracy and robustness of the model, thereby generating high-quality reconstructed images.

[0087] Furthermore, if Figure 9 As shown, based on the above-mentioned magnetic resonance imaging super-resolution image reconstruction method, the present invention also provides a magnetic resonance imaging super-resolution image reconstruction system, wherein the magnetic resonance imaging super-resolution image reconstruction system includes:

[0088] A feature extraction module 51 is configured to obtain an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features;

[0089] An image segmentation module 52 is configured to input the target image into a semantic segmentation network for semantic segmentation, and output a plurality of semantic segmentation maps containing shallow features;

[0090] A position encoding module 53 is configured to perform feature extraction on all the semantic segmentation maps after encoding them to obtain a plurality of position embedding vectors, and extract a semantic prior from each of the position embedding vectors;

[0091] The feature fusion module 54 is used to input all the shallow features and all the semantic priors into a semantic fusion model for feature fusion, and output the deep features of the initial image and a reconstructed image.

[0092] Furthermore, if Figure 10 As shown, based on the above-mentioned magnetic resonance imaging super-resolution image reconstruction method and system, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Figure 10 Only some of the components of the terminal are shown, but it should be understood that implementation of all of the shown components is not required, and more or fewer components may be implemented instead.

[0093] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard drive or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card, etc. equipped on the terminal. Furthermore, the memory 20 may include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software installed on the terminal and various types of data, such as program code of the terminal. The memory 20 may also be used to temporarily store data that has been output or is about to be output. In one embodiment, the memory 20 stores a magnetic resonance imaging super-resolution image reconstruction program 40, which can be executed by the processor 10, thereby implementing the magnetic resonance imaging super-resolution image reconstruction method of the present application.

[0094] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chip, configured to execute program codes or process data stored in the memory 20 , such as executing the magnetic resonance imaging super-resolution image reconstruction method.

[0095] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0096] In one embodiment, when the processor 10 executes the magnetic resonance imaging super-resolution image reconstruction program 40 in the memory 20, the following steps are implemented:

[0097] Acquire an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features;

[0098] Inputting the target image into a semantic segmentation network for semantic segmentation, and outputting multiple semantic segmentation maps containing shallow features;

[0099] After encoding all the semantic segmentation maps, a feature extraction operation is performed to obtain a plurality of position embedding vectors, and a semantic prior in each of the position embedding vectors is extracted;

[0100] All the shallow features and all the semantic priors are input into a semantic fusion model for feature fusion, and the deep features of the initial image and a reconstructed image are output.

[0101] The step of obtaining an initial image, inputting the initial image into a super-resolution network for feature extraction, outputting initial shallow features, and performing feature extraction on the initial shallow features to obtain a target image containing deep features specifically includes:

[0102] Obtaining an initial image input by a user, and inputting the initial image into a super-resolution network in a semantic fusion model;

[0103] The super-resolution network extracts features from the initial image and outputs initial shallow features of the initial image;

[0104] The initial shallow features are input into a semantic fusion model to extract deep features, and a target image containing deep features is output.

[0105] The step of inputting the target image into a semantic segmentation network for semantic segmentation and outputting a plurality of semantic segmentation maps containing shallow features specifically includes:

[0106] Inputting the target image into the semantic segmentation network, wherein an encoder in the semantic segmentation network extracts features of the target image according to multiple semantic labels and outputs shallow features of different image regions;

[0107] The decoder in the semantic segmentation network performs feature recovery on all the image regions and outputs high-level semantic information of each image region;

[0108] All the shallow features and all the high-level semantic information in each image region are matched and aligned through an alignment segmentation network, and multiple semantic segmentation maps are output.

[0109] The step of encoding all the semantic segmentation maps and performing a feature extraction operation to obtain a plurality of position embedding vectors and extracting the semantic prior in each position embedding vector specifically includes:

[0110] Performing position encoding on all pixels in each of the semantic segmentation images to obtain a rotation angle of each pixel;

[0111] Calculating a rotation position vector of each pixel according to the rotation angle, and embedding all the rotation position vectors into the corresponding semantic segmentation map to obtain a position embedding vector corresponding to each semantic segmentation map;

[0112] Feature extraction is performed on each of the position embedding vectors to obtain a semantic prior corresponding to each of the semantic segmentation maps.

[0113] The step of inputting all the shallow features and all the semantic priors into a semantic fusion model for feature fusion and outputting the deep features of the initial image and the reconstructed image specifically includes:

[0114] Inputting all the shallow features and all the semantic priors corresponding to each semantic segmentation map into a semantic fusion model;

[0115] When the semantic fusion model performs shallow feature extraction on all the semantic priors, a first parameter and a second parameter are generated;

[0116] The first parameter and the second parameter are weightedly fused with all the shallow features to generate deep features of the initial image and a reconstructed image.

[0117] Wherein, when the semantic fusion model performs shallow feature extraction on all the semantic priors, generating the first parameter and the second parameter specifically includes:

[0118] Performing shallow feature extraction on all the semantic priors through the convolutional layer of the semantic fusion model, and outputting corresponding semantic shallow features and a first modulation parameter corresponding to each of the semantic shallow features in the feature extraction process;

[0119] Generating a second modulation parameter corresponding to each of the semantic shallow features in the feature extraction process through a GELU activation function;

[0120] performing element-by-element multiplication on all the first modulation parameters to obtain a first parameter;

[0121] Perform element-by-element multiplication on all the second modulation parameters to obtain second parameters.

[0122] The method further comprises performing weighted fusion processing on all the parameters and all the shallow features to generate deep features of the initial image and a reconstructed image, and then:

[0123] The reconstructed image is used as an iterative target image and input into the semantic segmentation network again for image segmentation processing to obtain multiple iterative semantic segmentation maps;

[0124] Using the deep-level features as iterative shallow-level features, and generating corresponding iterative semantic priors based on all the iterative semantic segmentation maps after segmentation;

[0125] All the iterative shallow features and all the iterative semantic priors are input into the semantic fusion model, and the deep features of the reconstructed image are output until a final reconstructed image that meets the standards is obtained.

[0126] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an image reconstruction program for magnetic resonance imaging super-resolution, and when the image reconstruction program for magnetic resonance imaging super-resolution is executed by a processor, the steps of the magnetic resonance imaging super-resolution image reconstruction method as described above are implemented.

[0127] In summary, the present invention provides a method for super-resolution image reconstruction of magnetic resonance images and related equipment, the method comprising: obtaining an initial image, inputting the initial image into a super-resolution network for feature extraction, outputting initial shallow features, and performing feature extraction on the initial shallow features to obtain a target image containing deep features; inputting the target image into a semantic segmentation network for semantic segmentation, and outputting multiple semantic segmentation maps containing shallow features; encoding all the semantic segmentation maps and performing feature extraction operations to obtain multiple position embedding vectors, and extracting the semantic prior in each position embedding vector; inputting all the shallow features and all the semantic priors into a semantic fusion model for feature fusion, and outputting deep-level features of the initial image and a reconstructed image. The present invention utilizes the prior information within the target image to enable the super-resolution model to have semantic perception capabilities, utilizes higher-resolution data to learn stable semantic priors, and constructs a bidirectional optimization framework to further improve the accuracy and robustness of the model, thereby generating high-quality reconstructed images.

[0128] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or terminal comprising the element.

[0129] Of course, those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware (such as a processor, controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When executed, the program can include the processes in the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0130] It should be understood that the application of the present invention is not limited to the above examples. For those skilled in the art, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for super-resolution image reconstruction of magnetic resonance imaging, characterized in that: The magnetic resonance imaging super-resolution image reconstruction method comprises: Acquire an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features; Inputting the target image into a semantic segmentation network for semantic segmentation, and outputting multiple semantic segmentation maps containing shallow features; The target image is input into a semantic segmentation network for semantic segmentation, and a plurality of semantic segmentation maps containing shallow features are output, specifically comprising: Inputting the target image into the semantic segmentation network, wherein an encoder in the semantic segmentation network extracts features of the target image according to multiple semantic labels and outputs shallow features of different image regions; The decoder in the semantic segmentation network performs feature recovery on all the image regions and outputs high-level semantic information of each image region; Performing matching and alignment processing on all the shallow features and all the high-level semantic information in each of the image regions through an alignment and segmentation network, and outputting multiple semantic segmentation maps; After encoding all the semantic segmentation maps, a feature extraction operation is performed to obtain a plurality of position embedding vectors, and a semantic prior in each of the position embedding vectors is extracted; All the shallow features and all the semantic priors are input into a semantic fusion model for feature fusion, and the deep features of the initial image and a reconstructed image are output.

2. The method for super-resolution magnetic resonance imaging reconstruction according to claim 1, wherein: The method of obtaining an initial image, inputting the initial image into a super-resolution network for feature extraction, outputting initial shallow features, and performing feature extraction on the initial shallow features to obtain a target image containing deep features specifically includes: Obtaining an initial image input by a user, and inputting the initial image into a super-resolution network in a semantic fusion model; The super-resolution network extracts features from the initial image and outputs initial shallow features of the initial image; The initial shallow features are input into a semantic fusion model to extract deep features, and a target image containing deep features is output.

3. The method for super-resolution magnetic resonance imaging reconstruction according to claim 1, wherein: The encoding of all the semantic segmentation maps is followed by a feature extraction operation to obtain a plurality of position embedding vectors, and the extraction of the semantic prior in each position embedding vector specifically includes: Performing position encoding on all pixels in each of the semantic segmentation images to obtain a rotation angle of each pixel; Calculating a rotation position vector of each pixel according to the rotation angle, and embedding all the rotation position vectors into the corresponding semantic segmentation map to obtain a position embedding vector corresponding to each semantic segmentation map; Feature extraction is performed on each of the position embedding vectors to obtain a semantic prior corresponding to each of the semantic segmentation maps.

4. The method for super-resolution magnetic resonance imaging reconstruction according to claim 1, wherein: The step of inputting all the shallow features and all the semantic priors into a semantic fusion model for feature fusion, and outputting the deep features of the initial image and the reconstructed image, specifically includes: Inputting all the shallow features and all the semantic priors corresponding to each semantic segmentation map into a semantic fusion model; When the semantic fusion model performs shallow feature extraction on all the semantic priors, a first parameter and a second parameter are generated; The first parameter and the second parameter are weightedly fused with all the shallow features to generate deep features of the initial image and a reconstructed image.

5. The method for super-resolution magnetic resonance imaging reconstruction according to claim 4, wherein: When the semantic fusion model performs shallow feature extraction on all the semantic priors, generating the first parameter and the second parameter specifically includes: Performing shallow feature extraction on all the semantic priors through the convolutional layer of the semantic fusion model, and outputting corresponding semantic shallow features and a first modulation parameter corresponding to each of the semantic shallow features in the feature extraction process; Generating a second modulation parameter corresponding to each of the semantic shallow features in the feature extraction process through a GELU activation function; performing element-by-element multiplication on all the first modulation parameters to obtain a first parameter; Perform element-by-element multiplication on all the second modulation parameters to obtain second parameters.

6. The method for super-resolution magnetic resonance imaging reconstruction according to claim 4, wherein: The first parameter and the second parameter are weightedly fused with all the shallow features to generate deep features of the initial image and a reconstructed image, and then further comprising: The reconstructed image is used as an iterative target image and input into the semantic segmentation network again for image segmentation processing to obtain multiple iterative semantic segmentation maps; Using the deep-level features as iterative shallow-level features, and generating corresponding iterative semantic priors based on all the iterative semantic segmentation maps after segmentation; All the iterative shallow features and all the iterative semantic priors are input into the semantic fusion model, and the deep features of the reconstructed image are output until a final reconstructed image that meets the standards is obtained.

7. A magnetic resonance imaging super-resolution image reconstruction system, characterized in that: The magnetic resonance imaging super-resolution image reconstruction system is applied to the magnetic resonance imaging super-resolution image reconstruction method according to any one of claims 1 to 6, and the magnetic resonance imaging super-resolution image reconstruction system comprises: A feature extraction module is used to obtain an initial image, input the initial image into a super-resolution network for feature extraction, output initial shallow features, and perform feature extraction on the initial shallow features to obtain a target image containing deep features; An image segmentation module is used to input the target image into a semantic segmentation network for semantic segmentation, and output a plurality of semantic segmentation maps containing shallow features; A position encoding module is used to perform feature extraction on all the semantic segmentation maps after encoding them to obtain multiple position embedding vectors, and to extract semantic priors from each of the position embedding vectors; The feature fusion module is used to input all the shallow features and all the semantic priors into a semantic fusion model for feature fusion, and output the deep features of the initial image and a reconstructed image.

8. A terminal, characterized in that: The terminal includes: a memory, a processor, and a magnetic resonance imaging super-resolution image reconstruction program stored in the memory and executable on the processor. When the magnetic resonance imaging super-resolution image reconstruction program is executed by the processor, the steps of the magnetic resonance imaging super-resolution image reconstruction method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a magnetic resonance imaging super-resolution image reconstruction program, which, when executed by a processor, implements the steps of the magnetic resonance imaging super-resolution image reconstruction method according to any one of claims 1 to 6.

Images