Structural Magnetic Resonance Super-Resolution Image Reconstruction Model Training Method, Reconstruction Method and Device

Abstract

The present application provides a structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and equipment, which relate to the field of image processing technology. The training method includes: performing anisotropic downsampling on a structural magnetic resonance image to obtain a target low-resolution image for simulating a clinical structural magnetic resonance low-resolution image; generating image samples corresponding to the target low-resolution image based on a reference structural magnetic resonance image, and training a deep learning model based on the image samples and the structural magnetic resonance image, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image. The present application can improve the effectiveness and rationality of image samples in simulating low-resolution images of clinical medium-thick layer scans, can improve the effectiveness and reliability of using image samples to train a structural magnetic resonance super-resolution image reconstruction model, and can further improve the accuracy and reliability of using the model to reconstruct super-resolution images.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and equipment. Background Art

[0002] Structural Magnetic Resonance Imaging (sMRI) is used to characterize the three-dimensional structure of the human brain, including the volume of various brain tissues, cortical thickness, and surface area. To obtain more brain information, researchers primarily use high-resolution scans with smaller slice thicknesses, which allow for more precise calculation of various parameters. However, in clinical applications, due to time and cost constraints, hospitals primarily use low-resolution scans with larger slice thicknesses. This produces anisotropic images with lower resolution in the slice-selective direction than in the intra-slice direction. Due to the lack of preprocessing tools specifically for clinical sMRI sequences, structural MRI images of a large number of clinical patients cannot be used for scientific research, which to some extent limits the development of treatments for various diseases. Therefore, to bridge the gap between research and clinical structural MRI sequences, improving the resolution of clinical structural MRI images is particularly important.

[0003] Currently, the existing method to improve the resolution of structural MRI images is mainly the super-resolution (SR) reconstruction algorithm. It uses one or more low-resolution structural MRI images to eliminate degradation interference such as blur encountered during the signal acquisition process through signal processing, image processing and machine learning technologies, thereby predicting the real structural MRI super-resolution image to restore the details and high-frequency information lost during the acquisition of clinical low-resolution structural MRI images.

[0004] However, the training data used in existing deep learning-based structural MRI super-resolution image reconstruction methods are significantly different from the anisotropic structural MRI image features actually collected in clinical practice. Therefore, the accuracy and reliability of structural MRI super-resolution images reconstructed by the trained deep learning model cannot be guaranteed. Summary of the Invention

[0005] In view of this, embodiments of the present application provide a structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and device to eliminate or improve one or more defects in the prior art.

[0006] One aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction model training method, comprising:

[0007] performing anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image;

[0008] Based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, image samples corresponding to each of the target low-resolution images are generated respectively, and a deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image.

[0009] In some embodiments of the present application, performing anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image includes:

[0010] performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal plane direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images;

[0011] Cubic spline interpolation is performed on each of the low-resolution images to obtain a target low-resolution image corresponding to each of the low-resolution images and used to simulate a clinical structural magnetic resonance low-resolution image.

[0012] In some embodiments of the present application, performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images includes:

[0013] Dividing each layer of each structural magnetic resonance image in the sagittal plane direction into a plurality of first layer groups, wherein the number of layers in each first layer group is a preset super-resolution multiple;

[0014] performing inter-layer discarding processing on each of the first layer groups according to the super-resolution multiple and a preset inter-layer discarding ratio to reduce the number of layers in each of the first layer groups, thereby obtaining a second layer group corresponding to each of the first layer groups;

[0015] Mean downsampling processing is performed on each of the second layer groups respectively to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

[0016] In some embodiments of the present application, the generating of image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, and training a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model as a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image, includes:

[0017] Registering a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image with each of the target low-resolution images to obtain a reference image corresponding to each of the target low-resolution images;

[0018] generating image samples corresponding to the respective target low-resolution images according to the respective target low-resolution images and the respective corresponding reference images;

[0019] Based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, a deep learning model is trained with a preset target loss function to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

[0020] In some embodiments of the present application, generating image samples corresponding to the respective target low-resolution images according to the respective target low-resolution images and the respective corresponding reference images includes:

[0021] Dividing each of the target low-resolution images and each of the reference images into image blocks of the same size to obtain image blocks corresponding to each of the target low-resolution images and image blocks corresponding to each of the reference images;

[0022] Each of the target low-resolution images and its corresponding reference image is used as a sample group, and each image block corresponding to the target low-resolution image in each of the sample groups is spliced one-to-one with each image block corresponding to the reference image to obtain image samples corresponding to each of the sample groups.

[0023] In some embodiments of the present application, the deep learning model includes: a local feature extraction branch, a global feature extraction branch, and a feature addition layer;

[0024] Correspondingly, the deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples using a preset target loss function, including:

[0025] Performing preset iterative training steps in sequence for each sample group, wherein the iterative training steps include:

[0026] Inputting each of the image samples corresponding to the current sample group into the local feature extraction branch and the global feature extraction branch in the current deep learning model, respectively, so that the local feature extraction branch extracts local feature data corresponding to each of the image samples, respectively, so that the global feature extraction branch extracts global feature data corresponding to each of the image samples, and so that the feature addition layer adds the local feature data and the global feature data corresponding to each of the image samples, respectively, to obtain a super-resolution image corresponding to each of the image samples;

[0027] Recombining the super-resolution images of the image samples corresponding to the current sample group to obtain a target super-resolution image corresponding to the current sample group;

[0028] Based on the target super-resolution image and the structural magnetic resonance image corresponding to the sample group, the current loss of the deep learning model is calculated using a preset target loss function, and the deep learning model is optimized based on the loss to obtain the deep learning model in the next iterative training step, wherein the target loss function includes a weighted sum of a mean square error loss function and a perceptual loss function with different weights.

[0029] In some embodiments of the present application, the local feature extraction branch includes:

[0030] A first convolutional layer is used to perform high-dimensional feature space mapping on the image sample, adjust the format and dimension, and output corresponding first feature data;

[0031] a local feature extraction unit, configured to perform local feature extraction on the first feature data based on a plurality of stacked residual blocks, and output corresponding second feature data;

[0032] The second convolutional layer is used to adjust the number of channels of the second feature data and output corresponding local feature data.

[0033] In some embodiments of the present application, the global feature extraction branch includes:

[0034] An encoder, configured to encode the image sample based on three sequentially connected encoding layers and output corresponding encoded feature data; wherein the encoding layers include: a connected convolutional layer and a ReLU activation function;

[0035] A global feature extraction unit, configured to perform global feature extraction on the encoded feature data based on a plurality of sequentially connected Transformer layers, and output corresponding global encoded feature data;

[0036] A decoder is used to decode the global encoded feature data based on three sequentially connected decoding layers and output corresponding global feature data, wherein the encoding layer includes: an upsampling layer, a convolution layer and a ReLU activation function connected in sequence.

[0037] A second aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction method, comprising:

[0038] Registering the reference structural magnetic resonance image with the current clinical structural low-resolution magnetic resonance image to obtain a reference image corresponding to the clinical structural low-resolution magnetic resonance image;

[0039] Dividing the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image;

[0040] Performing one-to-one splicing of each image block corresponding to the clinical structure low-resolution magnetic resonance image and each image block corresponding to the reference image to obtain image samples corresponding to the clinical structure low-resolution magnetic resonance image;

[0041] Inputting each image sample corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model, respectively, so that the structural magnetic resonance super-resolution image reconstruction model outputs a super-resolution image of each image sample corresponding to the clinical structural magnetic resonance low-resolution image, respectively, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method mentioned in the first aspect;

[0042] The super-resolution images of the image samples corresponding to the clinical structure low-resolution magnetic resonance image are recombined to obtain a target super-resolution image corresponding to the clinical structure low-resolution magnetic resonance image.

[0043] A third aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction model training device, comprising:

[0044] a simulation data construction module, configured to perform anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used to simulate a clinical structural magnetic resonance low-resolution image;

[0045] A model training module is used to generate image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, and to train a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image.

[0046] A fourth aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction device, comprising:

[0047] An image registration module is used to register the reference structural magnetic resonance image with the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image;

[0048] an image segmentation module, configured to segment the clinical structure low-resolution MRI image and the reference image into image blocks of the same size, to obtain image blocks corresponding to the clinical structure low-resolution MRI image and image blocks corresponding to the reference image;

[0049] an image stitching module, configured to stitch each image block corresponding to the clinical structure low-resolution magnetic resonance image with each image block corresponding to the reference image one-to-one, so as to obtain each image sample corresponding to the clinical structure low-resolution magnetic resonance image;

[0050] an image reconstruction module, configured to input each image sample corresponding to the clinical structural MRI low-resolution image into a structural MRI super-resolution image reconstruction model, so that the structural MRI super-resolution image reconstruction model outputs a super-resolution image of each image sample corresponding to the clinical structural MRI low-resolution image, wherein the structural MRI super-resolution image reconstruction model is pre-trained based on the structural MRI super-resolution image reconstruction model training method provided in the first aspect;

[0051] The image recombining module is used to recombine the super-resolution images of the image samples corresponding to the clinical structure magnetic resonance low-resolution image to obtain a target super-resolution image corresponding to the clinical structure magnetic resonance low-resolution image.

[0052] The fifth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the structural magnetic resonance super-resolution image reconstruction model training method provided in the first aspect, and / or implements the structural magnetic resonance super-resolution image reconstruction method provided in the second aspect.

[0053] The sixth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the structural magnetic resonance super-resolution image reconstruction model training method, and / or implements the structural magnetic resonance super-resolution image reconstruction method provided in the second aspect.

[0054] The seventh aspect of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the structural magnetic resonance super-resolution image reconstruction model training method, and / or implements the structural magnetic resonance super-resolution image reconstruction method provided in the aforementioned second aspect.

[0055] The structural magnetic resonance super-resolution image reconstruction model training method provided in the present application performs anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each of the structural magnetic resonance images for simulating a clinical structural magnetic resonance low-resolution image; based on a reference structural magnetic resonance image with the same resolution as the structural magnetic resonance image, image samples corresponding to each of the target low-resolution images are generated respectively, and a deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image. By designing reasonable downsampling and degradation methods, simulating the resolution loss in clinical imaging, and constructing an appropriate training data set, the effectiveness and rationality of the image samples in simulating the low-resolution images of clinical medium-thick layer scans can be improved, so that the model has better robustness when processing anisotropic data. It also uses deep learning technology to automatically extract features and provide the model with prior knowledge of brain structure in the form of designed reference images. This can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects and loss of detailed textures, and improve the effectiveness and reliability of the structural magnetic resonance super-resolution image reconstruction model trained using image samples, thereby improving the accuracy and reliability of super-resolution images reconstructed using this model.

[0056] Additional advantages, purposes, and features of the present application will be described in part in the following description and will become apparent to those skilled in the art upon study of the following or may be learned from practice of the present application. The purposes and other advantages of the present application may be achieved and obtained by the structures specifically pointed out in the specification and drawings.

[0057] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The drawings described herein are intended to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger than other components in the exemplary device actually manufactured according to the present application. In the drawings:

[0059] Figure 1 This is a schematic diagram of a first flow chart of a structural magnetic resonance super-resolution image reconstruction model training method in one embodiment of the present application.

[0060] Figure 2 This is a second flow chart of the structural magnetic resonance super-resolution image reconstruction model training method in one embodiment of the present application.

[0061] Figure 3 Schematic diagram of the sagittal plane orientation of a structural magnetic resonance image.

[0062] Figure 4 This is a third flow chart of the structural magnetic resonance super-resolution image reconstruction model training method in one embodiment of the present application.

[0063] Figure 5 Schematic diagram of the overall architecture of the machine learning model in one embodiment of the present application.

[0064] Figure 6 Schematic diagram of the architecture of the local feature extraction branch in the machine learning model in one embodiment of the present application.

[0065] Figure 7 This is a schematic diagram of the data processing logic of the machine learning model in one embodiment of the present application.

[0066] Figure 8 Schematic diagram of the architecture of the global feature extraction branch in the machine learning model in one embodiment of the present application.

[0067] Figure 9 This is a flowchart of a structural magnetic resonance super-resolution image reconstruction model training method in an application example of the present application.

[0068] Figure 10 Schematic diagram of isotropic downsampling of a three-dimensional image in the prior art.

[0069] Figure 11 FIG. 1 is a schematic diagram of anisotropic downsampling of a structural magnetic resonance image in an application example of the present application.

[0070] Figure 12FIG. 4 is a flow chart of a structural magnetic resonance super-resolution image reconstruction method according to an embodiment of the present application.

[0071] Figure 13 Schematic diagram of the structure of a structural magnetic resonance super-resolution image reconstruction model training device in one embodiment of the present application.

[0072] Figure 14 FIG. 1 is a schematic structural diagram of a structural magnetic resonance super-resolution image reconstruction device in one embodiment of the present application. DETAILED DESCRIPTION

[0073] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail in conjunction with the embodiments and drawings. Here, the illustrative embodiments of this application and their descriptions are used to explain this application, but are not intended to limit this application.

[0074] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, the accompanying drawings only show structures and / or processing steps that are closely related to the scheme according to the present application, while other details that are not closely related to the present application are omitted.

[0075] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0076] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0077] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0078] It should be noted that the current super-resolution image reconstruction methods for structural magnetic resonance images are mainly divided into the following four categories:

[0079] 1. Interpolation-based algorithms: Interpolation methods use the linear relationship between adjacent pixels in a low-resolution image to estimate the grayscale values of missing pixels in a high-resolution reconstructed image. Common interpolation methods include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and B-spline interpolation.

[0080] 2. Reconstruction-based algorithms: These algorithms model the MRI imaging process and solve its inverse process through a combination of signal processing methods and optimization algorithms. This allows for image deblurring, upsampling, and denoising to reconstruct high-frequency details lost during MRI imaging. Magnetic resonance imaging (MRI) is a medical imaging technique based on the phenomenon of nuclear magnetic resonance. MRI scans require the acquisition of multiple signal sequences, resulting in long acquisition times. Due to the difficulty of subjects remaining still for extended periods, scanned images often contain artifacts, impacting the signal-to-noise ratio.

[0081] 3. Algorithms based on shallow learning: Model the inverse process of the MRI imaging process, learn the mapping relationship between high-resolution and low-resolution images from the training set, and optimize the model parameters to minimize the error between the reconstructed prediction results and the actual high-resolution images.

[0082] 4. Deep learning-based algorithms: Use deep neural networks to automatically learn the mapping function between low-resolution images and high-resolution images from large amounts of data, thereby achieving super-resolution of sMRI images.

[0083] Among them, since the model of the algorithm based on the interpolation method is too simple, there are problems such as blurred edges, jagged effects, loss of detailed textures, and inaccurate reconstruction of complex structures. It is difficult to ensure the optimality of the predicted image and its help in image reconstruction is limited. In addition, the computational complexity of the reconstruction-based algorithm is high, and the convergence of the algorithm depends on the initial estimate. It is difficult to ensure the stability of the solution without sufficient prior information constraints. In addition, the algorithm based on shallow learning is limited by the quality and quantity of training data, and the feature extraction process relies too much on the dictionary constructed by prior knowledge. Therefore, this application adopts a deep learning-based algorithm that can solve the above problems to perform super-resolution image reconstruction of structural magnetic resonance images. However, when constructing training data, the deep learning-based algorithm mostly uses isotropic downsampling to obtain low-resolution images, which is quite different from the anisotropic structural magnetic resonance image features actually collected in clinical practice. At the same time, the existing algorithms lack prior knowledge of the brain to improve the super-resolution effect.

[0084] In order to solve the problem that the training data used in the existing deep learning-based super-resolution image reconstruction method of structural magnetic resonance images is significantly different from the anisotropic structural magnetic resonance image features actually collected in clinical practice, and therefore the accuracy and reliability of the structural magnetic resonance super-resolution images reconstructed by the trained deep learning model cannot be guaranteed, the embodiments of the present application respectively provide a structural magnetic resonance super-resolution image reconstruction model training method, a structural magnetic resonance super-resolution image reconstruction model training device for executing the structural magnetic resonance super-resolution image reconstruction model training method, a structural magnetic resonance super-resolution image reconstruction model method, a structural magnetic resonance super-resolution image reconstruction device for executing the structural magnetic resonance super-resolution image reconstruction method, a physical device, a computer-readable storage medium and a computer program product. By designing reasonable downsampling and degradation methods, simulating the resolution loss in clinical imaging, and constructing an appropriate training data set, the model has better robustness when processing anisotropic data. Deep learning technology is also used to automatically extract features and provide the model with prior knowledge of brain structure in the form of designed reference images, which can improve the reconstruction accuracy of complex brain structures and reduce edge blurring, jagged effects and loss of detailed textures.

[0085] The details are described in detail through the following examples.

[0086] Based on this, the embodiment of the present application provides a structural magnetic resonance super-resolution image reconstruction model training method that can be implemented by a structural magnetic resonance super-resolution image reconstruction model training device, see Figure 1 The structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents:

[0087] Step 100: performing anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image.

[0088] In one or more embodiments of the present application, the structural MRI image refers to a high-resolution structural MRI image having a higher resolution than a low-resolution clinical structural MRI image. The low-resolution clinical structural MRI image refers to an anisotropic image obtained by clinically scanning with a low-resolution scan having a larger slice thickness and having a lower resolution in the slice-selective direction than in the intra-slice direction.

[0089] It is understandable that, in view of the lack of one-to-one positional relationship in real scenes, paired structural MRI high-resolution images and structural MRI low-resolution image data, it is necessary to construct a simulated data set during the model training process. The data set simulated by the prior art is down-sampled in an isotropic manner, and the low-resolution images produced in this way are difficult to simulate the low-resolution images of clinical medium and thick layer scans. Therefore, in step 100 of the embodiment of the present application, each structural MRI image is anisotropically down-sampled, that is, a mean down-sampling method is used to simulate anisotropic low-resolution images in a certain direction of the structural MRI image, thereby effectively simulating the anisotropic images produced by clinical medium and thick layer scans. Wherein, each direction refers to each axial direction in the three-dimensional coordinate system where the image is located.

[0090] Step 200: Based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, respectively generate image samples corresponding to each of the target low-resolution images, and train a deep learning model based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image.

[0091] In step 200, the reference structural magnetic resonance image refers to a structural magnetic resonance high-resolution image having the same resolution as each of the structural magnetic resonance images in step 100, and in one example, the reference structural magnetic resonance image can be a structural magnetic resonance high-resolution image of a healthy human brain that has been manually evaluated.

[0092] It can be understood that the structural magnetic resonance image corresponding to each of the image samples is the structural magnetic resonance image corresponding to the target low-resolution image corresponding to the image sample.

[0093] In one example of step 200, after iteratively training the deep learning model using each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, the simulated training set consisting of each of the image samples can be evaluated (i.e., the effect of the super-resolution images output by the deep learning model trained using the simulated training set is evaluated). The deep learning model is then further trained and fine-tuned using a constructed real clinical dataset, and the effect of the super-resolution images output by the fine-tuned deep learning model is then evaluated on the real clinical dataset. Finally, the deep learning model whose evaluation results meet the preset accuracy requirements is determined as the structural magnetic resonance super-resolution image reconstruction model used to reconstruct the image samples into super-resolution images. Thus, after training and evaluation on simulated data, the model can be fine-tuned using a small amount of real clinical data to improve the generalization performance of the model on real clinical data, effectively restore image quality, and retain image detail texture information to the greatest extent possible.

[0094] As can be seen from the above description, the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application, by designing reasonable downsampling and degradation methods, simulates the resolution loss in clinical imaging, and constructs an appropriate training data set, can improve the effectiveness and rationality of image samples in simulating low-resolution images of clinical thick layer scans, and make the model more robust when processing anisotropic data. It also uses deep learning technology to automatically extract features and provide the model with prior knowledge of brain structure in the form of designed reference images, which can improve the reconstruction accuracy of complex brain structures, reduce edge blurring, jagged effects, and loss of detailed textures, and can improve the effectiveness and reliability of using image samples to train the structural magnetic resonance super-resolution image reconstruction model, thereby improving the accuracy and reliability of reconstructing super-resolution images using the model.

[0095] In order to further improve the effectiveness and rationality of image samples simulating low-resolution images of clinical thick layer scans, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 2 Step 100 of the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following:

[0096] Step 110: performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

[0097] It is understood that the sagittal plane direction is the anatomical direction corresponding to the structural magnetic resonance super-resolution image, see Figure 3, which is perpendicular to the transverse and coronal planes and divides the plane of the human body from front to back (from ventral to dorsal), dividing the body into left and right parts. In brain anatomy, the sagittal plane is usually used to observe and study the structure and tissue between the left and right hemispheres of the brain. That is to say, the conventional single-direction mean downsampling only simulates the layer thickness parameters of the clinical scan, but does not take into account the physical intervals between layers in the actual scan, resulting in significant differences between the generated low-resolution images and the actual clinical scan data. Step 110 of the embodiment of the present application can add an inter-layer discarding strategy to solve the problem that the existing method does not take into account the intervals between layers by adopting a mean downsampling process based on an inter-layer discarding strategy.

[0098] Step 120: performing cubic spline interpolation on each of the low-resolution images to obtain a target low-resolution image corresponding to each of the low-resolution images for simulating a clinical structural magnetic resonance low-resolution image.

[0099] It should be noted that the downsampling method of step 110 destroys the spatial position correspondence between the structural magnetic resonance image and the low-resolution image, making it difficult for the deep learning model to establish an effective mapping relationship during the training process, which will significantly increase the difficulty of model training. Therefore, in order to solve this problem, step 120 of the embodiment of the present application further performs cubic spline interpolation on the low-resolution image to obtain an isotropic target low-resolution image and establishes a spatial correspondence between it and the structural magnetic resonance image. That is, when generating the isotropic target low-resolution image, its correspondence with the original high-resolution image (i.e., the structural magnetic resonance image) in the three-dimensional spatial coordinate system is strictly maintained, thereby effectively reducing the training difficulty of the deep learning model and improving the accuracy of super-resolution reconstruction.

[0100] In order to further improve the effectiveness and reliability of the mean downsampling process based on the inter-layer discarding strategy, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 4 Step 110 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents:

[0101] Step 111: Divide each layer of each structural magnetic resonance image in the sagittal plane into a plurality of first layer groups, wherein the number of layers in each first layer group is a preset super-resolution multiple.

[0102] Specifically, the total number of layers of the structural magnetic resonance image in the sagittal direction is N, the super-resolution factor is k, and each k layers constitutes a first layer group.

[0103] Step 112: performing inter-layer discarding processing on each of the first layer groups according to the super-resolution multiple and a preset inter-layer discarding ratio to reduce the number of layers in each of the first layer groups, thereby obtaining a second layer group corresponding to each of the first layer groups.

[0104] Specifically, if the inter-layer discard ratio is p, then the number of layers discarded (i.e., deleted) in each first-layer group is d=k×p; the number of layers in the resulting second-layer group is c=k×(1-p).

[0105] Step 113: performing mean downsampling processing on each of the second layer groups respectively to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

[0106] In order to further reduce the loss of prior knowledge as much as possible and input it into the model in a simple form, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 2 Step 200 of the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following:

[0107] Step 210: Registering a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image with each of the target low-resolution images to obtain a reference image corresponding to each of the target low-resolution images.

[0108] It will be understood that registering a reference structural MRI image having the same resolution as the structural MRI image with each of the target low-resolution images refers to applying a spatial transformation (or a series of spatial transformations) to the reference structural MRI image so that it is spatially aligned with corresponding points on the target low-resolution image. This alignment means that the same anatomical point has the same spatial position on both matched images. The registration result should ensure that all anatomical points on both images, or at least all points of diagnostic significance and surgical interest, are aligned.

[0109] Specifically, many existing super-resolution techniques for brain sMRI are derived from super-resolution techniques for natural images. However, this approach fails to fully consider the specific characteristics of brain sMRI images and underutilizes prior knowledge of the brain. To address this issue, a module is needed that can provide models with prior knowledge of the brain to guide them in performing brain image super-resolution. In neuroscience, prior knowledge of the brain is typically based on qualitative descriptions, while deep learning models require precise digital representations, resulting in inconsistencies in knowledge representation. Furthermore, how to minimize information loss and conveniently feed prior knowledge into the model is crucial. While segmentation results of brain anatomical structures (such as gray matter, white matter, and cerebrospinal fluid) can be used as additional input channels for the model, or MRI data from other modalities can be used as complementary information, there are currently no robust segmentation tools for low-resolution images, and training data with low-resolution images and corresponding segmentation labels is also scarce. Furthermore, since low-resolution images acquired in clinical settings are often monomodal, multimodal auxiliary information is difficult to obtain in practical applications. Therefore, step 210 of the embodiment of the present application designs an improved reference image strategy that does not require other modalities, and provides the model with prior knowledge of the brain in a simple way. Specifically, a reference structural magnetic resonance image of a fixed healthy person with the same modality as the structural magnetic resonance image is first denoised, and then aligned with the target low-resolution image as a reference image of the target low-resolution image. The reference image and the interpolated target low-resolution image have a point-to-point one-to-one correspondence, which can well provide the model with the approximate anatomical structure of each brain region, and can be input into the model together as a dual-channel image in a simple splicing manner.

[0110] In one example, the registration method can use the symmetric normalization mode in ANTSpy, which has high registration accuracy, supports the registration of both two-dimensional and three-dimensional images, and is highly robust and stable. ANTSpy is a powerful open-source Python library derived from the Advanced Neuroimaging with Templates and Statistics (ANTs) project and focuses on medical imaging data processing.

[0111] Step 220: Generate image samples corresponding to each target low-resolution image according to each target low-resolution image and the corresponding reference image.

[0112] Step 230: Based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, a deep learning model is trained with a preset target loss function to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

[0113] In order to further improve the execution reliability and effectiveness of the structural magnetic resonance super-resolution image reconstruction model training, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 4 Step 220 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents:

[0114] Step 221: Divide each of the target low-resolution images and each of the reference images into image blocks of the same size to obtain image blocks corresponding to each of the target low-resolution images and image blocks corresponding to each of the reference images.

[0115] Step 222: Each of the target low-resolution images and its corresponding reference image is used as a sample group, and each image block corresponding to the target low-resolution image in each sample group is spliced one-to-one with each image block corresponding to the reference image to obtain image samples corresponding to each sample group.

[0116] In one example, step 221 may adopt a sliding window strategy to cut the target low-resolution image and the reference image into The image blocks of different sizes are then stitched together in step 222 to form a dual-channel image as image samples, which are then fed into the deep learning model one by one for super-resolution reconstruction. The super-resolution images corresponding to all the image samples output by the deep learning model are then reassembled to obtain the target super-resolution image corresponding to each sample group. In order to further improve the accuracy and reliability of the structural magnetic resonance super-resolution image reconstruction model, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 5 , the deep learning model includes: a local feature extraction branch, a global feature extraction branch and a feature addition layer, the output ends of the local feature extraction branch and the global feature extraction branch are both connected to the input end of the feature addition layer;

[0117] For corresponding reference, see Figure 4 Step 230 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents:

[0118] Step 231: Execute a preset iterative training step for each sample group in sequence, wherein the iterative training step includes: inputting each of the image samples corresponding to the current sample group into the local feature extraction branch and the global feature extraction branch in the current deep learning model, so that the local feature extraction branch extracts local feature data corresponding to each of the image samples, the global feature extraction branch extracts global feature data corresponding to each of the image samples, and the feature addition layer adds the local feature data and the global feature data corresponding to each of the image samples to obtain super-resolution images corresponding to each of the image samples; recombining the super-resolution images of each of the image samples corresponding to the current sample group to obtain a target super-resolution image corresponding to the current sample group; calculating the current loss of the deep learning model based on the target super-resolution image corresponding to the sample group and the structural magnetic resonance image using a preset target loss function, and optimizing the deep learning model based on the loss to obtain a deep learning model in the next iterative training step, wherein the target loss function includes a weighted sum of a mean square error loss function and a perceptual loss function with different weights.

[0119] In one example, recombining the super-resolution images of the respective image samples corresponding to the current sample group can be as follows: The individual super-resolution images are stitched together to form The target super-resolution image.

[0120] Specifically, in the prior art, the super-resolution reconstruction task of structural magnetic resonance images usually uses a single mean square error (MSE) loss function for model training. However, the MSE loss function only focuses on the differences at the voxel level, which often leads to problems such as blurring and lack of details in the generated images, and cannot well maintain the high-frequency information and perceptual quality of the image. The model loss function of the embodiment of the present application is the weighted sum of the MSE loss function and the perceptual loss function, with weights of 1 and 0.1 respectively, where the perceptual loss uses a pre-trained Squeeze Net to calculate the feature map, where Squeeze Net is a convolutional neural network architecture that focuses on model lightweighting. Its core goal is to significantly reduce the number of parameters and the amount of calculation while ensuring model accuracy. And it is calculated in a "pseudo-three-dimensional" form slice by slice. This loss function ensures both the voxel-level accuracy and visual perception quality of the image. The formula of the target loss function is as follows:

[0121]

[0122] in, Indicates loss; represents the voxel value of the output super-resolution image, represents the voxel value of the structural magnetic resonance image, T represents the total number of voxels, Q represents the total number of elements of the feature map used, represents the g-th feature layer of Squeeze Net, G represents the number of feature layers used, and H represents the number of slices used. and represent the hth slice (cross-sectional direction) of the super-resolution image and structural magnetic resonance image, respectively.

[0123] In order to further improve the effectiveness and reliability of local feature extraction, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 6 and Figure 7 , the local feature extraction branch specifically includes a first convolutional layer, a local feature extraction unit and a second convolutional layer connected in sequence;

[0124] The first convolutional layer is used to perform high-dimensional feature space mapping on the image sample and adjust the format and dimension, and output corresponding first feature data.

[0125] Specifically, the image samples input to the local feature extraction branch and the global feature extraction branch are two-channel 3D images, which are the concatenation of a low-resolution interpolated image and a reference image. The shape of the input image sample is a 5-dimensional tensor [batch_size, channels, length, width, height], where batch_size is the batch size, channels is the number of image channels, length is the image length, width is the image width, and height is the image height.

[0126] On this basis, after entering the local feature extraction branch, the image sample first passes through a first convolution layer to map the image sample to a high-dimensional feature space and adjust it to be consistent with the input format and dimension expected by the residual module to ensure the effectiveness of residual learning. The shape of the first feature data output by the first convolution layer is [batch_size, num_filters, length, width, height], where num_filters is the number of convolution kernels and the number of channels for outputting the first feature data.

[0127] The local feature extraction unit is used to perform local feature extraction on the first feature data based on multiple stacked residual blocks, and output corresponding second feature data.

[0128] In one example, the local feature extraction unit includes 9 stacked residual blocks.

[0129] Specifically, the first feature data output by the first convolutional layer is input into the local feature extraction unit, and the 9 stacked residual blocks ResBlock in the local feature extraction unit are composed of a convolutional layer Conv, a ReLU activation function and another convolutional layer Conv connected in sequence, and have residual connections; the shape of the feature map output by each residual block ResBlock is [batch_size, num_filters, length, width, height], and the output of each residual block ResBlock enters another convolutional layer to aggregate the high-dimensional outputs of multiple residual blocks ResBlocks, and the shape of the output second feature data is [batch_size, num_filters, length, width, height].

[0130] The second convolutional layer is used to adjust the number of channels of the second feature data and output corresponding local feature data.

[0131] Specifically, the second feature data output by the local feature extraction unit is input into the second convolutional layer, so that the second convolutional layer adjusts the number of channels of the second feature data to 1, as the output of the local feature extraction branch, and the shape of the output local feature data is [batch_size, 1, length, width, height].

[0132] In order to further improve the effectiveness and reliability of global feature extraction, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 7 and Figure 8 The global feature extraction branch specifically includes an encoder, a global feature extraction unit and a decoder connected in sequence.

[0133] The encoder is used to encode the image sample based on three sequentially connected encoding layers and output corresponding encoding feature data; wherein the encoding layer includes: a connected convolution layer and a ReLU activation function. In one example, the convolution layers in the three sequentially connected encoding layers can respectively adopt 、 and 3D convolution kernel.

[0134] Specifically, the image sample is input into the global feature extraction branch and then passes through an encoder consisting of three convolutional layers to encode the original dual-channel image sample. The shape of the convolutional feature map output by each convolutional layer is [batch_size, num_filters, new_length, new_width, new_length], where new_length, new_width, and new_height represent the length, width, and height of the convolutional feature map, respectively. The length, width, and height of the convolutional feature map gradually decrease. After three layers of convolution, the shape of the encoded feature data output is [batch_size, feature_channels, feature_length, feature_width, feature_height]. Among them, feature_channels, feature_length, feature_width, and feature_height represent the number of feature channels, feature length, feature width, and feature height, respectively.

[0135] The global feature extraction unit is used to perform global feature extraction on the encoded feature data based on multiple sequentially connected Transformer layers and output corresponding global encoded feature data.

[0136] Specifically, the encoded feature data is input into multiple sequentially connected Transformer layers (Transformer Block) in the global feature extraction unit, and the global features of the image are extracted on the encoded feature data. The shape of the output global encoded feature data is [batch_size, feature_channels, feature_length, feature_width, feature_height].

[0137] Among them, Transformer is a sequence model based on the attention mechanism.

[0138] The decoder is used to decode the global encoding feature data based on three sequentially connected decoding layers and output corresponding global feature data, wherein the encoding layer includes: an upsampling layer, a convolution layer and a ReLU activation function connected in sequence.

[0139] Specifically, the encoded feature data output by the global feature extraction unit is input into the decoder, and the three sequentially connected decoding layers of the decoder gradually restore the feature map to the original image size, and the output global feature data shape is [batch_size, 1, length, width, height].

[0140] It is understandable that the feature extraction branch in the prior art only considers the local features of the image and lacks interaction with the overall information of the image. To address this problem, although the Transformer layer can be used to capture the image context information, since medical images are three-dimensional images, the convolutions within the model are all three-dimensional convolutions. Directly applying the Transformer layer will lead to problems such as high computational complexity and excessive memory usage. Therefore, this application designs an improved global feature extraction branch based on the Transformer layer, which is successively used 、 and The image is convolved with a three-dimensional convolution kernel and encoded to obtain a smaller feature map. An encoder with 8 Transformer layers is applied to the feature map to extract the global features of the image. Three upsampling and convolution modules are then used to restore the feature map to the original image size. The dimension of the hidden layer in each Transformer layer is 384, the dimension of the hidden layer in the feedforward neural network is 1536, and the number of heads in the multi-head attention mechanism is 8. Finally, the images of the two branches are added together to obtain the final super-resolution image. Compared with the existing sMRI super-resolution model that only focuses on local areas, the multi-scale feature extraction network proposed in this application fuses local and global features to achieve the complementarity of their information, which can improve the global consistency and overall quality of the image.

[0141] In addition, it should be noted that the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application uses the mean downsampling and discarding strategy to construct anisotropic data sets. Adding some simple strategies such as Gaussian blur or not discarding is also within the scope of protection of the application examples of the present application; the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application uses a fixed healthy human brain sMRI image to align with a low-resolution interpolated image as a reference image for the image. Using multiple images or other brain maps for alignment as reference images is also within the scope of protection of the application examples of the present application; the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application realizes multi-dimensional analysis of brain images from global features and local features, and stacking more repeated local feature extraction branches or global feature extraction branches is also within the scope of protection of the application examples of the present application; the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application adopts the strategy of training on a simulated data set and then fine-tuning on a real clinical data set. Using pre-training-fine-tuning multiple alternating training or only single-stage training on a simulated or clinical data set is also within the scope of protection of the application examples of the present application.

[0142] To further illustrate the above embodiments, this application also provides a specific application example of a structural magnetic resonance super-resolution image reconstruction model training method. This application example provides a low-resolution structural magnetic resonance image super-resolution algorithm based on deep learning. By designing reasonable downsampling and degradation methods, it effectively simulates clinical low-resolution brain sMRI, constructs a targeted data set, and improves the performance of the model when processing anisotropic data. After training and evaluating on simulated data, the model is fine-tuned using a small amount of real clinical data to improve the generalization performance of the model on real clinical data, effectively restore image quality, and retain image detail texture information to the greatest extent. See Figure 9 , the application examples of this application specifically include the following contents:

[0143] Step 1. Constructing a simulated dataset: Given the lack of one-to-one correspondence between high-resolution and low-resolution image data in real scenes, a simulated dataset needs to be constructed during model training. Figure 10 As shown, most of the downsampling is performed in an isotropic manner, and the low-resolution images produced in this way are difficult to simulate the low-resolution images of thick-slice scans in clinical practice.

[0144] For this issue, see Figure 11 This application first uses the method of mean downsampling in a certain direction of the structural magnetic resonance image to simulate anisotropic low-resolution images, and then constructs a training data set. However, this method will produce new technical defects when applied: First, the conventional single-direction mean downsampling only simulates the layer thickness parameters of the clinical scan, but does not consider the physical intervals between layers in the actual scan, resulting in significant differences between the generated low-resolution images and the actual clinical scan data. Second, this downsampling method destroys the spatial position correspondence between the high-resolution image and the low-resolution image, making it difficult for the deep learning model to establish an effective mapping relationship during the training process, significantly increasing the difficulty of model training.

[0145] Therefore, the application example of this application provides an improved solution for constructing a medical image super-resolution dataset based on clinical scanning principles:

[0146] 1) Improved downsampling algorithm for low-resolution thick-slice simulation: Based on mean downsampling, an inter-slice discarding strategy is added to address the problem that existing methods do not consider the gaps between layers. Specific mathematical expression: Assume that the total number of slices in the sagittal plane of the original high-resolution structural MRI image is N, the super-resolution factor is k, and the inter-slice discarding ratio is p. Then, the number of slices c actually involved in averaging in each k slice is:

[0147]

[0148] The corresponding number of discarded layers d is:

[0149]

[0150] The formula for low-resolution image generation is:

[0151]

[0152] in is the mth layer in the generated low-resolution image, is the nth layer in the original high-resolution structural magnetic resonance image.

[0153] 2) Improved super-resolution dataset construction method

[0154] Using only the downsampling algorithm in 1) to construct a super-resolution dataset results in a lack of spatial correspondence between the high-resolution and low-resolution images, greatly increasing the difficulty of model training. Therefore, this application example further performs cubic spline interpolation on the low-resolution image to obtain an isotropic target low-resolution image and establishes a spatial correspondence between it and the high-resolution image:

[0155]

[0156] in, 、 and are the three-dimensional coordinates of the target low-resolution image after interpolation; The interpolated target low-resolution image is at coordinates The voxel value at , i, j and q are integer coordinates in the low-resolution image, d, e and l are relative position indexes in the cubic spline interpolation, ranging from -1 to 2, is the voxel value of the anisotropic low-resolution image, u, v, and w all represent the offset of the decimal part in the interpolation calculation, where , and Represents the coordinates of the current calculation point in the x-axis, y-axis, and z-axis directions respectively With the sampling point (d+m, e+n, q+l The relative distance between them, C(x) is a one-dimensional interpolation kernel function, and the specific expression is:

[0157]

[0158] The interpolation kernel function C(x) is used to distribute the relative distances according to (du), (ev) and (lw). The weights C(du), C(ev) and C(lw) are given in the following way: the closer the point is, the greater the weight it gets.

[0159] By adopting a specific spatial position preservation strategy, when generating an isotropic target low-resolution image, its correspondence with the original high-resolution structural magnetic resonance image in the three-dimensional spatial coordinate system is strictly maintained, thereby effectively reducing the training difficulty of the deep learning model and improving the accuracy of super-resolution reconstruction.

[0160] Step 2: Design a deep learning model:

[0161] First, many existing super-resolution techniques for structural MRI are derived from super-resolution techniques for natural images. However, this approach fails to fully consider the specific characteristics of structural MRI and underutilizes prior knowledge of the brain. To address this issue, a module is needed that can provide models with prior knowledge of the brain to guide them in performing brain image super-resolution. In neuroscience, prior knowledge of the brain is typically based on qualitative descriptions, while deep learning models require precise digital representations, resulting in inconsistencies in knowledge representation. Furthermore, how to minimize information loss and conveniently feed prior knowledge into the model is crucial. While segmentation results of brain anatomical structures (such as gray matter, white matter, and cerebrospinal fluid) can be used as additional input channels for the model, or MRI data from other modalities can be used as complementary information, there are currently no robust segmentation tools for low-resolution images, and training data with low-resolution images and corresponding segmentation labels is also scarce. Furthermore, because low-resolution images acquired in clinical practice are often monomodal, multimodal auxiliary information is difficult to obtain in practical applications. Therefore, the application example of this application designs an improved reference image module that does not require other modalities. By introducing the strategy of reference images, the model is provided with prior knowledge of the brain in a simple way. Specifically, a fixed reference structural magnetic resonance image of a healthy person (the same modality as the image to be super-resolution) is first denoised, and then aligned with the isotropic target low-resolution image constructed in step 1 after cubic spline interpolation as the reference image of the target low-resolution image. The reference image and the interpolated target low-resolution image have a point-to-point one-to-one correspondence, which can well provide the model with the approximate anatomical structure of each brain region, and can be input into the model together as a dual-channel image sample in a simple splicing manner. Among them, the registration method used is the symmetric normalization mode in ANTSpy, which has high registration accuracy, supports the registration of two-dimensional and three-dimensional images, and has high robustness and stability. The specific process of reference image generation is as follows:

[0162]

[0163]

[0164]

[0165] Wherein, D represents the mean downsampling process proposed in the application example of this application, S represents cubic spline interpolation, and T represents registration; represents a high-resolution structural magnetic resonance image; Represents a low-resolution image; represents the target low-resolution image; represents the reference image; represents the reference structural magnetic resonance image.

[0166] In terms of the model backbone, in order to reduce the loss of detailed textures and improve the reconstruction effect of complex structures, the deep learning model used in the application example of this application adopts a multi-scale strategy for image feature extraction. In the local feature extraction branch, 9 residual blocks ResBlock in the EDSR architecture are stacked to capture the local features of the image. This module is also used in the prior art for image super-resolution, but this branch only considers the local features of the image and lacks interaction with the overall information of the image. In response to this problem, those skilled in the art may easily think of using the Transformer layer to capture image context information, but since medical images are three-dimensional images, the convolutions within the model are all three-dimensional convolutions, and directly applying the Transformer layer will lead to problems of excessive computational complexity and excessive video memory usage. Therefore, the application example of this application designs an improved global feature extraction branch based on the Transformer layer, which is used successively 、 and The image is convolved with a three-dimensional convolution kernel, and the image is encoded to obtain a smaller feature map. An encoder with 8 Transformer layers is applied to the feature map to extract the global features of the image. Three upsampling and convolution modules are then used to restore the feature map to the original image size. The dimension of the hidden layer in each Transformer layer is 384, the dimension of the hidden layer in the feedforward neural network is 1536, and the number of heads in the multi-head attention mechanism is 8. Finally, the images of the two branches are added together to obtain the final super-resolution image. Compared with the existing sMRI super-resolution model that only focuses on local areas, the multi-scale feature extraction network proposed in the application example of this application fuses local and global features to achieve the complementarity of their information, which can improve the global consistency and overall quality of the image.

[0167] The loss function of the deep learning model adopts the target loss function mentioned in the above embodiment. In addition, the application example of this application adopts a sliding window strategy to cut the image into The image blocks of different sizes are fed into the model one by one for super-resolution, and then all the image blocks output by the model (i.e., the super-resolution images corresponding to each of the image samples) are spliced together to regain the target super-resolution image.

[0168] Step 3: Training and Evaluation on a Simulated Dataset: Training is performed on a simulated dataset. Structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used to evaluate the structural accuracy of the super-resolution images. The visual quality of the super-resolution images is evaluated using LPIPS.Alex, LPIPS.vgg, and LPIPS.squeeze. Common brain sMRI preprocessing tools such as CAT12 and Freesurfer are used for preprocessing. Gray and white matter are segmented, and the correlation between the gray and white matter of the super-resolution images and the original high-resolution images is calculated. LPIPS.Alex is a perceptual similarity metric (Learned Perceptual Image Patch Similarity, LPIPS) based on the AlexNet network architecture. It is used to evaluate the visual similarity between the super-resolution image and the original high-resolution image. LPIPS.vgg and LPIPS.squeeze are similar to LPIPS.Alex, but use different neural network architectures. LPIPS.vgg is a perceptual similarity metric based on the VGG network architecture, while LPIPS.squeeze is a perceptual similarity metric based on the SqueezeNet network architecture. CAT12 and Freesurfer are both commonly used magnetic resonance imaging analysis software.

[0169] Step 4: Construct a real clinical dataset and fine-tune the model: Existing brain sMRI super-resolution technologies are all trained on simulated datasets, and there are still differences between the simulated low-resolution data and the real clinical low-resolution data, which leads to the problem of poor generalization of the model on real data. In order to better improve the generalization performance of the model on real clinical low-resolution images, the application example of this application uses the model trained on the simulated dataset as a pre-training model, and uses the parameters of the model as the initial parameters to perform training and fine-tuning on a small amount of clinical datasets. Similar to step 1, the low-resolution image is first interpolated using cubic spline interpolation. In order to ensure that the high-resolution image and the interpolated image correspond point-to-point, the high-resolution image is aligned with the interpolated image to obtain image data that can be used for fine-tuning. In addition, it is also necessary to use a fixed healthy human brain sMRI to align with the low-resolution interpolated image, and use it as a reference image to input the model together with the low-resolution image for fine-tuning.

[0170] Step 5: Evaluation on a real clinical dataset: Because super-resolution images and true high-resolution images do not have a one-to-one correspondence between voxel and voxel position, it is impossible to calculate the correlation between the super-resolution images and the original high-resolution images in gray matter and white matter using the evaluation method on the simulated dataset. Therefore, this application example uses the calculation of the correlation between the whole-brain gray matter volume and white matter volume of the super-resolution images and the original high-resolution images, and the calculation of the correlation between the gray and white matter volumes and cortical thickness of the brain regions by registering the images to the template space for evaluation.

[0171] That is to say, the application example of this application provides a method for constructing a training data set: previous deep learning models may have constructed isotropic data sets to train deep learning models. The mean downsampling and discarding strategies used in the application example of this application can better simulate clinical low-resolution images, and at the same time perform cubic spline interpolation on the low-resolution images to obtain isotropic low-resolution images, and establish a spatial correspondence between them and high-resolution images. The application example of this application also provides a reference image guidance module: the application example of this application uses a fixed healthy human brain sMRI image to align with the low-resolution interpolation image as the reference image of the image, and inputs the model together with the interpolation image as a dual-channel image. The application example of this application also provides a multi-scale feature extraction network: the multi-scale feature extraction network proposed in the application example of this application realizes multi-dimensional analysis of brain images from global features and local features, and captures the structure of the brain from more dimensions. The application example of this application also provides a strategy for training on a simulated dataset and fine-tuning on a real clinical dataset: the application example of this application first trains on a simulated dataset of brain sMRI, and then fine-tunes on a small amount of real brain sMRI clinical dataset, which improves the generalization effect of the model in real application scenarios.

[0172] In summary, the structural magnetic resonance super-resolution image reconstruction model training method provided by the application example of this application has the following beneficial effects:

[0173] 1. The application example of this application uses mean downsampling combined with the strategy of discarding certain layers to construct an anisotropic dataset for training, which can better simulate clinical low-resolution images.

[0174] 2. This application example designs a reference image guidance module, which provides a large amount of prior knowledge of the brain for the deep learning model, making the model more accurate in reconstructing complex structures.

[0175] 3. The application example of this application uses a multi-scale strategy for image feature extraction, realizing multi-dimensional analysis of brain images from global and local features, capturing the structure of the brain from more dimensions, improving the effect and global uniformity of the images generated by the model, and the fusion mechanism of multi-scale features can effectively balance the reconstruction quality of local details and global structure.

[0176] 4. This application example utilizes a fine-tuning strategy on a small number of real clinical datasets, improving the model's generalization in real-world application scenarios. Fine-tuning on a small amount of data allows the model to quickly learn the distribution of image features in specific scenarios. While maintaining its original performance, it further improves reconstruction in the target scenario, providing a more practical solution for clinical practice.

[0177] Based on the above embodiments and / or application examples of the structural magnetic resonance super-resolution image reconstruction model training method, the present application also provides an embodiment of a structural magnetic resonance super-resolution image reconstruction method, see Figure 12 The structural magnetic resonance super-resolution image reconstruction method specifically includes the following contents:

[0178] Step 300: registering the reference structural magnetic resonance image with the current clinical structural low-resolution magnetic resonance image to obtain a reference image corresponding to the clinical structural low-resolution magnetic resonance image;

[0179] Step 400: dividing the clinical structure low-resolution MRI image and the reference image into image blocks of the same size to obtain image blocks corresponding to the clinical structure low-resolution MRI image and image blocks corresponding to the reference image;

[0180] Step 500: performing one-to-one splicing of each image block corresponding to the clinical structure low-resolution MRI image and each image block corresponding to the reference image to obtain image samples corresponding to the clinical structure low-resolution MRI image;

[0181] Step 600: inputting each image sample corresponding to the clinical structural MRI low-resolution image into a structural MRI super-resolution image reconstruction model, so that the structural MRI super-resolution image reconstruction model outputs a super-resolution image of each image sample corresponding to the clinical structural MRI low-resolution image, wherein the structural MRI super-resolution image reconstruction model is pre-trained based on the structural MRI super-resolution image reconstruction model training method;

[0182] Step 700: recombining the super-resolution images of the image samples corresponding to the clinical structure MRI low-resolution image to obtain a target super-resolution image corresponding to the clinical structure MRI low-resolution image.

[0183] The structural magnetic resonance super-resolution image reconstruction model training method mentioned in the embodiment of the structural magnetic resonance super-resolution image reconstruction method provided in the embodiment of the present application can specifically adopt the processing flow of the embodiment of the structural magnetic resonance super-resolution image reconstruction model training method in the above-mentioned embodiment and / or application example. Its functions are not repeated here, and reference can be made to the detailed description of the above-mentioned structural magnetic resonance super-resolution image reconstruction model training method embodiment.

[0184] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction method provided in the embodiment of the present application can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects and loss of detailed textures, and can improve the effectiveness and reliability of using image samples to train the structural magnetic resonance super-resolution image reconstruction model, thereby improving the accuracy and reliability of reconstructing super-resolution images using the model.

[0185] From the software level, the present application also provides a structural magnetic resonance super-resolution image reconstruction model training device for executing all or part of the structural magnetic resonance super-resolution image reconstruction model training method, see Figure 13 The structural magnetic resonance super-resolution image reconstruction model training device specifically includes the following contents:

[0186] A simulation data construction module 10 is configured to perform anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used to simulate a clinical structural magnetic resonance low-resolution image;

[0187] The model training module 20 is used to generate image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, and to train a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image.

[0188] The embodiment of the structural magnetic resonance super-resolution image reconstruction model training device provided in this application can be specifically used to execute the processing flow of the embodiment of the structural magnetic resonance super-resolution image reconstruction model training method in the above-mentioned embodiment. Its functions will not be repeated here, and reference can be made to the detailed description of the above-mentioned structural magnetic resonance super-resolution image reconstruction model training method embodiment.

[0189] The part of the structural magnetic resonance super-resolution image reconstruction model training device that performs structural magnetic resonance super-resolution image reconstruction model training can be executed in the server or completed in the client device. The specific selection can be made based on the processing power of the client device and the limitations of the user's usage scenario. This application is not limited to this. If all operations are completed in the client device, the client device may also include a processor for the specific processing of the structural magnetic resonance super-resolution image reconstruction model training.

[0190] The aforementioned client device may include a communication module (i.e., a communication unit) capable of establishing a communication connection with a remote server to facilitate data transmission with the server. The server may include a server at the task scheduling center or, in other implementation scenarios, a server on an intermediate platform, such as a server on a third-party server platform that is communicatively linked to the task scheduling center server. The server may comprise a single computer device, a server cluster consisting of multiple servers, or a distributed server configuration.

[0191] The server and the client device may communicate using any suitable network protocol, including network protocols that have not yet been developed as of the filing date of this application. Examples of such network protocols include TCP / IP, UDP / IP, HTTP, and HTTPS. Furthermore, examples of such network protocols include RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols, which are used on top of the aforementioned protocols.

[0192] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction model training device provided in the embodiment of the present application can improve the effectiveness and rationality of image samples in simulating low-resolution images of clinical medium-thick layer scans, and can improve the effectiveness and reliability of using image samples to train the structural magnetic resonance super-resolution image reconstruction model, thereby improving the accuracy and reliability of reconstructing super-resolution images using this model.

[0193] From the software level, the present application also provides a structural magnetic resonance super-resolution image reconstruction device for executing all or part of the structural magnetic resonance super-resolution image reconstruction method, see Figure 14 The structural magnetic resonance super-resolution image reconstruction device specifically includes the following contents:

[0194] An image registration module 30 is configured to register the reference structural magnetic resonance image with the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image;

[0195] an image segmentation module 40 for segmenting the clinical structure low-resolution MRI image and the reference image into image blocks of the same size, to obtain image blocks corresponding to the clinical structure low-resolution MRI image and image blocks corresponding to the reference image;

[0196] An image stitching module 50 is configured to stitch each image block corresponding to the clinical structure low-resolution MRI image with each image block corresponding to the reference image in a one-to-one manner, so as to obtain image samples corresponding to the clinical structure low-resolution MRI image;

[0197] an image reconstruction module 60, configured to input each image sample corresponding to the clinical structural MRI low-resolution image into a structural MRI super-resolution image reconstruction model, so that the structural MRI super-resolution image reconstruction model outputs a super-resolution image of each image sample corresponding to the clinical structural MRI low-resolution image, wherein the structural MRI super-resolution image reconstruction model is pre-trained based on the structural MRI super-resolution image reconstruction model training method;

[0198] The image recombining module 70 is configured to recombine the super-resolution images of the image samples corresponding to the clinical structure MRI low-resolution image to obtain a target super-resolution image corresponding to the clinical structure MRI low-resolution image.

[0199] The embodiment of the structural magnetic resonance super-resolution image reconstruction device provided in the present application can be specifically used to execute the processing flow of the embodiment of the structural magnetic resonance super-resolution image reconstruction method in the above-mentioned embodiment. Its functions will not be described in detail here, and reference can be made to the detailed description of the above-mentioned structural magnetic resonance super-resolution image reconstruction method embodiment.

[0200] The portion of the structural MRI super-resolution image reconstruction apparatus that performs structural MRI super-resolution image reconstruction can be executed in a server or a client device. The specific selection can be based on the processing capabilities of the client device and the limitations of the user's usage scenario. This application does not impose any restrictions on this.

[0201] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction device provided in the embodiment of the present application can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects and loss of detailed textures, and can improve the effectiveness and reliability of using image samples to train the structural magnetic resonance super-resolution image reconstruction model, thereby improving the accuracy and reliability of reconstructing super-resolution images using the model.

[0202] The present application also provides an electronic device that may include a processor, a memory, a receiver, and a transmitter. The processor is configured to execute the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method mentioned in the above embodiments. The processor and the memory may be connected via a bus or other means, with bus connection being used as an example. The receiver may be connected to the processor and the memory via a wired or wireless means.

[0203] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0204] The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as the program instructions / modules corresponding to the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method in the embodiments of the present application. The processor executes the various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory, that is, implementing the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method in the above-mentioned method embodiments.

[0205] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0206] The one or more modules are stored in the memory, and when executed by the processor, perform the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method in the embodiment.

[0207] In some embodiments of the present application, the user equipment may include a processor, a memory and a transceiver unit, and the transceiver unit may include a receiver and a transmitter. The processor, memory, receiver and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0208] As an implementation method, the functions of the receiver and transmitter in this application can be considered to be implemented through a transceiver circuit or a dedicated transceiver chip, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general-purpose chip.

[0209] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiments of the present application. That is, the program code for implementing the functions of the processor, receiver, and transmitter is stored in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0210] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned structural magnetic resonance super-resolution image reconstruction model training method and / or structural magnetic resonance super-resolution image reconstruction method. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, floppy disk, hard disk, removable storage disk, CD-ROM, or any other form of storage medium known in the art.

[0211] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned structural magnetic resonance super-resolution image reconstruction model training method and / or structural magnetic resonance super-resolution image reconstruction method.

[0212] It should be understood by those skilled in the art that the various exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether it is implemented in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted on a transmission medium or communication link via a data signal carried in a carrier.

[0213] It should be understood that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated. Those skilled in the art can make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present application.

[0214] In this application, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0215] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Those skilled in the art will appreciate that various modifications and variations of the present embodiment are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A structural magnetic resonance super-resolution image reconstruction model training method, characterized in that: include: performing anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image; Registering a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image with each of the target low-resolution images to obtain a reference image corresponding to each of the target low-resolution images; generating image samples corresponding to the target low-resolution images according to the target low-resolution images and the corresponding reference images, and training a deep learning model based on the image samples and the structural magnetic resonance images corresponding to the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images; The anisotropic downsampling process is performed on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image, including: performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal plane direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images; performing cubic spline interpolation on each of the low-resolution images to obtain a target low-resolution image corresponding to each of the low-resolution images for simulating a clinical structural magnetic resonance low-resolution image; The performing of mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal plane direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images includes: Dividing each layer of each structural magnetic resonance image in the sagittal plane direction into a plurality of first layer groups, wherein the number of layers in each first layer group is a preset super-resolution multiple; performing inter-layer discarding processing on each of the first layer groups according to the super-resolution multiple and a preset inter-layer discarding ratio to reduce the number of layers in each of the first layer groups, thereby obtaining a second layer group corresponding to each of the first layer groups; Mean downsampling processing is performed on each of the second layer groups respectively to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

2. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 1, characterized in that: The step of training a deep learning model based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images, comprises: Based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, a deep learning model is trained with a preset target loss function to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

3. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 2, characterized in that: Generating image samples corresponding to the target low-resolution images according to the target low-resolution images and the corresponding reference images includes: Dividing each of the target low-resolution images and each of the reference images into image blocks of the same size to obtain image blocks corresponding to each of the target low-resolution images and image blocks corresponding to each of the reference images; Each of the target low-resolution images and its corresponding reference image is used as a sample group, and each image block corresponding to the target low-resolution image in each of the sample groups is spliced one-to-one with each image block corresponding to the reference image to obtain image samples corresponding to each of the sample groups.

4. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 3, characterized in that: The deep learning model includes: a local feature extraction branch, a global feature extraction branch and a feature addition layer; Correspondingly, the deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples using a preset target loss function, including: Performing preset iterative training steps in sequence for each sample group, wherein the iterative training steps include: Inputting each of the image samples corresponding to the current sample group into the local feature extraction branch and the global feature extraction branch in the current deep learning model, respectively, so that the local feature extraction branch extracts local feature data corresponding to each of the image samples, respectively, so that the global feature extraction branch extracts global feature data corresponding to each of the image samples, and so that the feature addition layer adds the local feature data and the global feature data corresponding to each of the image samples, respectively, to obtain a super-resolution image corresponding to each of the image samples; Recombining the super-resolution images of the image samples corresponding to the current sample group to obtain a target super-resolution image corresponding to the current sample group; Based on the target super-resolution image and the structural magnetic resonance image corresponding to the sample group, the current loss of the deep learning model is calculated using a preset target loss function, and the deep learning model is optimized based on the loss to obtain the deep learning model in the next iterative training step, wherein the target loss function includes a weighted sum of a mean square error loss function and a perceptual loss function with different weights.

5. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 4, characterized in that: The local feature extraction branch includes: A first convolutional layer is used to perform high-dimensional feature space mapping on the image sample, adjust the format and dimension, and output corresponding first feature data; a local feature extraction unit, configured to perform local feature extraction on the first feature data based on a plurality of stacked residual blocks, and output corresponding second feature data; The second convolutional layer is used to adjust the number of channels of the second feature data and output corresponding local feature data.

6. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 4, characterized in that: The global feature extraction branch includes: An encoder, configured to encode the image sample based on three sequentially connected encoding layers and output corresponding encoded feature data; wherein the encoding layers include: a connected convolutional layer and a ReLU activation function; A global feature extraction unit, configured to perform global feature extraction on the encoded feature data based on a plurality of sequentially connected Transformer layers, and output corresponding global encoded feature data; A decoder is used to decode the global encoded feature data based on three sequentially connected decoding layers and output corresponding global feature data, wherein the encoding layer includes: an upsampling layer, a convolution layer and a ReLU activation function connected in sequence.

7. A structural magnetic resonance super-resolution image reconstruction method, characterized in that: include: Registering the reference structural magnetic resonance image with the current clinical structural low-resolution magnetic resonance image to obtain a reference image corresponding to the clinical structural low-resolution magnetic resonance image; Dividing the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image; Performing one-to-one splicing of each image block corresponding to the clinical structure low-resolution magnetic resonance image and each image block corresponding to the reference image to obtain image samples corresponding to the clinical structure low-resolution magnetic resonance image; inputting the image samples corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model, respectively, so that the structural magnetic resonance super-resolution image reconstruction model outputs super-resolution images of the image samples corresponding to the clinical structural magnetic resonance low-resolution image, respectively, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method according to any one of claims 1 to 6; The super-resolution images of the image samples corresponding to the clinical structure low-resolution magnetic resonance image are recombined to obtain a target super-resolution image corresponding to the clinical structure low-resolution magnetic resonance image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the structural magnetic resonance super-resolution image reconstruction model training method according to any one of claims 1 to 6, and / or implements the structural magnetic resonance super-resolution image reconstruction method according to claim 7.

Images