Structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and equipment

Abstract

The invention provides a structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and equipment, and relates to the technical field of image processing, and the training method comprises the steps: carrying out the anisotropic down-sampling of a structural magnetic resonance image, so as to obtain a target low-resolution image for simulating a clinical structural magnetic resonance low-resolution image; and generating an image sample corresponding to the target low-resolution image based on the reference structure magnetic resonance image, and training a deep learning model based on the image sample and the structure magnetic resonance image so as to train the deep learning model into a structure magnetic resonance super-resolution image reconstruction model used for reconstructing the image sample into a super-resolution image. According to the invention, the effectiveness and rationality of the low-resolution image of the thick layer scanning in the image sample simulation clinic can be improved, the effectiveness and reliability of the magnetic resonance super-resolution image reconstruction model adopting the image sample training structure can be improved, and the precision and reliability of the super-resolution image reconstruction by adopting the model can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a structural magnetic resonance super-resolution image reconstruction model training method, reconstruction method and equipment. Background Art

[0002] Structural Magnetic Resonance Imaging (sMRI) is used to depict the three-dimensional structural information of the human brain, including the volume of various brain tissues, the thickness and surface area of ​​the cortex, etc. In order to obtain more information about the brain, researchers mostly use high-resolution scans with smaller scanning slice thickness, which can calculate various indicators more accurately. However, in clinical applications, due to time and money cost constraints, hospitals mostly use low-resolution scans with larger scanning slice thicknesses for patients, which produces anisotropic images with lower resolution in the slice selection direction than in the intra-slice direction. Due to the lack of preprocessing tools specifically for clinical sequence sMRI, the structural magnetic resonance images of a large number of clinical patients cannot be used for scientific research, which to some extent limits the development of treatment for various diseases. Therefore, in order to bridge the gap between structural magnetic resonance images in research sequences and clinical sequences, it is particularly important to improve the resolution of clinical structural magnetic resonance images.

[0003] At present, the existing methods to improve the resolution of structural MRI images are mainly super-resolution (SR) reconstruction algorithms, which use one or more low-resolution structural MRI images to eliminate degradation interference such as blur encountered during signal acquisition through signal processing, image processing, and machine learning techniques, thereby predicting the real structural MRI super-resolution image to restore the details and high-frequency information lost during the acquisition of clinical structural MRI low-resolution images.

[0004] However, the training data used in the existing deep learning-based structural MRI super-resolution image reconstruction methods are significantly different from the anisotropic structural MRI image characteristics actually collected in clinical practice. Therefore, the accuracy and reliability of the structural MRI super-resolution images reconstructed by the trained deep learning model cannot be guaranteed. Summary of the invention

[0005] In view of this, the embodiments of the present application provide a structural magnetic resonance super-resolution image reconstruction model training method, a reconstruction method and a device to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction model training method, comprising: Performing anisotropic downsampling processing on each structural magnetic resonance image respectively, so as to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used for simulating a clinical structural magnetic resonance low-resolution image; Based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, image samples corresponding to each of the target low-resolution images are generated respectively, and a deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

[0007] In some embodiments of the present application, the anisotropic downsampling process is performed on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image, including: Performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal plane direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images; Cubic spline interpolation is performed on each of the low-resolution images to obtain a target low-resolution image corresponding to each of the low-resolution images and used to simulate a clinical structural magnetic resonance low-resolution image.

[0008] In some embodiments of the present application, the step of performing mean downsampling processing based on an inter-layer discarding strategy on each layer of each structural magnetic resonance image in the sagittal plane direction to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images includes: Dividing each layer of each structural magnetic resonance image in the sagittal plane direction into a plurality of first layer groups, wherein the number of layers in each first layer group is a preset super-resolution multiple; According to the super-resolution multiple and the preset inter-layer discarding ratio, each of the first layer groups is subjected to inter-layer discarding processing to reduce the number of layers in each of the first layer groups, thereby obtaining a second layer group corresponding to each of the first layer groups; Mean downsampling processing is performed on each of the second layer groups respectively to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

[0009] In some embodiments of the present application, the method of generating image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, and training a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model as a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image, comprises: Registering a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image to each of the target low-resolution images, respectively, to obtain a reference image corresponding to each of the target low-resolution images; Generate image samples corresponding to each of the target low-resolution images according to each of the target low-resolution images and the corresponding reference images; Based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, a deep learning model is trained with a preset target loss function to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

[0010] In some embodiments of the present application, generating image samples corresponding to each of the target low-resolution images according to each of the target low-resolution images and the corresponding reference images respectively includes: Dividing each of the target low-resolution images and each of the reference images into image blocks of the same size to obtain image blocks corresponding to each of the target low-resolution images and image blocks corresponding to each of the reference images; Each of the target low-resolution images and the corresponding reference image are respectively used as sample groups, and each image block corresponding to the target low-resolution image in each of the sample groups is spliced ​​one-to-one with each image block corresponding to the reference image, so as to obtain each image sample corresponding to each of the sample groups.

[0011] In some embodiments of the present application, the deep learning model includes: a local feature extraction branch, a global feature extraction branch, and a feature addition layer; Correspondingly, the deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples with a preset target loss function, including: For each sample group, a preset iterative training step is sequentially performed, wherein the iterative training step includes: Inputting each of the image samples corresponding to the current sample group into the local feature extraction branch and the global feature extraction branch in the current deep learning model, respectively, so that the local feature extraction branch extracts local feature data corresponding to each of the image samples, respectively, so that the global feature extraction branch extracts global feature data corresponding to each of the image samples, and the feature addition layer adds the local feature data and the global feature data corresponding to each of the image samples, respectively, to obtain super-resolution images corresponding to each of the image samples; Recombining the super-resolution images of the respective image samples corresponding to the current sample group to obtain a target super-resolution image corresponding to the current sample group; Based on the target super-resolution image and the structural magnetic resonance image corresponding to the sample group, the current loss of the deep learning model is calculated using a preset target loss function, and the deep learning model is optimized based on the loss to obtain the deep learning model in the next iterative training step, wherein the target loss function includes a weighted sum of a mean square error loss function and a perceptual loss function with different weights.

[0012] In some embodiments of the present application, the local feature extraction branch includes: A first convolutional layer is used to perform high-dimensional feature space mapping on the image sample and adjust the format and dimension, and output corresponding first feature data; A local feature extraction unit, configured to perform local feature extraction on the first feature data based on a plurality of stacked residual blocks, and output corresponding second feature data; The second convolutional layer is used to adjust the number of channels of the second feature data and output corresponding local feature data.

[0013] In some embodiments of the present application, the global feature extraction branch includes: An encoder, configured to encode the image sample based on three sequentially connected encoding layers and output corresponding encoding feature data; wherein the encoding layer comprises: a connected convolutional layer and a ReLU activation function; A global feature extraction unit, used to extract global features from the encoded feature data based on a plurality of sequentially connected Transformer layers, and output corresponding global encoded feature data; A decoder is used to decode the global encoded feature data based on three sequentially connected decoding layers and output corresponding global feature data, wherein the encoding layer includes: an upsampling layer, a convolutional layer and a ReLU activation function connected in sequence.

[0014] The second aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction method, comprising: Registering the reference structural magnetic resonance image to the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image; Dividing the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size respectively, so as to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image; Respectively performing one-to-one splicing of each image block corresponding to the clinical structure low-resolution magnetic resonance image and each image block corresponding to the reference image to obtain each image sample corresponding to the clinical structure low-resolution magnetic resonance image; Inputting the image samples corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model respectively, so that the structural magnetic resonance super-resolution image reconstruction model outputs super-resolution images of the image samples corresponding to the clinical structural magnetic resonance low-resolution image respectively, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method mentioned in the first aspect; The super-resolution images of the image samples corresponding to the clinical structure magnetic resonance low-resolution image are recombined to obtain a target super-resolution image corresponding to the clinical structure magnetic resonance low-resolution image.

[0015] The third aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction model training device, comprising: A simulation data construction module is used to perform anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used to simulate a clinical structural magnetic resonance low-resolution image; A model training module is used to generate image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image with the same resolution as the structural magnetic resonance image, and to train a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image.

[0016] A fourth aspect of the present application provides a structural magnetic resonance super-resolution image reconstruction device, comprising: An image registration module, used for registering the reference structural magnetic resonance image with the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image; An image segmentation module, used for segmenting the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size, so as to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image; An image stitching module, used for stitching each image block corresponding to the clinical structure low-resolution magnetic resonance image with each image block corresponding to the reference image one-to-one, so as to obtain each image sample corresponding to the clinical structure low-resolution magnetic resonance image; An image reconstruction module, used for inputting the image samples corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model, so that the structural magnetic resonance super-resolution image reconstruction model outputs super-resolution images of the image samples corresponding to the clinical structural magnetic resonance low-resolution image, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method provided in the first aspect; The image recombining module is used to recombine the super-resolution images of the image samples corresponding to the clinical structure magnetic resonance low-resolution image to obtain a target super-resolution image corresponding to the clinical structure magnetic resonance low-resolution image.

[0017] The fifth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the structural magnetic resonance super-resolution image reconstruction model training method provided in the first aspect is implemented, and / or the structural magnetic resonance super-resolution image reconstruction method provided in the second aspect is implemented.

[0018] The sixth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it implements the structural magnetic resonance super-resolution image reconstruction model training method, and / or implements the structural magnetic resonance super-resolution image reconstruction method provided in the aforementioned second aspect.

[0019] The seventh aspect of the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the structural magnetic resonance super-resolution image reconstruction model training method, and / or implements the structural magnetic resonance super-resolution image reconstruction method provided in the aforementioned second aspect.

[0020] The structural magnetic resonance super-resolution image reconstruction model training method provided in the present application performs anisotropic downsampling processing on each structural magnetic resonance image respectively to obtain a target low-resolution image corresponding to each of the structural magnetic resonance images for simulating a clinical structural magnetic resonance low-resolution image; based on a reference structural magnetic resonance image with the same resolution as the structural magnetic resonance image, image samples corresponding to each of the target low-resolution images are generated respectively, and a deep learning model is trained based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image, and by designing reasonable downsampling and degradation methods, simulating the resolution loss in clinical imaging, and constructing an appropriate amount of training data sets, the effectiveness and rationality of the image samples in simulating the low-resolution images of clinical medium and thick layer scans can be improved, so that the model has better robustness when processing anisotropic data. Deep learning technology is used to automatically extract features and provide the model with prior knowledge of brain structure in the form of designed reference images. This can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects, and loss of detailed textures, and improve the effectiveness and reliability of the structural magnetic resonance super-resolution image reconstruction model trained using image samples, thereby improving the accuracy and reliability of super-resolution images reconstructed using this model.

[0021] Additional advantages, purposes, and features of the present application will be partially described in the following description, and will become partially apparent to those skilled in the art after studying the following, or may be learned from the practice of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the specification and the drawings.

[0022] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present application are not limited to the above specific description, and the above and other purposes that can be achieved by the present application will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application. The components in the drawings are not drawn to scale, but are only for the purpose of illustrating the principles of the present application. In order to facilitate the illustration and description of some parts of the present application, the corresponding parts in the drawings may be enlarged, that is, they may become larger relative to other components in the exemplary device actually manufactured according to the present application. In the drawings: Figure 1 This is a schematic diagram of a first flow chart of a structural magnetic resonance super-resolution image reconstruction model training method in one embodiment of the present application.

[0024] Figure 2 This is a second flow chart of the structural magnetic resonance super-resolution image reconstruction model training method in one embodiment of the present application.

[0025] Figure 3 Schematic diagram of the sagittal plane orientation of a structural magnetic resonance image.

[0026] Figure 4 This is a third flow chart of the structural magnetic resonance super-resolution image reconstruction model training method in one embodiment of the present application.

[0027] Figure 5 Schematic diagram of the overall architecture of the machine learning model in one embodiment of the present application.

[0028] Figure 6 Schematic diagram of the architecture of the local feature extraction branch in the machine learning model in one embodiment of the present application.

[0029] Figure 7 A schematic diagram of the data processing logic of a machine learning model in one embodiment of the present application.

[0030] Figure 8 Schematic diagram of the architecture of the global feature extraction branch in the machine learning model in one embodiment of the present application.

[0031] Fig. 9 This is a flow chart of a structural magnetic resonance super-resolution image reconstruction model training method in an application example of the present application.

[0032] Fig.10 A schematic diagram of isotropic downsampling of a three-dimensional image in the prior art.

[0033] Fig.11 It is a schematic diagram of anisotropic downsampling of a structural magnetic resonance image in an application example of the present application.

[0034] Fig.12 Schematic diagram of the process of structural magnetic resonance super-resolution image reconstruction method in one embodiment of the present application.

[0035] Fig.13 Schematic diagram of the structure of a structural magnetic resonance super-resolution image reconstruction model training device in one embodiment of the present application.

[0036] Fig.14 Schematic diagram of the structure of a structural magnetic resonance super-resolution image reconstruction device in one embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the implementation modes and the accompanying drawings. Here, the illustrative implementation modes and descriptions of the present application are used to explain the present application, but are not intended to limit the present application.

[0038] It should also be noted here that in order to avoid obscuring the present application due to unnecessary details, only the structures and / or processing steps closely related to the scheme according to the present application are shown in the accompanying drawings, while other details that are not very relevant to the present application are omitted.

[0039] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0040] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0041] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0042] It should be noted that the current super-resolution image reconstruction methods for structural magnetic resonance images are mainly divided into the following four categories: 1. Interpolation-based algorithms: Interpolation methods use the linear relationship between adjacent pixels in a low-resolution image to estimate the grayscale values ​​of missing pixels in a high-resolution reconstructed image. Common interpolation methods include nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, and B-spline interpolation.

[0043] 2. Reconstruction-based algorithm: Model the MRI imaging process, and solve its inverse process by combining signal processing methods and optimization algorithms, so as to achieve image deblurring, upsampling, and denoising, so as to reconstruct the high-frequency details lost during MRI imaging. Magnetic Resonance Imaging (MRI) is a medical imaging technology based on the nuclear magnetic resonance phenomenon. During the MRI scanning process, multiple sequence signals need to be collected, resulting in a long acquisition time. Due to the difficulty of the subject to remain still for a long time, the scanned image often has artifacts, which affects the signal-to-noise ratio.

[0044] 3. Algorithm based on shallow learning: Model the inverse process of MRI imaging process, learn the mapping relationship between high-resolution and low-resolution images from the training set, and optimize the model parameters to minimize the error between the reconstructed prediction result and the actual high-resolution image.

[0045] 4. Deep learning-based algorithm: Use deep neural network to automatically learn the mapping function between low-resolution images and high-resolution images from a large amount of data, thereby achieving super-resolution of sMRI images.

[0046] Among them, since the model of the algorithm based on the interpolation method is too simple, there are problems such as blurred edges, jagged effects, loss of detailed textures, and inaccurate reconstruction of complex structures, it is difficult to ensure the optimality of the predicted image, and the help for image reconstruction is limited. In addition, the algorithm based on reconstruction has high computational complexity, and the convergence of the algorithm depends on the initial estimate. It is difficult to ensure the stability of the solution without sufficient prior information constraints. In addition, the algorithm based on shallow learning is limited by the quality and quantity of training data, and the feature extraction process is too dependent on the dictionary constructed by prior knowledge. Therefore, this application adopts a deep learning-based algorithm that can solve the above problems to perform super-resolution image reconstruction of structural magnetic resonance images. However, when constructing training data, most algorithms based on deep learning use isotropic downsampling to obtain low-resolution images, which is quite different from the anisotropic structural magnetic resonance image features actually collected in clinical practice. At the same time, existing algorithms lack prior knowledge of the brain to improve the effect of super-resolution.

[0047] In order to solve the problem that the training data used in the existing deep learning-based super-resolution image reconstruction method of structural magnetic resonance images is quite different from the anisotropic structural magnetic resonance image features actually collected in clinical practice, and therefore the accuracy and reliability of the deep learning model obtained by training to reconstruct the structural magnetic resonance super-resolution image cannot be guaranteed, the embodiments of the present application respectively provide a structural magnetic resonance super-resolution image reconstruction model training method, a structural magnetic resonance super-resolution image reconstruction model training device for executing the structural magnetic resonance super-resolution image reconstruction model training method, a structural magnetic resonance super-resolution image reconstruction model method, a structural magnetic resonance super-resolution image reconstruction device for executing the structural magnetic resonance super-resolution image reconstruction method, a physical device, a computer-readable storage medium and a computer program product, by designing a reasonable downsampling and degradation method, simulating the resolution loss in clinical imaging, constructing an appropriate amount of training data sets, so that the model has better robustness when processing anisotropic data. And using deep learning technology, automatically extracting features, and providing the model with prior knowledge of brain structure in the form of designing reference images, it can improve the reconstruction accuracy of complex brain structures and reduce edge blur, jagged effects and loss of detail textures.

[0048] The details are described in detail through the following examples.

[0049] Based on this, the present application embodiment provides a structural magnetic resonance super-resolution image reconstruction model training method that can be implemented by a structural magnetic resonance super-resolution image reconstruction model training device, see Figure 1The structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents: Step 100: performing anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image for simulating a clinical structural magnetic resonance low-resolution image.

[0050] In one or more embodiments of the present application, the structural magnetic resonance image refers to a structural magnetic resonance high-resolution image having a higher resolution than a clinical structural magnetic resonance low-resolution image. The clinical structural magnetic resonance low-resolution image refers to an anisotropic image in which the resolution of the slice selection direction is lower than that of the intra-slice direction, obtained by clinically using a low-resolution scan with a larger scanning slice thickness.

[0051] It is understandable that, in view of the lack of paired structural MRI high-resolution images and structural MRI low-resolution image data with one-to-one positional relationship in real scenes, it is necessary to construct a simulated data set in the process of model training. The data set simulated by the prior art is down-sampled in an isotropic manner, so the low-resolution images produced in this way are difficult to simulate the low-resolution images of clinical medium and thick layer scans. Therefore, in step 100 of the embodiment of the present application, each structural MRI image is anisotropically down-sampled, that is, the mean down-sampling method is used to simulate anisotropic low-resolution images in a certain direction of the structural MRI image, thereby effectively simulating the anisotropic images produced by clinical medium and thick layer scans. Among them, each direction refers to each axial direction in the three-dimensional coordinate system where the image is located.

[0052] Step 200: Based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, respectively generate image samples corresponding to each of the target low-resolution images, and train a deep learning model based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

[0053] In step 200, the reference structural MRI image refers to a structural MRI high-resolution image having the same resolution as each of the structural MRI images in step 100, and in one example, the reference structural MRI image can be a structural MRI high-resolution image of a healthy human brain that has been artificially evaluated.

[0054] It can be understood that the structural magnetic resonance image corresponding to each of the image samples is the structural magnetic resonance image corresponding to the target low-resolution image corresponding to the image sample.

[0055] In an example of step 200, after the deep learning model is iteratively trained with each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, the simulated training set composed of each of the image samples can also be evaluated (i.e., the effect of the super-resolution image output by the deep learning model trained with the simulated training set is evaluated, etc.), and then the deep learning model is further trained and fine-tuned using the constructed real clinical data set, and then the effect of the super-resolution image output by the fine-tuned deep learning model is evaluated on the real clinical data set, and finally the deep learning model whose evaluation result meets the preset accuracy requirement data is determined as the structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images. Then, after training and evaluating on the simulated data, the model can be fine-tuned using a small amount of real clinical data to improve the generalization performance of the model on real clinical data, effectively restore the image quality, and retain the image detail texture information to the greatest extent.

[0056] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application can improve the effectiveness and rationality of image samples simulating low-resolution images of clinical thick layer scans by designing reasonable downsampling and degradation methods, and constructing an appropriate amount of training data sets, so as to make the model more robust when processing anisotropic data. And by using deep learning technology to automatically extract features, and in the form of designing reference images, provide the model with prior knowledge of brain structure, which can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects and loss of detailed textures, and improve the effectiveness and reliability of using image samples to train structural magnetic resonance super-resolution image reconstruction models, thereby improving the accuracy and reliability of super-resolution images reconstructed using the model.

[0057] In order to further improve the effectiveness and rationality of image samples simulating low-resolution images of clinical medium-thick layer scans, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 2 , step 100 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents: Step 110: performing mean downsampling processing based on an inter-layer discarding strategy on each layer of each structural magnetic resonance image in the sagittal plane direction, so as to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

[0058] It is understood that the sagittal plane direction is the anatomical direction corresponding to the structural magnetic resonance super-resolution image, see Figure 3, which is perpendicular to the transverse and coronal planes and divides the plane of the human body from front to back (from ventral to dorsal), dividing the body into left and right parts. In brain anatomy, the sagittal plane is usually used to observe and study the structure and tissue between the left and right hemispheres of the brain. In other words, the conventional single-direction mean downsampling only simulates the layer thickness parameters of the clinical scan, but does not take into account the physical intervals between layers in the actual scan, resulting in significant differences between the generated low-resolution images and the actual clinical scan data. Step 110 of the embodiment of the present application, by adopting a mean downsampling process based on an inter-layer discarding strategy, can add an inter-layer discarding strategy to solve the problem that the existing method does not take into account the intervals between layers.

[0059] Step 120: Performing cubic spline interpolation on each of the low-resolution images to obtain a target low-resolution image corresponding to each of the low-resolution images for simulating a clinical structural magnetic resonance low-resolution image.

[0060] It should be noted that the downsampling method of step 110 destroys the spatial position correspondence between the structural magnetic resonance image and the low-resolution image, making it difficult for the deep learning model to establish an effective mapping relationship during the training process, which will significantly increase the difficulty of model training. Therefore, in order to solve this problem, step 120 of the embodiment of the present application further performs cubic spline interpolation on the low-resolution image to obtain an isotropic target low-resolution image and establishes a spatial correspondence between it and the structural magnetic resonance image. That is, when generating an isotropic target low-resolution image, its correspondence with the original high-resolution image (i.e., the structural magnetic resonance image) in the three-dimensional space coordinate system is strictly maintained, thereby effectively reducing the training difficulty of the deep learning model and improving the accuracy of super-resolution reconstruction.

[0061] In order to further improve the effectiveness and reliability of the mean downsampling process based on the inter-layer discarding strategy, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 4 , step 110 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents: Step 111: Divide each layer of each structural magnetic resonance image in the sagittal plane direction into a plurality of first layer groups, wherein the number of layers in each first layer group is a preset super-resolution multiple.

[0062] Specifically, the total number of layers of the structural magnetic resonance image in the sagittal plane direction is N, and the super-resolution multiple is k, then every k layers constitute a first layer group.

[0063] Step 112: According to the super-resolution multiple and the preset inter-layer discarding ratio, inter-layer discarding processing is performed on each of the first layer groups to reduce the number of layers in each of the first layer groups, so as to obtain second layer groups corresponding to each of the first layer groups.

[0064] Specifically, if the inter-layer discarding ratio is p, then the number of layers discarded (i.e. deleted) in each first layer group is d=k×p; the number of layers in the resulting second layer group is c=k×(1-p).

[0065] Step 113: performing mean downsampling processing on each of the second layer groups respectively, so as to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

[0066] In order to further reduce the loss of prior knowledge as much as possible and input it into the model in a simple form, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 2 , step 200 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents: Step 210: Registering a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image with each of the target low-resolution images to obtain a reference image corresponding to each of the target low-resolution images.

[0067] It can be understood that the registration of the reference structural magnetic resonance image with the same resolution as the structural magnetic resonance image to each of the target low-resolution images refers to seeking a (or a series of) spatial transformations for the reference structural magnetic resonance image so that it can be spatially consistent with the corresponding points on the target low-resolution image. This consistency means that the same anatomical point has the same spatial position on the two matching images. The result of the registration should match all anatomical points on the two images, or at least all points of diagnostic significance and points of surgical interest.

[0068] Specifically, many existing super-resolution technologies for brain sMRI are migrated from super-resolution technologies on natural images, but this approach fails to fully consider the particularity of brain sMRI images and does not make full use of prior knowledge of the brain. In order to solve this problem, it is necessary to design a module that can provide the model with prior knowledge of the brain to guide the model to better perform the task of brain image super-resolution. In the field of neuroscience, prior knowledge of the brain is usually based on qualitative descriptions, while deep learning models require precise digital forms, and there is an inconsistency in knowledge expression between the two. At the same time, how to minimize the loss of prior knowledge and feed it into the model in a simple form is also a very critical issue. Although the segmentation results of the anatomical structure of the brain (such as gray matter, white matter, cerebrospinal fluid) can be used as additional input channels of the model or MRI data of other modalities can be used as complementary information, there is currently no tool for robust segmentation of low-resolution images, and there is also a lack of training data for low-resolution images and corresponding segmentation labels. In addition, since the low-resolution images collected clinically are almost only single-modality, multi-modal auxiliary information is difficult to obtain in actual application scenarios. Therefore, step 210 of the embodiment of the present application designs an improved reference image strategy that does not require other modalities, and provides the model with prior knowledge of the brain in a simple way. Specifically, a reference structural magnetic resonance image of a fixed healthy person with the same modality as the structural magnetic resonance image is first denoised, and then aligned with the target low-resolution image as a reference image of the target low-resolution image. The reference image and the interpolated target low-resolution image have a point-to-point one-to-one correspondence, which can provide the model with the approximate anatomical structure of each brain region, and can be input into the model together as a dual-channel image in a simple splicing manner.

[0069] In one example, the registration method can use the symmetric normalization mode in ANTSpy, which has high registration accuracy, supports registration of two-dimensional and three-dimensional images, and has high robustness and stability. ANTSpy is a powerful open source Python library that is derived from the Advanced Neuroimaging with Templates and Statistics (ANTs) and focuses on data processing for medical imaging.

[0070] Step 220: Generate image samples corresponding to each of the target low-resolution images according to each of the target low-resolution images and the corresponding reference images.

[0071] Step 230: Based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, a deep learning model is trained with a preset target loss function to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

[0072] In order to further improve the execution reliability and effectiveness of the structural magnetic resonance super-resolution image reconstruction model training, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 4 , step 220 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents: Step 221: Divide each of the target low-resolution images and each of the reference images into image blocks of the same size to obtain image blocks corresponding to each of the target low-resolution images and image blocks corresponding to each of the reference images.

[0073] Step 222: each of the target low-resolution images and the corresponding reference image are respectively used as sample groups, and each image block corresponding to the target low-resolution image in each of the sample groups is spliced ​​one-to-one with each image block corresponding to the reference image, so as to obtain each image sample corresponding to each of the sample groups.

[0074] In one example, step 221 may adopt a sliding window strategy to cut the target low-resolution image and the reference image into The image blocks of the corresponding size are then spliced ​​into a dual-channel image in step 222 as image samples, so as to send each image sample to the deep learning model for super-resolution reconstruction, and then reassemble the super-resolution images corresponding to all the image samples output by the deep learning model to obtain the target super-resolution image corresponding to each sample group. In order to further improve the accuracy and reliability of the structural magnetic resonance super-resolution image reconstruction model to reconstruct the structural magnetic resonance super-resolution image, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 5 , the deep learning model includes: a local feature extraction branch, a global feature extraction branch and a feature addition layer, and the output ends of the local feature extraction branch and the global feature extraction branch are both connected to the input end of the feature addition layer; For the corresponding Figure 4 , step 230 in the structural magnetic resonance super-resolution image reconstruction model training method specifically includes the following contents: Step 231: Execute a preset iterative training step for each sample group in sequence, wherein the iterative training step includes: inputting each of the image samples corresponding to the current sample group into the local feature extraction branch and the global feature extraction branch in the current deep learning model, so that the local feature extraction branch extracts the local feature data corresponding to each of the image samples, and the global feature extraction branch extracts the global feature data corresponding to each of the image samples, and the feature addition layer adds the local feature data and the global feature data corresponding to each of the image samples, so as to obtain the super-resolution images corresponding to each of the image samples; recombining the super-resolution images of each of the image samples corresponding to the current sample group to obtain the target super-resolution image corresponding to the current sample group; based on the target super-resolution image corresponding to the sample group and the structural magnetic resonance image, calculating the current loss of the deep learning model with a preset target loss function, and optimizing the deep learning model based on the loss to obtain the deep learning model in the next iterative training step, wherein the target loss function includes a weighted sum of a mean square error loss function and a perceptual loss function with different weights.

[0075] In one example, recombining the super-resolution images of the respective image samples corresponding to the current sample group may be: The individual super-resolution images are stitched together to form The target super-resolution image.

[0076] Specifically, in the prior art, the super-resolution reconstruction task of structural magnetic resonance images usually uses a single mean square error (MSE) loss function for model training. However, the MSE loss function only focuses on the differences at the voxel level, which often leads to problems such as blurring and lack of details in the generated images, and cannot well maintain the high-frequency information and perceptual quality of the image. The model loss function of the embodiment of the present application is the weighted sum of the MSE loss function and the perceptual loss function, with weights of 1 and 0.1 respectively, where the perceptual loss uses a pre-trained Squeeze Net to calculate the feature map, where Squeeze Net is a convolutional neural network architecture that focuses on model lightweighting. Its core goal is to significantly reduce the number of parameters and the amount of calculation while ensuring model accuracy. And it is calculated in a slice-by-slice "pseudo-three-dimensional" form. This loss function ensures both the voxel-level accuracy and visual perception quality of the image, and the target loss function formula is as follows: in, Indicates loss; represents the voxel value of the output super-resolution image, represents the voxel value of the structural magnetic resonance image, T represents the total number of voxels, Q represents the total number of elements of the feature map used, represents the g-th feature layer of Squeeze Net, G represents the number of feature layers used, and H represents the number of slices used. and represent the hth slice (cross-sectional direction) of the super-resolution image and the structural magnetic resonance image, respectively.

[0077] In order to further improve the effectiveness and reliability of local feature extraction, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 6 and Figure 7 , the local feature extraction branch specifically includes a first convolutional layer, a local feature extraction unit and a second convolutional layer connected in sequence; The first convolutional layer is used to perform high-dimensional feature space mapping on the image sample and adjust the format and dimension, and output corresponding first feature data.

[0078] Specifically, the image samples input to the local feature extraction branch and the global feature extraction branch are two-channel three-dimensional images spliced ​​by the low-resolution interpolation image and the reference image. The shape of the input image sample is a five-dimensional tensor [batch_size, channels, length, width, height], where batch_size is the batch size, channels is the number of image channels, length is the image length, width is the image width, and height is the image height.

[0079] On this basis, after entering the local feature extraction branch, the image sample first passes through a first convolution layer to map the image sample to a high-dimensional feature space and adjust it to be consistent with the input format and dimension expected by the residual module to ensure the effectiveness of residual learning. The shape of the first feature data output by the first convolution layer is [batch_size, num_filters, length, width, height], where num_filters is the number of convolution kernels and the number of channels for outputting the first feature data.

[0080] The local feature extraction unit is used to perform local feature extraction on the first feature data based on a plurality of stacked residual blocks, and output corresponding second feature data.

[0081] In one example, the local feature extraction unit includes 9 stacked residual blocks.

[0082] Specifically, the first feature data output by the first convolutional layer is input into the local feature extraction unit, and the 9 stacked residual blocks ResBlock in the local feature extraction unit are composed of a convolutional layer Conv, a ReLU activation function and another convolutional layer Conv connected in sequence, and have a residual connection; the shape of the feature map output by each residual block ResBlock is [batch_size, num_filters, length, width, height], and the output of each residual block ResBlock enters another convolutional layer to aggregate the high-dimensional outputs of multiple residual blocks ResBlocks, and the shape of the output second feature data is [batch_size, num_filters, length, width, height].

[0083] The second convolutional layer is used to adjust the number of channels of the second feature data and output corresponding local feature data.

[0084] Specifically, the second feature data output by the local feature extraction unit is input into the second convolutional layer, so that the second convolutional layer adjusts the number of channels of the second feature data to 1, as the output of the local feature extraction branch, and the shape of the output local feature data is [batch_size, 1, length, width, height].

[0085] In order to further improve the effectiveness and reliability of global feature extraction, in a structural magnetic resonance super-resolution image reconstruction model training method provided in an embodiment of the present application, see Figure 7 and Figure 8 The global feature extraction branch specifically includes an encoder, a global feature extraction unit and a decoder connected in sequence.

[0086] The encoder is used to encode the image sample based on three sequentially connected encoding layers and output corresponding encoding feature data; wherein the encoding layer includes: a connected convolution layer and a ReLU activation function. In an example, the convolution layers in the three sequentially connected encoding layers can respectively adopt , and 3D convolution kernel.

[0087] Specifically, the image samples are input into the global feature extraction branch and then passed through an encoder consisting of three convolutional layers to encode the original dual-channel image samples. The shape of the convolutional feature map output by each convolutional layer is [batch_size, num_filters, new_length, new_width, new_length], where new_length, new_width, and new_height represent the length, width, and height of the convolutional feature map, respectively. The length, width, and height of the convolutional feature map gradually decrease. The shape of the encoded feature data output after three layers of convolution is [batch_size, feature_channels, feature_length, feature_width, feature_height]. Where feature_channels, feature_length, feature_width, and feature_height represent the number of feature channels, feature length, feature width, and feature height, respectively.

[0088] The global feature extraction unit is used to perform global feature extraction on the encoded feature data based on multiple sequentially connected Transformer layers and output corresponding global encoded feature data.

[0089] Specifically, the encoded feature data is input into multiple sequentially connected Transformer layers (Transformer Block) in the global feature extraction unit, and the global features of the image are extracted on the encoded feature data. The shape of the output global encoded feature data is [batch_size, feature_channels, feature_length, feature_width, feature_height].

[0090] Among them, Transformer is a sequence model based on the attention mechanism.

[0091] The decoder is used to decode the global encoded feature data based on three sequentially connected decoding layers and output corresponding global feature data, wherein the encoding layer includes: an upsampling layer, a convolutional layer and a ReLU activation function connected in sequence.

[0092] Specifically, the encoded feature data output by the global feature extraction unit is input into the decoder, and the three sequentially connected decoding layers of the decoder gradually restore the feature map to the original image size, and the output global feature data shape is [batch_size, 1, length, width, height].

[0093] It is understandable that the feature extraction branch in the prior art only considers the local features of the image and lacks interaction with the overall information of the image. To address this problem, although the Transformer layer can be used to capture the image context information, since medical images are three-dimensional images, the convolutions within the model are all three-dimensional convolutions. Directly applying the Transformer layer will lead to problems such as high computational complexity and excessive video memory usage. Therefore, this application designs an improved global feature extraction branch based on the Transformer layer, which is successively used , and The image is convolved with a three-dimensional convolution kernel, and the image is encoded to obtain a feature map of smaller size. An encoder with 8 Transformer layers is applied to the feature map to extract the global features of the image. Then, three upsampling and convolution modules are used to restore the feature map to the original image size. The dimension of the hidden layer in each Transformer layer is 384, the dimension of the hidden layer in the feedforward neural network is 1536, and the number of heads in the multi-head attention mechanism is 8. Finally, the images of the two branches are added together to obtain the final super-resolution image. Compared with the existing sMRI super-resolution model that only focuses on the local area, the multi-scale feature extraction network proposed in this application fuses local and global features to achieve the complementarity of the information between the two, which can improve the global consistency and overall quality of the image.

[0094] It should also be noted that the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application uses the mean downsampling and discarding strategy to construct anisotropic data sets. Adding some simple strategies such as Gaussian blur or not discarding is also within the protection scope of the application example of the present application; the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application uses a fixed healthy human brain sMRI image to align with the low-resolution interpolation image as the reference image of the image. Using multiple images or other brain maps for alignment as reference images is also within the protection scope of the application example of the present application; the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application realizes multi-dimensional analysis of brain images from global features and local features, and stacking more repeated local feature extraction branches or global feature extraction branches is also within the protection scope of the application example of the present application; the structural magnetic resonance super-resolution image reconstruction model training method provided in the embodiment of the present application adopts the strategy of training on the current simulation data set and then fine-tuning on the real clinical data set, and adopts pre-training-fine-tuning multiple alternating training or only single-stage training on simulation or clinical data sets is also within the protection scope of the application example of the present application.

[0095] To further illustrate the above embodiments, the present application also provides a specific application example of a structural magnetic resonance super-resolution image reconstruction model training method. The application example of the present application provides a low-resolution structural magnetic resonance image super-resolution algorithm based on deep learning. By designing reasonable downsampling and degradation methods, it effectively simulates clinical low-resolution brain sMRI, constructs a targeted data set, and improves the performance of the model in processing anisotropic data. After training and evaluating on simulated data, the model is fine-tuned using a small amount of real clinical data to improve the generalization performance of the model on real clinical data, effectively restore image quality, and retain image detail texture information to the greatest extent. See Fig. 9 The application examples of this application specifically include the following contents: Step 1. Construct a simulated data set: In view of the lack of one-to-one position relationship in real scenes, it is necessary to construct a simulated data set during model training. The data set simulated by the existing technology, such as Fig.10 As shown, most of them are down-sampled in an isotropic manner, and the low-resolution images produced in this way are difficult to simulate the low-resolution images of thick-slice scans in clinical practice.

[0096] For this question, see Fig.11 , this application first adopts the method of mean downsampling in a certain direction of the structural magnetic resonance image to simulate anisotropic low-resolution images, and then constructs a training data set, but this method will produce new technical defects when applied: First, the conventional single-direction mean downsampling only simulates the layer thickness parameters of the clinical scan, but does not consider the physical interval between layers in the actual scan, resulting in significant differences between the generated low-resolution images and the actual clinical scan data. Second, this downsampling method destroys the spatial position correspondence between the high-resolution image and the low-resolution image, making it difficult for the deep learning model to establish an effective mapping relationship during the training process, significantly increasing the difficulty of model training.

[0097] Therefore, the application example of this application provides an improved medical image super-resolution dataset construction solution based on clinical scanning principles: 1) Improved downsampling algorithm for low-resolution thick layer simulation: Based on the mean downsampling, an inter-layer discarding strategy is added to solve the problem that the existing method does not consider the interval between layers. Specific mathematical expression: Assume that the total number of layers of the original high-resolution structural magnetic resonance image in the sagittal plane is N, the super-resolution multiple is k, and the inter-layer discarding ratio is p, then the number of layers c actually involved in the average in each k layer is: The corresponding number of discarded layers d is: The formula for low-resolution image generation is: in is the mth layer in the generated low-resolution image, is the nth layer in the original high-resolution structural magnetic resonance image.

[0098] 2) Improved super-resolution dataset construction method Using only the downsampling algorithm in 1) to construct a super-resolution algorithm dataset will result in a lack of spatial correspondence between high-resolution images and low-resolution images, greatly increasing the difficulty of model training. Therefore, the application example of this application further performs cubic spline interpolation on the low-resolution image to obtain an isotropic target low-resolution image and establish its spatial correspondence with the high-resolution image: in, , and are the three-dimensional coordinates of the target low-resolution image after interpolation; The interpolated target low-resolution image is at coordinates The voxel value at the position, i, j and q are integer coordinates in the low-resolution image, d, e and l are relative position indexes in the cubic spline interpolation, ranging from -1 to 2, is the voxel value of the anisotropic low-resolution image, u, v, and w all represent the offset of the decimal part in the interpolation calculation, where , and Respectively represent the coordinates of the current calculation point in the x-axis, y-axis, and z-axis directions With sampling point (d+m, e+n, q+l The relative distance between them, C(x) is a one-dimensional interpolation kernel function, and the specific expression is: The interpolation kernel function C(x) is used to distribute the relative distances (du), (ev) and (lw) The weights C(du), C(ev) and C(lw) are given in the formula. The closer the point is, the greater the weight it gets.

[0099] By adopting a specific spatial position preservation strategy, when generating an isotropic target low-resolution image, the correspondence between the image and the original high-resolution structural magnetic resonance image in the three-dimensional spatial coordinate system is strictly maintained, thereby effectively reducing the training difficulty of the deep learning model and improving the accuracy of super-resolution reconstruction.

[0100] Step 2: Design a deep learning model: First, many existing super-resolution technologies for structural MRI images are migrated from super-resolution technologies on natural images, but this approach fails to fully consider the particularity of structural MRI images and does not make full use of prior knowledge of the brain. In order to solve this problem, it is necessary to design a module that can provide the model with prior knowledge of the brain to guide the model to better perform the task of super-resolution of brain images. In the field of neuroscience, prior knowledge of the brain is usually based on qualitative descriptions, while deep learning models require precise digital forms, and there is an inconsistency in the expression of knowledge between the two. At the same time, how to minimize the loss of prior knowledge and feed it into the model in a simple form is also a very critical issue. Although the segmentation results of the anatomical structures of the brain (such as gray matter, white matter, and cerebrospinal fluid) can be used as additional input channels of the model or MRI data of other modalities can be used as complementary information, there is currently no tool for robust segmentation of low-resolution images, and there is also a lack of training data for low-resolution images and corresponding segmentation labels. In addition, since the low-resolution images collected clinically are almost only single-modal, multi-modal auxiliary information is difficult to obtain in actual application scenarios. Therefore, the application example of this application designs an improved reference image module that does not require other modalities. By introducing the strategy of reference images, the model is provided with prior knowledge of the brain in a simple way. Specifically, a fixed reference structural magnetic resonance image of a healthy person (the same modality as the image to be super-resolution) is first denoised, and then aligned with the isotropic target low-resolution image constructed in step 1 after cubic spline interpolation as the reference image of the target low-resolution image. The reference image and the interpolated target low-resolution image have a point-to-point one-to-one correspondence, which can provide the model with the approximate anatomical structure of each brain region, and can be input into the model together as a dual-channel image sample in a simple way of splicing. Among them, the registration method used is the symmetric normalization mode in ANTSpy. This registration method has a high registration accuracy, supports the registration of two-dimensional and three-dimensional images, and has high robustness and stability. The specific process of reference image generation is as follows: Wherein, D represents the mean downsampling process proposed in the application example of this application, S represents cubic spline interpolation, and T represents registration; represents a high-resolution structural magnetic resonance image; Represents a low-resolution image; represents the target low-resolution image; represents a reference image; represents a reference structural magnetic resonance image.

[0101] In terms of the model backbone, in order to reduce the loss of detailed textures and improve the reconstruction of complex structures, the deep learning model used in the application example of this application adopts a multi-scale strategy for image feature extraction. In the local feature extraction branch, 9 residual blocks ResBlock in the EDSR architecture are stacked to capture the local features of the image. This module is also used in the prior art for image super-resolution, but this branch only considers the local features of the image and lacks interaction with the overall image information. In response to this problem, those skilled in the art may easily think of using the Transformer layer to capture image context information, but since medical images are three-dimensional images, the convolutions within the model are all three-dimensional convolutions, and directly applying the Transformer layer will lead to problems such as high computational complexity and excessive video memory usage. Therefore, the application example of this application designs an improved global feature extraction branch based on the Transformer layer, which is used successively , and The image is convolved with a three-dimensional convolution kernel, and the image is encoded to obtain a feature map of smaller size. An encoder with 8 Transformer layers is applied to the feature map to extract the global features of the image. Three upsampling and convolution modules are used to restore the feature map to the original image size. The dimension of the hidden layer in each Transformer layer is 384, the dimension of the hidden layer in the feedforward neural network is 1536, and the number of heads in the multi-head attention mechanism is 8. Finally, the images of the two branches are added together to obtain the final super-resolution image. Compared with the existing sMRI super-resolution model that only focuses on the local area, the multi-scale feature extraction network proposed in the application example of this application fuses local and global features to achieve the complementarity of the information between the two, which can improve the global consistency and overall quality of the image.

[0102] The loss function of the deep learning model adopts the target loss function mentioned in the above embodiment. In addition, the application example of this application adopts a sliding window strategy to cut the image into The image blocks of different sizes are sent to the model one by one for super-resolution, and then all the image blocks output by the model (that is, the super-resolution images corresponding to each of the image samples) are spliced ​​to regain the target super-resolution image.

[0103] Step 3: Training and evaluation on simulated datasets: Train on simulated datasets and use structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) to evaluate the structural accuracy of super-resolution images. Use LPIPS.Alex, LPIPS.vgg, and LPIPS.squeeze to evaluate the visual effects of super-resolution images. Use common brain sMRI preprocessing tools such as CAT12 and Freesurfer for preprocessing, segment the gray matter and white matter of the image, and calculate the correlation between the gray and white matter of the super-resolution image and the original high-resolution image. Among them, LPIPS.Alex refers to the perceptual similarity index (Learned Perceptual Image Patch Similarity, LPIPS) based on the AlexNet network architecture, which is used to evaluate the visual perception similarity between super-resolution images and original high-resolution images. LPIPS.vgg and LPIPS.squeeze are similar to LPIPS.Alex, but use different neural network structures. LPIPS.vgg is a perceptual similarity index based on the VGG network architecture, and LPIPS.squeeze is a perceptual similarity index based on the Squeeze Net network architecture. CAT12 and Freesurfer are both commonly used magnetic resonance imaging analysis software.

[0104] Step 4: Construct a real clinical dataset and fine-tune the model: Existing brain sMRI super-resolution technologies are all trained on simulated datasets, and there are still differences between the simulated low-resolution data and the real clinical low-resolution data, which leads to the problem of poor generalization of the model on real data. In order to better improve the generalization performance of the model on real clinical low-resolution images, the application example of this application uses the model trained on a simulated dataset as a pre-training model, and uses the parameters of the model as the initial parameters to perform training and fine-tuning on a small amount of clinical datasets. Similar to step 1, first use cubic spline interpolation to interpolate the low-resolution image. To ensure point-to-point correspondence between the high-resolution image and the interpolated image, align the high-resolution image with the interpolated image to obtain image data that can be used for fine-tuning. In addition, it is also necessary to use a fixed healthy human brain sMRI to align with the low-resolution interpolated image, and use it as a reference image to input the model together with the low-resolution image for fine-tuning. Step 5: Evaluation on real clinical data sets: Since the super-resolution image and the real high-resolution image do not correspond to each other voxel by voxel, it is impossible to calculate the correlation between the super-resolution image and the original high-resolution image in gray matter and white matter in the same way as the evaluation on the simulated data set. Therefore, the application example of this application uses the calculation of the correlation between the whole brain gray matter volume and white matter volume of the super-resolution image and the original high-resolution image, and the registration of the image to the template space to calculate the correlation of the gray and white matter volume and cortical thickness of the brain region for evaluation.

[0105] That is to say, the application example of the present application provides a method for constructing a training data set: previous deep learning models may have constructed isotropic data sets to train deep learning models. The mean downsampling and discarding strategies used in the application example of the present application can better simulate clinical low-resolution images, and at the same time perform cubic spline interpolation on the low-resolution images to obtain isotropic low-resolution images, and establish a spatial correspondence between them and high-resolution images. The application example of the present application also provides a reference image guidance module: the application example of the present application uses a fixed healthy human brain sMRI image to align the low-resolution interpolation image as the reference image of the image, and together with the interpolation image as a dual-channel image input into the model. The application example of the present application also provides a multi-scale feature extraction network: the multi-scale feature extraction network proposed in the application example of the present application realizes multi-dimensional analysis of brain images from global features and local features, and captures the structure of the brain from more dimensions. The application example of this application also provides a strategy for training on a simulated dataset and fine-tuning on a real clinical dataset: the application example of this application first trains on a simulated dataset of brain sMRI, and then fine-tunes on a small amount of real brain sMRI clinical datasets, which improves the generalization effect of the model in real application scenarios.

[0106] In summary, the structural magnetic resonance super-resolution image reconstruction model training method provided by the application example of this application has the following beneficial effects: 1. The application example of this application uses mean downsampling combined with a strategy of discarding certain layers to construct an anisotropic dataset for training, which can better simulate clinical low-resolution images.

[0107] 2. The application example of this application designs a reference image guidance module, which provides a large amount of prior knowledge of the brain for the deep learning model, allowing the model to be more accurate in reconstructing complex structures.

[0108] 3. The application example of this application uses a multi-scale strategy to extract image features, and realizes multi-dimensional analysis of brain images from global features and local features respectively, captures the structure of the brain from more dimensions, improves the effect and global unity of the images generated by the model, and the fusion mechanism of multi-scale features can effectively balance the reconstruction quality of local details and global structures.

[0109] 4. The application example of this application adopts the strategy of fine-tuning on a small amount of real clinical data sets to improve the generalization effect of the model in real application scenarios. Through fine-tuning of a small amount of data, the model can quickly learn the distribution of image features in a specific scenario, and further improve the reconstruction effect in the target scenario while maintaining the original performance, providing a more practical solution for clinical practice.

[0110] Based on the above-mentioned embodiments and / or application examples of the structural magnetic resonance super-resolution image reconstruction model training method, the present application also provides an embodiment of a structural magnetic resonance super-resolution image reconstruction method, see Fig.12 The structural magnetic resonance super-resolution image reconstruction method specifically includes the following contents: Step 300: registering the reference structural magnetic resonance image with the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image; Step 400: dividing the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size, to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image; Step 500: performing one-to-one splicing of each image block corresponding to the clinical structure low-resolution magnetic resonance image and each image block corresponding to the reference image, so as to obtain each image sample corresponding to the clinical structure low-resolution magnetic resonance image; Step 600: inputting the image samples corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model, so that the structural magnetic resonance super-resolution image reconstruction model outputs super-resolution images of the image samples corresponding to the clinical structural magnetic resonance low-resolution image, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method; Step 700: recombining the super-resolution images of the image samples corresponding to the clinical structure MRI low-resolution image to obtain a target super-resolution image corresponding to the clinical structure MRI low-resolution image.

[0111] The structural magnetic resonance super-resolution image reconstruction model training method mentioned in the embodiment of the structural magnetic resonance super-resolution image reconstruction method provided in the embodiment of the present application can specifically adopt the processing flow of the embodiment of the structural magnetic resonance super-resolution image reconstruction model training method in the above-mentioned embodiment and / or application example. Its functions are not repeated here, and reference can be made to the detailed description of the above-mentioned structural magnetic resonance super-resolution image reconstruction model training method embodiment.

[0112] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction method provided in the embodiment of the present application can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects and loss of detail texture, and can improve the effectiveness and reliability of the structural magnetic resonance super-resolution image reconstruction model trained using image samples, thereby improving the accuracy and reliability of super-resolution images reconstructed using the model.

[0113] From the software level, the present application also provides a structural magnetic resonance super-resolution image reconstruction model training device for executing all or part of the structural magnetic resonance super-resolution image reconstruction model training method, see Fig.13 The structural magnetic resonance super-resolution image reconstruction model training device specifically includes the following contents: The simulation data construction module 10 is used to perform anisotropic downsampling processing on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used to simulate a clinical structural magnetic resonance low-resolution image; The model training module 20 is used to generate image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image with the same resolution as the structural magnetic resonance image, and to train a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image.

[0114] The embodiment of the structural magnetic resonance super-resolution image reconstruction model training device provided in the present application can be specifically used to execute the processing flow of the embodiment of the structural magnetic resonance super-resolution image reconstruction model training method in the above-mentioned embodiment. Its functions will not be repeated here, and reference can be made to the detailed description of the above-mentioned structural magnetic resonance super-resolution image reconstruction model training method embodiment.

[0115] The part of the structural magnetic resonance super-resolution image reconstruction model training device that performs structural magnetic resonance super-resolution image reconstruction model training can be executed in the server or completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application is not limited to this. If all operations are completed in the client device, the client device may also include a processor for specific processing of the structural magnetic resonance super-resolution image reconstruction model training.

[0116] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0117] The server and the client device may communicate with each other using any suitable network protocol, including network protocols that have not yet been developed on the date of filing this application. The network protocols may include, for example, TCP / IP, UDP / IP, HTTP, HTTPS, etc. Of course, the network protocols may also include, for example, RPC (Remote Procedure Call Protocol) and REST (Representational State Transfer) protocols used on top of the above protocols.

[0118] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction model training device provided in the embodiment of the present application can improve the effectiveness and rationality of image samples simulating low-resolution images of clinical medium and thick layer scanning, and can improve the effectiveness and reliability of using image samples to train the structural magnetic resonance super-resolution image reconstruction model, thereby improving the accuracy and reliability of reconstructing super-resolution images using the model.

[0119] From the software level, the present application also provides a structural magnetic resonance super-resolution image reconstruction device for executing all or part of the structural magnetic resonance super-resolution image reconstruction method, see Fig.14 The structural magnetic resonance super-resolution image reconstruction device specifically includes the following contents: An image registration module 30, for registering the reference structural magnetic resonance image with the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image; An image segmentation module 40 is used to segment the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size, so as to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image; An image stitching module 50 is used to stitch each image block corresponding to the clinical structure low-resolution magnetic resonance image with each image block corresponding to the reference image one-to-one, so as to obtain each image sample corresponding to the clinical structure low-resolution magnetic resonance image; An image reconstruction module 60 is used to input each image sample corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model, so that the structural magnetic resonance super-resolution image reconstruction model outputs a super-resolution image of each image sample corresponding to the clinical structural magnetic resonance low-resolution image, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method; The image recombining module 70 is used to recombine the super-resolution images of the image samples corresponding to the clinical structure MRI low-resolution image to obtain a target super-resolution image corresponding to the clinical structure MRI low-resolution image.

[0120] The embodiment of the structural magnetic resonance super-resolution image reconstruction device provided in the present application can be specifically used to execute the processing flow of the embodiment of the structural magnetic resonance super-resolution image reconstruction method in the above-mentioned embodiment. Its functions will not be repeated here, and reference can be made to the detailed description of the above-mentioned structural magnetic resonance super-resolution image reconstruction method embodiment.

[0121] The part of the structural magnetic resonance super-resolution image reconstruction device that performs structural magnetic resonance super-resolution image reconstruction can be executed in the server or in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this.

[0122] From the above description, it can be seen that the structural magnetic resonance super-resolution image reconstruction device provided in the embodiment of the present application can improve the reconstruction accuracy of complex brain structures, reduce edge blur, jagged effects and loss of detail texture, and can improve the effectiveness and reliability of the structural magnetic resonance super-resolution image reconstruction model trained using image samples, thereby improving the accuracy and reliability of super-resolution images reconstructed using the model.

[0123] The present application also provides an electronic device, which may include a processor, a memory, a receiver and a transmitter, wherein the processor is used to execute the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method mentioned in the above embodiment, wherein the processor and the memory may be connected via a bus or other means, with the bus connection being taken as an example. The receiver may be connected to the processor and the memory via a wired or wireless manner.

[0124] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0125] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method in the embodiments of the present application. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, implementing the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method in the above method embodiments.

[0126] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The one or more modules are stored in the memory, and when executed by the processor, perform the structural magnetic resonance super-resolution image reconstruction model training method and / or the structural magnetic resonance super-resolution image reconstruction method in the embodiment.

[0128] In some embodiments of the present application, the user equipment may include a processor, a memory, and a transceiver unit, which may include a receiver and a transmitter. The processor, memory, receiver, and transmitter may be connected through a bus system. The memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver unit to send and receive signals.

[0129] As an implementation method, the functions of the receiver and the transmitter in the present application can be considered to be implemented through a transceiver circuit or a dedicated chip for transceiver, and the processor can be considered to be implemented through a dedicated processing chip, a processing circuit or a general chip.

[0130] As another implementation method, it is possible to use a general-purpose computer to implement the server provided in the embodiment of the present application, that is, to store the program code for implementing the functions of the processor, receiver, and transmitter in a memory, and the general-purpose processor implements the functions of the processor, receiver, and transmitter by executing the code in the memory.

[0131] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned structural magnetic resonance super-resolution image reconstruction model training method and / or structural magnetic resonance super-resolution image reconstruction method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0132] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the aforementioned structural magnetic resonance super-resolution image reconstruction model training method and / or structural magnetic resonance super-resolution image reconstruction method.

[0133] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0134] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0135] In the present application, features described and / or illustrated for one embodiment may be used in the same manner or in a similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0136] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A structural magnetic resonance super-resolution image reconstruction model training method, characterized in that: include: Performing anisotropic downsampling processing on each structural magnetic resonance image respectively, so as to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used for simulating a clinical structural magnetic resonance low-resolution image; Based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, image samples corresponding to each of the target low-resolution images are generated respectively, and a deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

2. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 1, characterized in that: The anisotropic downsampling process is performed on each structural magnetic resonance image to obtain a target low-resolution image corresponding to each structural magnetic resonance image and used to simulate a clinical structural magnetic resonance low-resolution image, including: Performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal plane direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images; Cubic spline interpolation is performed on each of the low-resolution images to obtain a target low-resolution image corresponding to each of the low-resolution images and used to simulate a clinical structural magnetic resonance low-resolution image.

3. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 2, characterized in that: The step of performing mean downsampling processing on each layer of each structural magnetic resonance image in the sagittal plane direction based on an inter-layer discarding strategy to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images comprises: Dividing each layer of each structural magnetic resonance image in the sagittal plane direction into a plurality of first layer groups, wherein the number of layers in each first layer group is a preset super-resolution multiple; According to the super-resolution multiple and the preset inter-layer discarding ratio, each of the first layer groups is subjected to inter-layer discarding processing to reduce the number of layers in each of the first layer groups, thereby obtaining a second layer group corresponding to each of the first layer groups; Mean downsampling processing is performed on each of the second layer groups respectively to obtain a low-resolution image corresponding to each of the structural magnetic resonance super-resolution images.

4. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 1, characterized in that: The method of generating image samples corresponding to each of the target low-resolution images based on a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image, and training a deep learning model based on each of the image samples and the structural magnetic resonance image corresponding to each of the image samples, so as to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into a super-resolution image, comprises: Registering a reference structural magnetic resonance image having the same resolution as the structural magnetic resonance image to each of the target low-resolution images, respectively, to obtain a reference image corresponding to each of the target low-resolution images; Generate image samples corresponding to each of the target low-resolution images according to each of the target low-resolution images and the corresponding reference images; Based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples, a deep learning model is trained with a preset target loss function to train the deep learning model into a structural magnetic resonance super-resolution image reconstruction model for reconstructing the image samples into super-resolution images.

5. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 4, characterized in that: The step of generating image samples corresponding to each of the target low-resolution images according to each of the target low-resolution images and the corresponding reference images comprises: Dividing each of the target low-resolution images and each of the reference images into image blocks of the same size to obtain image blocks corresponding to each of the target low-resolution images and image blocks corresponding to each of the reference images; Each of the target low-resolution images and the corresponding reference image are respectively used as sample groups, and each image block corresponding to the target low-resolution image in each of the sample groups is spliced ​​one-to-one with each image block corresponding to the reference image, so as to obtain each image sample corresponding to each of the sample groups.

6. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 5, characterized in that: The deep learning model includes: a local feature extraction branch, a global feature extraction branch and a feature addition layer; Correspondingly, the deep learning model is trained based on each of the image samples and the structural magnetic resonance images corresponding to each of the image samples with a preset target loss function, including: For each sample group, a preset iterative training step is sequentially performed, wherein the iterative training step includes: Inputting each of the image samples corresponding to the current sample group into the local feature extraction branch and the global feature extraction branch in the current deep learning model, respectively, so that the local feature extraction branch extracts local feature data corresponding to each of the image samples, respectively, so that the global feature extraction branch extracts global feature data corresponding to each of the image samples, and the feature addition layer adds the local feature data and the global feature data corresponding to each of the image samples, respectively, to obtain super-resolution images corresponding to each of the image samples; Recombining the super-resolution images of the respective image samples corresponding to the current sample group to obtain a target super-resolution image corresponding to the current sample group; Based on the target super-resolution image and the structural magnetic resonance image corresponding to the sample group, the current loss of the deep learning model is calculated using a preset target loss function, and the deep learning model is optimized based on the loss to obtain the deep learning model in the next iterative training step, wherein the target loss function includes a weighted sum of a mean square error loss function and a perceptual loss function with different weights.

7. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 6, characterized in that: The local feature extraction branch includes: A first convolutional layer is used to perform high-dimensional feature space mapping on the image sample and adjust the format and dimension, and output corresponding first feature data; A local feature extraction unit, configured to perform local feature extraction on the first feature data based on a plurality of stacked residual blocks, and output corresponding second feature data; The second convolutional layer is used to adjust the number of channels of the second feature data and output corresponding local feature data.

8. The structural magnetic resonance super-resolution image reconstruction model training method according to claim 6, characterized in that: The global feature extraction branch includes: An encoder, configured to encode the image sample based on three sequentially connected encoding layers and output corresponding encoding feature data; wherein the encoding layer comprises: a connected convolutional layer and a ReLU activation function; A global feature extraction unit, used to extract global features from the encoded feature data based on a plurality of sequentially connected Transformer layers, and output corresponding global encoded feature data; A decoder is used to decode the global encoded feature data based on three sequentially connected decoding layers and output corresponding global feature data, wherein the encoding layer includes: an upsampling layer, a convolutional layer and a ReLU activation function connected in sequence.

9. A structural magnetic resonance super-resolution image reconstruction method, characterized in that: include: Registering the reference structural magnetic resonance image to the current clinical structural magnetic resonance low-resolution image to obtain a reference image corresponding to the clinical structural magnetic resonance low-resolution image; Dividing the clinical structure low-resolution magnetic resonance image and the reference image into image blocks of the same size respectively, so as to obtain image blocks corresponding to the clinical structure low-resolution magnetic resonance image and image blocks corresponding to the reference image; Respectively performing one-to-one splicing of each image block corresponding to the clinical structure low-resolution magnetic resonance image and each image block corresponding to the reference image to obtain each image sample corresponding to the clinical structure low-resolution magnetic resonance image; Inputting the image samples corresponding to the clinical structural magnetic resonance low-resolution image into a structural magnetic resonance super-resolution image reconstruction model respectively, so that the structural magnetic resonance super-resolution image reconstruction model outputs super-resolution images of the image samples corresponding to the clinical structural magnetic resonance low-resolution image respectively, wherein the structural magnetic resonance super-resolution image reconstruction model is pre-trained based on the structural magnetic resonance super-resolution image reconstruction model training method according to any one of claims 1 to 8; The super-resolution images of the image samples corresponding to the clinical structure magnetic resonance low-resolution image are recombined to obtain a target super-resolution image corresponding to the clinical structure magnetic resonance low-resolution image.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the structural magnetic resonance super-resolution image reconstruction model training method as described in any one of claims 1 to 8, and / or implements the structural magnetic resonance super-resolution image reconstruction method as described in claim 9.

Images