Super-resolution reconstruction method based on multi-direction nuclear magnetic image

Through deep learning and 3D multi-band fusion technology, multi-directional MRI information is used for super-resolution reconstruction of 3D-MRI images, solving the problem of algorithm design difficulty and insufficient mining of context information, and achieving high-quality high-resolution MRI Image reconstruction.

CN119027313APending Publication Date: 2024-11-26NANJING TECH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410748365.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-11
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The prior art has problems such as difficult algorithm design and insufficient context information mining in super-resolution reconstruction of 3D-MRI images, resulting in unsatisfactory reconstruction results and blurred local details of MRI images.

Method used

Deep learning method combined with 3D multi-band fusion, multi-directional MRI information is used to interpolate data into isotropic data through network model, and 3D Gaussian pyramid and 3D Laplace pyramid are constructed to calculate feature weights for fusion, and the output is high MRI data of resolution.

Benefits of technology

The reconstruction of high-resolution MRI images is realized, which improves the overall quality of the image and provides a new super-resolution reconstruction method, which has significantly improved the generation time and image quality compared with other methods.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027313A_ABST
    Figure CN119027313A_ABST
Patent Text Reader

Abstract

The invention relates to a super-resolution reconstruction method based on a multi-direction nuclear magnetic image, and belongs to the field of image processing. Magnetic resonance imaging (MRI) is an important medical imaging technology, but the image resolution is generally low due to hardware and physical limitations. In order to improve the spatial resolution of an MRI image, the invention provides a multi-direction nuclear magnetic resonance imaging super-resolution reconstruction method based on deep learning. According to the method, a traditional method and a deep learning method are fused, 3D multi-band fusion and deep learning image interpolation are utilized, the advantages of multi-direction MRI information are effectively utilized, and a high-resolution and high-quality MRI image can be generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a super-resolution reconstruction method based on multi-directional nuclear magnetic resonance images, and belongs to the field of image processing. Background Art

[0002] Magnetic resonance imaging (MRI) is an important medical imaging technique for non-invasive diagnosis. However, the resolution of MRI images is often limited due to hardware and physical limitations. Obtaining high-resolution 3D MRI requires the patient to remain still for a long time, which can cause patient discomfort and increase the possibility of motion-induced image artifacts. In order to eliminate motion artifacts, it is common practice to acquire anisotropic MRI from different directions in a relatively short scan. Super-resolution reconstruction based on multi-directional nuclear magnetic imaging is an important way to improve the spatial resolution of MRI images, providing an important basis for early diagnosis.

[0003] In recent years, the powerful representation and learning capabilities of deep learning have provided new solutions for the research of super-resolution reconstruction algorithms. It does not rely on artificially designed features, overcomes the shortcomings of traditional super-resolution reconstruction algorithms, and greatly promotes the development of super-resolution reconstruction algorithm research. However, the application of deep learning to 3D-MRI image super-resolution reconstruction research still faces some difficulties and challenges, such as: the difficulty of algorithm design, insufficient context information mining resulting in unsatisfactory reconstruction results, and the local details of the reconstructed MRI images are still relatively blurred.

[0004] Fusion of traditional and deep learning to achieve high-resolution MRI reconstruction: This method combines traditional methods and deep learning methods. It utilizes 3D multi-band fusion and deep learning image interpolation, and effectively utilizes the advantages of multi-directional MRI information to achieve high-resolution MRI reconstruction. Summary of the invention

[0005] In the first aspect, the purpose of the present invention is to disclose a super-resolution reconstruction method based on multi-directional nuclear magnetic resonance imaging, firstly using a deep learning method to interpolate MRI into isotropic data, and then using a network model such as Figure 1 shown.

[0006] Secondly, we use the advantages of multi-directional nuclear magnetic information to reconstruct high-resolution isotropic MRI data using 3D multi-band fusion. The process is as follows: Figure 2 shown.

[0007] In an embodiment of the present disclosure, the super-resolution reconstruction method based on multi-directional nuclear magnetic resonance imaging includes the following steps:

[0008] Step 1: Data preparation and preprocessing

[0009] We use the public online dataset ProstateX to group the cross-sectional, sagittal, and coronal data in the dataset and scale them to 320*320 pixels to facilitate sending them to the network for training.

[0010] Step 2: Model training

[0011] Put the processed data set into the network for training, and use the trained parameter model after the training is completed.

[0012] Step 3: Data processing before 3D multi-band fusion

[0013] Select one of the cross-section, sagittal and coronal planes as the target reconstruction plane. Taking the cross-section as an example, first use the trained parameter model to interpolate the three planes into isotropic data. Then use the spatial affine formula: Based on the cross section of the target reconstruction surface, the other two surfaces are fully transformed into the physical space of the cross section.

[0014] Among them, represents the serial number of the i-th feature point, R represents the rotation matrix, t represents the translation vector, represents the pixel coordinates of the i-th feature point on the plane, and represents the three-dimensional physical coordinates of the i-th feature point.

[0015] Step 4: 3D multi-band fusion.

[0016] Using the data processed in step 3, the next step is to fuse 3D multi-bands to construct 3D Gaussian pyramids and 3D Laplacian pyramids, and assign weights to the feature calculations of each band. Finally, the features of each band are fused together according to the weights to output a high-resolution reconstruction. The complete specific process is as follows Figure 3 shown. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments:

[0018] Figure 1 Network model diagram

[0019] Figure 2 3D multi-band fusion diagram

[0020] Figure 3 Method flow chart DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0022] Example:

[0023] Please refer to Figure 3 As shown, the embodiment of the present application provides a super-resolution reconstruction method based on multi-directional nuclear magnetic resonance images, and the method includes the following steps:

[0024] Data preparation and preprocessing: The public dataset ProstateX was sorted, and the cross-sectional, sagittal, and coronal data were extracted and uniformly scaled to 320*320 and saved as training format. The data was divided into training set, validation set, and test set in a ratio of 6:2:2.

[0025] Model training: reference Figure 1 The dataset is sent to the image and then sent to the network model for training. The network is generally divided into two parts: the generator and the discriminator. The generator is responsible for generating the generated image, and the discriminator is responsible for identifying the generated image. The judgment result of the generated sample will be fed back to the generator as the basis for its training. The generator continuously adjusts its parameters according to the feedback from the discriminator to generate more realistic samples to deceive the discriminator. The two compete against each other, driving both sides to make continuous progress and eventually generate more realistic images or other data. A deep residual network is formed by using several 3×3 convolution kernels. The residual network is easy to optimize and can improve the quality of generation by increasing the depth. The internal residual block uses jump connections to alleviate the gradient vanishing problem caused by increasing the depth in the deep neural network.

[0026] Furthermore, the sum of the losses generated by the tasks is used for reverse gradient propagation to update the parameters of the feature extraction network until the loss reaches the minimum value and the model converges. The final model is saved for the next step.

[0027] 3D multi-band fusion: Use the trained model to convert the input data into isotropic data. Then transform them into a unified physical space of the target reconstruction surface, construct 3D Gaussian pyramid and 3D Laplacian pyramid. Calculate the weights for the features of each frequency band, and finally obtain high-resolution MRI data through 3D fusion output.

[0028] The present invention can generate isotropic high-resolution MRI data through an advance pre-trained model and 3D multi-band fusion technology. Compared with other methods, the present invention improves the generation time and the overall quality of the image, and provides a new method for MRI super-resolution reconstruction.

[0029] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A super-resolution reconstruction method based on multi-directional nuclear magnetic resonance imaging, characterized in that: The method comprises the following steps: Step 1: Organize the public dataset ProstateX, extract the cross-sectional, sagittal and coronal data, and resize them to 320*320 size and save them as training format; Step 2: Put the processed data set into the network for training, and wait for the training to complete the parameter model; Step 3: Use the trained model to perform interpolation operations on the data to be super-resolved; Step 4: Select the reconstructed section and transfer other surface data to a unified physical space; Step 5: Construct Gaussian and Laplacian pyramids to calculate the fusion parameters of each layer and perform fusion.

2. The method for super-resolution reconstruction based on multi-directional nuclear magnetic resonance imaging according to claim 1, characterized in that: The step 1 includes the following steps: Step 1: Select the cross-sectional, sagittal and coronal data and scale them to 320*320 size and save them in training format. Step 2: Classify the data set into training set, validation set and test set in a ratio of 6:2:

2.

3. The method for super-resolution reconstruction based on multi-directional nuclear magnetic resonance imaging according to claim 1, characterized in that: The step 2 model training includes the following steps: Step 1: Input a set of 3 images of size 320*320, with the first and third images as input and the second image as the true value. Step 2: Then feed the first and third images into the generator to produce an image for comparison between the discriminator and the true value. Step 3: Use the mean square error (MSE) between the generated image and the true value as the calculation loss for reverse gradient propagation, update the parameters of the feature extraction network in step 1 until the loss reaches the minimum value and the model converges, and save the final model for the next step.

4. The method for super-resolution reconstruction based on multi-directional nuclear magnetic resonance imaging according to claim 1, characterized in that: The step 3 of unifying the physical space comprises the following steps: Step 1: Input a set of MRI data and convert it into model input format. Step 2: Use the trained model to perform interpolation operations.

5. The method for super-resolution reconstruction based on multi-directional nuclear magnetic resonance imaging according to claim 1, characterized in that: The step 4 model usage includes the following steps: Step 1: Select the reconstruction section. Step 2: Use the affine transformation formula The other two surfaces are fully transformed into the physical space of the cross section based on the target reconstruction surface. Among them, represents the serial number of the i-th feature point, R represents the rotation matrix, t represents the translation vector, represents the pixel coordinates of the i-th feature point on the plane, and represents the three-dimensional physical coordinates of the i-th feature point.

6. The method for super-resolution reconstruction based on multi-directional nuclear magnetic resonance imaging according to claim 1, characterized in that: The super-resolution reconstruction in step 5 comprises the following steps: Step 1: Use the data processed in step 4. Step 2: Construct 3D Gaussian and Laplacian pyramids to calculate the fusion parameters of each layer and perform fusion.