Super-resolution image reconstruction method for dynamic contrast enhancement based on deep learning
Through the super-resolution image reconstruction method based on deep learning, the trade-off between spatial resolution and temporal resolution in dynamic contrast-enhanced magnetic resonance imaging is solved, and the image is improved without extending the scanning time is achieved. The effect of resolution.
Patent Information
- Application Number
- CN202411177358.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2025-01-10
AI Technical Summary
In dynamic contrast-enhanced magnetic resonance imaging, in order to obtain high spatial resolution, time resolution is often required to sacrifice, resulting in extended scanning time.
Using a deep learning-based super-resolution image reconstruction method, high-resolution static data is collected before injection of contrast agent and low-resolution dynamic data is collected after injection, deep learning model is used to reconstruct images, and high-resolution dynamics are output. Reconstruct images to improve spatial resolution without sacrificing temporal resolution.
It effectively improves the spatial resolution with enhanced dynamic contrast, and improves the resolution quality of the image without extending the scanning time.
Smart Images
Figure CN119295575A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of magnetic resonance imaging, and in particular to a super-resolution image reconstruction method with dynamic contrast enhancement based on deep learning. Background Art
[0002] Dynamic Contrast Enhanced (DCE) MRI usually requires the injection of contrast agents, such as gadolinium contrast agents, and repeated acquisition of T1-weighted images of the area of interest within a period of time, such as 5-10 minutes, after the injection of the contrast agent. During this period, the gadolinium contrast agent accumulates in the corresponding tissues and organs, and the gadolinium contrast agent significantly shortens the T1 of the corresponding area, thereby achieving image contrast enhancement.
[0003] However, if you want to obtain dynamic contrast-enhanced images with high spatial resolution, you need to collect more data at each time point, such as repeated scanning and extended scanning time. Therefore, in dynamic contrast-enhanced MRI, in order to obtain images with high spatial resolution, it is often necessary to sacrifice temporal resolution. Summary of the invention
[0004] The purpose of the present invention is to provide a super-resolution image reconstruction method with dynamic contrast enhancement based on deep learning to solve the above technical problems;
[0005] The technical problem solved by the present invention can be achieved by adopting the following technical solutions:
[0006] The super-resolution image reconstruction method based on dynamic contrast enhancement based on deep learning includes:
[0007] Step S1, before injecting contrast agent, imaging the part of interest to collect static data of a first resolution;
[0008] Step S2, after contrast agent is injected, repeatedly collecting N frames of imaging of the part of interest to obtain dynamic data of a second resolution;
[0009] Step S3, inputting the static data and the dynamic data into a deep learning model, and the deep learning model outputs a dynamic reconstructed image with the first resolution;
[0010] Wherein N is a positive integer.
[0011] Preferably, the first resolution in step S1 is higher than the second resolution in step S2.
[0012] Preferably, in step S1, the static data has structural information of the part of interest.
[0013] Preferably, in step S2, N frames of imaging are repeatedly acquired for the part of interest, all of which are k-space data of the central part of the region of interest in the phase encoding direction, and the imaging sequence adopts a T1-weighted three-dimensional spoiled gradient echo sequence.
[0014] Preferably, in step S3, the method for inputting the static data into the deep learning model is to obtain a first channel input image with the first resolution after inverse Fourier transform of the static data, and input the first channel of the deep learning model.
[0015] Preferably, in step S3, the method for inputting the dynamic data into the deep learning model is to perform zero-padding on the dynamic data with the second resolution of each frame in k-space until the resolution of the dynamic data is consistent with the first resolution, and perform an inverse Fourier transform on the zero-padding dynamic data to obtain a second channel input image with the first resolution, which is input into the second channel of the deep learning model.
[0016] Preferably, in step S3, the deep learning model uses multiple first convolutional layers and multiple pooling layers to extract features from the input static data and the dynamic data, and uses a linear rectification function as an activation function.
[0017] Preferably, in step S3, the deep learning model uses multiple second convolutional layers and multiple upsampling layers to restore the extracted features, and a jump connection is performed between the output of the second convolutional layer and the part of the output of the upsampling layer that has the same size.
[0018] Preferably, in step S3, the training method of the deep learning model is to collect training static images and training dynamic images with the first resolution, downsample the training dynamic images to obtain the training dynamic images with the second resolution, use the training static images with the first resolution and the training dynamic images with the second resolution as inputs of the deep learning model to train the deep learning model, and use the training dynamic images with the first resolution as the ideal output of the deep learning model.
[0019] Preferably, when training the deep learning model, a mean square error function or a structural similarity index measurement function is used as a loss function, and an adaptive moment estimation optimizer is used as an optimizer.
[0020] Beneficial effects of the present invention: Due to the adoption of the above technical scheme, the present invention performs super-resolution image reconstruction based on a deep learning model, effectively improving the spatial resolution of dynamic contrast enhancement without sacrificing temporal resolution. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 Schematic diagram of collecting static data and dynamic data in an embodiment of the present invention;
[0022] Figure 2 Schematic diagram of the steps of a super-resolution image reconstruction method in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0024] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0025] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0026] Super-resolution image reconstruction method based on dynamic contrast enhancement based on deep learning, such as Figure 1 , Figure 2 As shown, including,
[0027] Step S1, before injecting contrast agent, imaging the part of interest to collect static data 1 of the first resolution;
[0028] Step S2, after contrast agent is injected, repeatedly collecting N frames of imaging of the part of interest to obtain dynamic data 2 of a second resolution;
[0029] Step S3, inputting the static data and the dynamic data into a deep learning model, and the deep learning model outputs a dynamic reconstructed image with a first resolution;
[0030] Wherein N is a positive integer.
[0031] Specifically, the present invention proposes a super-resolution image reconstruction method based on deep learning, and uses the same imaging parameters to collect a set of static image data with high resolution, i.e., first resolution, for the current part of interest before injecting the contrast agent to reflect the structural information of the part of interest. Dynamic imaging after the injection of the contrast agent only requires repeated acquisition of N frames of low-resolution, i.e., second-resolution image data, and then the high-resolution static data and low-resolution dynamic data are input into the deep learning model, and the deep learning model outputs a high-resolution dynamically reconstructed image. The present invention can effectively improve the spatial resolution of dynamic contrast enhancement (DCE) without sacrificing temporal resolution.
[0032] In a preferred embodiment, the first resolution in step S1 is higher than the second resolution in step S2.
[0033] Specifically, the high resolution and low resolution of the present invention are relative concepts, and the first resolution can be higher than the second resolution. No specific limitation is made to the specific resolution.
[0034] Assume that the field of view (FOV) size of the second resolution image is 256×256mm 2 , the sampling matrix size is 64×256 (where 64 is the size of the phase encoding direction), then its resolution is 4×1mm 2 If we want to encode along the phase direction, that is, Figure 1 The super-resolution image is reconstructed 4 times in the kx and ky directions. The reconstructed image matrix size should be 256x256 and the resolution is 1x1mm. 2 , the acquired static image data with the first resolution should have the same matrix size and resolution.
[0035] In a preferred embodiment, in step S1, the static data has structural information of the part of interest.
[0036] Specifically, the static data is obtained by using the same imaging parameters to collect a set of static image data with the first resolution for the current part of interest before injecting the contrast agent, so as to reflect the structural information of the part of interest.
[0037] In a preferred embodiment, in step S2, N frames of imaging are repeatedly acquired for the part of interest, all of which are k-space data of the central part of the region of interest in the phase encoding direction, and the imaging sequence adopts a T1-weighted three-dimensional spoiled gradient echo sequence.
[0038] Specifically, the dynamic data is obtained by repeatedly acquiring N frames of dynamic data 2 of the second resolution after contrast agent injection, that is, only acquiring k-space data of the central part in the phase encoding direction: ky direction. The imaging sequence can use T1-weighted 3D spoiled gradient echo technology.
[0039] T1-weighted imaging is a magnetic resonance imaging (MRI) imaging mode whose images are more sensitive to the T1 relaxation time in tissues. T1 relaxation time refers to the time it takes for a nucleus or molecule to return to its ground state from an excited state. Different tissues have different T1 relaxation times, so T1-weighted imaging can show contrast differences between tissues and help distinguish different types of tissue. For example, in T1-weighted images, fat usually appears as a high signal intensity (bright), while water and other soft tissues appear as a low signal intensity (dark).
[0040] 3D Spoiled Gradient Recalled Echo Technique is an MRI imaging sequence that acquires images using a gradient echo pulse sequence. In 3D Spoiled Gradient Echo Technique, gradient fields are used to generate images instead of RF pulses. "Spoiled" means that the gradient pulses used in the imaging sequence will cause phase distortion of the gradient echo signal, "echo" means that images are formed by detecting echo signals, and "3D" means that image data of the entire volume is acquired through three-dimensional imaging.
[0041] In a preferred embodiment, in step S3, the method of inputting static data into the deep learning model is to obtain a first channel input image with a first resolution after inverse Fourier transform of the static data, and input the first channel of the deep learning model.
[0042] Specifically, the static data of the first resolution, i.e., high resolution, is input into the deep learning model. The static k-space data of high resolution, assuming that its size is 256×256, is inverse Fourier transformed to obtain a high resolution static image as one channel of input;
[0043] In a preferred embodiment, in step S3, the method for inputting dynamic data into the deep learning model is to pad the dynamic data 2 with the second resolution of each frame with zeros in the k-space until the resolution of the dynamic data is consistent with the first resolution, and obtain a second channel input image with the first resolution by inverse Fourier transforming the dynamic data after zero padding, and input it into the second channel of the deep learning model.
[0044] Specifically, a frame of dynamic data of the second resolution, i.e., low resolution, is assumed to be 256×64 in size. Then, zero padding is performed in the k-space, i.e., all unsampled areas are filled with 0s to make the size consistent with the size of the static data of the first resolution, i.e., 256×256. After inverse Fourier transformation, it is used as another input channel. Thus, an input of 256×256×2 is obtained.
[0045] Specifically, the zero padding operation ensures that the resolution of the dynamic data in k-space is consistent with the static data of the first resolution 1. It ensures that the input images of the two channels have the same spatial resolution, so as to better perform joint processing in the deep learning model.
[0046] Specifically, the zero-padding operation maintains the spatial range of dynamic data, and even low-resolution dynamic data can cover the same spatial range as high-resolution static data after zero-padding, ensuring the integrity of important information in the image and avoiding the loss of important details in low-resolution data.
[0047] In a preferred embodiment, in step S3, the deep learning model uses multiple first convolutional layers and multiple pooling layers to extract features of input static data and dynamic data, and uses a linear rectification function as an activation function.
[0048] Specifically, the deep learning model uses multiple first convolutional layers and pooling layers to extract features from the input data, the convolution kernel size is set to 3×3, and the linear rectification function (ReLU) is used as the activation function. In this embodiment, the output dimensions of each first convolutional layer are:
[0049] The first convolutional layer outputs 256×256×16;
[0050] The output of the first convolutional layer is 256×256×32;
[0051] The pooling layer output is 128×128×32;
[0052] The first convolutional layer outputs 128×128×64;
[0053] The pooling layer outputs 64×64×64;
[0054] The first convolutional layer outputs 64×64×128;
[0055] The pooling layer outputs 32×32×128.
[0056] The above layers are used to extract features from a 256×256×2 input.
[0057] In a preferred embodiment, in step S3, the deep learning model uses multiple second convolutional layers and multiple upsampling layers to restore the extracted features, and a jump connection is performed between the output of the second convolutional layer and the parts of the same size in the output of the upsampling layer.
[0058] Specifically, in this stage, the extracted features are restored through the second convolutional layer and the upsampling layer, and a skip connection is used to connect the output of the second convolutional layer for feature extraction with the output of the upsampling layer for feature restoration.
[0059] The output dimensions of each second convolutional layer are:
[0060] The second convolutional layer outputs 32×32×128;
[0061] The upsampling layer outputs 64×64×128
[0062] The second convolutional layer outputs 64×64×64;
[0063] Upsampling layer output 128×128×64
[0064] The second convolutional layer outputs 128×128×32;
[0065] The upsampling layer outputs 256×256×32
[0066] The second convolutional layer outputs 256×256×16;
[0067] The second convolutional layer outputs 256×256×4;
[0068] The second convolutional layer outputs 256×256×1;
[0069] In a preferred embodiment, in step S3, the training method of the deep learning model is to collect training static images and training dynamic images with a first resolution, downsample the training dynamic images to obtain training dynamic images with a second resolution, use the training static images with the first resolution and the training dynamic images with the second resolution as inputs of the deep learning model to train the deep learning model, and use the training dynamic images with the first resolution as the ideal output of the deep learning model.
[0070] Specifically, the training data in the present invention can be obtained by sacrificing the temporal resolution, and collecting static images and dynamic images of the first resolution, i.e., high resolution, as training static images and training dynamic images, respectively. The high-resolution training dynamic images are downsampled to obtain low-resolution training dynamic images, and the low-resolution training dynamic images and the high-resolution training static images are used as inputs of the deep learning model. The high-resolution dynamic images are used as ideal outputs of the deep learning model.
[0071] Specifically, by simultaneously collecting training static images with a first resolution and training dynamic images with a second resolution and using them as the input of the model, and using high-resolution dynamic images as the ideal output of the deep learning model, the deep learning model can simultaneously use static and dynamic information to learn image reconstruction and super-resolution tasks. This improves the deep learning model's ability to understand complex dynamic scenes and improves the quality and accuracy of the reconstructed image.
[0072] In a preferred embodiment, a mean square error function or a structural similarity index measurement function is used as a loss function when training a deep learning model, and an adaptive moment estimation optimizer is used as an optimizer.
[0073] Specifically, the architecture of the deep learning model can use any deep learning model suitable for image super-resolution, such as a convolutional neural network (CNN) model. The deep learning model contains multiple convolutional layers and upsampling layers to learn the mapping from a low-resolution input image to a high-resolution output image.
[0074] When training the model, the mean square error (MSE) or the similarity index measurement (SSIM) can be used as the loss function. The adaptive moment estimation optimizer (Adam) can be used as the optimizer.
[0075] Specifically, using the mean square error function or the structural similarity index measurement function as the loss function and the adaptive moment estimation optimizer as the optimizer can effectively guide the model training process. The mean square error function is suitable for measuring the pixel-level differences between images, while the structural similarity index measurement function can better consider the structural similarity. The adaptive moment estimation optimizer can automatically adjust the learning rate, accelerate model convergence and improve training efficiency.
[0076] The above description is only a preferred embodiment of the present invention, and does not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.
Claims
1. A super-resolution image reconstruction method based on deep learning for dynamic contrast enhancement, characterized in that: include, Step S1, before injecting contrast agent, imaging the part of interest to collect static data of a first resolution; Step S2, after contrast agent is injected, repeatedly collecting N frames of imaging of the part of interest to obtain dynamic data of a second resolution; Step S3, inputting the static data and the dynamic data into a deep learning model, and the deep learning model outputs a dynamic reconstructed image with the first resolution; Wherein N is a positive integer.
2. The super-resolution image reconstruction method according to claim 1, characterized in that: The first resolution in step S1 is higher than the second resolution in step S2.
3. The super-resolution image reconstruction method according to claim 1, characterized in that: In step S1, the static data has structural information of the part of interest.
4. The super-resolution image reconstruction method according to claim 1, characterized in that: In step S2, N frames of imaging are repeatedly collected for the region of interest, all of which are k-space data of the central part of the region of interest in the phase encoding direction, and the imaging sequence adopts a T1-weighted three-dimensional spoiled gradient echo sequence.
5. The super-resolution image reconstruction method according to claim 1, characterized in that: In step S3, the method for inputting the static data into the deep learning model is to obtain a first channel input image with the first resolution after inverse Fourier transform of the static data, and input the first channel of the deep learning model.
6. The super-resolution image reconstruction method according to claim 1, characterized in that: In step S3, the method for inputting the dynamic data into the deep learning model is to pad the dynamic data of each frame with the second resolution with zeros in k-space until the resolution of the dynamic data is consistent with the first resolution, and subject the zero-padded dynamic data to an inverse Fourier transform to obtain a second channel input image with the first resolution, which is input into the second channel of the deep learning model.
7. The super-resolution image reconstruction method according to claim 1, characterized in that: In step S3, the deep learning model uses multiple first convolutional layers and multiple pooling layers to extract features from the input static data and the dynamic data, and uses a linear rectification function as an activation function.
8. The super-resolution image reconstruction method according to claim 7, characterized in that: In step S3, the deep learning model uses multiple second convolutional layers and multiple upsampling layers to restore the extracted features, and a jump connection is performed between the output of the second convolutional layer and the part of the output of the upsampling layer that has the same size.
9. The super-resolution image reconstruction method according to claim 1, characterized in that: In step S3, the training method of the deep learning model is to collect training static images and training dynamic images with the first resolution, downsample the training dynamic images to obtain the training dynamic images with the second resolution, use the training static images with the first resolution and the training dynamic images with the second resolution as inputs of the deep learning model to train the deep learning model, and use the training dynamic images with the first resolution as the ideal output of the deep learning model.
10. The super-resolution image reconstruction method according to claim 9, characterized in that: When training the deep learning model, a mean square error function or a structural similarity index measurement function is used as a loss function, and an adaptive moment estimation optimizer is used as an optimizer.