A method for LDCT image reconstruction based on deep dual-domain joint guided learning
By employing a deep dual-domain joint guided learning approach, which combines global and local feature extraction, the noise and artifact problems in low-dose CT image reconstruction are resolved, achieving high-quality LDCT image reconstruction suitable for various datasets and clinical applications.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2023-05-29
- Publication Date
- 2026-08-04
AI Technical Summary
Existing low-dose CT image reconstruction algorithms, while reducing radiation dose, suffer from noise and artifact issues, making it difficult to achieve high-quality image reconstruction.
A deep dual-domain joint guided learning approach is adopted. By constructing neural networks in the projection domain and image domain, combining global and local feature extraction, and training the network using a differential fusion module and multiple loss functions, high-quality reconstruction of LDCT images is achieved.
It effectively removes noise and artifacts, improves the reconstruction quality of LDCT images, and preserves spatial texture, making it suitable for different LDCT datasets and clinical application scenarios.
Smart Images
Figure CN116863014B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an LDCT image reconstruction algorithm based on deep dual-domain joint guided learning, belonging to the field of computer image processing. Background Technology
[0002] Computed Tomography (CT) is a medical imaging technique that uses X-rays and computer algorithms to generate three-dimensional images of the body's interior. A CT scan passes X-rays through the body from different angles, and the computer then combines this data to create a high-resolution three-dimensional image that can be used to diagnose many different diseases and injuries. Compared to MRI, CT technology has the advantage of faster imaging speed and is particularly effective in detecting bone tissue and lung diseases. However, it is well known that excessive X-ray radiation doses are harmful to patients and may lead to genetic disorders, cancer, or other diseases.
[0003] In the clinical use of CT technology, the ALARA (as low as reasonably achievable) principle is widely accepted. However, radiation dose directly affects X-ray image quality, a quantum accumulation process used to control the signal-to-noise ratio (SNR) of CT scans. Clearly, reducing the X-ray dose will decrease the quality of the reconstructed image and introduce noise and artifacts. To address this contradictory problem, many algorithms have been proposed to improve the image quality of low-dose CT (LDCT) and avoid unnecessary radiation doses. These algorithms can be categorized into three types: sinusoidal domain filtering methods, iterative reconstruction methods, and image domain denoising methods.
[0004] (1) Sine wave domain filtering method:
[0005] The sinusoidal image domain filtering method primarily utilizes a sinusoidal transform to convert projected data into a sinusoidal image, then filters this image, and finally uses an arcsine transform to convert the processed sinusoidal image back into a reconstructed image. This method is mainly suitable for reconstruction under low noise and low scattering conditions, and can quickly reconstruct relatively clear images with low computational cost. However, because the sinusoidal transform may lose some high-frequency information when the projection angle is small, it leads to a deterioration in image quality. Therefore, this method performs poorly under high noise and high scattering conditions.
[0006] (2) Iterative reconstruction method:
[0007] Iterative reconstruction methods refer to obtaining the final reconstructed image through multiple iterations of computation. The reconstruction process typically consists of two steps: updating the projection data and updating the image. In the projection data update step, a projection backpropagation method is usually used to update the projection data; in the image update step, a gradient-based optimization algorithm is typically used to update the image. This method can progressively optimize the quality of the reconstructed image through multiple iterations and allows for flexible parameter adjustment as needed. However, this method requires a long computation time, which may lead to excessively long reconstruction times.
[0008] (3) Image domain denoising methods:
[0009] Image domain denoising methods primarily target reconstructed images, removing noise and artifacts to improve image quality. Commonly used denoising methods include wavelet transform-based methods, total variation-based methods, dictionary learning-based methods, and deep learning-based methods. The main advantage of these methods is that they can preserve more high-frequency information and improve image resolution and contrast. The disadvantage is that they may result in the loss of some detail, requiring a trade-off between noise and signal strength.
[0010] With the development of deep learning, and in order to better utilize the advantages of sinusoidal image domain filtering methods and image domain denoising methods, while simultaneously leveraging information from both the projection and image domains, dual-domain learning based on deep learning (DL) and knowledge models has been applied to obtain projection-reconstructed CT images from non-ideal scenes. These methods have proven to have good performance in noise suppression during LDCT reconstruction. The traditional dual-domain approach is as follows, and its flowchart is attached. Figure 2 :
[0011] (1) Process the data in the projection domain;
[0012] (2) The FBP algorithm is used to reconstruct the data after the projection domain is processed to obtain the image domain data;
[0013] (2) Reconstruction is performed in the image domain; Summary of the Invention
[0014] To address the noise and artifact issues in low-dose computed tomography (LDCT) image reconstruction, this invention provides an LDCT image reconstruction method based on deep dual-domain joint guided learning. This method solves the problems of secondary artifacts and structural degradation that exist in previous dual-domain learning methods, fully utilizes the global and local features of the data, and achieves high-quality reconstruction of LDCT images.
[0015] To achieve the above objectives, the present invention provides the following technical solution:
[0016] A method for LDCT image reconstruction based on deep dual-domain joint guided learning includes the following steps:
[0017] Step 1: Prepare training data and obtain low-dose CT clinical data and its corresponding normal dose data. The low-dose CT is 1 / 4 of the normal dose.
[0018] Step 2: Build a neural network for projection domain reconstruction, and set the projection domain network as a residual learning network.
[0019] Step 3: The low-dose projection domain data and the projection domain data output by the projection domain reconstruction network are reconstructed using the FBP algorithm to obtain image domain data respectively.
[0020] Step 4: Build a neural network for image domain reconstruction. Using the two image domain data obtained in Step 3 as input, the final low-dose CT reconstructed image is obtained after network processing.
[0021] Step 5: Construct the loss function and train the projection domain and image domain networks.
[0022] Step 6: Testing and Evaluation. Train the converged network on the training sets of the two datasets respectively, and then test and evaluate it on the corresponding test sets.
[0023] Furthermore, in step 1, the training data comes from AAPM challenge data and real data collected on a Siemens SomtomSensation 64-slice CT scanner.
[0024] Furthermore, in step 2, the neural network for projection domain reconstruction is a novel global-local Transformer-Convolution U-Net, which is based on a typical U-Net architecture and is constructed from four collaborative elements: GSTEncoder, LCEncoder, Differentiated Fusion Model (DFM), and TransDecoder.
[0025] (1)Global Swin Transformer Encoder(GSTEncoder)
[0026] GSTDencoder performs context-aware global long-range feature extraction. Inspired by the great success of the Transformer in computer vision tasks, the Swin Transformer is used for LDCT reconstruction. Unlike typical ViTs, the Swin Transformer introduces hierarchical feature map extraction between adjacent windows to extract global content-based features.
[0027] (2) The LCEncoder performs hierarchical short-range local feature extraction. Corresponding to the GSTDecoder, the LCEncoder consists of four convolutional blocks. Each block includes a convolutional layer with a kernel size of 3×3, batch normalization (BN), and rectified linear unit (ReLU) activation functions. A maxpooling layer with a stride of 2 is used for downsampling.
[0028] (3) The Differential Fusion Model (DFM) is designed to be non-linear, tightly connecting and transmitting hierarchical global and local features. These hierarchical feature maps differ in size for the designed GSTEncoder and LCEncoder. Combining the global features generated by the GSTEncoder with the local features generated by the LCEncoder is crucial and determines the final reconstruction performance.
[0029] (4) TransDecoder upsamples the extracted deep features and obtains the final reconstruction result. Corresponding to the patch merging layer in GSTEncoder, the patch expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps and reduces the feature dimensions to half of the original dimensions accordingly.
[0030] Furthermore, in step 3, the FBP (Filtered Back Projection) algorithm is a filtering-based reconstruction algorithm used to convert X-ray projection data into two-dimensional or three-dimensional images. The FBP algorithm mainly consists of two steps: filtering and back projection. Filtering involves performing a Fourier transform on the projection data and filtering the transform result. Filtering methods can include Butterworth filtering, Ram-Lak filtering, etc. Back projection involves projecting the filtered data back into points in space along different angles, forming a three-dimensional dataset. Specifically, for each angle, the projection data passing through that angle is superimposed at the corresponding position and multiplied by a weighting factor to finally obtain the pixel value of each spatial point.
[0031] Furthermore, step 4 specifically includes the following process: constructing a neural network for image domain reconstruction, the structure of which is exactly the same as that of the neural network for projection domain reconstruction; stitching the two projection domain data obtained in step 3 together as input to the image domain network; and obtaining the final low-dose CT reconstructed image after network processing.
[0032] Furthermore, in step 5, two loss functions L are designed for the proposed DUGL-Net. s and L IThese constraints are applied to the projection domain network Sino-Net and the image domain network Image-Net, respectively. They are used to adjust all involved network components, guiding their updates towards high-quality reconstruction. The specific expressions are:
[0033]
[0034] in:
[0035]
[0036] M P M represents the projection domain network. I This represents an image domain network, where [a,b] denotes the concatenation operation between a and b, and R is the Filter Back Projection (FBP) operator. α and β are the weight parameters of these two loss functions. Specifically:
[0037] L s (M p (P LD ,P RD )=L1(M p (P LD ,P RD )
[0038]
[0039] L1 loss is used in our network to reconstruct LDCT images and accelerate training. The L1 loss formula is expressed as follows:
[0040]
[0041] Adversarial loss is employed to improve the visual quality of the denoised image. In the discriminator, the distribution difference between the generated and real images is calculated using Wasserstein distance, and the Lipschitz constraint is calculated using gradient penalty. The specific formula is as follows:
[0042]
[0043] The network is continuously trained using this loss function.
[0044] Furthermore, in step 6, ablation experiments were conducted on the two datasets used, and the proposed method was compared with state-of-the-art methods. Experimental results show that the proposed DUGL-Net outperforms other state-of-the-art methods, indicating that the method is effective in LDCT reconstruction.
[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0046] 1. This invention implements a deep dual-domain joint guided learning framework, which utilizes the advantages of both projection domain and image domain networks to reconstruct LDCT images. Unlike previous domain-based progressive solutions, this invention adopts a domain-guided strategy, in which the projection domain network performs noise and artifact reduction, and the LDCT image serves as a supplement, jointly guiding the image domain network to reconstruct LDCT images, thus making full use of the information from LDCT.
[0047] 2. This invention implements a hierarchical combination of the Swing Transformer and CNN mechanisms into an encoder-decoder architecture to capture comprehensive and complementary features. A GSTEncoder is designed, containing multiple hierarchical Swing Transformer blocks to extract global long-range features; simultaneously, an LCEncoder is designed, containing multiple CNN blocks to extract local features in the dual-encoder U-shaped structure.
[0048] 3. This invention implements a novel differentiated fusion module for hierarchical combination between GSTEncoder and LCEncoder, so as to fully aggregate and transfer global and local features, enabling more full utilization of global and local features during LDCT reconstruction.
[0049] 4. This invention uses a joint training approach, which trains the model simultaneously using multiple loss functions, thereby improving the model's stability and generalization ability, making it applicable to different LDCT datasets and clinical application scenarios. Attached Figure Description
[0050] Figure 1 The overall flowchart of LDCT image reconstruction based on deep dual-domain joint guided learning provided by the present invention;
[0051] Figure 2 A flowchart for LDCT reconstruction using the traditional two-domain method;
[0052] Figure 3 This is a diagram showing the overall structure of the GL-TCUNet implemented in this invention.
[0053] Figure 4 Here is a structural diagram of the differential fusion module (DFM) implemented in this invention:
[0054] Figure 5 To compare the reconstruction performance of this invention on LDCT with other state-of-the-art algorithms on the aapm dataset, the display window is [-360, 40] HU:
[0055] Figure 6 for Figure 5 Enlarged ROI of the medium rectangular region:
[0056] Figure 7 The image shows the axial reconstruction results of the actual data acquired by the Siemens Somtom Sensation 64-slice CT scanner using different reconstruction methods. The image reconstructed by the conventional dose projection of FBP is used as a reference, and the display window is [0,250]HU. Detailed Implementation
[0057] The technical solutions provided by the present invention will be described in detail below with reference to specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention.
[0058] This invention provides a method for LDCT image reconstruction based on deep dual-domain joint guided learning. First, low-dose computed tomography (CT) projection domain data is acquired from clinical scans. This data is then input into a projection domain network for processing. By extracting and fusing local and global features, excellent processing results are achieved. The output is then transformed to the image domain using the FBP algorithm, serving as the image to be reconstructed. Furthermore, the low-dose CT projection domain data acquired from clinical scans is directly transformed to the projection domain using the FBP algorithm, serving as a guide image. The image to be reconstructed and the guide image are used as input to the image domain network to obtain the final reconstruction result, thus realizing LDCT image reconstruction based on deep dual-domain joint guided learning. Finally, the LDCT image reconstruction result based on deep dual-domain joint guided learning is compared with results obtained using other algorithms to evaluate the performance of the network model.
[0059] Specifically, the overall process of this invention is as follows: Figure 1 As shown, it includes the following steps:
[0060] Step 1, Prepare training data: Obtain low-dose CT clinical data and its corresponding normal dose data. The low-dose CT is 1 / 4 of the normal dose. The training data comes from AAPM challenge data and real data collected on a Siemens SomtomSensation 64-slice CT scanner.
[0061] Step 2: Construct the neural network for projection domain reconstruction, setting the projection domain network as a residual learning network. The neural network for projection domain reconstruction is a novel global-local Transformer-Convolution U-Net, which is based on a typical U-Net architecture and is constructed from four collaborative elements: GSTEncoder, LCEncoder, Differentiated Fusion Model (DFM), and TransDecoder.
[0062] (1)Global Swin Transformer Encoder(GSTEncoder)
[0063] GSTDencoder performs context-aware global long-range feature extraction. Inspired by the great success of the Transformer in computer vision tasks, the Swin Transformer is used for LDCT reconstruction. Unlike typical ViTs, the Swin Transformer introduces hierarchical feature map extraction between adjacent windows to extract global content-based features.
[0064] (2) The LCEncoder performs hierarchical short-range local feature extraction. Corresponding to the GSTDecoder, the LCEncoder consists of four levels of convolutional blocks. Each block includes a convolutional layer with a kernel size of 333, batch normalization (BN), and rectified linear unit (ReLU) activation functions. A maxpooling layer with a stride of 2 is used for downsampling.
[0065] (3) The Differential Fusion Module (DFM) is designed to be non-linear, tightly connecting and transmitting hierarchical global and local features. These hierarchical feature maps differ in size for the designed GSTEncoder and LCEncoder. Combining the global features generated by the GSTEncoder with the local features generated by the LCEncoder is crucial and determines the final reconstruction performance.
[0066] (4) TransDecoder upsamples the extracted deep features and obtains the final reconstruction result. Corresponding to the patch merging layer in GSTEncoder, the patch expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps and reduces the feature dimensions to half of the original dimensions accordingly.
[0067] Step 3: The low-dose projection domain data and the projection domain data output by the projection domain reconstruction network are reconstructed using the FBP algorithm to obtain image domain data. The FBP algorithm (Filtered Back Projection Algorithm) is a filtering-based reconstruction algorithm used to convert X-ray projection data into two-dimensional or three-dimensional images. The FBP algorithm mainly consists of two steps: filtering and back projection. Filtering involves performing a Fourier transform on the projection data and filtering the transform result. Filtering methods can include Butterworth filtering, Ram-Lak filtering, etc. Back projection projects the filtered data back into points in space along different angles, forming a three-dimensional dataset. Specifically, for each angle, the projection data passing through that angle is superimposed at the corresponding position and multiplied by a weighting factor to obtain the pixel value of each spatial point.
[0068] Step 4: Construct a neural network for image domain reconstruction. Using the two image domain data obtained in Step 3 as input, the final low-dose CT reconstructed image is obtained after network processing. The neural network for image domain reconstruction has the same structure as the neural network for projection domain reconstruction. The two projection domain data obtained in Step 3 are stitched together and used as input to the image domain network. After network processing, the final low-dose CT reconstructed image is obtained.
[0069] Step 5: Construct the loss function and train the projection domain and image domain networks. For the proposed DUGL-Net, two loss functions L are designed. s and L I These constraints are applied to Sino-Net and Image-Net, respectively. They are used to adjust all involved network components, guiding their updates towards high-quality reconstruction. The specific expressions are:
[0070]
[0071] in:
[0072]
[0073] M P M represents the projection domain network. I This represents an image domain network, where [a,b] denotes the concatenation operation between a and b, and R is the Filter Back Projection (FBP) operator. α and β are the weight parameters of these two loss functions. Specifically:
[0074] L s (M p (P LD ,P RD )=L1(M p (P LD ,P RD )
[0075]
[0076] L1 loss is used in our network to reconstruct LDCT images and accelerate training. The L1 loss formula is expressed as follows:
[0077]
[0078] Adversarial loss is employed to improve the visual quality of the denoised image. In the discriminator, the distribution difference between the generated and real images is calculated using Wasserstein distance, and the Lipschitz constraint is calculated using gradient penalty. The specific formula is as follows:
[0079]
[0080] The network is continuously trained using this loss function.
[0081] Step 6, Testing and Evaluation: Train the network on the training sets of the two datasets respectively, and test and evaluate it on the corresponding test sets. Compare the LDCT image reconstruction results based on deep dual-domain joint guided learning with the reconstruction results of traditional algorithms, and evaluate the performance of the network model.
[0082] To verify the effectiveness of the LDCT image reconstruction algorithm based on deep dual-domain joint guided learning disclosed in this invention, data from the AAPM challenge and real data acquired on a Siemens Somtom Sensation 64-slice CT scanner were used. A comparative experiment was conducted using human chest data as an example. Figure 6 This paper presents a comparison chart of LDCT reconstruction results obtained using several other state-of-the-art methods and the reconstruction algorithm of this invention on the AAPM challenge dataset. Figure 7 The presentation shows a comparison of LDCT reconstruction results obtained using several state-of-the-art methods and the reconstruction algorithm of this invention on real data. The comparison demonstrates that our proposed DUGL-Net is effective in noise removal and restoring good visual quality and clear images. Furthermore, due to its domain-guided strategy, our DUGL-Net preserves spatial texture and avoids introducing artifacts in the projection domain. The proposed DUGL-Net consistently outperforms state-of-the-art methods in LDCT reconstruction tasks, significantly improving reconstruction results and possessing profound significance for clinical medical research.
[0083] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications are also considered within the scope of protection of this invention.
Claims
1. A method of LDCT image reconstruction based on deep dual-domain joint guided learning, characterized in that, Includes the following steps: Step 1: Prepare training data and obtain low-dose CT clinical data and its corresponding normal dose data. The low-dose CT is 1 / 4 of the normal dose. Step 2: Build a neural network for projection domain reconstruction, and set the projection domain network as a residual learning network; Step 3: The FBP algorithm is used to reconstruct the low-dose projection domain data and the projection domain data output by the projection domain reconstruction network to obtain image domain data respectively. Step 4: Build a neural network for image domain reconstruction. Using the two image domain data obtained in Step 3 as input, the final low-dose CT reconstructed image is obtained after network processing. Step 5: Construct the overall image reconstruction network DUGL-Net, design the loss function, and train the projection domain and image domain networks in DUGL-Net; Step 6: Testing and evaluation. Train the converged network on the training sets of the two datasets respectively, and test and evaluate it on the corresponding test sets. In step 2, the neural network for projection domain reconstruction is a novel global-local Transformer-Convolution U-Net, which is based on a typical U-Net architecture and is constructed from four cooperative elements: GSTEncoder, LCEncoder, differential fusion model, and TransDecoder. GSTEncoder performs context-based global remote feature extraction; inspired by the great success of Transformer in computer vision tasks, Swing Transformer is used for LDCT reconstruction; unlike typical ViT, Swing Transformer introduces hierarchical feature map extraction between adjacent windows to extract global content-based features. LCEncoder performs hierarchical short-range local feature extraction; corresponding to GSTEncoder, LCEncoder consists of four levels of convolutional blocks; each block includes a convolutional layer with a kernel size of 3×3, batch normalization and corrected linear unit activation functions; a Maxpooling layer with a stride of 2 is used for downsampling; The differential fusion model is designed to be non-linear, tightly connecting and transmitting hierarchical global and local features; for the designed GSTEncoder and LCEncoder, these hierarchical feature maps are of different sizes; combining the global features generated by GSTEncoder with the local features generated by LCEncoder is the key and determines the final reconstruction performance. TransDecoder upsamples the extracted deep features and obtains the final reconstruction result; corresponding to the patch merging layer in GSTEncoder, the patch expansion layer reshapes the feature maps of adjacent dimensions into higher resolution feature maps and reduces the feature dimensions to half of the original dimensions accordingly.
2. The LDCT image reconstruction method based on deep dual-domain joint guided learning of claim 1, wherein, In step 1, the training data comes from AAPM challenge data and real data collected on a Siemens Somtom Sensation 64-slice CT scanner.
3. The LDCT image reconstruction method based on deep dual-domain joint guided learning of claim 1, wherein, In step 3, the FBP algorithm is a filtering-based reconstruction algorithm used to convert X-ray projection data into two-dimensional or three-dimensional images. The FBP algorithm consists of two steps: filtering and backprojection. Filtering involves performing a Fourier transform on the projection data and filtering the transformation result. Filtering methods include Butterworth filtering and Ram-Lak filtering. Backprojection involves backprojecting the filtered data back to points in space along different angles to form a three-dimensional dataset. Specifically, for each angle, the projection data passing through that angle is superimposed at the corresponding position and multiplied by a weighting factor to finally obtain the pixel value of each spatial point.
4. The LDCT image reconstruction method based on deep dual-domain joint guided learning of claim 1, wherein, Step 4 specifically includes the following processes: building a neural network for image domain reconstruction, the structure of which is exactly the same as that of the neural network for projection domain reconstruction; stitching the two image domain data obtained in step 3 together as the input to the image domain network; and obtaining the final low-dose CT reconstructed image after network processing.
5. The LDCT image reconstruction method based on deep dual-domain joint guided learning of claim 1, wherein, In step 5, two loss functions are designed for the proposed DUGL-Net and , which restrict the projection domain network and the image domain network respectively; they are used to adjust all the involved network components to update towards the direction of high-quality reconstruction; the specific expressions are as follows: in: denotes a projection domain network, denotes an image domain network, denotes a concatenation operation, and is a filtered back-projection (FBP) operator; and are weight parameters for the two loss functions; in particular: loss for network reconstruction low-dose CT images and accelerate training, The loss formula is expressed as follows: Adversarial loss is employed to improve the visual quality of the denoised image. In the discriminator, the distribution difference between the generated image and the real image is calculated using Wasserstein distance, and the Lipschitz constraint is calculated using gradient penalty. The specific formula is as follows: Finally, the above loss function is used to train the network.
6. The LDCT image reconstruction method based on deep dual-domain joint guided learning of claim 1, wherein, In step 6, ablation experiments were conducted on two datasets, and the proposed method was compared with state-of-the-art methods. The experimental results show that the proposed DUGL-Net outperforms other state-of-the-art methods, indicating that the method is effective in low-dose CT image reconstruction.