A deep learning sparse angular CT reconstruction method based on frequency domain and image domain degradation perception
By adopting frequency domain and image domain degradation perception modules in CT reconstruction, combining multi-degradation-level data sets and supervised training, the poor performance of deep learning CT reconstruction methods on data of different degradation degrees is solved, and higher quality is achieved CT image reconstruction effect.
Patent Information
- Application Number
- CN202111472842.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-01
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-01
AI Technical Summary
The existing deep learning CT reconstruction methods have poor performance when processing data of different degrees of degradation, and the increase in training computation and parameter storage volume is not scalable, which leads to difficulty in deployment in practical applications.
Using the degeneration perception module based on frequency domain and image domain, by constructing multi-degradation level data sets and supervised training, we learn the characteristics of data at different degradation levels, and design the frequency domain and image domain reconstruction model, combining iterative algorithms and attention mechanisms , improve the prediction effect and stability of the model.
The quality of CT image reconstruction is significantly improved, noise and artifacts are suppressed, the detailed structure and texture of reconstruction are improved, and the reconstruction effect is achieved.
Smart Images

Figure CN114140442B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing technology, and more specifically, relates to a deep learning sparse angle CT reconstruction method based on frequency domain and image domain degradation perception. Background Art
[0002] CT reconstruction is the process of reconstructing CT images from projection data obtained through CT scanning through algorithms. In the past fifty years, CT images have been widely used in clinical diagnosis, nondestructive testing and biological research due to their high resolution and high sensitivity. With the continuous development of medical CT technology, people have put forward requirements for faster, safer and more accurate CT technology. However, high doses of radiation can cause headaches, and in severe cases can even cause cancer and leukemia. In addition, long scanning times and high scanning frequencies can further increase the harm. Sparse angle CT and limited angle CT reduce the number of measurements by sparse projection and controlling the projection angle range, respectively. However, due to insufficient data acquisition, the reconstructed CT images inevitably have serious streak artifacts and directional artifacts. Therefore, the research on low-dose CT reconstruction has received widespread attention from researchers.
[0003] CT reconstruction methods can be divided into three categories: projection domain reconstruction, iterative reconstruction, and image domain reconstruction. In the projection domain reconstruction method, the traditional filtering algorithm has the advantages of small computational complexity and fast reconstruction speed, but it cannot achieve satisfactory performance when the original data is severely missing. Dictionary-based and deep learning-based methods often lead to undesirable artifacts or over-smoothing in the reconstruction results. With the help of prior knowledge, iterative methods can achieve better reconstruction results, but at the same time they sacrifice a lot of computing resources.
[0004] In recent years, deep learning methods have become particularly important in image-domain CT reconstruction. Many convolutional neural network-based models have significantly outperformed iterative algorithms at a specific degree of damage. However, current deep learning reconstruction methods often apply supervised learning on data with a single degree of degradation (e.g., sparse projection data at 120 angles), and therefore cannot achieve good reconstruction performance on data with other degrees of degradation (e.g., sparse projection data at 60 angles). A direct approach to this problem is to train a set of parameters for each degradation level, but deployment in practice is a huge challenge due to the growth in training computation and parameter storage. In addition, as the number of degradation levels increases, the training cost and parameter storage increase linearly, which is not scalable in practical applications. Another approach is to alleviate this problem by mixing data from multiple degradation levels to construct a training dataset, but this will cause the model to predict compromised reconstruction results, and there is still room for performance improvement. Summary of the invention
[0005] The purpose of the present invention is to overcome the shortcomings of the existing deep learning-based CT reconstruction method and provide a deep learning CT reconstruction algorithm based on degradation perception. By designing degradation perception modules in the frequency domain and image domain respectively, the characteristics of data with different degradation levels are learned, and different operations are performed on data with different degradation levels accordingly. In addition, by combining CT data sets containing multiple degradation levels with supervised training, the prediction effect and prediction stability of the model are improved.
[0006] To achieve the above purpose, this paper proposes a deep learning sparse angular CT reconstruction method based on frequency domain and image domain degradation perception, comprising the following steps:
[0007] (1) Construct a dataset containing multiple degradation levels
[0008] Collect full-angle CT images of different parts of the patient, forward project the CT images to obtain sufficient projection data, then perform sparse sampling of the projection data at different levels (for example, 60 angles, 120 angles, 240 angles, etc.) and add noise, and use an iterative algorithm to reconstruct the sparse projection data into sparse angle CT images. Each sparse angle CT image and full angle CT image constitute a training pair;
[0009] (2) Constructing a frequency domain reconstruction model
[0010] (2.1) Construct a discrete cosine transform module and convert the block discrete cosine transform into a convolutional layer. The block size is N×N, and the convolutional layer consists of N 2 The input of the discrete cosine transform module is the sparse angle CT image, and the output is the frequency feature.
[0011] (2.2) Construct a frequency domain attention module. The input of the frequency domain attention module is the frequency feature, and the output is the weighted frequency feature. The network structure between the input layer and the output layer is as follows:
[0012] The first layer is the global average pooling layer;
[0013] The second layer consists of a fully connected layer and a linear rectification function (ReLU);
[0014] The third layer consists of a fully connected layer and a Sigmoid function;
[0015] The fourth layer is a pixel-by-pixel multiplication operation: the input of the first layer and the output of the third layer are multiplied pixel by pixel;
[0016] (2.3) Construct a frequency domain reconstruction module. The frequency domain reconstruction module can be designed as any neural network model. The input of the frequency domain reconstruction module is the weighted frequency feature, and the output is the frequency prediction result.
[0017] (2.4) Construct an inverse discrete cosine transform module to convert the block inverse discrete cosine transform into a transposed convolution layer. The block size is N×N, and the convolution layer consists of N 2 The input of the discrete cosine transform module is the frequency prediction result, and the output is the frequency domain reconstructed image.
[0018] (3) Constructing image domain reconstruction model
[0019] (3.1) Construct an image domain attention module. The image domain attention module can be designed as any neural network model. The input of the image domain attention module is the splicing of the sparse angle CT image and the frequency domain reconstructed image (channel splicing), and the output is the image domain attention prediction image;
[0020] (3.2) Construct an image domain reconstruction module. The image domain reconstruction module can be designed as any neural network model. The input of the image domain reconstruction module is the concatenation of the frequency domain reconstructed image and the image domain attention prediction image, and the output is the final reconstruction result.
[0021] (4) Training frequency domain model
[0022] Use training pairs (sparse angle CT images, full angle CT images) to train the weight parameters of the frequency domain model;
[0023] (5) Construct a training dataset for the image domain attention module
[0024] (5.1) Use edge detection algorithm to extract the edge of the full angle CT image to obtain an edge detection image;
[0025] (5.2) Using the weight parameters trained in (4), the sparse angle CT image is reconstructed into a frequency domain reconstructed image using the frequency domain model. The difference between the frequency domain reconstructed image and the full angle CT image is calculated and then the absolute value is taken. The pixel points with a difference greater than or equal to r are set to 1, and the pixel points with a difference less than r are set to 0 to obtain the reconstructed difference image.
[0026] (5.3) Take the intersection of the edge detection image and the reconstructed difference image to obtain the image domain attention ideal image. Each (the splicing of the sparse angle CT image and the frequency domain reconstructed image, the image domain attention ideal image) constitutes a training sample for the image domain attention module;
[0027] (6) Training the image domain attention module
[0028] The weight parameters of the image domain attention module are trained using training pairs (the concatenation of sparse angle CT images and frequency domain reconstructed images, and the image domain attention ideal image). The loss function uses the binary cross entropy loss function.
[0029] (7) Training image domain reconstruction module
[0030] Use training pairs (the concatenation of frequency domain reconstructed images and image domain attention prediction images, and fully angled CT images) to train the weight parameters of the image domain reconstruction module;
[0031] (8) Overall training
[0032] The weight parameters of the image domain attention module are frozen, and the weight parameters of the frequency domain model and the image domain reconstruction module are trained as a whole using the training pairs (sparse angle CT images, full angle CT images). BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a general diagram of the network structure;
[0034] Figure 2 This is a flowchart of the steps of a deep learning sparse angle CT reconstruction method based on frequency domain and image domain degradation perception of the method of the present invention;
[0035] Figure 3 Schematic diagram of the network structure of the discrete cosine transform module and the frequency domain attention module in the embodiment;
[0036] Figure 4 is a schematic diagram of a network structure diagram of a frequency domain reconstruction module in an embodiment;
[0037] Figure 5 is a schematic diagram of the network structure of the image domain attention module in the embodiment;
[0038] Figure 6 is a schematic diagram of the network structure of the image domain reconstruction module in the embodiment;
[0039] Figure 7 is a schematic diagram of a training data set of an image domain attention module in an embodiment;
[0040] Figure 8 1 and 2 are image comparison results of different processing methods in the embodiment. DETAILED DESCRIPTION
[0041] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.
[0042] Example:
[0043] In this embodiment, the present invention provides a deep learning sparse angle CT reconstruction method based on frequency domain and image domain degradation perception, comprising the following steps:
[0044] (1) Construct a dataset containing multiple degradation levels
[0045] Full-angle CT images of different parts of the patient are collected, downsampled to 512×512, and fan-beam projection is simulated to forward project each CT image to obtain full-angle projection data of 360 projection angles. Gaussian and Poisson noise are added to the projection data, and then randomly and uniformly sparsely sampled to 60, 120, and 240 projection angles. Finally, the filtered back-projection reconstruction algorithm is used to reconstruct the sparse projection data into sparse angle CT images. The training set, validation set, and test set contain 9203, 300, and 1000 training pairs, respectively;
[0046] (2) Constructing a frequency domain reconstruction model
[0047] (2.1), construct discrete cosine transform module, such as Figure 3 As shown in the figure, a discrete cosine transform with a block size of 8×8 is used. The convolution layer consists of 64 convolution kernels with a size of 8×8. The weight parameters are initialized as discrete cosine basis functions. The step size of the convolution operation is 8 to ensure that there is no overlap between image blocks.
[0048] (2.2) Construct the frequency domain attention module. Figure 3 As shown, the network structure between the input layer and the output layer is as follows:
[0049] The first layer is the global average pooling layer;
[0050] The second layer consists of a fully connected layer (containing 32 neurons) and a linear rectification function (ReLU);
[0051] The third layer consists of a fully connected layer (containing 64 neurons) and a Sigmoid function;
[0052] The fourth layer is a pixel-by-pixel multiplication operation: the input of the first layer and the output of the third layer are multiplied pixel by pixel;
[0053] (2.3) Construct the frequency domain reconstruction module. Figure 4 As shown in the figure, the frequency domain reconstruction module is designed as a U-Net structure, which includes operations such as convolution, maximum pooling, and maximum anti-pooling;
[0054] (2.4) Construct an inverse discrete cosine transform module to convert the block inverse discrete cosine transform into a transposed convolution layer. The block size is 8×8, the convolution layer consists of 64 convolution kernels of size 8×8, and the initial weight parameter is the discrete cosine basis function;
[0055] (3) Construct an image domain reconstruction model.
[0056] (3.1) Construct the image domain attention module. Figure 5 As shown in the figure, the image domain attention module is designed as a U-Net structure, which includes operations such as convolution, downsampling, and upsampling;
[0057] (3.2) Construct the image domain reconstruction module. Figure 6 As shown in the figure, the image domain reconstruction module is designed as a residual network model, which includes 6 residual modules. The network includes operations such as convolution and ReLU.
[0058] (4) Training frequency domain model
[0059] The weight parameters of the frequency domain model were trained using training pairs (sparse angle CT images, full angle CT images), using mean square error as the loss function, batch size of 32, using ADAM optimizer for gradient update, and the initial learning rate of 10 -4 , every 10 5 The learning rate was reduced by half in each iteration, and training was stopped when the validation set loss no longer decreased. The training process was completed on a GPU (GeForce GTX 1080Ti);
[0060] (5) Construct a training dataset for the image domain attention module
[0061] (5.1), such as Figure 7 As shown, the Canny edge detection operator is used to extract the edge of the full angle CT image to obtain an edge detection image;
[0062] (5.2) Using the weight parameters trained in (4), the sparse angle CT image is reconstructed into a frequency domain reconstructed image using the frequency domain model. The difference between the frequency domain reconstructed image and the full angle CT image is calculated and then the absolute value is taken (normalized to between 0 and 1). The pixels with a difference greater than or equal to 0.01 are set to 1, and the pixels with a difference less than r are set to 0 to obtain the reconstructed difference image.
[0063] (5.3) Take the intersection of the edge detection image and the reconstructed difference image to obtain the ideal image of image domain attention;
[0064] (6) Training the image domain attention module
[0065] The weight parameters of the image domain attention module are trained using training pairs (the concatenation of sparse angle CT images and frequency domain reconstructed images, and the image domain attention ideal image). The loss function uses the binary cross entropy loss function, the batch size is 4, and the ADAM optimizer is used for gradient update. The initial learning rate is 10 -4 , every 10 5The learning rate was reduced by half in each iteration, and training was stopped when the validation set loss no longer decreased. The training process was completed on a GPU (GeForce GTX 1080Ti);
[0066] (7) Training image domain reconstruction module
[0067] The weight parameters of the image domain reconstruction module are trained using training pairs (the concatenation of frequency domain reconstructed images and image domain attention prediction images, and full angle CT images); the loss function uses the L1 loss function, the batch size is 8, the ADAM optimizer is used for gradient update, and the initial learning rate is 10 -4 , every 10 5 The learning rate was reduced by half in each iteration, and training was stopped when the validation set loss no longer decreased. The training process was completed on a GPU (GeForce GTX 1080Ti);
[0068] (8) Overall training
[0069] The weight parameters of the image domain attention module are frozen, and the weight parameters of the frequency domain model and the image domain reconstruction module are trained as a whole using training pairs (sparse angle CT images, full angle CT images). The loss function uses the L1 loss function, the batch size is 1, and the ADAM optimizer is used for gradient update. The initial learning rate is 10 -4 , every 10 5 The learning rate was reduced by half in each iteration, and the training was stopped when the validation set loss no longer decreased. The training process was completed on a GPU (GeForce GTX 1080Ti).
[0070] The embodiment is tested on the test set, and PSNR and SSIM are selected as evaluation criteria, and the calculation method is as follows:
[0071]
[0072]
[0073]
[0074] Where I(i,j) and K(i,j) represent the final reconstruction result and the full angle CT image of size m×n, MAX is the maximum pixel value of the image, μ x , μ y Respectively represent the average values of I(i,j) and K(i,j), σ x , σ y , σ xy Respectively represent the variance of I(i,j), the variance of K(i,j) and their covariance, c1=(0.01*L) 2, c2=(0.03*L) 2 , L is the range of pixel values.
[0075] For the convenience of description, the relevant professional terms that appear are explained:
[0076] FBP: Filtered Back Projection Reconstruction Algorithm
[0077] TU-Net: A reconstruction algorithm based on deep learning: Han Y, Ye J C. Framing U-Net via deep convolutional framelets: Application to sparse-view CT[J]. IEEE transactions on medical imaging, 2018, 37(6): 1418-1429.
[0078] The reconstruction effect is shown in Table 1:
[0079] Table 1 Reconstruction effect comparison table
[0080]
[0081]
[0082] Figure 8 The reconstruction results of different methods at 120 projection angles are shown. The results show that the method of the embodiment can effectively suppress noise and artifacts, reconstruct detailed structures and textures, and achieve better reconstruction effects.
Claims
1. A deep learning sparse angular CT reconstruction method based on frequency domain and image domain degradation perception, The following steps are involved: (1) Construct a dataset containing multiple degradation levels Collect full-angle CT images of different parts of the patient, forward project the CT images to obtain sufficient projection data, then perform sparse sampling at different levels on the projection data and add noise, and use an iterative algorithm to reversely reconstruct the sparse projection data into sparse angle CT images. Each sparse angle CT image and full angle CT image constitute a training pair; (2) Constructing a frequency domain reconstruction model (2.1) Construct a discrete cosine transform module to convert the block discrete cosine transform into a convolutional layer. Its specific characteristics are that the block size is N×N and the convolutional layer consists of N 2 The input of the discrete cosine transform module is the sparse angle CT image, and the output is the frequency feature. (2.2) Construct a frequency domain attention module. The specific feature of the frequency domain attention module is that the input is the frequency feature and the output is the weighted frequency feature. The network structure between the input layer and the output layer is as follows: The first layer is the global average pooling layer; The second layer consists of a fully connected layer and a linear rectification function (ReLU); The third layer consists of a fully connected layer and a Sigmoid function; The fourth layer is a pixel-by-pixel multiplication operation: the input of the first layer is multiplied pixel by pixel by the output of the third layer; (2.3) Construct a frequency domain reconstruction module, wherein the specific characteristics of the frequency domain reconstruction module are that the frequency domain reconstruction module is a neural network model, the input of the frequency domain reconstruction module is a weighted frequency feature, and the output is a frequency prediction result; (2.4) Construct an inverse discrete cosine transform module to convert the block inverse discrete cosine transform into a transposed convolutional layer. Its specific characteristics are that the block size is N×N, and the convolutional layer consists of N 2 The input of the discrete cosine transform module is the frequency prediction result, and the output is the frequency domain reconstructed image. (3) Constructing image domain reconstruction model (3.1) Construct an image domain attention module. The specific features of the image domain attention module are as follows: the image domain attention module is a neural network model, the input of the image domain attention module is the channel concatenation of the sparse angle CT image and the frequency domain reconstructed image, and the output is the image domain attention predicted image; (3.2) construct an image domain reconstruction module, whose specific features are that the image domain reconstruction module is a neural network model, the input of the image domain reconstruction module is the splicing of the sparse angle CT image and the image domain attention prediction image, and the output is the final reconstruction result; (4) Training frequency domain model The sparse angle CT images and the full angle CT images are used as training pairs to train the weight parameters of the frequency domain model; (5) Construct a training dataset for the image domain attention module (5.1) Use edge detection algorithm to extract the edge of the full angle CT image to obtain an edge detection image; (5.2) Using the weight parameters trained in (4), the sparse angle CT image is reconstructed into a frequency domain reconstructed image using the frequency domain model. The difference between the frequency domain reconstructed image and the full angle CT image is calculated and then the absolute value is taken. The pixel points with a difference greater than or equal to r are set to 1, and the pixel points with a difference less than r are set to 0 to obtain the reconstructed difference image. (5.3) Take the intersection of the edge detection image and the reconstructed difference image to obtain the image domain attention ideal image, perform channel splicing on the sparse angle CT image and the frequency domain reconstructed image to obtain the input image, and construct the input image and the image domain attention ideal image as the image domain attention module training sample; (6) Training the image domain attention module The sparse angle CT image and the frequency domain reconstructed image are channel-stitched, and the channel-stitched image and the image domain attention ideal image are used as training pairs to train the weight parameters of the image domain attention module. The loss function uses the binary cross entropy loss function. (7) Training image domain reconstruction module The frequency domain reconstructed image and the image domain attention prediction image are channel-wise stitched as the input image, and the input image and the full angle CT image are used as training pairs to train the weight parameters of the image domain reconstruction module; (8) Overall training The weight parameters of the image domain attention module are frozen, and the sparse angle CT images and full angle CT images are used as training pairs. The weight parameters of the frequency domain model and the image domain reconstruction module are trained simultaneously.
