Single-view CT reconstruction method based on progressive learning
By combining attention mechanism and adaptive instance normalization in CT reconstruction, the progressive learning strategy is adopted to solve the limitations of existing CT reconstruction methods in image quality and radiation, and the single-sheet High-quality CT image reconstruction in X-ray images reduces radiation risk and improves reconstruction efficiency.
Patent Information
- Application Number
- CN202110308254.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-23
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-03-23
AI Technical Summary
Existing CT reconstruction methods reduce image quality when reducing the number of X-ray images, and the single-view CT method based on deep learning has limitations in training and image quality, especially in scenarios with very few angles and extreme scanning angles, which are difficult to meet medical conditions. Diagnostic requirements.
A convolutional neural network combining attention mechanism and adaptive instance normalization was designed, and a progressive learning strategy was used to reconstruct from a single X-ray image through feature encoding and decoding, 2D to 3D feature transformation and feature fusion structure. High-quality CT images use multi-scale resolution images to train the network and perform network amplification and parameter transfer to improve image quality.
It effectively improves the quality of single-view CT reconstruction images, reduces the total amount of X-ray radiation, reduces the radiation damage and potential risks of patients, and provides cancer patients with image guidance information for precise radiotherapy.
Smart Images

Figure CN113052935B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computed tomography (CT) image reconstruction, and in particular to a single-view CT reconstruction method of progressive learning using deep learning technology. Background Art
[0002] With the increasing importance of CT technology in the field of medical auxiliary diagnosis and treatment, CT has become a very common examination technology. However, the radiation damage and risks caused by X-rays during the examination have also attracted more and more attention and attention. The current common CT scanning reconstruction method requires obtaining a large number of X-ray images of the patient. For more advanced 4D CT, more X-ray images are required. This will invisibly increase the radiation damage to the patient, especially for those cancer patients and pneumonia patients who need to undergo multiple CT examinations. In order to reduce radiation, some methods have been tried to reconstruct CT from low-dose X-ray images, but this will also reduce the quality of the reconstructed image. More research is to reduce the total radiation by reducing the number of X-ray images obtained. The Chinese patent "A limited angle CT image reconstruction method based on DTw-SART-TV iterative process" (CN112381904A) proposes a corresponding limited angle, and the Chinese patent "A sparse angle X-ray CT imaging method" (CN103136773A) proposes a sparse angle CT reconstruction method. Among them, Chinese patent CN112381904A combines the SART and IRS steps, and continuously iterates the calculation through two stages to reconstruct the CT image from the X-ray projection collected from a limited angle. Chinese patent CN103136773A, on the other hand, needs to collect X-ray images at all angles and perform CT reconstruction in advance, and then use the optimization model to iteratively solve the final CT image. These two types of CT reconstruction algorithms need to be combined with a priori models to reconstruct images in an iterative manner. When reconstructing high-resolution images, these two methods require longer calculation time, and the reconstruction efficiency is also lower than that of traditional analytical CT reconstruction methods. In addition, the limited-angle method will not be able to iteratively reconstruct high-quality images in some scenarios where the scanning angle is very limited due to severe data loss. Solving the problem of CT reconstruction at very few angles, thereby greatly reducing the radiation risk of patients, is an objective of the present invention.
[0003] In recent years, the rapid development of deep learning technology has opened up new directions for traditional CT technology. Limited-angle and sparse-angle CT reconstruction methods based on deep learning continue to emerge. Since the learning-based model has a reconstruction efficiency far exceeding that of traditional CT after training, this type of method has a very broad development prospect. However, the problem with this type of method is that on the one hand, a large amount of CT data about patients is required for training the model, which is usually difficult to obtain. On the other hand, in some scenarios where the CT scanning angle is limited or the number of collected projections is close to the limit, the quality of CT images reconstructed by the above-mentioned deep learning-based methods may not meet the requirements of medical diagnosis. Therefore, this type of method also has certain limitations. However, the powerful ability of deep learning technology in extracting image information and constructing nonlinear models has led some researchers to further study it. CT reconstruction is achieved when the scanning range and the number of X-ray images are both minimized, that is, CT image reconstruction is achieved from a single X-ray image (referred to as single-view CT), which greatly reduces the total amount of radiation. However, since this extremely challenging problem is complex and has pathological solutions, it is not easy to build a suitable deep learning network and design a training strategy that can stabilize network training. This means that although the current single-view based deep learning method can reconstruct a CT result that looks similar from a single X-ray image, the quality and structural details of the reconstructed CT image still need to be improved. Therefore, further improving the quality of CT reconstructed images is another goal of this invention. By combining the characteristics of CT images, the present invention constructs a more suitable network. In addition, the present invention also designs a corresponding network training strategy to make network training more stable and effectively improve image quality. Summary of the invention
[0004] In order to improve the quality of single-view CT reconstruction images, the present invention designs a convolutional neural network (CT reconstruction network) that combines attention mechanism and adaptive instance normalization. The network is trained by the progressive learning strategy proposed by the present invention, which can make the network training more stable and effectively improve the quality of reconstructed images.
[0005] To achieve the above-mentioned object of the invention, the technical solution of the present invention is as follows:
[0006] A progressive learning single-view CT reconstruction method comprises the following steps:
[0007] Step 1: Dataset creation and preprocessing.
[0008] The dataset contains a large number of paired X-ray images and corresponding CT volume images. The paired image data are processed by first calculating the mean μ and variance σ, and then normalizing them by (x-μ) / σ, where x represents the X-ray image or CT volume image to be standardized; the dataset is then divided into a training set and a test set for network training and prediction, respectively.
[0009] Step 2: Build a CT reconstruction network to reconstruct CT images from a single X-ray image.
[0010] The network includes three structures: a feature encoding and decoding structure, a 2D to 3D feature conversion structure and a feature fusion structure.
[0011] The feature encoding and decoding structure includes two parts: encoding and decoding. The encoding part uses multiple convolution modules to extract and downsample the input single X-ray image, and encodes the extracted features into the hidden space. The decoding part uses multiple convolution operation modules to convolute upsample the encoded information in the hidden space, and finally restores the 3D CT volume image.
[0012] The 2D to 3D feature conversion structure is a part of the feature encoding and decoding structure, which is used to connect the features of the same level in the encoding and decoding structure to compensate for the information loss during the encoding process. The 2D to 3D feature conversion structure includes two parts: feature channel attention and feature conversion. The feature channel attention part re-assigns different weights to the features of different channels in the encoding process, and gives more attention to the features with high weights. The feature conversion part converts the features after weight assignment to make them closer to the features of the CT image.
[0013] The feature fusion structure fuses the result of the 2D to 3D feature conversion structure and the intermediate result of the decoding part in the feature encoding and decoding structure, thereby helping to improve the quality of the reconstructed CT image. The feature fusion structure includes three steps of operation, the first step of calculating the mean and variance of the output result of the 2D to 3D feature conversion structure, the second step of calculating the mean and variance of the intermediate result output by the encoding part in the feature encoding and decoding structure, and the third step of fusing the output features of the structures in the first two steps according to the mean and variance calculated in the first and second steps.
[0014] Step 3: Design a progressive learning (training) strategy for the CT reconstruction network described in step 2
[0015] The progressive training strategy is a strategy for inputting multi-scale resolution images into the CT reconstruction network and training the network, which includes two processes: network expansion and parameter transfer. Network training starts with low-resolution image input until the network training process converges, and then the network is expanded to obtain a larger new network by increasing the number of network layers. At the same time, the parameter transfer process transfers the parameters of the trained network to the new network, and then inputs a higher-resolution image to the new network, and continues network training. By continuously repeating the network expansion and parameter transfer process, the network gradually transitions from learning large-scale structures in small-resolution images to learning fine structures in larger-resolution images, thereby helping to improve the quality of reconstructed CT images.
[0016] Step 4: Use the CT reconstruction network trained in step 3 to test on the test data set. A CT volume image can be reconstructed by inputting a single X-ray image into the network.
[0017] The beneficial effects of the present invention are:
[0018] The present invention designs a single-view CT reconstruction method for reconstructing a CT volume image from a single X-ray image. By training the CT reconstruction network designed by the present invention in a progressive learning manner, on the one hand, the quality of the reconstructed CT image can be effectively improved, and effective image guidance information can be provided for precise radiotherapy of cancer patients in the future. On the other hand, the total amount of X-ray radiation can be greatly reduced, reducing the radiation damage and potential risks to patients during multiple CT examinations. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a flow chart of the steps of the method of the present invention.
[0020] Figure 2 It is the encoding and decoding structure in the CT reconstruction network of the present invention.
[0021] Figure 3 It is the feature conversion structure in the CT reconstruction network of the present invention.
[0022] Figure 4 It is the overall diagram of the progressive training strategy of the present invention.
[0023] Figure 5(a) to Figure 5(c) 5(a) is the input X-ray image, FIG5(b) is the reconstructed CT image, and FIG5(c) is the real CT image. DETAILED DESCRIPTION
[0024] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.
[0025] like Figure 1 As shown, a progressive learning single-view CT reconstruction method includes the following steps:
[0026] Step 1, data acquisition and preprocessing. This embodiment obtains a CT data set of the lungs, abdomen and brain of cancer patients from a public medical image data set, establishes a data set containing paired X-ray images and CT volume images by means of simulated projection, and divides the data set into a training set and a test set in a 7:3 manner. Each pair of data in the data set is processed by first calculating the mean and variance, and then standardizing. For example, for an X-ray image, first calculate its mean μ and variance σ, and then standardize it by (x-μ) / σ, where x represents the X-ray image to be standardized. The CT volume image is processed in the same manner.
[0027] Step 2: Build a CT reconstruction network for reconstructing CT images from a single X-ray image. The network structure is as follows: Figure 2 As shown. It includes feature encoding and decoding structure, 2D to 3D feature conversion structure, and feature fusion structure. The network fully extracts the important features of the input single X-ray image. The specially designed feature conversion network can make up for the information loss in the feature extraction process. Through the feature fusion structure, some information is fused into the output CT features, thereby improving the quality of the reconstructed CT image. The process includes the following steps:
[0028] (1) The feature encoding and decoding structure encodes the input 128×128 resolution image through a convolutional layer and a universal downsampled residual block to obtain a hidden layer encoding feature. During the encoding process, the number of feature channels gradually increases from 1 of the input to 1280, and the feature resolution is gradually downsampled from 128x128 to 8x8. During the upsampling process, the hidden layer encoding feature is decoded through the upsampling residual block and the convolutional layer, where the number of channels gradually decreases from 1280 to 128, and the feature resolution is gradually upsampled to 128×128, thereby outputting a CT volume image (128×128×128).
[0029] (2) In the codec structure, the feature downsampling process causes information loss because the resolution is continuously reduced. Therefore, the feature conversion structure converts the features before downsampling and connects the converted features to the upsampling results of the same level, thereby compensating for the information loss to a certain extent. Figure 3As shown, the dimension of the input feature is C×W×H, where C represents the number of channels, and W and H represent the width and height of the image respectively. The input features are assigned different weights to different channel features through the channel attention part, and the weights are automatically learned by the network during network training, so that the network pays attention to some important features. The channel attention part has different implementation methods. In this embodiment, the channel attention part is implemented by adaptive instance normalization. This part includes three processes: channel compression, weight activation and weighted operation, which are implemented by 1×1 convolution, full connection and dot product operation respectively. These important features are then converted to CT-like features through feature transformation. The implementation of the feature transformation is not unique. In this embodiment, the transformation includes three processes: 2×2 convolution downsampling, two-dimensional convolution and 2×2 convolution upsampling.
[0030] (3) The feature fusion structure fuses the result x of the feature conversion structure with the up-sampled result y. In this embodiment, a general adaptive instance normalization method is used for fusion. The method is as follows:
[0031]
[0032] Among them, μ(·) represents the mean and σ(·) represents the standard deviation.
[0033] Step 3: The CT reconstruction network designed by the present invention is trained on the training set by the progressive learning method. The progressive learning process is as follows: Figure 4 As shown. The training of the network starts with low-resolution input, so that the network first focuses on the large-scale structure in the low-resolution image. After the training is completed, some learned model parameters are obtained, and the model parameters are transferred to the corresponding part of the new network through the parameter transfer process. The new network expands the network by adding some new network layers, thereby improving the network's ability to learn details in larger resolution images. After continuous network training, network amplification and parameter transfer processes, the network is finally guided to improve the quality of the reconstructed CT image step by step. In this embodiment, the resolutions of the paired X-ray images and CT volume images in the training set are 128×128 and 128×128×128, respectively. X-ray images are used for network input, and CT volume images are used to supervise the results generated by the network. In the low-resolution image training stage, the input low-resolution image (16×16) is obtained by downsampling the high-resolution image (128×128). After 3 training, network amplification and parameter transfer processes, the final amplified network model outputs a CT volume image with a resolution of 128×128×128.
[0034] Step 4: Train the CT reconstruction network through the progressive learning strategy described in step 3, and finally obtain the trained model parameters. In the test set, we input X-ray images that are not used for network training into the trained model and compare them with the corresponding real CT volume images to verify the effectiveness of the present invention. Figure 5(a) to Figure 5(c) Some reconstructed CT image results are shown. It can be seen that the reconstructed CT images are very close to the real CT images and have good performance in details. Table 1 shows the quantitative evaluation results of the feature conversion structure, feature fusion structure and progressive learning strategy based on the attention mechanism in the network, which verifies the effectiveness of the present invention. It can be seen that when all network structures and strategies are applied, the reconstructed CT results of the present invention perform best in two commonly used image quality evaluation indicators (peak signal-to-noise ratio (PSNR) and structural similarity (SSIM)). In addition, when the trained network model is actually deployed and applied, it only needs to collect one X-ray image of the patient to realize the corresponding CT image reconstruction, and the time to complete a reconstruction is about 0.1 seconds.
[0035] Table 1 Quantitative evaluation results
[0036]
[0037] PG.: Progressive training strategy; Ada.: Fusion structure; Att.: Attention-based feature transformation structure.
Claims
1. A progressive learning single-view CT reconstruction method, characterized in that: Here are the steps: Step 1: Dataset establishment and preprocessing; The dataset contains multiple pairs of X-ray images and corresponding CT volume images. The paired image data are processed by first calculating the mean μ and variance σ, and then normalizing them by (x-μ) / σ, where x represents the X-ray image or CT volume image to be standardized; the dataset is then divided into a training set and a test set for network training and prediction, respectively; Step 2: Build a CT reconstruction network to reconstruct a CT image from a single X-ray image; The network includes three structures: feature encoding and decoding structure, 2D to 3D feature conversion structure and feature fusion structure; The feature encoding and decoding structure includes two parts: encoding and decoding. The encoding part uses multiple convolution modules to extract and downsample the input single X-ray image, and encodes the extracted features into the hidden space. The decoding part uses multiple convolution operation modules to convolute upsample the encoded information in the hidden space, and finally restores the 3D CT volume image. The 2D to 3D feature conversion structure is a part of the feature encoding and decoding structure, which is used to connect the features of the same level in the encoding and decoding structure, so as to make up for the information loss in the encoding process; the 2D to 3D feature conversion structure includes two parts: feature channel attention and feature conversion; the feature channel attention part re-assigns different weights to the features of different channels in the encoding process; the feature conversion part converts the features after weight distribution to make them closer to the features of the CT image; The feature fusion structure performs feature fusion on the result of the 2D to 3D feature conversion structure and the intermediate result of the encoding part in the feature encoding and decoding structure to improve the quality of the reconstructed CT image; the feature fusion structure includes three steps of operation, the first step is to calculate the mean and variance of the output result of the 2D to 3D feature conversion structure, the second step is to calculate the mean and variance of the intermediate result output of the encoding part in the feature encoding and decoding structure, and the third step is to fuse the output features of the structures in the first two steps according to the mean and variance calculated in the first and second steps; Step 3: Design a progressive learning strategy for the CT reconstruction network described in step 2 The progressive learning strategy is a strategy for inputting multi-scale resolution images into a CT reconstruction network and training the network, which includes two processes: network augmentation and parameter transfer. The network training starts with low-resolution image input until the network training process converges, and then the network is augmented to obtain a larger new network by increasing the number of network layers. At the same time, the parameter transfer process transfers the parameters of the trained network to the new network, and then inputs a larger resolution image into the new network, and continues the network training. By continuously repeating the network augmentation and parameter transfer process, the network gradually transitions from learning large-scale structures in small-resolution images to learning fine structures in larger-resolution images, thereby helping to improve the quality of reconstructed CT images. Step 4: Use the CT reconstruction network trained in step 3 to test on the test data set; the CT volume image can be reconstructed by inputting a single X-ray image into the network.
