Single-view CT reconstruction method integrating FDK and deep learning

By combining the two-stage framework of FDK algorithm and deep learning, the problem of lack of theoretical basis and interpretability in single-view CT reconstruction is solved, and high-quality tomographic image reconstruction under low dose conditions is achieved, reducing X-ray radiation to patients The impact of

CN113052936BActive Publication Date: 2024-11-15DALIAN UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110339612.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-30
Publication Date
2024-11-15
Estimated Expiration
2041-03-30

AI Technical Summary

Technical Problem

The existing single-view CT reconstruction methods lack theoretical basis and interpretability, and it is difficult to generate high-quality tomographic images under low dose conditions, especially when medical data resources are limited.

Method used

Using a two-stage deep learning framework, firstly, the single-view reconstruction problem is transformed into general CT reconstruction problem through a full-view generation network, and the FDK algorithm is used to achieve accurate mapping from 2D to 3D, and then fine-tune the back projection results through the CT fine-tuning network. , improve the quality of tomographic images.

Benefits of technology

Provide theoretical support for analysis, reduces the model parameters of the neural network, improves computational efficiency and image quality, enhances the interpretability of the method, and reduces the radiation of X-rays to patients under low dose conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113052936B_ABST
    Figure CN113052936B_ABST
Patent Text Reader

Abstract

The present invention relates to a single-view CT reconstruction method integrating FDK and deep learning, and belongs to the field of computer vision and deep learning. In order to use the FDK algorithm to complete accurate 2D to 3D mapping, the method of the present invention proposes a two-stage deep learning framework. In the first stage, a full-view generation network is used to convert the single-view reconstruction pathological problem into a general CT reconstruction problem. In the second stage, the FDK algorithm is used to achieve accurate mapping of 2D projection to 3D volume, and then a CT fine-tuning network is built to fine-tune the CT volume obtained by back-projection to improve the quality of the tomographic image.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a single-view CT reconstruction method integrating FDK and deep learning, and belongs to the field of computer vision and deep learning. Background Art

[0002] Computer tomography (CT) is now widely used in clinical medicine, nondestructive testing, and public security inspection. Especially in the medical field, due to its extremely high imaging quality, it provides a reliable image basis for the diagnosis of diseases. With the large-scale popularization of CT technology, people are beginning to pay attention to the damage caused by X-rays to the human body. According to CT reconstruction theory, in order to obtain the internal structure of the human body, high-energy X-rays are needed to penetrate the human body and perform rotational scanning to obtain sufficient projection data. This will cause huge electronic radiation to the human body. Studies have shown that patients will significantly increase their risk of cancer after multiple CT examinations.

[0003] Therefore, in the field of CT research, low-dose reconstruction has always been an important topic. Reducing radiation dose can be achieved through sparse angle or limited angle sampling, but according to CT theory, sufficient projections need to be collected to reconstruct the tomography, otherwise incomplete projection information will cause artifacts in the reconstructed tomography, affecting the observation quality. However, combined with prior structural knowledge, CT reconstruction can still be completed with limited projection information, such as the Chinese patent "Sparse reconstruction of lung CT images at low sampling rate" (CN112070855A), and the Chinese patent "An image reconstruction method based on Bayesian estimation" (CN110458907A). However, these reconstruction methods are only based on some shallow features or regression functions, and cannot grasp the complex mapping relationship from 2D projection to 3D volume. With the widespread application of neural networks that can fit complex functional relationships in computer vision, the field of CT reconstruction has also launched limited or sparse angle reconstruction research based on deep learning. By extracting the prior structure of CT through deep learning, the neural network can use limited projection information to complete the complex 2D to 3D mapping relationship, and finally output the CT volume, which greatly reduces the artifacts caused by limited viewing angle reconstruction and improves the quality of tomographic images. For example, the Chinese patent "A method for removing artifacts from limited-angle CT reconstruction based on generative adversarial networks" (CN110648376A) and the Chinese patent "A low-dose CT reconstruction method based on interpolation convolutional neural network" (CN112489156A). Even in the case of only binocular projection, neural networks can achieve CT reconstruction through deep learning, such as the Chinese patent "CT image generation method, device, equipment and storage medium" (CN109745062A).

[0004] In order to further reduce the harm to the human body, some scholars have proposed to reconstruct CT from single-view projections under the lowest dose. Existing single-view CT studies are all based on deep learning, using neural networks combined with model prior knowledge to reconstruct CT slices based on the input single-view projections. Although neural networks can fit complex mapping relationships, the lack of interpretability will become an obstacle to clinical application due to the lack of theoretical basis for traditional algorithms. Moreover, methods that are completely data-driven can only perform deep learning under the supervision of a large amount of labeled data, and it is difficult to converge to an effective model due to the difficulty in acquiring medical data resources. Therefore, the quality of the final generated slices needs to be further improved. Summary of the invention

[0005] In order to achieve the lowest dose of CT reconstruction, the present invention proposes a single-view CT reconstruction method that integrates FDK and deep learning based on an X-ray projection image. In order to use the FDK algorithm to complete accurate 2D to 3D mapping, this method proposes a two-stage deep learning framework. In the first stage, the single-view reconstruction pathological problem is converted into a general CT reconstruction problem through a full-view generation network. In the second stage, the FDK algorithm is used to achieve accurate mapping of 2D projection to 3D volume, and then a CT fine-tuning network is built to fine-tune the CT volume obtained by back-projection to improve the quality of the tomographic image.

[0006] To achieve the above-mentioned object of the invention, the technical solution of the present invention is as follows:

[0007] A single-view CT reconstruction method integrating FDK and deep learning, comprising the following steps:

[0008] Step 1: Data acquisition and preprocessing.

[0009] The dataset contains X-ray body anteroposterior projection images and their 360° full-view projection images at 1° intervals, as well as CT volumes. The X-ray body anteroposterior projection images and their corresponding full-view projections are the training data for the full-view generation network in the first stage, and the CT volumes are the supervisory data for the CT fine-tuning network in the second stage. All the data used in the training are processed by first calculating the mean μ and variance σ, and then normalizing them by (x-μ) / σ, where x represents the training data to be standardized. The dataset is divided into a training set and a test set for network training and testing, respectively.

[0010] Step 2: Build a full-view generation network.

[0011] The full-view generation network consists of a feature codec, a residual module and a feature jump cascade structure. The feature codec consists of a continuously down-sampled convolutional neural network and a continuously up-sampled convolutional neural network connected after obtaining the encoded features at the bottom layer. The residual module is a component of the deepening convolutional neural network. By introducing a short connection to connect the input features to the output of the nonlinear layer, the network obtains a stronger ability to fit high-dimensional functions. The feature jump cascade structure cascades the encoding features and decoding features of the same scale in the codec to achieve feature fusion.

[0012] First, the full-view generation network uses a residual module to extract features from the input X-ray body projection image, and then sends it to the codec for encoding. During decoding, a feature skipping cascade structure is used to achieve feature fusion and finally generate a full-view projection image.

[0013] Step 3: Train the full-view generation network.

[0014] X-ray body projection images are used as input, the corresponding full-view projections are used as supervision, the consistency loss function is used as the optimization target, and the gradient descent algorithm is used for deep learning to train the constructed full-view generation network.

[0015] Step 4: Generate predicted coarse CT volume data.

[0016] Use the full-view generation network trained in step 3, input the X-ray body orthogonal projection image in the data set, obtain the predicted full-view projection and execute the FDK algorithm on it. The FDK algorithm is a filtered back-projection CT reconstruction algorithm for cone-beam projection images, and the CT volume data can be obtained through the weighted filtered back-projection step. At this time, the full-view projection used in the back-projection is the prediction result of the neural network, and there is a certain error relative to the true value, and these errors are also back-projected into the generated CT volume through the FDK algorithm. Therefore, the coarse CT generated by the back-projection needs to be further processed to remove the artifacts in the CT section caused by the projection error. The back-projected coarse CT is standardized, and then formed into a training pair with the corresponding CT volume true value, and sent to the CT fine-tuning network for training to remove the artifacts in the coarse CT volume.

[0017] Step 5: Build the CT fine-tuning network.

[0018] The CT fine-tuning network consists of a feature encoder-decoder, a residual module and a feature skip cascade structure, and its structure is as described in step 2.

[0019] First, the CT fine-tuning network uses a residual module to extract features from the input coarse CT volume, and then sends it to the codec for encoding. During decoding, a feature skipping cascade structure is used to realize feature fusion to achieve fine-tuning of the coarse CT volume, remove artifacts in the coarse CT volume, and output a more accurate CT volume.

[0020] Step 6: Train the CT fine-tuning network.

[0021] Use the back-projected coarse CT obtained in step 4 as input, the CT true value as supervision, the consistency loss function as the optimization target, and the gradient descent algorithm for deep learning to train the constructed CT fine-tuning network, complete the fine-tuning of the coarse CT, and finally generate a clear, artifact-free CT reconstructed image.

[0022] The beneficial effects of the present invention are:

[0023] (1) The present invention is the first to introduce a filtered back-projection algorithm into the single-view CT reconstruction problem, providing an analytical theoretical support for solving this pathological problem.

[0024] (2) The method of the present invention introduces the FDK algorithm to accurately complete the 2D to 3D mapping, eliminating the model parameters required for training the neural network to complete the conversion, and greatly saving computing resources.

[0025] (3) The two-stage training method of the present invention can visualize each part of the neural network, improving the interpretability of single-view CT. In addition, this method has strong scalability and can design complex neural network structures and targeted loss functions for two-stage tasks.

[0026] (4) The method of the present invention improves the quality of single-view CT reconstructed tomographic images. On the one hand, it can provide doctors with tomographic images of the patient's body structure based on the projection images to provide guidance information during interventional treatment. On the other hand, it can provide single-view projection for tomographic reconstruction based on the DR examination images of patients with chronic diseases such as tumors during long-term treatment, thereby avoiding excessive examination to a certain extent, thereby greatly reducing X-ray radiation to patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a flow chart of the steps of the method of the present invention.

[0028] Figure 2 It is a neural network structure diagram based on the U-net structure combined with the residual module of the method of the present invention.

[0029] Figure 3 This is a first-stage method of the present invention for generating a neural network visualization projection image from a full-view perspective.

[0030] Figure 4It is a schematic diagram of the FDK back-projection algorithm used in the method of the present invention.

[0031] Figure 5 The two-stage CT fine-tuning neural network of the method of the present invention is used to visualize CT tomographic images. DETAILED DESCRIPTION

[0032] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0033] like Figure 1 As shown, a single-view CT reconstruction method integrating FDK and deep learning includes the following steps:

[0034] Step 1, data acquisition and preprocessing. This embodiment obtains a 4DCT data set of a cancer patient's lung from a public medical image data set, uses a simulated projection method to generate deep learning sample pairs, and simulates the patient's offset on the scanning bed by displacing and rotating the CT volume to complete data augmentation. The deep learning samples include X-ray body orthotopic projection images, and their corresponding 1° interval 360° full-view projection images, and the corresponding patient's lung CT volume. This embodiment divides the data set into a training set and a test set in a 7:3 manner. Among them, the X-ray body orthotopic projection image and its corresponding full-view projection are the first-stage full-view generation network training data, and the CT volume is the supervision data of the second-stage CT fine-tuning network. The projection image resolution is 128×128, and the CT volume resolution is 128×128×128. The data in the data set is processed by first calculating the mean and variance, and then standardizing. For example, for the input X-ray image, first calculate its mean μ and variance σ, and then standardize it in the manner of (x-μ) / σ, where x represents the X-ray image to be standardized. The 1° interval 360° projection images and the corresponding CT volumes used for supervision were also normalized in the same way.

[0035] Step 2: Build a full-view projection network. The full-view projection generation neural network uses a U-net structure combined with a residual module neural network (such as Figure 2 As shown in the figure, U-net is a neural network feature codec with a feature skip cascade structure. The number of channels of the input and output of the neural network is set to 1 for the input projection and 360 for the output projection. The network first uses a 7×7 large-size convolution kernel at the input to extract features, and uses a residual structure to enhance feature channels and deepen the network structure. Before each downsampling and after each upsampling of U-net, the residual fast is used to deepen the network and extract complex features. Finally, the feature fusion is performed step by step through the skip cascade, and 360 full-view projection images are encoded and output.

[0036] Step 3: Train the full-view generation network. Use the X-ray body projection image data of the training set as input, and the corresponding standardized full-view projection image as supervision to train the full-view projection generation network. This embodiment uses the consistency loss function L1 norm for training:

[0037]

[0038] p f is the true value of 360° full viewing angle projection at 1° intervals, The full-view projection predicted by the network is trained iteratively using the Adam optimizer. After training, the error between the generated projection and the true value is calculated on the test set to verify the effectiveness of the full-view generation network. Figure 3 .

[0039] Step 4: Generate predicted coarse CT volume data. Use the trained full-view generation network to take the X-ray body projection image data in the data set as input to generate predicted full-view projection data. Use the FDK algorithm to perform filtered back-projection on the generated full-view projection data to obtain coarse CT volume data (see Figure 4 ). The FDK algorithm consists of three steps. First, the projection image is weighted. The formula is as follows:

[0040]

[0041] P β (s,h) is the projection data, (s,h) is the coordinate in the detector coordinate system, is the weighting coefficient, and D is the distance from the ray source to the rotation axis. Then the weighted projection is filtered:

[0042] Q β (s,h)=P′ β (s,h)×g(s)

[0043] P′ β (s,h) is the weighted projection data, and g(s) is the 'Ram-Lak' one-dimensional ramp filter. Back-project the filtered data:

[0044]

[0045] 1 / U(x,y,β) 2 is the weighting coefficient, where:

[0046] U(x,y,β)=(D+xcosβ+ysinβ) / D

[0047] (s,h) are the coordinates in the world coordinate system, β is the angle between the line from the ray source to the rotation axis and the x-axis, and the voxel with the reconstruction point R(x,y,z) can be obtained by back-projecting the full view from 0 to 2π. After executing the FDK algorithm, we obtain the CT volume, but since the predicted full view projection has errors relative to the true value, these errors are also introduced into the CT volume, resulting in artifacts, such as Figure 5 The crude CT volume generated at this time is standardized for further training.

[0048] Step 5: Build a CT fine-tuning network. The CT fine-tuning network uses a U-net structure combined with a residual module neural network (such as Figure 2 As shown in Figure 2). We set the number of input and output channels of the neural network to the corresponding number of CT slices, 128. The network model first encodes the input coarse CT volume structure features through a cascaded U-net encoder-decoder structure, removes artifacts during decoding, completes fine-tuning, and outputs clear CT slice images.

[0049] Step 6: Train the CT fine-tuning network. The coarse CT obtained in step 4 is used as the input of the CT fine-tuning network, and the corresponding true value is used as supervision. This embodiment uses the L1 norm to train the CT fine-tuning network:

[0050]

[0051] The CT volume predicted by the CT fine-tuned network, V ct is the true value of the CT volume. After deep learning, the CT fine-tuning network can complete the fine-tuning of the coarse CT and generate artifact-free CT results. The coarse CT of the test set is sent to the CT fine-tuning neural network to verify the effectiveness of the fine-tuning network. Figure 5 shown.

[0052] Finally, the X-ray body projection images of the test set were sequentially used to complete the trained input full-view projection generation network, and the FDK algorithm was executed on the predicted full-view projection to obtain a rough CT. Finally, the rough CT was sent to the trained CT fine-tuning network for fine-tuning to obtain a clear artifact-free CT reconstructed image. The effectiveness of the deep learning single-view CT method combined with the FDK algorithm was verified. The structural similarity SSIM of the predicted CT volume on the entire test set reached 0.9329, and the peak signal-to-noise ratio PSNR reached 30.25dB.

Claims

1. A single-view CT reconstruction method integrating FDK and deep learning, characterized in that: The following steps are involved: Step 1: Data acquisition and preprocessing The dataset contains X-ray body anteroposterior projection images and their 360° full-view projection images at 1° intervals, as well as CT volumes; the X-ray body anteroposterior projection images and their corresponding full-view projections are the training data for the full-view generation network in the first stage, and the CT volumes are the supervision data for the CT fine-tuning network in the second stage; all the data used in the training are processed by first calculating the mean μ and variance σ, and then normalizing them by (x-μ) / σ, where x represents the training data to be standardized; the dataset is divided into a training set and a test set for network training and testing, respectively; Step 2: Build a full-view generation network The full-view generation network is composed of a feature codec, a residual module and a feature skip cascade structure; the feature codec is composed of a continuously down-sampled convolutional neural network and a continuously up-sampled convolutional neural network connected after obtaining the encoded features at the bottom layer; the residual module is a component of the deepening convolutional neural network, which introduces a short connection to connect the input features to the output of the nonlinear layer, so that the network has a stronger ability to fit high-dimensional functions; the feature skip cascade structure is to cascade the encoding features and decoding features of the same scale in the codec to achieve feature fusion; First, the full-view generation network uses a residual module to extract features from the input X-ray body projection image, and then sends it to the codec for encoding. During decoding, a feature skipping cascade structure is used to achieve feature fusion and finally generate a full-view projection image. Step 3: Train the full-view generation network Use X-ray body projection images as input, the corresponding full-view projection as supervision, use the consistency loss function as the optimization target, use the gradient descent algorithm for deep learning, and train the constructed full-view generation network; use the consistency loss function L1 norm for training: p f is the true value of 360° full viewing angle projection at 1° intervals, The full-view projection predicted by the network; the Adam optimizer is used to perform gradient descent iterative training on the neural network. After the training is completed, the error between the generated projection and the true value is calculated on the test set to verify the effectiveness of the full-view generation network; Step 4: Generate predicted coarse CT volume data Using the full-view generation network trained in step 3, input the X-ray body orthogonal projection image in the data set, obtain the predicted full-view projection and execute the FDK algorithm on it; the FDK algorithm obtains the CT volume data through the weighted filtering back-projection step; Step 5: Build a CT fine-tuning network The CT fine-tuning network consists of a feature encoder-decoder, a residual module and a feature jump cascade structure, and its structure is as described in step 2; First, the CT fine-tuning network uses a residual module to extract features from the input coarse CT volume, and then sends it to the codec for encoding. During decoding, a feature skip cascade structure is used to achieve feature fusion to achieve fine-tuning of the coarse CT volume, remove artifacts in the coarse CT volume, and output a more accurate CT volume. Step 6: Train the CT fine-tuning network Use the back-projected coarse CT volume obtained in step 4 as input, the true value of the CT volume as supervision, use the consistency loss function as the optimization target, use the gradient descent algorithm for deep learning, train the constructed CT fine-tuning network, complete the fine-tuning of the coarse CT volume, and finally generate a clear and artifact-free CT volume reconstructed image; use the L1 norm to train the CT fine-tuning network: The CT volume predicted by the CT fine-tuned network, V ct is the true value of CT volume; After deep learning, the CT fine-tuning network is able to complete the fine-tuning of the coarse CT volume and generate artifact-free CT volume results. The coarse CT volume of the test set is sent to the CT fine-tuning neural network to verify the effectiveness of the fine-tuning network.