Cone beam CT sparse reconstruction method based on attention mechanism and deep learning

Abstract

According to the cascaded CBCT projection data restoration method, complete projection data can be generated through CBCT sparse projection data, and then a high-quality reconstructed image is obtained through an FDK method. The method is composed of two star-shaped attention networks. The continuity of the generated projection data in the sine domain and the projection domain can be fully ensured through the reasonable deployment of the two single-stage networks, and finally the original projection data and the corresponding generated projection data are replaced to be fused into high-quality projection data.

Description

Technical Field

[0001] The present invention uses the FDK (Feldkamp-Davis-Kress, FDK) algorithm to obtain high-quality reconstructed images, belongs to the field of medical image processing, and particularly relates to a cone-beam CT sparse reconstruction method based on attention mechanism and deep learning. Background Art

[0002] Cone Beam Computed Tomography (CBCT) based on flat panel detectors has been widely used in the fields of dentistry, orthopedics, radiotherapy, etc. in clinical practice. Because X-rays have ionizing radiation characteristics, excessive CBCT scanning doses will cause radiation damage to the scanned object. Sparse projection, which reduces exposure parameters and increases the projection angle interval, are two commonly used scanning methods to reduce doses. Compared with reducing exposure parameters, the sparse projection method can also speed up CBCT scanning and avoid motion artifacts caused by movement or deformation of the imaging object during scanning. At present, the FDK (Feldkamp-Davis-Kress) algorithm is still the standard algorithm for commercial CBCT scanners. It has the advantages of short reconstruction time and reconstructed images that conform to doctors' diagnostic habits. The FDK algorithm is a three-dimensional extension of the two-dimensional filtered back-projection algorithm (FBP). Both are based on Fourier transform theory and have high requirements for the completeness of projection data. Under sparse projection conditions, the reconstructed image will have severe stripe artifacts accompanied by high levels of noise. This defect seriously limits the application of CBCT sparse projection scanning in clinical practice. Therefore, how to improve the quality of reconstructed images under sparse projection conditions has become a key issue in CBCT imaging.

[0003] The projection domain method uses interpolation technology to complete the projection data of missing angles in sparse projections to improve the completeness of the projection data, and then uses the traditional FDK method to reconstruct the completed projection data to indirectly improve the quality of the reconstructed image, such as linear interpolation, directional interpolation, etc. However, due to the inaccuracy of the generated projection data, the projection domain method is prone to introduce secondary artifacts while removing artifacts and reducing noise, making it difficult to achieve ideal results.

[0004] In recent years, deep learning methods have achieved remarkable results in improving the quality of generated projection data due to their ability to extract high-dimensional features of images and generate high-quality images. Generating high-quality projection data through deep learning can obtain reconstruction results of higher quality than interpolation methods. At the same time, the advantage of deep learning methods in terms of short time consumption has improved the feasibility of deep learning-based reconstruction algorithms in clinical applications. Summary of the invention

[0005] In order to retain the original projection data collected by the CT device for reconstruction as much as possible, after generating the complete projection data, existing research usually uses the original projection data to replace the generated projection data of the corresponding angle to further improve the accuracy of the reconstruction results. However, this approach cannot guarantee the continuity between the generated projection data and the original projection data, and will introduce secondary artifacts in the reconstructed image. At the same time, since the CBCT projection data is three-dimensional data composed of a series of two-dimensional projection images, directly using the repair method for the two-dimensional projection data cannot guarantee the continuity of the three-dimensional data reconstructed from the repaired two-dimensional projection images. In response to the above technical problems, the present invention provides a cone beam CT sparse reconstruction method based on attention mechanism and deep learning, including:

[0006] Step 1: Use interpolation methods to expand the dimension of sparse projection data.

[0007] Step 2: The interpolated projection data is used as input into the first star attention network;

[0008] Step 3, after being repaired by the first star attention network, the repaired projection data is replaced with the sparse projection data;

[0009] Step 4: The replaced projection data is used as input into the second star attention network;

[0010] Step 5, after secondary restoration by the second star attention network, the obtained secondary restored projection data is replaced with the sparse projection data;

[0011] Step six, reconstruct the projection data after the second replacement using the FDK method to obtain a reconstructed image.

[0012] The star attention network contains four encoding modules and four decoding modules. The encoding modules and decoding modules are symmetrically distributed and connected by skip connections.

[0013] The star attention layer is used to establish correlations in the channel dimension and spatial dimension of the input feature map, thereby extracting the global correlation information in the input feature map and capturing the correlation information of each pixel in the input feature map in three-dimensional space.

[0014] The encoding module consists of an encoding layer and a downsampling layer, which is used to extract local feature information in the image;

[0015] The decoding module consists of an upsampling layer, a decoding layer, and a star attention layer, which is used to fuse the feature map extracted by the encoding module with the feature map output by the upsampling layer.

[0016] The encoding layer and the decoding layer have the same structure, which is composed of a 3×3 convolutional layer, a Batch Norm (BN) and an activation function;

[0017] The downsampling layer consists of a 2×2 convolutional layer, Batch Norm (BN), and an activation function to extract feature maps of different scales;

[0018] The upsampling layer consists of a 2×2 deconvolution layer, Batch Norm (BN), and an activation function to restore the original size of the feature map.

[0019] The star attention layer consists of two parts: the channel attention module CA (Channel Attention) and the spatial attention module SA (Spatial Attention). The channel attention module CA and the spatial attention module SA are respectively used to establish correlations in the channel dimension and spatial dimension of the input feature map, thereby extracting the global correlation information in the input feature map. The combination of the two can effectively capture the correlation information of each pixel in the input feature map in three-dimensional space.

[0020] The channel attention module CA consists of an adaptive pooling layer, two 1x1 convolutional layers and a Softmax layer;

[0021] The input of the channel attention module CA is multiplied by the output of the Softmax layer to obtain the output of the channel attention module CA. The output of the channel attention module CA is then sent to the two spatial attention modules SA after passing through a 3x3 convolutional layer.

[0022] Each spatial attention module SA includes three 1x1 convolutional layers and a Softmax layer. The input of the spatial attention module SA is first mapped into three feature map matrices through the convolutional layer;

[0023] The output obtained by multiplying two feature map matrices through the Softmax layer is multiplied by the third feature map matrix, and then connected to the input of the spatial attention module SA through a residual connection to obtain the output of the spatial attention module SA.

[0024] The output of the spatial attention module SA is multiplied by the output of the channel attention module CA to obtain the final output;

[0025] For a pixel in the input feature map, the receptive field of the channel attention module CA is cylindrical in the channel dimension, and the receptive field of the spatial attention module SA is cross-shaped in the spatial dimension;

[0026] After the two are combined, their receptive field is star-shaped in the three-dimensional feature map, and the star-shaped attention layer can effectively capture the correlation information of each pixel in the input feature map in the three-dimensional space.

[0027] Construct the loss function L and generate the projection data T(P e) and the Frobenius norm between the real projection data P as the loss function:

[0028]

[0029] By constructing a cascaded deep learning network, the two-dimensional projection data is repaired multiple times using the Star Attention Network from the perspective of the sinusoidal graph and the projection graph, fully maintaining the continuity between the generated projection data and the original projection data. The Star Attention Network used uses a new type of star attention mechanism, which has a star-shaped receptive field, which can fully capture the correlation between any pixel in the feature map and all pixels in the same row or column, as well as the correlation between all pixels in the corresponding spatial positions in different channels, ensuring the accuracy of the restored projection data, thereby improving the quality of the reconstruction results.

[0030] In summary, the present invention can fully mine the original correlation information in CBCT projection data, enhance the accuracy of network-generated projection data, and thus improve the quality of sparse reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Shown is a schematic diagram of the workflow of the cascade projection data repair method;

[0032] Figure 2 Shown is a diagram of the star-shaped attention network structure;

[0033] Figure 3 Shown is the specific combination of the CA module and the SA module and their respective structural schematics. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1 The workflow of the cascade projection data repair method proposed in the present invention includes:

[0036] Step 1: Use interpolation methods to expand the dimension of sparse projection data.

[0037] Step 2: The interpolated projection data is used as input into the first star attention network;

[0038] Step 3, after being repaired by the first star attention network, the repaired projection data is replaced with the sparse projection data;

[0039] Step 4: The replaced projection data is used as input into the second star attention network;

[0040] Step 5, after secondary restoration by the second star attention network, the obtained secondary restored projection data is replaced with the sparse projection data;

[0041] Step six, reconstruct the projection data after the second replacement using the FDK method to obtain a reconstructed image.

[0042] The star-shaped attention network architecture proposed in this invention is as follows Figure 2 The specific explanation is as follows:

[0043] The star attention network contains four encoding modules and four decoding modules. The encoding modules and decoding modules are symmetrically distributed and connected by skip connections.

[0044] The star attention layer is used to establish correlations in the channel dimension and spatial dimension of the input feature map, thereby extracting the global correlation information in the input feature map and capturing the correlation information of each pixel in the input feature map in three-dimensional space.

[0045] The encoding module consists of an encoding layer and a downsampling layer, which is used to extract local feature information in the image;

[0046] The decoding module consists of an upsampling layer, a decoding layer, and a star attention layer, which is used to fuse the feature map extracted by the encoding module with the feature map output by the upsampling layer.

[0047] The encoding layer and the decoding layer have the same structure, which is composed of a 3×3 convolutional layer, a Batch Norm (BN) and an activation function;

[0048] The downsampling layer consists of a 2×2 convolutional layer, Batch Norm (BN), and an activation function to extract feature maps of different scales;

[0049] The upsampling layer consists of a 2×2 deconvolution layer, Batch Norm (BN), and an activation function to restore the original size of the feature map.

[0050] The star attention layer consists of two parts: the channel attention module CA (Channel Attention) and the spatial attention module SA (Spatial Attention). The channel attention module CA and the spatial attention module SA are used to establish correlation in the channel dimension and spatial dimension of the input feature map respectively, so as to extract the global correlation information in the input feature map. The combination of the two can effectively capture the correlation information of each pixel in the input feature map in three-dimensional space.

[0051] The specific combination of CA module and SA module and their respective structures are as follows: Figure 3 As shown:

[0052] The channel attention module CA consists of an adaptive pooling layer, two 1x1 convolutional layers and a Softmax layer;

[0053] The input of the channel attention module CA is multiplied by the output of the Softmax layer to obtain the output of the channel attention module CA. The output of the channel attention module CA is then sent to the two spatial attention modules SA after passing through a 3x3 convolutional layer.

[0054] Each spatial attention module SA includes three 1x1 convolutional layers and a Softmax layer. The input of the spatial attention module SA is first mapped into three feature map matrices through the convolutional layer;

[0055] The output obtained by multiplying two feature map matrices through the Softmax layer is multiplied by the third feature map matrix, and then connected to the input of the spatial attention module SA through a residual connection to obtain the output of the spatial attention module SA.

[0056] The output of the spatial attention module SA is multiplied by the output of the channel attention module CA to obtain the final output;

[0057] For a pixel in the input feature map, the receptive field of the channel attention module CA is cylindrical in the channel dimension, and the receptive field of the spatial attention module SA is cross-shaped in the spatial dimension;

[0058] After the two are combined, their receptive field is star-shaped in the three-dimensional feature map, and the star-shaped attention layer can effectively capture the correlation information of each pixel in the input feature map in the three-dimensional space.

[0059] Construct the loss function L and generate the projection data T(P e ) and the Frobenius norm between the real projection data P as the loss function:

[0060]

[0061] The technical effects of the present invention are:

[0062] The continuity between the original projection data and the generated projection data is fully considered. By building a cascade restoration network, the continuity between the two is maintained to improve the quality of the reconstructed image.

[0063] The network's projection data generation capability is fully considered, and an attention module is introduced to fully capture the correlation in the projection data and generate more accurate projection data.

[0064] Therefore, the present invention fully exploits the correlation information in the CBCT projection data, improves the accuracy of generating the CBCT projection data, and thus improves the quality of the reconstructed image, and provides a new means for high-quality reconstruction of sparse CBCT projection data.

[0065] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is limited by the appended claims and their equivalents.

Claims

1. A cone-beam CT sparse reconstruction method based on attention mechanism and deep learning, characterized in that: include: Step 1: Use interpolation methods to expand the dimension of sparse projection data. Step 2: The interpolated projection data is used as input into the first star attention network; Step 3, after being repaired by the first star attention network, the repaired projection data is replaced with the sparse projection data; Step 4: The replaced projection data is used as input into the second star attention network; Step 5, after secondary restoration by the second star attention network, the obtained secondary restored projection data is replaced with the sparse projection data; Step six, reconstruct the projection data after the second replacement using the FDK method to obtain a reconstructed image.

2. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 1, characterized in that: The star-shaped attention network includes four encoding modules and four decoding modules, which are symmetrically distributed and connected by skip connections. The star attention layer is used to establish correlation in the channel dimension and spatial dimension of the input feature map, so as to extract the global correlation information in the input feature map and capture the correlation information of each pixel in the input feature map in three-dimensional space.

3. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 2, characterized in that: The encoding module is composed of an encoding layer and a downsampling layer, and is used to extract local feature information in an image; The decoding module consists of an upsampling layer, a decoding layer and a star attention layer, and is used to fuse the feature map extracted by the encoding module with the feature map output by the upsampling layer.

4. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 3, characterized in that: The encoding layer and the decoding layer have the same structure, which is composed of a 3×3 convolutional layer, a Batch Norm (BN) and an activation function; The downsampling layer is composed of a 2×2 convolutional layer, a Batch Norm (BN) and an activation function, and is used to extract feature maps of different scales; The upsampling layer consists of a 2×2 deconvolution layer, a Batch Norm (BN) and an activation function, and is used to restore the original size of the feature map.

5. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 4, characterized in that: The star attention layer consists of two parts: a channel attention module CA and a spatial attention module SA. The channel attention module CA and the spatial attention module SA are respectively used to establish correlations in the channel dimension and spatial dimension of the input feature map, thereby extracting the global correlation information in the input feature map. The combination of the two can effectively capture the correlation information of each pixel in the input feature map in three-dimensional space.

6. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 5, characterized in that: The channel attention module CA includes an adaptive pooling layer, two 1x1 convolutional layers and a Softmax layer; The input of the channel attention module CA is multiplied by the output of the Softmax layer to obtain the output of the channel attention module CA. The output of the channel attention module CA is then sent to two spatial attention modules SA after passing through a 3x3 convolutional layer.

7. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 6, characterized in that: Each spatial attention module SA includes three 1x1 convolutional layers and a Softmax layer. The input of the spatial attention module SA is first mapped into three feature map matrices through the convolutional layer; The output obtained by multiplying two feature map matrices through the Softmax layer is multiplied by the third feature map matrix, and then connected to the input of the spatial attention module SA through a residual connection to obtain the output of the spatial attention module SA.

8. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 7, characterized in that: The output of the spatial attention module SA is multiplied by the output of the channel attention module CA to obtain the final output; For a pixel in the input feature map, the receptive field of the channel attention module CA is cylindrical in the channel dimension, and the receptive field of the spatial attention module SA is cross-shaped in the spatial dimension; After the two are combined, their receptive field is star-shaped in the three-dimensional feature map, and the star-shaped attention layer can effectively capture the correlation information of each pixel in the input feature map in the three-dimensional space.

9. The cone-beam CT sparse reconstruction method based on attention mechanism and deep learning according to claim 8, characterized in that: Construct the loss function L and generate the projection data T(P e ) and the Frobenius norm between the real projection data P as the loss function:

Images