Sparse projection CT reconstruction method based on projection domain super-resolution method

Abstract

According to the sparse projection CT reconstruction method based on the projection domain super-resolution method, by constructing a projection domain completion model based on super-resolution, the completion precision of sparse projection data is effectively improved, a complex structure can be accurately restored, and detail performance can be improved. Through the multi-stage information compensation module and the bottom-layer feature fusion module, efficient fusion of multi-scale and multi-channel information can be realized, the capturing capability of the model to a complex structure is enhanced, and generation of artifacts is reduced. Besides, the proposed network structure has a highly modular design, can be conveniently embedded into other image domain networks, adapts to diversified CT imaging tasks, and significantly improves the universality and applicability of the method. Through end-to-end optimization, the problem of error accumulation of a traditional step-by-step method is avoided, the reconstruction quality is improved, the calculation efficiency is kept while the performance is improved, and the method has high practical application value.

Description

Technical Field

[0001] The invention belongs to the technical field of CT reconstruction, and relates to sparse projection CT reconstruction, and in particular to a sparse projection CT reconstruction method based on a projection domain super-resolution method. Background Art

[0002] CT (computed tomography) plays an important role in modern medicine, and its image quality directly affects diagnosis and treatment. However, in order to reduce the radiation dose received by patients, sparse projection sampling technology is widely used. However, the incompleteness and insufficient resolution of sparse projection data seriously limit the reconstruction quality of CT images, which has become an urgent problem to be solved.

[0003] Current sparse projection completion technologies are mainly divided into two categories: interpolation methods and deep learning-based methods. Interpolation methods such as linear interpolation and spline interpolation are simple to implement and computationally efficient, but since they only use local information, it is difficult to effectively restore complex structures and it is easy to introduce artifacts. Deep learning-based methods can capture global features and improve the completion effect by learning the mapping relationship between sparse projections and full-view projections.

[0004] Problems with existing technologies:

[0005] 1. Most methods use lightweight networks, which have limited ability to restore details.

[0006] 2. Some studies adopt a step-by-step approach of interpolation followed by optimization, which results in the problem of error accumulation.

[0007] 3. Most studies focus on feature learning in a single domain (projection domain or image domain). This approach has limitations in information linkage and optimization goals. The correlation information between the projection domain and the image domain has not been fully mined, and end-to-end full-process optimization cannot be achieved, which limits the reconstruction quality and the diversity of application scenarios.

[0008] 4. Insufficient completion of sparse projection data: The interpolation method in the existing technology is difficult to accurately restore complex projection details, and the deep learning method is limited by lightweight networks, resulting in limited completion effect and serious loss of detail information.

[0009] 5. Insufficient utilization of multi-scale and multi-channel information: Existing methods are not capable of fusing and optimizing the multi-scale features and channel information of sparse projection data, which limits the final reconstruction effect.

[0010] 6. Lack of dual-domain collaborative optimization: Most current technologies only perform single optimization on the projection domain or the image domain, failing to fully utilize the correlation information between the two domains, making it difficult to achieve full-process optimization and affecting the overall CT reconstruction quality.

[0011] 7. Lack of versatility and flexibility of the methods: Most existing methods are designed for specific scenarios and are difficult to embed into other network structures, and cannot meet diverse practical needs.

[0012] At present, dual-domain parallel training provides a solution path, which makes full use of the complementary information between the two domains by learning features in the projection domain and the image domain at the same time. Compared with the single-domain method, the dual-domain method has significant advantages in information interaction and optimization effect. However, the current dual-domain method has limited application scenarios, its network design is not flexible and universal, and the interaction mechanism of multi-scale features is not efficient enough, which limits its practical application in diverse scenarios.

[0013] Therefore, sparse projection CT reconstruction technology requires a new solution that can simultaneously improve the quality of projection domain completion and image domain reconstruction, and achieve the ability to flexibly embed other task networks through modular design. This solution needs to break through the limitations of existing technologies, especially in terms of multi-scale information utilization, artifact suppression, and dual-domain collaborative optimization, to provide effective support for the comprehensive improvement of CT imaging quality. Summary of the invention

[0014] In order to overcome the above-mentioned deficiencies of the prior art, the purpose of the present invention is to provide a sparse projection CT reconstruction method based on a projection domain super-resolution method, solve the above-mentioned problems existing in the prior art, realize dual-domain collaborative optimization of the projection domain and the image domain, and improve the reconstruction quality and technical versatility.

[0015] In order to achieve the above object, the technical solution adopted by the present invention is:

[0016] A sparse projection CT reconstruction method based on a projection domain super-resolution method, characterized by comprising the following steps;

[0017] Step 1: Construct sparse projection data, preprocess the projection domain data, divide them into 2x, 4x, 8x, 15x and full view sinogram data sets, and divide them into training set, test set and validation set;

[0018] Step 2: construct a super-resolution network structure, which includes 4 modules: a downlink channel branch module, a bottom module, an uplink channel branch module, and a channel compensation and recovery module;

[0019] Step 3: For the downstream channel branch module, the network first inputs the input features into the downstream channel branch for feature extraction on different channels. In the downstream channel branch, there are n layers for feature extraction in n different channel cases, and n-1 downstream channel modules for channel compression between adjacent layers. The input features are first input into the feature extraction module with m-dimensional channel number to extract features, and then compressed by the downstream channel module to change the channel number to m / 2, and features are extracted in the feature extraction module with m / 2-dimensional channel number, until the next level branch performs feature extraction with m / 2n-dimensional channel number. The operation process is as follows;

[0020] ;

[0021] Among them, X represents the input feature, It represents the features obtained after the input features are operated through n layers of downstream channel branches. and Represents the feature extraction operations at different layers, and Represents different downstream channel modules. The downstream channel module is composed of convolutions. The input of the convolution layer is the output of the previous layer, and the output is the input of the next layer. The structure of the feature extraction module at different layers consists of multiple residual attention blocks with local residual connections.

[0022] Step 4: For the bottom layer module, the features extracted by the downlink channel branch through different layers are the features of different channels. There is a lack of information association between these features. For this reason, a bottom layer module is constructed to process these features between different channels. Specifically, the bottom layer module fuses the features of different channels obtained by the downlink channel branch and extracts the relevant features between different channels. The operation process is as follows.

[0023]

[0024] in, Represents the operation of the underlying module, Represents the features obtained through the operation of the underlying module, used to branch from the downstream channel Different channel characteristics in advance.

[0025] Step 5: For the uplink channel branch module, since the underlying module has less feature information fused and extracted in the case of small channels, if these features are directly used to reconstruct the sinusoidal image with full viewing angle, the information reconstruction of the sinusoidal image will be insufficient. Because the uplink channel branch is constructed, different levels of channel recovery are performed on these features, and feature extraction is performed in different channel recovery processes. The structure of the uplink channel branch is similar to that of the downlink channel branch, with n layers and n-1 uplink channel modules, which are used for feature extraction in n different channel conditions and feature recovery between adjacent channel layers, respectively. Unlike the downlink channel module, the uplink channel branch uses n-1 uplink channel modules to recover the feature information of the uplink channel under the condition of n different channels. The features of the dimensional channel are restored to the features of the m-dimensional channel. The operation process is as follows.

[0026] ;

[0027] in, It indicates the features extracted by the bottom module after being operated by n layers of upstream channel branches. and Indicates different upstream channel modules. The upstream channel module is composed of deconvolution, the input channel value of the deconvolution is the output channel value of the next level, and the output channel value is the input channel value of the previous level.

[0028] Step 6: For the channel information compensation and recovery module, during the channel compression of different levels of U-Net, it is difficult for the image features to obtain important feature information in the feature extraction of the next level because some smaller feature information is compressed, which means that some feature information is lost in the channel compression. Similarly, it is difficult to recover rich feature information during the channel recovery process at different levels because some feature information has been lost after channel compression. Therefore, in order to solve these problems, a communication information compensation and recovery module is proposed to compensate for the feature information lost during the compression process of the downstream channel branch at different levels. The operation is as follows: During the channel compression process at different layers of the downstream branch channel, the features before compression are first pooled, and then the pooled features are subjected to 1x1 convolution operation. Finally, these features are cascaded and input into the underlying module, and then equation (2) is updated as follows,

[0029] (4);

[0030] in, represents the pooling operation, represents a 1x1 convolution operation, Indicates a cascade operation. The input channel value of the bottom module is , output channel value .

[0031] The upstream channel branch performs compensation operations in the recovery process of different levels. First, a 1x1 convolution operation is performed on the features after the operation of the bottom module. Then, the features obtained by the operation are cascaded with the features before the channel recovery of different levels. Finally, the channel recovery operation is performed on the cascaded features. Then, equation (3) is updated as follows:

[0032] ;

[0033] Step 7: The overall operation is mainly divided into shallow feature extraction, deep feature extraction, sub-pixel upsampling and image reconstruction operations, and its model is shown in the figure. Assume is the sinusoidal graph of the input sparse angle, It outputs the sinusoidal graph of the full viewing angle. The network model first extracts shallow features from the sinusoidal graph of the sparse viewing angle through the initial convolutional layer, as follows.

[0034] (6);

[0035] in, represents the initial convolutional layer operation, Represents the extracted shallow features, and through multiple cycles of U-net as follows, the shallow features are extracted into deep features of the image.

[0036] ;

[0037] in, represents the extracted deep features, It represents multiple cycles of the U-net network, and then the shallow features and deep features are sub-pixel upsampled, and the features obtained by sub-pixel upsampling are further subjected to 1x1 convolution to obtain the final full-view sinusoidal map.

[0038] ;

[0039] in, represents the features obtained by sub-pixel upsampling, represents the sub-pixel upsampling operation, represents 1x1 convolution, Sine graph representing the full viewing angle.

[0040] Step 8: Perform a filtering back-projection module on the full-view sinusoidal image to obtain a CT image.

[0041] The beneficial effects of the present invention are:

[0042] The present invention has significant beneficial effects compared to the prior art. First, by constructing a super-resolution-based projection domain completion model, the completion accuracy of sparse projection data is effectively improved, and complex structures can be accurately restored and detail performance can be improved. Through a multi-level information compensation module and an underlying feature fusion module, the present invention can achieve efficient fusion of multi-scale and multi-channel information, enhance the model's ability to capture complex structures, and reduce the generation of artifacts. In addition, the proposed network structure has a highly modular design and can be easily embedded in other image domain networks to adapt to a variety of CT imaging tasks, significantly improving the versatility and applicability of the method. Through end-to-end optimization, the error accumulation problem of the traditional step-by-step method is avoided, the reconstruction quality is improved, and the computational efficiency is maintained while the performance is improved, which has a high practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic diagram of the overall CT reconstruction process of the present invention.

[0044] Figure 2 These are the full-view projection data of the CT image and sinusoidal graph of the present invention and the projection data samples of 2x, 4x, 8x and 15x downsampling.

[0045] Figure 3 It is a schematic diagram of the overall network structure of the present invention.

[0046] Figure 4 It is a schematic diagram of the feature extraction network at each stage of the present invention.

[0047] Figure 5 It is a schematic diagram of the feature fusion network of the underlying modules of the present invention.

[0048] Figure 6 It is a comparison chart of the present invention with other methods. DETAILED DESCRIPTION

[0049] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0050] Example

[0051] A sparse projection CT reconstruction method based on a projection domain super-resolution method comprises the following steps:

[0052] Step 1.1. Select 2000 lung DICOM format CT images of 10 patients from the TCIA database (200 images for each patient). The size of the CT images is 512x512. The full-view projection data is obtained from normal-dose CT images under fan-beam geometry using a standard ray-driven numerical forward projection program. That is, the full-view projection data size is 650x360, 650 detectors, and 360 projection views (sampling interval = 1 degree). The projection data on the projection view is downsampled to a sparse view. Sparse view projection data with sparsity levels S = 2, 4, 8, and 15 are obtained. For each sparsity level, 1600 images are used as a training set, 200 images are used as a validation set, and 200 images are used as a test set. Figure 2 shown.

[0053] Step 1.2, preprocessing the projection data set is to specify cropping, 90-degree random rotation, horizontal flipping, and vertical flipping of the data to obtain standardized input data that is compatible with the super-resolution network model, which helps to improve the training efficiency and effect of the network model.

[0054] Step 1.3: Divide the preprocessed projection data set into a training set, a validation set, and a test set for batch import into the super-resolution network model to train, test, and validate the network model.

[0055] Step 2: Build Figure 3 The projection domain super-resolution network model shown in FIG. 1 includes four modules, namely, the downlink channel branch module, the bottom layer module, the uplink channel branch module, and the channel information compensation and recovery module. The downlink channel branch module is divided into four stages of feature extraction, which are named , , , The network first projects the input sparse angle data Perform shallow feature SF extraction and then input it into each stage for feature extraction, such as Figure 4 shown.

[0056] First, the input information is subjected to a 3x3 convolution layer, a ReLU activation function, a 3x3 convolution layer, and then a skip connection. The result is then subjected to an Average Pooling layer, a 1x1 convolution layer, a ReLU activation function, a 1x1 convolution layer, a Sigmoid activation function, and then multiplied with the initial input to obtain the result, perform channel compression, and input it into the feature extraction of the next layer.

[0057] Step 4: In each stage of the downlink channel branch, a Pooling layer and a 1x1 convolution layer are performed before channel compression for channel information compensation. The channel information of each stage is cascaded and input into the underlying module for channel fusion, such as Figure 5 shown.

[0058] Low-level modules First, the channel information is subjected to a 1x1 convolution layer, a ReLU activation function, a 1x1 convolution layer, a Sigmoid activation function, and then multiplied with the initial input to obtain the result, which is input into the uplink channel branch and each stage of the uplink channel branch.

[0059] Step 5: The uplink channel branch corresponds to the downlink channel branch and is also divided into 4 stages, which are: , , , Each stage also performs feature extraction and cascades with the results of the underlying modules, followed by channel recovery.

[0060] Step 6: After the above steps, the deep feature DF is obtained, added to the shallow feature extracted previously, and then sub-pixel rearrangement PS is performed to obtain the full-view projection data. .

[0061] Step 7. Finally, generate the full-view projection data With real full-view projection data For comparison, the mean square error (MSE) is calculated as the loss function to guide the optimization of the network model. During the model optimization process, the network parameters are adjusted through the back propagation algorithm to gradually minimize the loss value, thereby improving the quality of the full-view data reconstructed by sparse projection.

[0062] (9);

[0063] Where N represents the total amount of data. and represent real data and generated data respectively.

[0064] In order to verify the advantages of the sparse projection CT reconstruction method based on the projection domain super-resolution method proposed in the present invention in CT image reconstruction, it is compared with the existing methods for completing missing data in the projection domain. The existing methods include linear interpolation, SDS-Net, and Res-Unet. The standards used are sinusoidal peak signal-to-noise ratio PSNR, CT image peak signal-to-noise ratio PSNR and CT image structural similarity index SSIM for quantitative analysis.

[0065] from Figure 6 , shows some test CT slices reconstructed by four methods. The reconstructed images of the three deep learning based models are close to and significantly better than the linear interpolated images. As the sparsity level increases, the quality of all reconstructed CT images degrades significantly, and the linear interpolation method degrades most significantly due to not using the prior knowledge in the training set. Our network performs slightly better than the two baseline networks (SDS-net and Res-Unet) at higher sparsity levels (s = 8, 15). As shown in the local zooms corresponding to s = 8 and 15, the proposed network is better in edge preservation of CT images.

[0066] Table 1 Sinogram PSNR (dB), the average PSNR value (dB) in the sinusoidal domain reflects the similarity between the estimated sinusoidal graph and the full view sinusoidal graph. It can be seen that the three deep learning-based methods are significantly better than linear interpolation in sinusoidal domain prediction. Under 2x, 4x, 8x, and 15x sparseness, the network structure we proposed is better than other methods.

[0067] Table 2 CT PSNR (dB), Table 3 CT SSIM (%),Tables 2 and 3 show the average PSNR (dB) and SSIM (%) in the CT image domain, which reflects the similarity between the reconstructed CT image and the expected (full dose) CT image. In the CT image domain, combining the PSNR and SSIM metrics, our proposed network still performs well.

[0068] It can be seen that the projection domain super-resolution method proposed in the present invention is better in both CT image visual effects and quantitative indicators.

[0069]

Claims

1. A sparse projection CT reconstruction method based on projection domain super-resolution method, characterized in that: The steps include: Step 1: Construct sparse projection data, preprocess the projection domain data, divide them into 2x, 4x, 8x, 15x and full view sinogram data sets, and divide them into training set, test set and validation set; Step 2: construct a super-resolution network structure, which includes 4 modules: a downlink channel branch module, a bottom module, an uplink channel branch module, and a channel compensation and recovery module; Step 3, for the downlink channel branch module, the network first inputs the input features into the downlink channel branch for feature extraction on different channels. In the downlink channel branch, there are n layers for feature extraction under n different channel conditions, and n-1 downlink channel modules for channel compression between adjacent layers. The input features are first input into the feature extraction module with m-dimensional channel number to extract features, and then compressed by the downlink channel module to change the channel number to m / 2, and features are extracted in the feature extraction module with m / 2-dimensional channel number, until the next level branch performs feature extraction with m / 2n-dimensional channel number, as shown in formula (1); ； Among them, X represents the input feature, It represents the features obtained after the input features are operated through n layers of downstream channel branches. and Represents the feature extraction operations at different layers, and Represents different downstream channel modules. The downstream channel module is composed of convolutions. The input of the convolution layer is the output of the previous layer, and the output is the input of the next layer. The structure of the feature extraction module at different layers consists of multiple residual attention blocks with local residual connections. Step 4: The bottom layer module fuses the features of different channels obtained by the downlink channel branch and extracts the relevant features between different channels, as shown in formula (2); ； in, Represents the operation of the underlying module, Represents the features obtained through the operation of the underlying module, used to branch from the downstream channel Different channel characteristics in advance; Step 5: For the uplink channel branch module, an uplink channel branch is constructed, and different levels of channel recovery are performed on the feature information fused and extracted by the bottom module in the case of small channels, and feature extraction is performed in different channel recovery processes. The structure of the uplink channel branch is similar to that of the downlink channel branch, with n layers and n-1 uplink channel modules, which are used for feature extraction in n different channel conditions and feature recovery between adjacent channel layers, respectively. The uplink channel branch uses n-1 uplink channel modules to extract the feature information in the case of small channels. The features of the dimensional channel are restored to the features of the m-dimensional channel, as shown in formula (3); ； in, It indicates the features extracted by the bottom-level module after being operated by n layers of upstream channel branches. and Represents different uplink channel modules. The uplink channel module is composed of deconvolution. The input channel value of the deconvolution is the output channel value of the next level, and the output channel value is the input channel value of the previous level. Step 6: For the channel information compensation and recovery module, compensate for the feature information lost during the compression process of different levels of the downlink channel branch, as follows: In the process of channel compression at different layers of the downstream branch channel, the features before compression are first pooled, and then a 1x1 convolution operation is performed on the pooled features. Finally, these features are cascaded and input into the underlying module, and formula (2) is updated as follows; ； in, represents the pooling operation, represents a 1x1 convolution operation, Indicates a cascade operation, the input channel value of the bottom module is , output channel value ; The upstream channel branch performs compensation operations in the recovery process of different levels. First, a 1x1 convolution operation is performed on the features after the operation of the bottom module. Then, the features obtained by the operation are cascaded with the features before the channel recovery of different levels. Finally, the channel recovery operation is performed on the cascaded features. Formula (3) is updated as follows: ； Step 7: In summary, it can be divided into shallow feature extraction, deep feature extraction, sub-pixel upsampling and image reconstruction operations. Assume is the sinusoidal graph of the input sparse angle, The network model first extracts shallow features from the sparse angle sinusoidal graph through the initial convolutional layer, as shown in formula (6); ； in, represents the initial convolutional layer operation, Represents the extracted shallow features. Through the following U-net multiple cycles, the shallow features are extracted into the deep features of the image, as shown in formula (7); ； in, represents the extracted deep features, represents multiple cycles of the U-net network, and then the shallow features and deep features are sub-pixel upsampled, and the features obtained by sub-pixel upsampling are further subjected to 1x1 convolution to obtain the final full-view sinusoidal map, as shown in formula (8); ； in, represents the features obtained by sub-pixel upsampling, represents the sub-pixel upsampling operation, represents 1x1 convolution, A sinusoidal diagram representing the full viewing angle; Step 8: Perform filtering back projection on the full-view sinogram to obtain a CT image. With real full-view projection data For comparison, the mean square error (MSE) is calculated as the loss function to guide the optimization of the network model.

2. The sparse projection CT reconstruction method based on the projection domain super-resolution method according to claim 1, characterized in that: In the optimization process of the network model, the network parameters are adjusted by the back propagation algorithm to gradually minimize the Loss value, thereby improving the quality of the full-view data reconstructed by sparse projection, as shown in formula (8); （9）； Where N represents the total amount of data. and represent real data and generated data respectively.

Images