A multi-stage processing neural network based on a two-domain joint mode for extremely sparse angle CT reconstruction

Abstract

The application belongs to the field of industrial CT image processing, and discloses a kind of extremely sparse angle CT reconstruction method of multi-stage processing neural network based on dual-domain joint mode, for solving the problem that traditional reconstruction method cannot reconstruct high-quality CT image under extremely sparse sampling view angle, the reconstruction method of the application first carries out extremely sparse sampling to the full-angle projection data A obtained by computer tomography system acquisition and obtains projection data B;Then projection data B is input to the trained sinusoidal domain network after interpolation to obtain optimized projection data C;Then projection data C is primarily reconstructed into CT image D using fast iterative soft threshold shrinkage algorithm (FISTA);Finally, a trained image domain network is used to optimize image D to obtain high-quality CT image E.The method of the application can reconstruct CT image under extremely sparse angle sampling, and the reconstructed CT image has higher structural integrity and clarity.

Description

Technical Field

[0001] This invention relates to an industrial CT image reconstruction method, specifically to an extremely sparse angular CT reconstruction method based on a multi-stage processing neural network with a dual-domain joint mode. Background Technology

[0002] CT (Computed Tomography) is an imaging technique widely used in medical diagnosis and industrial inspection. It typically uses X-rays to perform a circular scan of an object to obtain cross-sectional information. Thanks to the high chemical specificity of X-rays, the internal structure and information of a sample can be obtained without damaging its shape and structure. This technology can intuitively and accurately reflect the defects in the sample being inspected, providing strong technical support for analyzing the structure of industrial materials. However, full-angle CT scans are time-consuming, making it difficult to meet the real-time inspection needs of industry. One effective solution is to reduce the number of angles in CT sampling, using sparse angle sampling. However, sparse angle reconstruction does not satisfy the Nyquist sampling theorem, resulting in a large number of artifacts and noise during the reconstruction process, which worsens as the angle decreases. Currently, analytical and iterative reconstruction algorithms are commonly used for sparse angle CT image reconstruction, but these algorithms suffer from beam hardening and high computational complexity when reconstructing extremely sparse angle CT images. Subsequently, deep learning-based CT reconstruction algorithms emerged, which can eliminate artifacts while maintaining the integrity of the CT image structure. However, for extremely sparse angles, most deep learning algorithms suffer from severe loss of image details and edge information, resulting in poor quality of reconstructed images. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of current technology and provide an extremely sparse angle CT reconstruction method based on a multi-stage processing neural network with a dual-domain joint mode. The industrial CT reconstruction algorithm can directly reconstruct the projection data of extremely sparse angles, and can preserve the complete contour structure and rich internal details.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0005] An extremely sparse angular CT reconstruction method based on a multi-stage processing neural network with dual-domain joint mode includes the following steps:

[0006] S1. Use the detector element of the computed tomography system to rotate 180 degrees to collect full-angle projection data and put it into image set A;

[0007] S2. Extract the image data in image set A at equal intervals to obtain extremely sparse undersampled projection data, and then use bilinear interpolation to fill in the initial full-angle projection data and put it into image set B.

[0008] S3, input the images in the image set B into the sinusoidal domain network of the encoder and decoder structured as symmetry, the depth of which is 4 layers, including an enhanced convolution module, a maximum pooling down-sampling layer, a sub-pixel level convolution up-sampling layer and channel splicing extracted from the corresponding dimensional features of the encoder and decoder; wherein the enhanced convolution module introduces asymmetric convolution of cascaded convolution kernels, including 3x3, 1x3 and 3x1 convolution kernels, for directional feature extraction and dynamic adjustment of channel weights in combination with a channel attention mechanism; the maximum pooling down-sampling layer performs 2-pixel down-sampling; the sub-pixel convolution up-sampling layer uses a 3x3 convolution kernel, a linear rectifier activation function and a batch normalization layer to obtain 2-pixel information through 4-dimensional information compensation; the down-sampled dimension is increased in the form of 2 n , and the up-sampled dimension is decreased in the form of 2 n , and finally the optimized projection data image set C is obtained;

[0009] S4, the FISTA algorithm built in Pytorch is used to initially reconstruct the optimized projection data in the image set C into a CT image set D;

[0010] S5, the CT image set D is input into the trained image domain network, the image domain network adopts a multi-stage step-by-step optimization structure design, the first stage outputs a low-resolution reconstructed image, the second stage splices and further optimizes the details of the output reconstructed image and the input data, and the last stage outputs a high-quality CT reconstructed image; each stage includes an ASE-UNet, a CCA convolution channel attention mechanism and an IAM information attention mechanism; wherein the CCA convolution channel attention module dynamically adjusts the channel features through convolution, thereby highlighting the key information; the IAM information attention module adjusts the weight by detecting the feature distribution of the artifact area, effectively suppresses the noise and strengthens the key details, and finally obtains a high-quality CT image set E.

[0011] The multi-stage processing neural network based on a dual-domain joint mode for extremely sparse angle CT reconstruction method according to claim 1, wherein in step S3, the sinusoidal domain network finally outputs an image set C and an image set A to calculate a loss function value.

[0012] The multi-stage processing neural network based on a dual-domain joint mode for extremely sparse angle CT reconstruction method according to claim 1, wherein in step S5, the training of the image domain network includes the following steps:

[0013] S5-1, using the FISTA algorithm to reconstruct the CT image in the image set A collected in step S1 into an image set Image-Label;

[0014] S5-2, calculating the loss function value between the image set E output by the image domain network and the image set Image-Label;

[0015] S5-3, combining the loss functions of S3 and S5-2 and updating the network weights and parameters in reverse propagation, the optimizer for training is Adam, and the updating of the model weights and bias parameters is stopped when the set round is reached, so that the trained multi-stage processing neural network is obtained.

[0016] Preferably, in step S1, the projection data corresponding to the equidistant angles are collected as 180 full angles required in CT reconstruction.

[0017] Preferably, in step S2, the image set A is further extracted equidistantly to 1 / 20 or 1 / 30 of 180 full angles required in CT reconstruction.

[0018] Preferably, the loss function for training the network includes three, which are as follows: the loss function shared by the sine domain network and the image domain network, which is composed of L1 loss and mean square error loss, and the loss function combined with the Sobel edge characteristics in the image domain network, the L1 loss function is:

[0019]

[0020] The mean square error loss function is:

[0021]

[0022] The Sobel edge loss function is:

[0023]

[0024] Wherein, Z is the convolution kernel of the Sobel operator, that is

[0025] The final network loss function is:

[0026]

[0027] Wherein, z is the true value of the network output; is the predicted value of the network output; x and respectively represent the true value and the predicted value of the CT image; y and respectively represent the true value and the predicted value of the projection image.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] 1. The multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application, CT image data is input into the multi-stage network architecture, and the consistency of multi-scale feature extraction and reconstruction in the image reconstruction process is ensured through step-by-step optimization, and false signals introduced due to insufficient information in the image generation process are avoided, thereby improving the accuracy and quality of image reconstruction.

[0030] 2. The multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application, through periodic feature optimization and iterative image repair, the image quality of the reconstruction is significantly improved, the encoder-decoder structure and dynamic attention mechanism are used in each stage, and high and low frequency information in the sinusoidal domain and the image domain is gradually fused, so that the edge details and texture information of the image are recovered under extremely sparse angle data, and the interference of artifacts is reduced.

[0031] 3. The multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application, combined with multi-stage step-by-step optimization and edge information strengthening technology, realizes fine processing of sparse data, and in each stage of the network, respectively pays attention to the recovery of low frequency information and the refinement of high frequency information, and through cross-stage residual learning ensures the edge sharpness and detail restoration ability of the image, and is suitable for industrial CT scenes with high requirements for distinguishing small structures. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The flow chart of the multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application.

[0033] Figure 2 The sinusoidal domain network schematic diagram of the multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application.

[0034] Figure 3 The image domain network schematic diagram of the multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application. DETAILED DESCRIPTION

[0035] The application will be further described in detail below in combination with embodiments and drawings, but the embodiments of the application are not limited thereto.

[0036] Reference Figure 1 The multi-stage processing neural network based on the dual-domain joint mode of the extremely sparse angle CT reconstruction method of the application includes the following steps:

[0037] S1, using the detection element of the computer tomography imaging system, 180 angle rotation is used to collect full angle projection data and put into image set A, the projection image resolution size is N1XN2, which represents the projection angle X the number of detectors;

[0038] S2, the image data in the image set A is further extracted at equal intervals to obtain extremely sparse undersampling projection data, and an initial full angle projection data is obtained by further using a bilinear interpolation method to fill in the image set B, and the resolution size is 180XN2;

[0039] S3, the image in the image set B is input into the sinusoidal domain network with symmetrical encoder and decoder, the depth is 4 layers, including enhanced convolution module, maximum pooling down sampling layer, sub-pixel level convolution up sampling layer and channel splicing extracted by corresponding dimension feature of encoder and decoder; wherein, the enhanced convolution module introduces the asymmetric convolution of cascaded convolution kernel, including 3X3, 1X3, 3X1 convolution kernel, which carries out directional feature extraction, and combines with channel attention mechanism to dynamically adjust channel weight; the maximum pooling down sampling layer carries out 2 times pixel down sampling; the sub-pixel convolution up sampling layer uses 3X3 convolution kernel, linear rectification activation function and batch normalization layer, and obtains 2 times pixel information through 4 times dimension information compensation; the dimension of down sampling is increased in the form of 2 n , and the dimension of up sampling is decreased in the form of 2 n , finally the optimized projection data image set C is obtained;

[0040] S4, based on the FISTA algorithm built in Pytorch, the optimized projection data in the image set C is initially reconstructed into CT image set D, and the resolution size is 256X256;

[0041] S5, the CT image set D is input into the trained image domain network, and the image domain network adopts the structure design of multi-stage step-by-step optimization, the first stage outputs low resolution reconstructed image, the second stage splices and further optimizes the details of the output reconstructed image and the input data, and the last stage outputs high quality CT reconstructed image; each stage includes ASE-UNet, CCA convolution channel attention mechanism and IAM information attention mechanism; wherein, the CCA convolution channel attention module dynamically adjusts the channel features through the form of convolution, so as to highlight the key information; the IAM information attention module adjusts the weight by detecting the feature distribution of the artifact area, effectively suppresses the noise and strengthens the key details, and finally obtains the high quality CT image set E.

[0042] Referring to Figure 1 , in step S3, the sinusoidal domain network finally outputs the image set C and the image set A to calculate the loss function value.

[0043] Referring to Figure 1In step S5, the training of the image domain network includes the following steps:

[0044] S5-1, the CT image reconstructed by the image set A collected in step S1 is put into the image set Image-Label using the FISTA algorithm;

[0045] S5-2, the loss function value between the image set E output by the image domain processing network and the image set Image-Label is calculated;

[0046] S5-3, the loss functions of S3 and S5-2 are combined and the network weights and parameters are updated by back propagation, the optimizer for training is Adam, the initial momentum parameters of the optimizer are set to (0.9, 0.999), the initial learning rate is 0.0001, the batch size is 2, the iteration rounds are 100 rounds, the model weight and bias parameters are stopped from being updated when the set rounds are reached, and the trained dual-domain joint mode multi-stage processing neural network is obtained.

[0047] In the CT image reconstruction task of extremely sparse angle sampling, the data collected mainly contains global low-frequency feature information, and for the content of the unsampled region, the traditional iterative algorithm cannot accurately compensate, resulting in serious superposition of artifacts and noise, thereby affecting the reconstruction quality of the image. Due to the lack of collaborative constraints on data between different domains, the single-domain reconstruction network is easy to generate distorted structure information, which weakens the accuracy of reconstruction, and the existing dual-domain reconstruction method has insufficient extraction ability for global high-frequency features when processing extremely sparse data, which limits the full recovery of image details, especially in the reconstruction of edge details and texture information.

[0048] In addition, the loss function for training the network includes three, which are as follows: the loss function shared by the sinusoidal domain network and the image domain network, which is composed of L1 loss and mean square error loss, and the loss combined with the Sobel edge characteristics in the image domain network, the L1 loss function is:

[0049]

[0050] The mean square error loss function is:

[0051]

[0052] The Sobel edge loss function is:

[0053]

[0054] Wherein, Z is the convolution kernel of the Sobel operator, that is

[0055] The final network loss function is:

[0056]

[0057] Wherein, z is the real value of the network output; is the predicted value of the network output; x and respectively represent the real value and the predicted value of the CT image; y and respectively represent the real value and the predicted value of the projection image.

[0058] The following describes the industrial CT reconstruction method of the present application through a specific case:

[0059] First, the corresponding sparse angle projection data is collected by CT acquisition, and the projection data is put into the image set A; the images in the image set A are sampled at extremely sparse angles and supplemented with unsampled angles using a bilinear interpolation method and then put into the image set B; then the image set B is input into the trained sinusoidal domain network to obtain the optimized sinusoidal image set C; then the FISTA algorithm is used to perform CT reconstruction on the image set C and the image set A respectively to obtain the image set D and the image set Image-Label; the image set D is input into the trained image domain network to obtain the CT image set E; finally, the loss is calculated for the sinusoidal image set C and the image set A, and the loss is calculated for the image set E and the image set Image-Label; the loss value is combined to update the network parameters.

[0060] The above is the preferred embodiment of the present application, but the embodiments of the present application are not limited by the above, any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application, all should be equivalent replacement methods, all are included in the protection scope of the present application.

Claims

1. A method for extremely sparse angular CT reconstruction based on a two-domain joint mode multi-stage processing neural network, characterized in that, The method comprises the following steps: S1, using the detection elements of the computer tomography system to rotate 180 degrees to collect full-angle projection data and put into image set A; S2, further using a bilinear interpolation method to fill in the initial full-angle projection data obtained by equally spaced extraction of the image data in the image set A to obtain extremely sparse undersampling projection data and put into image set B; S3, input the images in the image set B into the sinusoidal domain network of the encoder and decoder with symmetrical structure, the depth of which is 4 layers, including an enhanced convolution module, a maximum pooling down-sampling layer, a sub-pixel level convolution up-sampling layer and channel splicing extracted from the corresponding dimensional features of the encoder and decoder; wherein the enhanced convolution module introduces asymmetric convolution of cascaded convolution kernels, including 3x3, 1x3 and 3x1 convolution kernels, for directional feature extraction, and dynamically adjusts the channel weight in combination with the channel attention mechanism; the maximum pooling down-sampling layer performs 2-pixel down-sampling; the sub-pixel convolution up-sampling layer uses a 3x3 convolution kernel, a linear rectifier activation function and a batch normalization layer to obtain 2-pixel information through 4-dimensional information compensation; the down-sampled dimension is increased by 2 n in form, and the up-sampled dimension is decreased by 2 n in form, and finally the optimized projection data image set C is obtained; S4, based on the FISTA algorithm built in Pytorch, the optimized projection data in the image set C is initially reconstructed into a CT image set D; S5, the CT image set D is input into the trained image domain network, the image domain network adopts a multi-stage step-by-step optimization structure design, the first stage outputs a low-resolution reconstructed image, the second stage splices and further optimizes the details of the output reconstructed image and the input data, and the last stage outputs a high-quality CT reconstructed image; each stage comprises an ASE-UNet, a CCA convolution channel attention mechanism and an IAM information attention mechanism; wherein the CCA convolution channel attention module dynamically adjusts the channel features through convolution, thereby highlighting the key information; the IAM information attention module adjusts the weight by detecting the feature distribution of the artifact area, effectively suppresses the noise and strengthens the key details, and finally obtains a high-quality CT image set E.

2. The method of claim 1, wherein the method is a two-domain joint pattern based multi-stage processing neural network for extremely sparse angle CT reconstruction. In step S3, the sine domain network finally outputs the image set C and calculates the loss function value between the image set A.

3. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network with a dual-domain joint mode according to claim 1, characterized in that, In step S5, the training of the image domain network comprises the following steps: S5-1, using the FISTA algorithm to reconstruct the CT image of the image set A collected in step S1 and put into the image set Image-Label; S5-2, calculating the loss function value between the image set E output by the image domain network and the image set Image-Label; S5-3, combining the loss functions of S3 and S5-2 and updating the network weight and parameter through back propagation, the optimizer for training is Adam, and the model weight and bias parameter are stopped from being updated when the set number of rounds is reached, thereby obtaining a trained multi-stage processing neural network.

4. The method of claim 1, wherein the method is a two-domain joint pattern based multi-stage processing neural network for extremely sparse angular CT reconstruction. In step S2, the image set A is equally spaced extracted to 1 / 20, 1 / 30, i.e., 9, 6, of the 180 full angles required in CT reconstruction.

5. The method of claim 2 or claim 3, wherein the method is a two-domain joint pattern based multi-stage processing neural network for extremely sparse angle CT reconstruction, characterized in that, The loss function for the training network includes three functions, as follows: a loss function shared by the sinusoidal domain network and the image domain network, which is... Loss and The mean squared error loss consists of the following components, and the loss function in the image domain network that incorporates Sobel edge characteristics is as follows: The mean square error loss function is: The Sobel edge loss function is: wherein Z is a convolution kernel of a Sobel operator, i.e. The final network loss function is: where z is the network output real value; is the network output predicted value; x and respectively represent the real value and the predicted value of the CT image; y and respectively represent the real value and the predicted value of the projection image.

Images