Extremely sparse angle CT reconstruction method of multi-stage processing neural network based on double-domain joint mode

Abstract

The invention belongs to the field of industrial CT image processing, discloses an extremely sparse angle CT reconstruction method of a multi-stage processing neural network based on a double-domain joint mode, and is used for solving the problem that a traditional reconstruction method cannot reconstruct a high-quality CT image at an extremely sparse sampling view angle. The reconstruction method comprises the following steps: firstly, carrying out extremely sparse sampling on full-angle projection data A acquired by a computed tomography system to obtain projection data B; interpolating the projection data B, and inputting the interpolated projection data B into the trained sine domain network to obtain optimized projection data C; secondly, preliminarily reconstructing the projection data C into a CT image D by using a fast iterative soft threshold shrinkage algorithm (FISTA); and finally, optimizing the image D through a trained image domain network to obtain a high-quality CT image E. According to the method, the CT image under extremely sparse angle sampling can be reconstructed, and the reconstructed CT image has higher structural integrity and definition.

Description

Technical Field

[0001] The present invention relates to an industrial CT image reconstruction method, and in particular to an extremely sparse angle CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode. Background Art

[0002] CT is an imaging method widely used in fields such as medical diagnosis and industrial inspection. Generally, it obtains cross-sectional information by performing circular scanning of objects through X-rays or the like. Thanks to the high chemical specificity provided by X-rays or the like, the internal structure and information of the sample can be obtained without destroying the shape and structure of the sample workpiece. This technology can intuitively and accurately reflect the defect status of the sample being inspected, and provide strong technical support for analyzing the structure in industrial materials. However, full-angle CT scanning takes a long time and is difficult to meet the needs of real-time detection in industry. One way to effectively solve the above problem is to reduce the number of angles during CT sampling and perform sparse angle sampling. However, since sparse angle reconstruction does not satisfy the Nyquist sampling theorem, its reconstruction process will generate a large number of artifacts and noise, which will become more serious as the angle decreases. At present, the algorithms commonly used for sparse angle CT image reconstruction are analytical reconstruction algorithms and iterative reconstruction algorithms, but these two types of algorithms will have problems such as beam hardening effect and high computational complexity when reconstructing extremely sparse angle CT images. Subsequently, the birth of CT reconstruction algorithms based on deep learning can eliminate artifacts while maintaining the complete structure of CT images. However, for extremely sparse angles, most deep learning algorithms suffer from severe loss of image details and edge information, resulting in poor quality of reconstructed images. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of existing technologies and provide an extremely sparse angle CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode. The industrial CT reconstruction algorithm can directly reconstruct extremely sparse angle projection data and retain the complete contour structure and rich internal details.

[0004] The technical solution of the present invention to solve the above technical problems is:

[0005] A very sparse angle CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode comprises the following steps:

[0006] (S1), using the detection element of the computed tomography system to rotate 180 degrees to collect full-angle projection data and put them into image set A;

[0007] (S2), extracting the image data in the image set A at equal intervals to obtain extremely sparse undersampled projection data, and further filling the data using a bilinear interpolation method to obtain initial full-angle projection data and putting it into the image set B;

[0008] (S3), inputting the images in the image set B into the trained sine domain network to obtain an optimized projection data image set C;

[0009] (S4), based on the FISTA algorithm built in Pytorch, the optimized projection data in the image set C are initially reconstructed into the CT image set D;

[0010] (S5) Input the CT image set D into the trained image domain network to obtain a high-quality CT image set E.

[0011] According to the extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode of claim 1, it is characterized in that in step (S3), the sinusoidal domain network finally outputs the loss function value between the image set C and the image set A.

[0012] The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 1 is characterized in that, in step (S5), the training of the image domain network comprises the following steps:

[0013] (S5-1), using the FISTA algorithm to reconstruct the CT image of the image set A acquired in step (S1) and put it into the image set Image-Label;

[0014] (S5-2), calculating the loss function value between the image set E output by the image domain processing network and the image set Image-Label;

[0015] (S5-3) Combine the loss functions of (S3) and (S5-2) and back-propagate to update the network weights and parameters. The training optimizer is Adam. When the set rounds are reached, the model weights and bias parameters are updated, and a fully trained multi-stage processing neural network is obtained.

[0016] Preferably, in step (S1), projection data of corresponding angles are collected at equal intervals, which is 180 full angles required for CT reconstruction.

[0017] Preferably, in step (S2), image set A is further extracted at equal intervals to obtain 1 / 20 and 1 / 30 of the full angles of 180 required for CT reconstruction.

[0018] Preferably, according to the extremely sparse angle CT reconstruction method of the multi-stage processing neural network based on the dual-domain joint mode of claim 2, it is characterized in that, in step (S3), the sinusoidal domain network structure is composed of a symmetrical encoder and decoder, and its depth is 4 layers, including an enhanced convolution module, a maximum pooling downsampling layer, a sub-pixel convolution upsampling layer, and channel splicing extracted from the corresponding dimensional features of the encoder and decoder; wherein, the enhanced convolution module introduces asymmetric convolution of cascaded convolution kernels (3×3, 1×3, 3×1 convolution kernels) to extract directional features, and dynamically adjusts the channel weights in combination with the channel attention mechanism to enhance the feature expression capability; the downsampling layer performs 2x pixel downsampling by maximum pooling; the sub-pixel convolution upsampling layer uses a 3×3 convolution kernel size, a linear rectification activation function, and a batch normalization layer to obtain 2x pixel information through 4x dimensional information compensation; the dimension of the sinusoidal domain network downsampling is obtained by 2 n The upsampling dimension is 2 n Decreasing form.

[0019] Preferably, according to the extremely sparse angle CT reconstruction method of the multi-stage processing neural network based on the dual-domain joint mode of claim 3, it is characterized in that the image domain network adopts a multi-stage step-by-step optimization structure design, and each stage includes ASE-UNet, CCA convolution channel attention mechanism and IAM information attention mechanism, etc.; wherein, the CCA convolution channel attention module dynamically adjusts the channel features in the form of convolution to highlight key information; the IAM information attention module adjusts the weights by detecting the feature distribution of the artifact area, effectively suppressing noise and enhancing key details; the first stage of image domain network reconstruction outputs a low-resolution reconstructed image, the second stage splices the output reconstructed image with the input data and further optimizes the details, and the last stage outputs a high-quality CT reconstructed image.

[0020] Preferably, the loss function of the training network includes three, which are as follows: the loss function shared by the sine domain network and the image domain network is given by Loss and The mean square error loss composition, as well as the loss function combined with the Sobel edge characteristics in the image domain network, the L1 loss function is:

[0021]

[0022] Among them, y is the true value; is the predicted value;

[0023] The mean square error loss function is:

[0024]

[0025] Among them, x is a truth value; is the predicted value;

[0026] The Sobel edge loss function is:

[0027]

[0028] Among them, x is the gradient strength of the real image; is the gradient strength of the predicted image; Z is the convolution kernel of the Sobel operator, that is

[0029] The final network loss function is:

[0030]

[0031] Among them, z is the true value of the network output; is the network output prediction value; x and Represent the true value and predicted value of CT images respectively; y and Represent the true value and predicted value of the projection image respectively.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. The extremely sparse angle CT reconstruction method of the present invention based on a multi-stage processing neural network in a dual-domain joint mode inputs CT image data into a multi-stage network architecture, ensures the consistency of multi-scale feature extraction and reconstruction in the image reconstruction process by a step-by-step optimization method, and avoids false signals introduced due to insufficient information in the image generation process, thereby improving the accuracy and quality of image reconstruction.

[0034] 2. The extremely sparse angle CT reconstruction method of the present invention based on a multi-stage processing neural network in a dual-domain joint mode significantly improves the reconstructed image quality through staged feature optimization and iterative image restoration. In each stage, an encoder-decoder structure and a dynamic attention mechanism are adopted to gradually integrate the high and low frequency information of the sinusoidal domain and the image domain, thereby restoring the edge details and texture information of the image under extremely sparse angle data and reducing the interference of artifacts.

[0035] 3. The extremely sparse angle CT reconstruction method of the present invention based on a multi-stage processing neural network in a dual-domain joint mode combines multi-stage step-by-step optimization and edge information enhancement technology to achieve refined processing of sparse data. In each stage of the network, it focuses on the recovery of low-frequency information and the refinement of high-frequency information respectively. Through cross-stage residual learning, the edge sharpness and detail restoration ability of the image are ensured, which is suitable for industrial CT scenarios with high requirements for distinguishing tiny structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1This is a flow chart of the extremely sparse angle CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode of the present invention.

[0037] Figure 2 It is a schematic diagram of a sinusoidal domain network of an extremely sparse angle CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode of the present invention.

[0038] Figure 3 It is a schematic diagram of the image domain network of the extremely sparse angle CT reconstruction method based on the multi-stage processing neural network of the dual-domain joint mode of the present invention. DETAILED DESCRIPTION

[0039] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0040] See also Figure 1 The extremely sparse angle CT reconstruction method based on a multi-stage processing neural network of a dual-domain joint mode of the present invention comprises the following steps:

[0041] (S1) Use the detection element of the computed tomography system to rotate 180 degrees to collect full-angle projection data and put it into image set A. The projection image resolution is N 1 ×N 2 (projection angle × number of detectors);

[0042] (S2) The image data in image set A are extracted at equal intervals to obtain extremely sparse undersampled projection data, and then the initial full-angle projection data is filled using the bilinear interpolation method and placed in image set B, whose resolution is 180×N. 2 ;

[0043] (S3), placing the images in the image set B into the trained sine domain network to obtain an optimized projection data image set C;

[0044] (S4), based on the FISTA algorithm built in Pytorch, the optimized projection data in the image set C are initially reconstructed into a CT image set D with a resolution of 256×256;

[0045] (S5) Put the CT image set D into the trained image domain network to obtain a high-quality CT image set E.

[0046] See also Figure 1 In step (S3), the sinusoidal domain network finally outputs the loss function value between the image set C and the image set A.

[0047] See also Figure 1 In step (S5), the training of the image domain network includes the following steps:

[0048] (S5-1), using the FISTA algorithm to reconstruct the CT image of the image set A acquired in step (S1) and put it into the image set Image-Label;

[0049] (S5-2), calculating the loss function value between the image set E output by the image domain processing network and the image set Image-Label;

[0050] (S5-3), combine the loss functions of (S3) and (S5-2) and back-propagate to update the network weights and parameters. The training optimizer is Adam, the initial momentum parameter of the optimizer is set to (0.9, 0.999), the initial learning rate is 0.0001, the batch size is 2, the number of iterations is 100, and when the set number of rounds is reached, the model weights and bias parameters are updated, and a fully trained dual-domain joint mode multi-stage processing neural network is obtained.

[0051] In the task of reconstructing CT images with extremely sparse angular sampling, the collected data mainly contains global low-frequency feature information. For the content of the unsampled area, the traditional iterative algorithm cannot accurately compensate, resulting in serious superposition of artifacts and noise, thus affecting the quality of image reconstruction. The single-domain reconstruction network is prone to generate distorted structural information due to the lack of collaborative constraints on data between different domains, which weakens the accuracy of reconstruction. The existing dual-domain reconstruction method is insufficient in the extraction of global high-frequency features when processing extremely sparse data, which limits the full restoration of image details, especially in the reconstruction of edge details and texture information.

[0052] For this purpose, see Figure 2 The sine domain network proposed in this example. The sine domain network structure is composed of a symmetrical encoder and decoder with a depth of 4 layers, including an enhanced convolution module, a maximum pooling downsampling layer, a sub-pixel convolution upsampling layer, and channel splicing extracted from the corresponding dimensional features of the encoder and decoder; among them, the enhanced convolution module introduces asymmetric convolution of cascaded convolution kernels (3×3, 1×3, 3×1 convolution kernels) for directional feature extraction, and dynamically adjusts the channel weights in combination with the channel attention mechanism to enhance the feature expression capability; the downsampling layer performs 2x pixel downsampling by maximum pooling; the sub-pixel convolution upsampling layer uses a 3×3 convolution kernel size, a linear rectification activation function, and a batch normalization layer to obtain 2x pixel information through 4x dimensional information compensation; the dimension of the sine domain network downsampling is obtained by 2 n The upsampling dimension is 2 n Decreasing form.

[0053] See also Figure 3The image domain processing model proposed in this example. The image domain network adopts a multi-stage step-by-step optimization structure design, each stage includes ASE-UNet, CCA convolution channel attention mechanism and IAM information attention mechanism, etc. Among them, the CCA convolution channel attention module dynamically adjusts the channel features in the form of convolution to highlight key information; the IAM information attention module adjusts the weights by detecting the feature distribution of the artifact area, effectively suppressing noise and strengthening key details; the first stage of image domain network reconstruction outputs a low-resolution reconstruction image, the second stage splices the output reconstructed image with the input data and further optimizes the details, and the last stage outputs a high-quality CT reconstructed image.

[0054] In addition, there are three loss functions for training the network, which are as follows: The loss function shared by the sine domain network and the image domain network is given by Loss and The mean square error loss composition, as well as the loss combined with the Sobel edge characteristics in the image domain network, the L1 loss function is:

[0055]

[0056] Among them, y is the true value; is the predicted value;

[0057] The mean square error loss function is:

[0058]

[0059] Among them, x is a truth value; is the predicted value;

[0060] The Sobel edge loss function is:

[0061]

[0062] Among them, x is the gradient strength of the real image; is the gradient strength of the predicted image; Z is the convolution kernel of the Sobel operator, that is

[0063] The final network loss function is:

[0064]

[0065] Among them, z is the true value of the network output; is the network output prediction value; x and Represent the true value and predicted value of CT images respectively; y and Represent the true value and predicted value of the projection image respectively.

[0066] The industrial CT reconstruction method of the present invention is described below with a specific case:

[0067] First, the projection data of the corresponding sparse angles are collected through CT and placed in image set A; the images in image set A are sampled at extremely sparse angles and the unsampled angles are supplemented using the bilinear interpolation method and placed in image set B; then image set B is input into the trained sinusoidal domain network to obtain the optimized sinusoidal image set C; then the FISTA algorithm is used to perform CT reconstruction on image set C and image set A respectively to obtain image set D and image set Image-Label; image set D is input into the trained image domain network to obtain CT image set E; finally, the loss is calculated for the sinusoidal image set C and image set A, and the loss is calculated for image set E and image set Image-Label; the loss values ​​are combined to update the network parameters.

[0068] The above is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited to the above content. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A very sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode, characterized in that: The following steps are involved: (S1), using the detection element of the computed tomography system to rotate 180 degrees to collect full-angle projection data and put them into image set A; (S2), extracting the image data in the image set A at equal intervals to obtain extremely sparse undersampled projection data, and further filling the data using a bilinear interpolation method to obtain initial full-angle projection data and putting it into the image set B; (S3), inputting the images in the image set B into the trained sine domain network to obtain an optimized projection data image set C; (S4), based on the FISTA algorithm built in Pytorch, the optimized projection data in the image set C are initially reconstructed into the CT image set D; (S5) Input the CT image set D into the trained image domain network to obtain a high-quality CT image set E.

2. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 1 is characterized in that: In step (S3), the sinusoidal domain network finally outputs the loss function value between the image set C and the image set A.

3. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 1 is characterized in that: In step (S5), the training of the image domain network includes the following steps: (S5-1), using the FISTA algorithm to reconstruct the CT image of the image set A acquired in step (S1) and put it into the image set Image-Label; (S5-2), calculating the loss function value between the image set E output by the image domain network and the image set Image-Label; (S5-3) Combine the loss functions of (S3) and (S5-2) and back-propagate to update the network weights and parameters. The training optimizer is Adam. When the set rounds are reached, the model weights and bias parameters are updated, and a fully trained multi-stage processing neural network is obtained.

4. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 1, characterized in that: In step (S2), image set A is further extracted at equal intervals to obtain 1 / 20 and 1 / 30 of the full angles of 180 required for CT reconstruction, that is, 9 and 6.

5. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 2, characterized in that: In step (S3), the sine domain network structure is composed of a symmetrical encoder and decoder, with a depth of 4 layers, including an enhanced convolution module, a maximum pooling downsampling layer, a sub-pixel convolution upsampling layer, and channel splicing extracted from the corresponding dimensional features of the encoder and decoder; wherein the enhanced convolution module introduces asymmetric convolution of cascaded convolution kernels (3×3, 1×3, 3×1 convolution kernels) to extract directional features, and dynamically adjusts the channel weights in combination with the channel attention mechanism to enhance the feature expression capability; the downsampling layer performs 2x pixel downsampling by maximum pooling; the sub-pixel convolution upsampling layer uses a 3×3 convolution kernel size, a linear rectification activation function, and a batch normalization layer to obtain 2x pixel information through 4x dimensional information compensation; the dimension of the sine domain network downsampling is obtained by 2 n The upsampling dimension is 2 n Decreasing form.

6. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 3, characterized in that: The image domain network adopts a multi-stage step-by-step optimization structure design. The first stage outputs a low-resolution reconstructed image. The second stage splices the output reconstructed image with the input data and further optimizes the details. The last stage outputs a high-quality CT reconstructed image.

7. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 6, characterized in that: Each of the described stages includes ASE-UNet, CCA convolutional channel attention mechanism and IAM information attention mechanism, etc.; among them, the CCA convolutional channel attention module dynamically adjusts the channel features in the form of convolution to highlight key information; the IAM information attention module adjusts the weights by detecting the feature distribution of the artifact area, effectively suppressing noise and enhancing key details.

8. The extremely sparse angular CT reconstruction method based on a multi-stage processing neural network in a dual-domain joint mode according to claim 2 and claim 3 is characterized in that: The loss function of the training network includes three functions, which are as follows: the loss function shared by the sine domain network and the image domain network is given by Loss and The mean square error loss composition, as well as the loss function combined with the Sobel edge characteristics in the image domain network, the L1 loss function is: Among them, y is the true value; is the predicted value; The mean square error loss function is: Among them, x is a truth value; is the predicted value; The Sobel edge loss function is: Among them, x is the gradient strength of the real image; is the gradient strength of the predicted image; the convolution kernel of the ZSobel operator, that is The final network loss function is: Among them, z is the true value of the network output; is the network output prediction value; x and Represent the true value and predicted value of CT images respectively; y and Represent the true value and predicted value of the projection image respectively.

Images