Multi-scale generative adversarial network correction method for industrial CT image coupling artifacts

Abstract

The invention provides a multi-scale generative adversarial network correction method for industrial CT image coupling artifacts, and belongs to the field of digital image processing. The invention provides a multi-scale generative adversarial network correction method aiming at the problem that image degradation caused by the coupling effect of multiple artifacts in industrial CT imaging is difficult to effectively correct, a feature pyramid structure is utilized to help a network to capture more comprehensive artifact feature information, and a multi-scale discriminator is matched to build a generative adversarial network framework, so that the image quality degradation is effectively corrected. And the generated artifact-removed image is clearer and more real. The method has strong coupling artifact correction capability, and can better restore detail information of the image, so that the accuracy of the geometric dimension and the contour edge of the part in the CT image is ensured.

Description

Technical Field

[0001] The invention belongs to the field of digital image processing and relates to a multi-scale generative adversarial network correction method for industrial CT image coupling artifacts. Background Art

[0002] Nowadays, high-end manufacturing represented by 3D printing, precision casting, and preparation of new composite materials has an urgent need for industrial CT (Computed Tomography) detection. However, since industrial CT is affected by factors such as mathematics, equipment, and physics (such as reconstruction algorithms, CT system characteristics, characteristics of the object being inspected, and characteristics of the action of radiation substances, etc.) during the imaging process, especially many physical assumptions used to derive reconstruction calculations can only be approximately met during the actual imaging process, resulting in serious degradation of image quality, greatly reducing image contrast and signal-to-noise ratio, making the image contour blurred, noise prominent, resolution insufficient, and information lost in some areas, greatly increasing the possibility of misjudgment and missed detection, and also greatly limiting the application of industrial CT in some fields.

[0003] Images that do not match the actual situation in industrial CT images are usually called artifacts. According to the actual imaging process of industrial CT, artifacts mainly include blurring artifacts caused by the point spread degradation of the imaging system, ring artifacts caused by inconsistent pixel response of the detector, scattering artifacts caused by ray scattering, and beam hardening artifacts caused by the beam hardening effect of multi-color rays. Noise is bound to exist in industrial CT imaging, and it is usually a mixture of multiple modes of noise. In many cases, it is also treated as an artifact. In the industrial CT images of the inspected object, the artifacts that cause serious degradation of image quality are usually the coupling effects of the above-mentioned artifacts. In practice, it is difficult to completely separate the various artifacts and correct them one by one.

[0004] At present, the technologies for coupling artifact correction of CT images can be roughly divided into three types: physical model-based methods, image processing-based methods and deep learning methods. Physical model-based methods include hardware blocking suppression method and hardware pre-hardening method, which focus on reducing the scattering and hardening effects caused by metal objects, but such methods will lead to a decrease in signal-to-noise ratio and also have system limitations that need to be solved. Image processing-based methods include preprocessing and postprocessing. Linearization is the most commonly used preprocessing correction method to convert multi-color projection data into monochrome projection data. The post-processing method is based on image segmentation and reprojection. Although it has greatly improved the imaging quality in terms of applicability, effect, performance and accuracy, there is no effective correction algorithm that can simultaneously meet the requirements for coupling artifacts caused by multiple causes. The deep learning decoupling artifact algorithm can realize the two operations of blur kernel estimation and non-blind deconvolution to restore the image, and can also use an end-to-end method to perform blind deblurring, thereby realizing the restoration of coupling artifact images. The current deep learning-based coupled artifact correction method for industrial CT images still has problems such as poor applicability of the correction object, poor image detail restoration effect, and long processing time. It is necessary to further develop relevant correction methods to improve the application effect and scope of industrial CT. Summary of the invention

[0005] In view of the problem that blurring artifacts, ring artifacts, scattering artifacts, beam hardening artifacts, noise and their coupling effects in industrial CT imaging cause degradation of CT images, which is difficult to correct effectively, the present invention provides a multi-scale generative adversarial network correction method for coupled artifacts of industrial CT images. The feature pyramid structure is used to help the network capture more comprehensive artifact feature information, and a generative adversarial network framework is built in conjunction with a multi-scale discriminator, so that the generated de-artifacted image is clearer and more realistic.

[0006] The technical solution adopted by the present invention to solve the technical problem comprises the following steps:

[0007] Step 1: According to the type of parts that need to be corrected for industrial CT image coupling artifacts, select part design models and actual parts of the same or similar types, and construct a fused CT image dataset that fuses the simulated CT image dataset and the actual CT image dataset;

[0008] Step 2: Construct a multi-scale generative adversarial network suitable for industrial CT image coupled artifact correction, which mainly includes a feature extraction module, a feature fusion and regression module, and a feature discrimination module;

[0009] Step 3: Use the fused CT image dataset to train the constructed multi-scale generative adversarial network for coupled artifact correction. The loss function used in network training consists of three parts: pixel space loss, perceptual loss, and discriminator loss. The pixel space loss includes two parts: structural similarity pixel space loss and mean square error pixel space loss. The discriminator loss is a fusion of global and local discriminator losses.

[0010] Step 4: Using peak signal-to-noise ratio and structural similarity as evaluation indicators, the fused CT image dataset is used to adjust the network hyperparameters, and the optimal hyperparameter settings are found through multiple iterations to achieve network model optimization;

[0011] Step 5: Use the trained multi-scale generative adversarial network to correct the industrial CT images of parts that need coupling artifact correction.

[0012] Furthermore, the actual CT image data set in step 1 is obtained by industrial CT scanning and reconstruction of the actual parts, including the actual coupling artifact CT image without any correction, and the corresponding actual ideal CT image after necessary correction using the existing method. Since the existing method cannot completely correct the coupling artifact, the actual ideal CT image is only an approximation of the ideal state. In the industrial CT scanning of the actual parts, the scattering model (using grating scanning, array plate scanning, etc.) and the beam hardening model (using beam hardening curve fitting, etc.) are obtained and used in the processing of the simulated CT image data set.

[0013] Furthermore, the simulated CT image data set in step 1 includes a simulated ideal CT image and a corresponding simulated coupled artifact CT image. The simulated ideal CT image is obtained by performing a single-energy simulated projection and reconstruction on the part design model by the industrial CT simulation software, which is a truly ideal state without artifacts. The method for obtaining the simulated coupled artifact CT image is as follows: in the corresponding single-energy simulated projection, the actual scattering is simulated according to the scattering model, the actual multi-energy projection is simulated according to the beam hardening model, the response inconsistent pixels are added to simulate the actual ring artifacts, the Poisson noise and Gaussian noise are added to simulate the actual mixed noise, and then the processed single-energy simulated projection is reconstructed to obtain the simulated coupled artifact CT image. Since the imaging information of the actual parts is fully utilized, the obtained simulated coupled artifact CT image can effectively approximate the actual coupled artifact CT image.

[0014] Furthermore, the feature extraction module of the multi-scale generative adversarial network in step 2 has the following characteristics: the backbone network of the feature extraction module is composed of four scale dense blocks, and the scale dense blocks are connected by transition layers to reduce the dimension of the extracted feature map, and finally the feature integration and optimization are completed through the fusion layer; specifically, the first scale dense block contains 6 convolutional layers, the second scale dense block contains 12 convolutional layers, the third scale dense block contains 24 convolutional layers, and the fourth scale dense block contains 24 convolutional layers; the transition layer includes a normalization layer, a 1×1 convolutional layer and a 2×2 average pooling layer; the fusion layer includes an upsampling layer and a fully connected layer with softmax activation.

[0015] Furthermore, the feature fusion and regression module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature fusion and regression module adopts a feature pyramid structure, which connects the fuzzy feature maps of adjacent scales through a lateral connection structure, reduces the number of convolution kernels and the number of channels of the feature map while keeping the size of the feature map unchanged, and helps the model capture local and global artifact features; the module combines features through addition between pixels, and repeats the iterative process to generate a feature map of the intermediate layer, thereby achieving effective feature fusion.

[0016] Furthermore, the feature discrimination module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature discrimination module consists of an initialization layer and a cyclic increase layer; the feature map generates a feature map of one scale through the initialization layer, and then generates two feature maps of different scales through two cyclic increase layers; specifically, the initialization layer includes a 3×3 convolution layer and a ReLU activation layer, and the cyclic increase layer includes a 3×3 convolution layer, a normalization layer and a ReLU activation layer.

[0017] The beneficial effect of the present invention is that the multi-scale generative adversarial network correction method for industrial CT image coupling artifacts provided by the present invention is suitable for industrial CT images containing several or all of blurring artifacts, ring artifacts, scattering artifacts, beam hardening artifacts, noise, and coupling artifacts. While having a strong coupling artifact correction capability, it can better restore the image detail information, thereby ensuring the accuracy of the part geometric dimensions and contour edges in the CT image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of the present invention.

[0019] Figure 2 This is a schematic diagram of the multi-scale generative adversarial network structure of the present invention.

[0020] Figure 3 The overall and detail comparison before and after coupling artifact correction. DETAILED DESCRIPTION

[0021] The present invention will now be further described with reference to the accompanying drawings and embodiments.

[0022] like Figure 1 As shown, it is a flow chart of the present invention, a multi-scale generative adversarial network correction method for industrial CT image coupling artifacts, comprising the following steps:

[0023] Step 1: According to the types of parts that need to be corrected for industrial CT image coupling artifacts, select part design models and actual parts of the same or similar types, and construct a fused CT image dataset that fuses the simulated CT image dataset and the actual CT image dataset.

[0024] The actual CT image data set in step 1 is obtained by industrial CT scanning and reconstruction of the actual parts, including the actual coupling artifact CT image without any correction, and the corresponding actual ideal CT image after necessary correction using the existing method. Since the existing method cannot completely correct the coupling artifact, the actual ideal CT image is approximately ideal. In the industrial CT scanning of the actual parts, the scattering model (using grating scanning, array plate scanning, etc.) and the beam hardening model (using beam hardening curve fitting, etc.) are obtained and used in the processing of the simulated CT image data set.

[0025] The simulated CT image data set in step 1 includes a simulated ideal CT image and a corresponding simulated coupled artifact CT image. The simulated ideal CT image is obtained by performing a single-energy simulated projection and reconstruction on the part design model by the industrial CT simulation software, which is a truly ideal state without artifacts. The method for obtaining the simulated coupled artifact CT image is as follows: in the corresponding single-energy simulated projection, the actual scattering is simulated according to the scattering model, the actual multi-energy projection is simulated according to the beam hardening model, the response inconsistent pixels are added to simulate the actual ring artifact, the Poisson noise and Gaussian noise are added to simulate the actual mixed noise, and then the processed single-energy simulated projection is reconstructed to obtain the simulated coupled artifact CT image. Since the imaging information of the actual part is fully utilized, the obtained simulated coupled artifact CT image can effectively approximate the actual coupled artifact CT image.

[0026] The fused CT image dataset should contain various types and degrees of artifacts to reflect different situations in the real world. The fused CT image dataset of this embodiment contains 1000 pairs of CT images, of which the ratio of simulated CT images to actual CT images is 7:3, which not only takes advantage of the advantages of both but also makes up for the shortcomings of both. The input CT image size is 256×256. 80% of them are used as training sets and 20% are used as validation sets.

[0027] Step 2: Construct a multi-scale generative adversarial network suitable for industrial CT image coupling artifact correction, which mainly includes feature extraction module, feature fusion and regression module and feature discrimination module, such as Figure 2 shown.

[0028] The feature extraction module of the multi-scale generative adversarial network in step 2 has the following characteristics: the backbone network of the feature extraction module consists of four scale dense blocks, which are connected by transition layers to reduce the dimension of the extracted feature map, and finally the integration and optimization of features are completed through the fusion layer. Specifically, the first scale dense block contains 6 convolutional layers, the second scale dense block contains 12 convolutional layers, the third scale dense block contains 24 convolutional layers, and the fourth scale dense block contains 24 convolutional layers. The transition layer includes a normalization layer, a 1×1 convolutional layer, and a 2×2 average pooling layer. The fusion layer includes an upsampling layer and a fully connected layer with softmax activation.

[0029] The feature fusion and regression module of the multi-scale generative adversarial network in step 2 has the following features: The feature fusion and regression module adopts a feature pyramid structure, which connects the blurred feature maps of adjacent scales through a lateral connection structure. Reducing the number of convolution kernels and the number of channels of the feature map while keeping the size of the feature map unchanged helps the model capture local and global artifact features. Subsequently, the module combines features by adding pixels and repeats the iterative process to generate feature maps of the intermediate layer, thereby achieving effective feature fusion.

[0030] The feature discrimination module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature discrimination module consists of an initialization layer and a cyclic increase layer. First, the feature map is generated through the initialization layer to generate a feature map of one scale, and then it is generated through two cyclic increase layers to generate two feature maps of different scales. Specifically, the initialization layer includes a 3×3 convolution layer and a ReLU activation layer, and the cyclic increase layer includes a 3×3 convolution layer, a normalization layer, and a ReLU activation layer.

[0031] Step 3: Use the fused CT image dataset to train the constructed multi-scale generative adversarial network for coupled artifact correction. The loss function used in network training consists of three parts: pixel space loss, perceptual loss, and discriminator loss. The pixel space loss includes two parts: structural similarity pixel space loss and mean square error pixel space loss. The discriminator loss is a fusion of global and local discriminator losses. Loss function L G The specific calculation is as follows:

[0032] L G =0.5*L p1 +0.5*L p2 +0.006*L X +0.01*Ladν

[0033] Where, L p1 is the pixel space loss of structural similarity, L p2 is the mean square error pixel space loss, L X is the perceptual loss, L adv is the discriminator loss; Adam is used as the optimizer in the training process, the initial learning rate is 0.0001, and the number of training rounds is 3000.

[0034] Step 4: Using peak signal-to-noise ratio and structural similarity as evaluation indicators, the fused CT image dataset is used to adjust the network hyperparameters. The optimal hyperparameter settings are found through multiple iterations to achieve network model optimization.

[0035] Step 5: Use the trained multi-scale generative adversarial network to correct the industrial CT images of parts that need coupling artifact correction. Figure 3 The overall and detailed comparison of the two parts before and after the coupling artifact correction is shown. The multi-scale generative adversarial network proposed in the present invention can effectively eliminate the coupling artifacts of industrial CT images while ensuring the accuracy of geometric dimensions and contour edges, better restore the detailed information of the image without introducing new artifacts, and achieve a balance between preserving structural details and removing artifacts.

Claims

1. A multi-scale generative adversarial network correction method for industrial CT image coupling artifacts, characterized in that The following steps are involved: Step 1: According to the type of parts that need to be corrected for industrial CT image coupling artifacts, select part design models and actual parts of the same or similar types, and construct a fused CT image dataset that fuses the simulated CT image dataset and the actual CT image dataset; Step 2: Construct a multi-scale generative adversarial network suitable for industrial CT image coupled artifact correction, which mainly includes a feature extraction module, a feature fusion and regression module, and a feature discrimination module; Step 3: Use the fused CT image dataset to train the constructed multi-scale generative adversarial network for coupled artifact correction. The loss function used in network training consists of three parts: pixel space loss, perceptual loss, and discriminator loss. The pixel space loss includes two parts: structural similarity pixel space loss and mean square error pixel space loss. The discriminator loss is a fusion of global and local discriminator losses. Step 4: Using peak signal-to-noise ratio and structural similarity as evaluation indicators, the fused CT image dataset is used to adjust the network hyperparameters, and the optimal hyperparameter settings are found through multiple iterations to achieve network model optimization; Step 5: Use the trained multi-scale generative adversarial network to correct the industrial CT images of parts that need coupling artifact correction.

2. According to claim 1, a multi-scale generative adversarial network correction method for industrial CT image coupling artifacts is characterized in that: The actual CT image data set in step 1 is obtained by reconstructing industrial CT scanning of actual parts, including an actual coupled artifact CT image without any correction, and a corresponding actual ideal CT image after necessary correction using existing methods; a scattering model and a beam hardening model are obtained in the industrial CT scanning of actual parts for the processing of the simulation CT image data set.

3. The multi-scale generative adversarial network correction method for industrial CT image coupling artifacts according to claim 1 is characterized in that: The simulated CT image data set in step 1 includes a simulated ideal CT image and a corresponding simulated coupled artifact CT image; the simulated ideal CT image is obtained by performing single-energy simulation projection and reconstruction on the part design model using industrial CT simulation software; the method for obtaining the simulated coupled artifact CT image is as follows: in the corresponding single-energy simulation projection, the actual scattering is simulated according to the scattering model, the actual multi-energy projection is simulated according to the beam hardening model, inconsistent response pixels are added to simulate actual ring artifacts, Poisson noise and Gaussian noise are added to simulate actual mixed noise, and then the processed single-energy simulation projection is reconstructed to obtain the simulated coupled artifact CT image.

4. According to the multi-scale generative adversarial network correction method for industrial CT image coupling artifacts according to claim 1, the feature extraction module of the multi-scale generative adversarial network in step 2 has the following characteristics: The backbone network of the extraction module consists of four scale dense blocks, which are connected by transition layers to reduce the dimension of the extracted feature maps, and finally the features are integrated and optimized through the fusion layer; specifically, the first scale dense block contains 6 convolutional layers, the second scale dense block contains 12 convolutional layers, the third scale dense block contains 24 convolutional layers, and the fourth scale dense block contains 24 convolutional layers; the transition layer includes a normalization layer, a 1×1 convolutional layer, and a 2×2 average pooling layer; the fusion layer includes an upsampling layer and a fully connected layer with softmax activation.

5. According to the multi-scale generative adversarial network correction method for industrial CT image coupling artifacts according to claim 1, the feature fusion and regression module of the multi-scale generative adversarial network in step 2 has the following characteristics: Fusion The regression module adopts a feature pyramid structure, which connects the fuzzy feature maps of adjacent scales through a lateral connection structure, reduces the number of convolution kernels and the number of channels of the feature map while keeping the size of the feature map unchanged, and helps the model capture local and global artifact features; the module combines features by adding pixels and repeats the iterative process to generate the feature map of the intermediate layer, thereby achieving effective feature fusion.

6. According to the multi-scale generative adversarial network correction method for industrial CT image coupling artifacts according to claim 1, the feature discrimination module of the multi-scale generative adversarial network in step 2 has the following characteristics: feature discrimination The module consists of an initialization layer and a cyclic increase layer; the feature map is generated into a feature map of one scale through the initialization layer, and then it is generated into two feature maps of different scales through two cyclic increase layers; Specifically, the initialization layer includes a 3×3 convolution layer and a ReLU activation layer, and the recurrent increase layer contains a 3×3 convolution layer, a normalization layer, and a ReLU activation layer.

Images