A multi-scale generative adversarial network correction method for industrial CT image coupling artifacts

Abstract

The application provides an industrial CT image coupling artifact multi-scale generative adversarial network correction method, and belongs to the field of digital image processing. The method aims at the problem that image degradation caused by the coupling of various artifacts in industrial CT imaging is difficult to correct effectively, and provides a multi-scale generative adversarial network correction method. The feature pyramid structure is used to help the network capture more comprehensive artifact feature information, and a multi-scale discriminator is used to build a generative adversarial network framework, so that the generated artifact-removed image is clearer and more real. The method has strong coupling artifact correction capability, and can better restore the detail information of the image, thereby ensuring the accuracy of the geometric size and contour edge of the part in the CT image.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of digital image processing, and relates to a multi-scale generative adversarial network correction method for coupled artifacts of industrial CT images. BACKGROUND

[0002] Nowadays, high-end manufacturing represented by 3D printing, precision casting and preparation of new composite materials has very urgent detection needs for industrial CT (Computed Tomography). However, due to the comprehensive influence of mathematics, equipment and physics (for example, reconstruction algorithm, CT system characteristics, characteristics of the object to be detected, characteristics of the action of the ray material, etc.) and other factors in the imaging process of industrial CT, especially the many physical assumptions used to derive the reconstruction calculation can only be approximately satisfied in the actual imaging process, which leads to serious degradation of image quality, greatly reduces the image contrast and signal-to-noise ratio, makes the outline fuzzy, the noise prominent, the resolution insufficient, and part of the region information lost, greatly increases the possibility of false positives and false negatives in detection, and greatly limits the application of industrial CT in some fields.

[0003] The image that does not conform to the actual situation in the industrial CT image is usually called artifact. According to the actual imaging process of industrial CT, the artifact mainly includes the blur artifact caused by the point spread degradation of the imaging system, the ring artifact caused by the inconsistent response of the detector pixels, the scattering artifact caused by the ray scattering, the beam hardening artifact caused by the beam hardening effect of the multi-color ray, etc. Noise inevitably exists in industrial CT imaging, and is usually a mixture of multiple mode noises, and is also treated as an artifact in many cases. In the industrial CT image of the object to be detected, the artifact that causes serious degradation of image quality is usually the coupling effect of various artifacts, and it is very difficult to completely separate various artifacts and correct them one by one in practice.

[0004] The current correction techniques for coupling artifacts in CT images can be roughly divided into three types: physical model-based methods, image processing-based methods, and deep learning-based methods. The physical model-based methods include hardware blocking suppression and hardware pre-hardening, which focus on reducing the scattering and hardening effects caused by metal objects. However, these methods can lead to a decrease in signal-to-noise ratio and have system limitations that need to be addressed. The image processing-based methods include preprocessing and post-processing. Linearization is the most commonly used preprocessing correction method for converting multi-color projection data into single-color projection data. Post-processing methods are based on image segmentation and re-projection. Although they have improved the imaging quality in terms of applicability, effectiveness, performance, and accuracy, none of them can effectively correct coupling artifacts caused by multiple factors. Deep learning decoupling artifact algorithms can achieve image restoration by estimating the blur kernel and performing non-blind deconvolution. They can also use an end-to-end approach for blind deblurring to restore the coupling artifact image. However, current deep learning-based coupling artifact correction methods for industrial CT images still have issues such as poor applicability of the correction object, poor image detail restoration, and long processing time. It is necessary to further develop related correction methods to improve the application effect and scope of industrial CT. SUMMARY

[0005] To address the problem of effectively correcting the degradation of CT images caused by blurring artifacts, ring artifacts, scattering artifacts, beam hardening artifacts, noise, and their coupling effects in industrial CT imaging, the present application provides a multi-scale generative adversarial network correction method for industrial CT image coupling artifacts. The feature pyramid structure helps the network capture more comprehensive artifact feature information, and the multi-scale discriminator is used to build a generative adversarial network framework, making the generated de-artifact image clearer and more realistic.

[0006] The technical solution adopted by the present application to solve its technical problems includes the following steps:

[0007] Step 1: According to the type of parts that need to be corrected for industrial CT image coupling artifacts, select the same or similar type of part design model and actual part, and construct a fusion CT image dataset that fuses simulation CT image dataset and actual CT image dataset;

[0008] Step 2: Construct a multi-scale generative adversarial network suitable for industrial CT image coupling artifact correction, mainly including a feature extraction module, a feature fusion and regression module, and a feature discrimination module;

[0009] Step 3: Coupling artifact correction training is performed on the constructed multi-scale generative adversarial network using the fusion CT image dataset. The loss function used in network training is composed of three parts: pixel space loss, perception loss, and discriminator loss. The pixel space loss includes structure similarity pixel space loss and mean square error pixel space loss. The discriminator loss is the fusion of global and local discriminator losses.

[0010] Step 4: Using the fusion CT image dataset, network hyperparameter adjustment is performed using peak signal-to-noise ratio and structural similarity as evaluation indicators. Through multiple iterations, the optimal hyperparameter setting is found to optimize the network model.

[0011] Step 5: Using the trained multi-scale generative adversarial network, the industrial CT image of the part that needs to be corrected for coupling artifacts is corrected.

[0012] Further, the actual CT image dataset in step 1 is obtained by industrial CT scanning and reconstruction of the actual part, including actual coupling artifact CT images without any correction, and actual ideal CT images corrected using existing methods. Since existing methods cannot completely correct coupling artifacts, the actual ideal CT image is only an approximate ideal state. In the actual industrial CT scanning of the part, the scatter model (using grating scanning, array hole plate scanning, etc.) and the beam hardening model (using beam hardening curve fitting, etc.) are obtained and used in the processing of the simulated CT image dataset.

[0013] Further, the simulated CT image dataset in step 1 includes a simulated ideal CT image and a corresponding simulated coupling artifact CT image. The simulated ideal CT image is obtained by single-energy simulation projection and reconstruction of the part design model using industrial CT simulation software, which is a truly artifact-free ideal state. The simulated coupling artifact CT image is obtained by: in the corresponding single-energy simulation projection, first simulate the actual scattering according to the scatter model, then simulate the actual multi-energy projection according to the beam hardening model, add response inconsistent pixels to simulate the actual ring artifact, add Poisson noise and Gaussian noise to simulate the actual mixed noise, and then reconstruct the processed single-energy simulation projection to obtain the simulated coupling artifact CT image. Since the imaging information of the actual part is fully utilized, the simulated coupling artifact CT image obtained can effectively approximate the actual coupling artifact CT image.

[0014] Further, the feature extraction module of the multi-scale generative adversarial network in step 2 has the following characteristics: the backbone network of the feature extraction module is composed of four scale dense blocks, the scale dense blocks are connected by transition layers to reduce the dimension of the extracted feature maps, and finally the integration and optimization of the features are completed through a fusion layer; specifically, the first scale dense block contains 6 convolution layers, the second scale dense block contains 12 convolution layers, the third scale dense block contains 24 convolution layers, and the fourth scale dense block contains 24 convolution layers; the transition layer includes a normalization layer, a 1*1 convolution layer and a 2*2 average pooling layer; the fusion layer includes an upsampling layer and a fully connected layer with a softmax activation.

[0015] Further, the feature fusion and regression module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature fusion and regression module adopts a feature pyramid structure, which connects the adjacent scale feature maps through a horizontal connection structure, reduces the number of convolution kernels and the number of channels of the feature maps while keeping the size of the feature maps unchanged, and helps the model to capture local and global artifact features; the module combines features through pixel addition and repeated iteration to generate intermediate layer feature maps, thereby realizing effective fusion of features.

[0016] Further, the feature discrimination module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature discrimination module is composed of an initialization layer and a cyclic increase layer; the feature map generates a feature map of one scale through the initialization layer, and then generates two feature maps of different scales through two cyclic increase layers; specifically, the initialization layer includes a 3*3 convolution layer and a ReLU activation layer, and the cyclic increase layer includes a 3*3 convolution layer, a normalization layer and a ReLU activation layer.

[0017] The beneficial effects of the present application are: the multi-scale generative adversarial network correction method for industrial CT image coupling artifacts provided by the present application is suitable for the case that the industrial CT image contains some or all of the fuzzy artifacts, ring artifacts, scattering artifacts, beam hardening artifacts, noise and coupling artifacts, and has strong coupling artifact correction capability, can better restore the detail information of the image, and further ensure the accuracy of the geometric size and contour edge of the parts in the CT image. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of the present application.

[0019] Figure 2 The multi-scale generative adversarial network structure diagram of the present application.

[0020] Figure 3 The overall and detail comparison chart before and after coupling artifact correction. DETAILED DESCRIPTION

[0021] The application will be further described in conjunction with the drawings and embodiments.

[0022] As Figure 1 shown, a flowchart of the present application, a multi-scale generative adversarial network correction method for industrial CT image coupling artifacts, includes the following steps:

[0023] Step 1: According to the need for industrial CT image coupling artifact correction of the part type, select the same or similar type of part design model and actual part, and construct a fusion CT image data set that fuses simulation CT image data set and actual CT image data set.

[0024] The actual CT image data set in step 1 is obtained by industrial CT scanning and reconstruction of the actual part, including actual coupling artifact CT images without any correction, and actual ideal CT images corrected by existing methods. Since the existing method cannot completely correct the coupling artifacts, the actual ideal CT image is an approximate ideal state. In the industrial CT scanning of the actual part, the scatter model (using grating scanning, array hole plate scanning, etc.) and the beam hardening model (using beam hardening curve fitting, etc.) are obtained and used in the processing of the simulation CT image data set.

[0025] The simulation CT image data set in step 1 includes simulation ideal CT images and corresponding simulation coupling artifact CT images. The simulation ideal CT image is obtained by single-energy simulation projection and reconstruction of the part design model by industrial CT simulation software, which is a truly artifact-free ideal state. The simulation coupling artifact CT image is obtained by: in the corresponding single-energy simulation projection, first simulate the actual scattering according to the scatter model, then simulate the actual multi-energy projection according to the beam hardening model, add response inconsistent pixels to simulate the actual ring artifact, add Poisson noise and Gaussian noise to simulate the actual mixed noise, and then reconstruct the processed single-energy simulation projection to obtain the simulation coupling artifact CT image. Since the imaging information of the actual part is fully utilized, the simulation coupling artifact CT image obtained can effectively approximate the actual coupling artifact CT image.

[0026] The fusion CT image data set should contain various types and degrees of artifacts to reflect different situations in the real world. The fusion CT image data set of this embodiment contains 1000 pairs of CT images, of which the proportion of simulation CT images to actual CT images is 7:3, both taking advantage of the two and making up for the shortcomings of the two, and the input CT image size is 256x256. Of which 80% as training set, 20% for validation set.

[0027] Step 2: Construct a multi-scale generative adversarial network suitable for coupling artifact correction of industrial CT images, mainly including a feature extraction module, a feature fusion and regression module, and a feature discrimination module, as shown in Figure 2 .

[0028] The feature extraction module of the multi-scale generative adversarial network in step 2 has the following characteristics: the backbone network of the feature extraction module is composed of four scale dense blocks, and the scale dense blocks are connected by transition layers to reduce the dimension of the extracted feature maps. Finally, the integration and optimization of the features are completed through the fusion layer. Specifically, the first scale dense block contains 6 convolution layers, the second scale dense block contains 12 convolution layers, the third scale dense block contains 24 convolution layers, and the fourth scale dense block contains 24 convolution layers. The transition layer includes a normalization layer, a 1x1 convolution layer, and a 2x2 average pooling layer. The fusion layer includes an upsampling layer and a fully connected layer with a softmax activation.

[0029] The feature fusion and regression module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature fusion and regression module adopts a feature pyramid structure. This module connects the blurred feature maps of adjacent scales through a horizontal connection structure. Reducing the number of convolution kernels and the number of channels of the feature maps while keeping the size of the feature maps unchanged helps the model capture local and global artifact features. Subsequently, the module combines features through pixel-wise addition and iterates the process repeatedly to generate feature maps of intermediate layers, thereby achieving effective fusion of features.

[0030] The feature discrimination module of the multi-scale generative adversarial network in step 2 has the following characteristics: the feature discrimination module is composed of an initialization layer and a cyclic increase layer. First, the feature map generates a feature map of one scale through the initialization layer, and then generates two feature maps of different scales through two cyclic increase layers. Specifically, the initialization layer includes a 3x3 convolution layer and a ReLU activation layer, and the cyclic increase layer contains a 3x3 convolution layer, a normalization layer, and a ReLU activation layer.

[0031] Step 3: Use the fused CT image dataset to train the multi-scale generative adversarial network for coupling artifact correction. The loss function used in network training is composed of three parts: pixel space loss, perceptual loss, and discriminator loss. The pixel space loss includes structure similarity pixel space loss and mean square error pixel space loss, and the discriminator loss is the fusion of global and local discriminator losses. The specific calculation of the loss function L G is as follows:

[0032] L G = 0.5*L p1 + 0.5*L p2 + 0.006*L X + 0.01*Ladν

[0033] wherein L p1 is the structural similarity pixel space loss, L p2 is the mean square error pixel space loss, L X is the perceptual loss, L adv is the discriminator loss; the training process uses Adam as the optimizer, the initial learning rate is 0.0001, and the training rounds are 3000 rounds.

[0034] Step 4: Taking the peak signal-to-noise ratio and the structural similarity as evaluation indexes, the network hyperparameter adjustment is performed by using the fused CT image dataset, and the optimal hyperparameter setting is found through multiple iterations to realize the network model optimization.

[0035] Step 5: The trained multi-scale generative adversarial network is used to correct and process the part industrial CT image needing to be coupled artifact corrected. Figure 3 As shown in the overall and detail comparison of the two parts before and after the coupled artifact correction, the multi-scale generative adversarial network proposed in the application can effectively eliminate the coupled artifacts of the industrial CT image, meanwhile, the accuracy of the geometric size and the contour edge is ensured, the image detail information is better restored, no new artifacts are introduced, and the balance of saving the structural details and removing the artifacts is achieved.

Claims

1.A method for correcting multi-scale generative adversarial network of industrial CT image coupling artifacts, characterized in that Comprising the following steps: Step 1: According to the need of industrial CT image coupling artifact correction of the part category, the same or similar category of part design model and actual part is selected, and the fusion CT image data set of the fusion simulation CT image data set and the actual CT image data set is constructed, and the specific construction method is as follows: (1) The actual CT image data set is obtained by industrial CT scanning and reconstruction of the actual part, including the actual coupling artifact CT image without any correction, and the actual ideal CT image corrected by the existing method; the scattering model and the beam hardening model are obtained in the actual part industrial CT scanning, which are used for processing of the simulation CT image data set; (2) The simulation CT image data set includes simulation ideal CT image and corresponding simulation coupling artifact CT image; The simulation ideal CT image is obtained by single-energy simulation projection and reconstruction of the part design model by industrial CT simulation software; the simulation coupling artifact CT image is obtained by: in the corresponding single-energy simulation projection, the actual scattering is simulated according to the scattering model, the actual multi-energy projection is simulated according to the beam hardening model, the actual ring artifact is simulated by adding pixels with inconsistent response, and the actual mixed noise is simulated by adding Poisson noise and Gaussian noise, and then the processed single-energy simulation projection is reconstructed to obtain the simulation coupling artifact CT image; Step 2: Construct a multi-scale generative adversarial network suitable for industrial CT image coupling artifact correction, mainly including a feature extraction module, a feature fusion and regression module and a feature discrimination module: (1) The feature extraction module has the following characteristics: the backbone network of the feature extraction module is composed of four scale dense blocks, which are connected by transition layers to reduce the dimension of the extracted feature map, and finally the integration and optimization of the features are completed through the fusion layer; Specifically, the first scale dense block contains 6 convolution layers, the second scale dense block contains 12 convolution layers, the third scale dense block contains 24 convolution layers, and the fourth scale dense block contains 24 convolution layers; The transition layer includes a normalization layer, a 1x1 convolution layer and a 2x2 average pooling layer; The fusion layer includes an up-sampling layer and a fully connected layer with softmax activation; (2) The feature fusion and regression module has the following characteristics: the feature fusion and regression module adopts a feature pyramid structure, which connects the adjacent scale blurred feature maps through a horizontal connection structure, reduces the number of convolution kernels and the number of channels of the feature map under the premise of keeping the size of the feature map unchanged, and helps the model to capture local and global artifact features; The module combines features by adding pixels, and iterates to generate intermediate layer feature maps, so as to realize effective fusion of features; (3) The feature discrimination module has the following characteristics: the feature discrimination module is composed of an initialization layer and a cyclic increase layer; The feature map generates a feature map of one scale through the initialization layer, and then generates two feature maps of different scales through two cyclic increase layers; Specifically, the initialization layer includes a 3x3 convolution layer and a ReLU activation layer, and the cyclic increase layer includes a 3x3 convolution layer, a normalization layer and a ReLU activation layer; Step 3: Coupling artifact correction training is performed on the constructed multi-scale generative adversarial network using the fused CT image dataset, and the loss function used in network training is composed of three parts, namely, pixel space loss, perception loss and discriminator loss, wherein the pixel space loss includes two parts, structural similarity pixel space loss and mean square error pixel space loss, and the discriminator loss is the fusion of global and local discriminator loss; Step 4: Using peak signal-to-noise ratio and structural similarity as evaluation indexes, network hyperparameter adjustment is performed using the fused CT image dataset, and the optimal hyperparameter setting is found through multiple iterations to realize network model optimization; Step 5: The trained multi-scale generative adversarial network is used to correct the industrial CT image of the part that needs to be corrected for coupling artifacts.

Images