Multi-module fusion self-supervision denoising method for industrial CT image

Abstract

The invention provides a multi-module fusion self-supervision denoising method for an industrial CT image, and belongs to the field of digital image processing and industrial nondestructive testing. In order to solve the problems of detail loss, poor denoising effect and the like in an industrial CT image denoising process, the invention provides a self-supervised denoising model which takes U-net as a trunk network and fuses an edge enhancement module, a convolution block attention module and a convolution attention fusion module. Meanwhile, by means of a mixed loss function composed of structural similarity index loss, mean square error loss and ResNet perception loss, the noise level of the industrial CT image can be remarkably reduced, meanwhile, the integrity of the object structure and the contour edge is accurately kept, and the image quality is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of digital image processing and industrial non-destructive testing, and relates to a multi-module fusion self-supervised denoising method for industrial CT images. BACKGROUND

[0002] Industrial computed tomography (CT) technology is an advanced non-destructive testing method integrating radiology, computer science, optics, electronics and image processing, and has the advantages of not being affected by the shape, structure and composition of the measured object, fast imaging speed and high resolution. However, in the actual industrial X-ray detection process, due to the limitations of the equipment and the combined effects of quantum noise, dark field noise and other factors, the obtained CT images often have image degradation problems such as noise and blur, which seriously interfere with subsequent CT image segmentation, point cloud acquisition and defect detection, and therefore it is necessary to suppress the noise of CT images in practical applications.

[0003] In order to improve the signal-to-noise ratio of industrial CT images, one method is to increase the scanning power of the X-ray tube or prolong the scanning time, but this method has high cost in terms of space for improvement, equipment loss and upgrade cost under the current physical and technological limitations of the X-ray source. Another method is to directly suppress the noise of low-quality CT images through software algorithms, which provides a new way of thinking for image denoising problems, does not rely on high-cost hardware equipment, and can obtain high-quality CT images while reducing the cost of equipment use.

[0004] Currently, the CT image noise suppression algorithms and techniques are mainly carried out in the reconstruction domain, which can be roughly divided into traditional methods and deep learning based methods. The main representatives of traditional methods are non-local means (NLM) and block-matching and 3D filtering (BM3D). The NLM algorithm makes full use of the redundancy information of the image, and estimates the current position information by using the self-similar structure block weighted. The algorithm complexity is closely related to the image size. The BM3D denoising algorithm first generates a three-dimensional matrix through similar block matching, and then realizes denoising in the three-dimensional transform domain. This method has poor flexibility, and needs to manually adjust parameters for different images, and the algorithm efficiency is low. Compared with traditional methods, the deep learning based method learns the complex mapping relationship between images through the powerful feature extraction ability of the network, which can be roughly divided into supervised learning, semi-supervised learning and self-supervised learning. Supervised learning uses noisy images and corresponding ideal label images for network training, semi-supervised learning uses part of the label images in the data set, and self-supervised learning can complete network training without label images. Since the label images are generally difficult to obtain in actual situations, the self-supervised learning method is more in line with the direction of technology development. However, most of the current algorithms are mainly used for denoising natural images and medical images, and the research on industrial part CT images is relatively scarce. Industrial part CT images often have unique imaging characteristics, which are specifically manifested as: complex metal texture distribution, subtle structural defects (such as cracks, pores, etc.) and metal material caused artifacts interference. The existing algorithms are difficult to adapt to these characteristics, resulting in the situation that the image is too smooth in the actual network test, and the details of the part are erased, and the problem of being unable to effectively balance the denoising processing and edge preservation performance. SUMMARY

[0005] To solve the above problems, the application provides a multi-module fusion self-supervised denoising method for industrial CT images. The method takes U-net as the backbone network, and fuses an edge enhancement module, a convolutional block attention module (CBAM), and a convolution and attention fusion module (CAFM). The edge enhancement module focuses on extracting image edge detail features; the CBAM deeply mines image information from the channel and spatial dimensions; and the CAFM is dedicated to efficiently fusing global and local features. At the same time, the model introduces a hybrid loss function composed of a structural similarity index (SSIM) loss, a mean squared error (MSE) loss, and a ResNet perception loss: the SSIM loss covers information in three dimensions of brightness, contrast, and structure; the MSE loss is accurate for pixel-level reconstruction and can directly reflect pixel value differences; and the ResNet perception loss extracts deep features with the help of a pre-trained CNN and focuses more on image texture and structure information. The U-net network, with its unique symmetric encoding and decoding structure, cooperates with each module to achieve complementary advantages, and is trained by the hybrid loss function, so as to finally generate high-quality denoised images with clear edges and complete structures.

[0006] The technical solution adopted by the application to solve its technical problems includes the following steps:

[0007] Step 1: For parts of the same type or similar to the parts to be denoised, the same industrial CT is scanned twice with the same parameters, and the starting positions of the parts are the same in the two scans, to obtain a pair of industrial CT images that differ only in random noise, and to establish an industrial CT image dataset for training the self-supervised denoising network;

[0008] Step 2: Taking the U-net structure as the overall framework, embedding the edge enhancement module, the convolutional block attention module, and the convolutional attention fusion module, a multi-module fusion self-supervised denoising model for multi-scale / multi-dimensional structural detail information learning is constructed;

[0009] Step 3: A hybrid loss function LOSS = a x SSIM + b x MSE + c x ResNet is constructed, which is composed of a structural similarity index (SSIM) loss, a mean squared error (MSE) loss, and a ResNet perception loss, where a, b, and c are corresponding weight coefficients;

[0010] Step 4: The multi-module fusion self-supervised denoising network constructed is trained using the industrial CT image dataset established in step 1 and using a data augmentation method.

[0011] Step 5: The trained multi-module fusion self-supervised denoising network is used to denoise the industrial CT image of the part to be denoised, and a high signal-to-noise ratio CT image is obtained.

[0012] In the above step 2, the multi-module fusion self-supervised denoising model for learning multi-scale / multi-dimensional structural detail information includes 1 edge enhancement module, 18 groups of Conv3x3 convolution block and ReLU activation function block, 2 convolution block attention modules, 4 2x2 maximum pooling units, 4 jump connections, 1 convolution attention fusion module, 4 2x2 up-sampling units, and 1 group of Conv1x1 convolution block and Sigmoid activation function block.

[0013] The above edge enhancement module includes four directions of trainable Sobel operators for extracting edge features of industrial CT images, i.e., vertical, horizontal and two diagonal lines. Through network training, the parameters are adaptively optimized to extract edge information of different intensities.

[0014] The above convolution block attention module is composed of a channel attention module and a spatial attention module. The channel attention module includes two parallel branches. One branch performs global maximum pooling operation on the input feature map, and then performs feature transformation through shared multi-layer perceptron (MLP). The other branch performs average pooling on the input feature map, and also processes it through shared multi-layer perceptron. Then, the weight outputs of the two branches are added element by element, and normalized by Sigmoid activation function to obtain the attention weight value in channel dimension. The spatial attention module takes the output of the channel attention module as input, and uses a double-channel processing mechanism inside. One branch performs global maximum pooling operation on the input feature map based on channel dimension; the other branch performs average pooling on the input feature map based on channel dimension; the feature maps output by the two branches are concatenated in channel dimension, and then processed by a 7x7 convolution layer for feature fusion, and finally the Sigmoid activation function is used to generate the attention weight map in spatial dimension.

[0015] The above convolution attention fusion module is composed of a local branch and a global branch. The processing flow of the local branch is: after the input feature is convolved by 1x1, the order in channel dimension is shuffled, and finally the local output feature is generated by 3x3x3 convolution. The processing flow of the global branch is: three feature tensors of query Q, key K and value V are generated by 1x1 convolution and 3x3 deep convolution, the dot product of Q and K is normalized by Softmax to generate an attention weight map, the weighted sum of the weight map and V is connected with the residual of the original input feature through 1x1 convolution to generate the global output feature. The local branch and the global branch capture feature information in different dimensions, and the outputs of the two branches are added element by element to form the output feature map of the convolution attention fusion module.

[0016] The beneficial effects of the present application are: (1) using a self-supervised denoising training strategy, only a noisy image pair is needed to complete network training, solving the problem of difficulty in obtaining ideal labels of industrial CT images; (2) through the design of multi-module fusion and the application of a hybrid loss function, the trained network can greatly reduce the noise level of industrial CT images and accurately maintain the integrity of object structure and contour edges, showing outstanding performance in reliability, stability and applicability. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 is a flowchart of the present application.

[0018] Figure 2 is a multi-module fusion self-supervised denoising network structure diagram.

[0019] Figure 3 is an edge enhancement module diagram.

[0020] Figure 4 is a convolution block attention module diagram.

[0021] Figure 5 is a convolution attention fusion module diagram.

[0022] Figure 6 is an overall and local comparison diagram of industrial CT image denoising. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present application will be further described in detail below in conjunction with the drawings and examples.

[0024] As shown in Figure 1 is a flowchart of the present application, a multi-module fusion self-supervised denoising method for industrial CT images, comprising the following steps:

[0025] Step 1: For parts of the same type or similar to the part to be denoised, the same industrial CT is scanned twice with the same parameters, and the starting position of the part rotation is the same, to obtain an industrial CT image pair that is only different in random noise, and to establish an industrial CT image dataset for self-supervised denoising network training. The industrial CT scanning parameters used are: a ray source voltage of 350kV and a current of 1.10mA, a ray source to detector distance of 1220.00mm, a ray source to rotating platform center distance of 800.00mm, and 720 projection data obtained by one circumferential scan. Finally, 1734 pairs of 512x512 resolution industrial CT image datasets of 9 parts are obtained.

[0026] Step 2: Embedding edge enhancement module, convolution block attention module and convolution attention fusion module into the overall framework of U-net structure, constructing a multi-module fusion self-supervised denoising model for multi-scale / multi-dimensional structural detail information learning, as shown in FIG. 2. The model includes 1 edge enhancement module, 18 groups of Conv3x3 convolution block and ReLU activation function block, 2 convolution block attention modules, 4 2x2 max pooling units, 4 skip connections, 1 convolution attention fusion module, 4 2x2 up-sampling units, and 1 group of Conv1x1 convolution block and Sigmoid activation function block. Figure 2

[0027] As shown in FIG. 3, it is a schematic diagram of the edge enhancement module. It uses the trainable Sobel operator in 4 directions of vertical, horizontal and two diagonal lines to extract the edge features of the industrial CT image. The feature map learned by the module is superimposed with the input CT image in the channel dimension to obtain the final feature output. The mathematical expression of the processing process is: Figure 3

[0028]

[0029]

[0030] wherein is the input feature map, is the output feature map, H, W and C are height, width and channel number respectively; K i represents the i-th Sobel kernel; and a is a trainable parameter. Through network training, the parameter is adaptively optimized to extract edge information of different intensities.

[0031] As shown in FIG. 4, it is a schematic diagram of the convolution block attention module CBAM. The module is composed of a channel attention module and a spatial attention module. The channel attention module includes two parallel branches. One of them performs global max pooling operation on the input feature map, and then transforms the features through a shared multi-layer perception (MLP). The other branch performs average pooling on the input feature map, which is also processed through a shared multi-layer perception. Then the weights of the outputs of the two branches are added element by element, and normalized by the Sigmoid activation function, so as to obtain the attention weight value in the channel dimension. The spatial attention module takes the output of the channel attention module as input, and uses a double-channel processing mechanism inside. One branch performs global max pooling operation on the input feature map based on the channel dimension; the other branch performs average pooling in the channel dimension; after the feature maps output by the two branches are spliced in the channel dimension, a 7x7 convolution layer is used for feature fusion processing, and finally the Sigmoid activation function is used to generate the attention weight map in the spatial dimension. The mathematical expression of the processing process is: Figure 4

[0032] ​​​

[0033] M S (X′)=σ{f 7×7 ([AvgPool(X′);MaxPool(X′)])}

[0034] wherein is an input feature map, is an intermediate feature map, is an output feature map, is a channel attention output, is a spatial attention output, H, W, C are height, width and channel number respectively; σ is a Sigmoid activation function.

[0035] As shown in FIG. 1, a convolutional attention fusion module (CAFM) is composed of a local branch and a global branch. The processing flow of the local branch is as follows: the input feature is shuffled in the channel dimension after 1x1 convolution, and finally the local output feature is generated through 3x3x3 convolution; the processing flow of the global branch is as follows: three feature tensors of query Q, key K and value V are generated by using 1x1 convolution and 3x3 depth convolution, the dot product of Q and K is normalized by Softmax to generate an attention weight map, the weighted sum of the weight map and V is connected with the original input feature through 1x1 convolution to generate the global output feature. The local branch and the global branch capture different dimensional feature information, and the outputs of the two branches are added element by element to finally form the output feature map of the convolutional attention fusion module. The mathematical expression of the processing process is as follows: Figure 5

[0036]

[0037] Y′=f 3×3×3 (C s (f 1×1 (X)))

[0038]

[0039]

[0040] wherein is an input feature map, is a local branch output, is a global branch output, is an overall output, H, W, C are height, width and channel number respectively; β is a learnable scaling parameter.

[0041] ​Step 3: A hybrid loss function LOSS = a x SSIM + b x MSE + c x ResNet is constructed by structural similarity index SSIM loss, mean square error MSE loss and ResNet perceptual loss, wherein a, b and c are corresponding weight coefficients. The SSIM loss considers three-dimensional information of brightness, contrast and structure; the MSE loss is a pixel-level accurate reconstruction, which directly reflects the pixel value difference; the ResNet perceptual loss extracts deep features through a pre-trained CNN, and pays more attention to the texture and structure information of the image. The combination of the three makes the network generate an image with clear edges, texture and complete structure at the pixel level. In this embodiment, a = 0.39, b = 0.6 and c = 0.01 are selected.

[0042] Step 4: The multi-module fusion self-supervised denoising network constructed is trained by using the industrial CT image data set established in step 1 and using a data augmentation method. The 1734 pairs of 512 x 512 resolution industrial CT image data sets of 9 parts obtained in step 1 are split into 1524 pairs of images of 7 parts and 210 pairs of images of 2 parts. Among them, 76200 pairs of 256 x 256 resolution images are obtained by randomly cropping the 1524 pairs of images, which are used for network training; and the other 210 pairs of images are used as an actual test set. Before model training, random angle rotation, vertical and horizontal displacement, vertical and horizontal flip and other data augmentation operations are performed on all training images. The Adam optimizer is used in the training process, the learning rate is set to 0.0001, and a total of 100 rounds of training are performed.

[0043] Step 5: As shown in Figure 6 , the trained multi-module fusion self-supervised denoising network is used to denoise the industrial CT image of the part to be denoised to obtain a high signal-to-noise ratio CT image, thereby verifying the effectiveness of the present application.

Claims

1. A multi-module fusion self-supervised denoising method for industrial CT images, characterized in that... Includes the following steps: Step 1: For parts that are the same type or similar to the parts to be denoised, scan them twice using the same industrial CT with the same parameters, and the starting position of the rotation of the parts is the same in both scans. Obtain industrial CT image pairs that differ only in random noise, and establish the industrial CT image dataset used for training the self-supervised denoising network. Step 2: Using the U-net structure as the overall framework, embed edge enhancement module, convolutional block attention module and convolutional attention fusion module to construct a multi-module fusion self-supervised denoising model for learning multi-scale / multi-dimensional structural detail information; Step 3: Construct a hybrid loss function LOSS = a×SSIM + b×MSE + c×ResNet, consisting of Structural Similarity Index (SSIM) loss, Mean Squared Error (MSE) loss, and ResNet perceptual loss, where a, b, and c are the corresponding weight coefficients. Step 4: Using the industrial CT image dataset established in Step 1, train the constructed multi-module fusion self-supervised denoising network using the data augmentation method; Step 5: Use the trained multi-module fusion self-supervised denoising network to denoise the industrial CT image of the part to be denoised, and obtain its high signal-to-noise ratio CT image.

2. The multi-module fusion self-supervised denoising method for industrial CT images according to claim 1, characterized in that, The multi-module fusion self-supervised denoising model for learning multi-scale / multi-dimensional structural detail information described in step 2 has the following characteristics: the model consists of 1 edge enhancement module, 18 sets of Conv3×3 convolutional blocks and ReLU activation function blocks, 2 convolutional block attention modules, 4 2×2 max pooling units, 4 skip connections, 1 convolutional attention fusion module, 4 2×2 upsampling units, and 1 set of Conv1×1 convolutional blocks and Sigmoid activation function blocks.

3. The edge enhancement module according to claim 2, characterized in that, This module contains trainable Sobel operators in four directions: vertical, horizontal, and two diagonals, enabling adaptive optimization of parameters through network training.

4. The convolutional block attention module according to claim 2, characterized in that, This module includes a channel attention module and a spatial attention module. The channel attention module performs max pooling and average pooling on the input and performs shared multilayer perceptron feature transformation. The output weights of the two branches are summed element by element and then normalized by the Sigmoid function. The spatial attention module concatenates the max pooling and average pooling results based on the channel dimension, performs a 7×7 convolution operation, and then normalizes them by the Sigmoid function.

5. The convolutional attention fusion module according to claim 2, characterized in that, The module consists of local branches and global branches. The input features of the local branch are shuffled in the channel dimension after being convolved by 1×1, and finally the local output features are generated by 3×3×3 convolution. The global branch uses 1×1 convolution and 3×3 depth convolution to generate query Q, key K and value V. The dot product of Q and K is normalized by Softmax to generate an attention weight map. This weight map is weighted with V and then connected to the residual of the original input feature through 1×1 convolution to generate the global output feature.

Images