Target detection-based weld infrared thermal imaging image defect detection method

Abstract

The invention provides a target detection-based weld infrared thermal imaging image defect detection method. The target detection-based weld infrared thermal imaging image defect detection method comprises the following steps of: selecting an optimal filtering algorithm (Gaussian / median / mean / wavelet filtering) through peak signal-to-noise ratio comparison to pre-process an image; an improved YOLO-AttCrossW model is constructed, a backbone network of the improved YOLO-AttCrossW model fuses high and low dimension features by adopting a cross auxiliary connection module, a neck network is embedded into a deformable channel attention module to realize sampling position adaptive offset, and a detection head applies a dynamic weighting loss function; during training, efficiency and precision are calculated through lightweight convolution and hyper-parameter fine tuning balance. Multi-scale features are fused by means of the spanning auxiliary connection module, the detection confidence of the tiny pores is greatly improved, and the detection confidence is obviously improved compared with original YOLOv5s; the deformable channel attention module can dynamically track the defect contour, and even if interference such as splashing and oxide skin exists, a high average precision mean value can still be kept. And by dynamically adjusting the confidence coefficient weight of the background complex region, the occurrence of false detection conditions is remarkably reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of weld defect detection, in particular to a weld infrared thermal imaging image defect detection method based on target detection. BACKGROUND

[0002] Welding is essentially a non-steady-state rapid melting and crystallization process. Its inherent nonlinear characteristics, strong coupling relationship between various links, and the complex factors involved in the entire process make it difficult to find a universal automatic detection solution for weld defects in actual production. Because of the lack of such universal automatic detection means, the current industrial production of weld defect detection still mainly relies on manual detection. However, this manual detection mode has many problems to be solved:

[0003] 1. The workload of manual detection is extremely large, and the efficiency is relatively low. With the acceleration of modern industrial production rhythm, the number of weld images processed by enterprises every day increases explosively, and the detection task becomes extremely heavy. It is difficult to complete all detection work within the given production cycle by relying on manual viewing of each image, which not only slows down the overall production progress, but also may affect the quality control of the final product due to delayed detection.

[0004] 2. The detection results are easily affected by subjective factors, and the consistency is difficult to guarantee. Different detection personnel, due to differences in experience accumulation, observation angle, and even personal judgment habits, may give completely different detection conclusions when facing weld images with only slight differences. More importantly, the detection of some specific types of weld images must rely on experienced experts in this field who have worked for many years, which will undoubtedly increase the personnel cost of enterprises.

[0005] 3. The working state of the detection personnel directly affects the accuracy of the detection. Long-term exposure to a large number of images can easily cause visual fatigue in the detection personnel, which not only leads to misjudgment or missed detection of defects, allowing unqualified products to flow into subsequent links, but also causes potential damage to the eye health of the detection personnel due to the inherent irritability of thermal imaging images.

[0006] These problems make the manual detection mode more and more difficult to adapt to the production needs of modern industry in terms of efficiency, accuracy, and low cost, and highlight the urgency of developing universal automatic detection technology. SUMMARY

[0007] In view of the above technical deficiencies, the purpose of the present application is to provide a weld infrared thermal imaging image defect detection method based on target detection, which realizes accurate identification of weld thermal imaging images.

[0008] To achieve the above purpose, the present application adopts the following technical solutions:

[0009] A target detection-based weld infrared thermography image defect detection method, comprising the following steps:

[0010] The weld infrared thermography image is preprocessed by using a filtering algorithm, and the optimal filtering mode is selected from median filtering, Gaussian filtering, mean filtering and wavelet filtering through peak signal-to-noise ratio comparison;

[0011] A YOLO-AttCrossW target detection model is constructed, which comprises:

[0012] The backbone network: the cross-aiding connection module is introduced to splice and fuse the input features and the features output by the spatial pyramid pooling fast layer, and then add the output features of the serially connected bottleneck operation;

[0013] The neck network: the deformable channel attention module is connected to dynamically adjust the feature sampling position through the deformable convolution kernel;

[0014] The detection head: the weighted loss function is used to dynamically adjust the weight coefficients of the positioning loss, the classification loss and the confidence loss;

[0015] The target detection model is trained by using the preprocessed image to realize defect recognition and positioning.

[0016] Preferably, the execution of the cross-aiding connection module comprises:

[0017] The spatial pyramid pooling fast operation and the serial bottleneck operation are performed on the input features in parallel;

[0018] The features output by the spatial pyramid pooling fast layer are spliced with the original input features along the channel dimension;

[0019] The spliced result is element-wise added to the output features of the serial bottleneck operation.

[0020] Preferably, the execution of the deformable channel attention module comprises:

[0021] The batch normalization and the nonlinear activation function are sequentially performed on the input features;

[0022] The dynamic sampling grid is generated through the learnable offset parameter, so that the convolution kernel sampling position is adaptively offset along the defect geometric contour.

[0023] Preferably, the execution of the weighted loss function comprises:

[0024] The positioning loss adopts the smooth L1 loss function, the classification loss adopts the cross-entropy loss function, and the confidence loss adopts the binary cross-entropy loss function;

[0025] The weight coefficient of the positioning loss is increased for small size defects, and the weight coefficient of the confidence loss is reduced for complex background regions.

[0026] Preferably, the selection of the filtering algorithm comprises:

[0027] Randomly selecting sample images containing cracks, incomplete penetration, incomplete fusion, pores and no defects;

[0028] Respectively calculating the peak signal-to-noise ratio after processing by each filtering algorithm;

[0029] Selecting the filtering algorithm with the highest peak signal-to-noise ratio.

[0030] Preferably, the optimization of the target detection model comprises:

[0031] Replacing the standard convolution layer with a lightweight convolution module;

[0032] Embedding an attention guide feature fusion module in the neck network to suppress background noise features.

[0033] Preferably, it further comprises:

[0034] Denoising the data set and image enhancement based on the deep learning model;

[0035] Balancing the calculation cost and recognition accuracy by adjusting the number of network layers and channels.

[0036] Compared with the prior art, the beneficial effects of the present application are as follows:

[0037] The detection accuracy is significantly improved

[0038] With the help of the cross-aiding connection module to fuse multi-scale features, the detection confidence of micro pores is greatly improved, which is obviously improved compared with the original YOLOv5s; the deformable channel attention module can dynamically track the defect outline, and even if there are splashes, oxide skin and other interference, it can still maintain a high average precision mean.

[0039] Robustness of complex working conditions is enhanced

[0040] The weighted loss function increases the positioning loss weight for small defects, effectively reducing the missed detection rate; by dynamically adjusting the confidence weight of the background complex region, the occurrence of false detection is significantly reduced.

[0041] Optimization of computing efficiency

[0042] Lightweight convolution is used to replace the standard convolution layer, which greatly reduces the calculation amount; after compressing the channel number, the inference speed on the edge device is improved, the single piece detection time is shorter, and the high efficiency detection demand is met.

[0043] Breakthrough in industrial applicability

[0044] Gaussian filter preprocessing can effectively suppress arc light, thermal noise and other interference; the generative adversarial network data enhancement technology expands the sample quantity, solves the problem of small sample defect training; the system can maintain a low and stable false alarm rate in various scenes such as pressure vessels and mining machinery, and has a wide range of applications. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The flowchart of the present application. DETAILED DESCRIPTION

[0046] The present application will be further described below in conjunction with the drawings.

[0047] As Figure 1 shown, a welding infrared thermal imaging image defect detection method based on target detection, comprising:

[0048] 1. Welding infrared thermal imaging image data set preparation

[0049] Data acquisition: using FLIR T865 infrared thermal imager (resolution 640x480) to collect special equipment welding seam images, containing four defect types of cracks, incomplete penetration, incomplete fusion, pores and defect-free samples, and the environmental temperature is controlled at 20±5℃.

[0050] Pretreatment process:

[0051] Filtering and denoising: applying median filtering (kernel size 5x5), Gaussian filtering (σ=1.5), mean filtering (kernel size 3x3) and wavelet filtering (sym4 wavelet basis) to the original image respectively, calculating the peak signal-to-noise ratio (PSNR) of each filtering result;

[0052] Optimal selection: selecting the highest PSNR Gaussian filter as the standard preprocessing method (measured PSNR≥38dB);

[0053] Data enhancement: using random rotation (±15°), brightness adjustment (±20%) and generative adversarial network to generate synthetic defect samples, expanding the data set to 10,000.

[0054] 2. YOLO-AttCrossW model construction

[0055] (1) Improvement of main network

[0056] Cross-aided connection module implementation:

[0057] The input feature map is input into the spatial pyramid pooling fast layer (SPPF) and three serial bottleneck layers (Bottleneck) at the same time;

[0058] The SPPF layer output is spliced along the channel dimension with the original input feature.

[0059] The concatenation result and the bottleneck layer output are executed element-wise addition, formula is:

[0060] Output feature = Concat (Input feature, SPPF (Input feature)) + Bottleneck (Input feature)

[0061] Lightweight transformation: replace C3 layer with depth separable convolution layer, calculation amount reduces by 62%.

[0062] (2) Neck network improvement

[0063] Deformable channel attention module implementation:

[0064] The input feature is processed by batch normalization (BN) and SiLU activation function;

[0065] Generate dynamic offset through 3x3 deformable convolution, make sampling points adaptive offset along defect contour (offset amplitude Δx, Δy ∈ [-3, 3] pixels);

[0066] Output weighted feature map, focus on strengthening the response of hot anomaly area.

[0067] (3) Detection head improvement

[0068] Weighted loss function design:

[0069] Positioning loss (Smooth L1), classification loss (cross entropy), confidence loss (binary cross entropy) weight dynamic adjustment;

[0070] 3. Model training and optimization

[0071] Training parameters:

[0072] Hardware: NVIDIA Tesla V100 GPU

[0073] Optimizer: Adam (initial learning rate 0.01, cosine decay)

[0074] Batch size: 16, iteration times: 300 epochs

[0075] Hyperparameter fine-tuning:

[0076] Channel number from [64, 128, 256] to [48, 96, 192], balance calculation cost;

[0077] Embedded attention guided feature fusion module, suppress background noise caused by splashing and oxide skin.

[0078] 4. Performance verification

[0079] Test environment:

[0080] Dataset: 2,000 industrial field weld images (1,200 defects)

[0081] Contrast model: Faster R-CNN, original YOLOv5s

[0082] Results:

[0083]

[0084] Typical case: The detection confidence of micro pores (diameter ≤1mm) reaches 96.7%, and no false detection occurs in complex weld background.

[0085] 5. Industrial deployment

[0086] System integration:

[0087] Hardware: Infrared thermal imager + Intel NUC 11th generation i7 edge computing unit

[0088] Software: PyTorch model converted to TensorRT engine, C++ developed detection interface

[0089] Online detection process:

[0090] Infrared image acquisition - Gaussian filter preprocessing - defect judgment: defective - output position / type; no defect - mark qualified

[0091] Actual effect: In the detection of pressure vessel welds, the single piece detection time is ≤0.8 seconds, and the missed detection rate is stable ≤2%.

[0092] Among them, the auxiliary skip connection module and the improved backbone structure:

[0093] The main function of the backbone network in YOLO is to extract information from the input image, so it is usually called a feature extraction network. However, complex operations such as convolution, batch normalization and sigmoid weighted linear unit will inevitably dilute the spatial information in the image. In view of the problem of large aspect ratio and uncertain features in the weld detection task, in order to solve this problem, inspired by the residual block in ResNet, an auxiliary skip connection module (ASCM) is designed. This connection aims to provide more information. Since deep layers may involve larger feature sizes, down-sampling is done through convolution operations. Based on the above motivation, the relationship between the input feature and the output characteristics F of the backbone layer can be expressed as: B

[0094] ​

[0095] wherein: SPPF (spatial pyramid pooling fast, SPPF) represents a spatial pyramid pooling fast layer, which avoids image distortion caused by cropping and scaling in image processing, and improves the selection speed of candidate anchor boxes; Concat represents a concatenation operation, which connects the features along the channel. At the same time, CBS and C 3-n satisfy the following conditions.

[0096] CBS(X) = Conv(BN(SiLU(X))) (5)

[0097] C 3-n (X) = CBS(Concat(CBS(X), Bottleneck n (CBS(X))) (6)

[0098] wherein: Conv represents a convolution operation, which uses a sliding kernel to extract information from an input image, generates a feature map to capture local features; BN represents batch normalization, which adjusts the feature distribution by standardization, scaling and moving, so as to speed up the training process and improve the stability of the model; SiLU represents Sigmoid weighted linear unit, which combines the nonlinear characteristics of Sigmoid and the simplicity of linear unit, can effectively alleviate the gradient vanishing problem, and improve the performance of the model; Bottleneck n represents n∈N serially connected bottleneck operations. It satisfies the following formula:

[0099] Bottleneck(X) = X + CBS(CBS(X)) (7)

[0100] wherein the deformable channel attention module:

[0101] One of the important contributions of YOLOv5 is the use of an efficient spatial pyramid pooling layer in the backbone, which concatenates the features produced by the max-pooling operation. This layer provides additional information from a broader spatial hierarchy. Unfortunately, the neck does not adopt a similar design. To address this issue, a deformable coordinate attention module (DCAM) is created in the neck of YOLO-AttCrossW. It aims to deeply fuse the feature screening ability of channel attention with the spatial adaptability of deformable mechanism, combining flexibility and accuracy: on the one hand, it inherits the advantages of coordinate attention, which can dynamically allocate channel weights by learning, allowing the model to focus on feature channels carrying key information and suppress redundant channel interference; on the other hand, the deformable feature makes it able to dynamically adjust the receptive field and feature sampling position according to the shape, scale, and spatial distribution of the target, especially for irregular, non-rigid, or occluded targets, it can adaptively capture their local key features, avoiding feature mismatch caused by fixed receptive fields.

[0102] The relationship between the input and output features of the DCAM module can be represented as:

[0103] M C (F)=Conv(Concat(AvgPool(F) X ,AvgPool(F) Y )) (8)

[0104] M S (F)=σ(BN(M C (F))) (9)

[0105] M D (F) 1 =Sigmoid(Conv(M S (F))) (10)

[0106] M D (F) 2 =Sigmoid(Conv(M S (F))) (11)

[0107]

[0108] where BN denotes batch normalization, σ(·) denotes a nonlinear transformation activation function, and the symbol represents element-wise multiplication.

[0109] Deformable convolution breaks the fixed receptive field limitation by introducing learnable offset parameters to dynamically adjust the sampling position of each convolution kernel according to the geometric shape of the input features. When encountering a pose-distorted pedestrian, the sampling points will focus on the natural bending trajectory of the limbs. When dealing with occluded objects, the sampling points will actively avoid the occluded area and focus on the visible part. This flexible spatial sampling mechanism can not only accurately capture the key geometric features of the target, but also effectively filter out redundant background interference, significantly improving the performance robustness in complex scenes.

[0110] Weighted loss function

[0111] Class imbalance is common in object detection: for example, the number of background samples (negative samples) is much larger than that of foreground targets (positive samples), or the number of samples of some niche classes (such as "welds" in "defects") is much smaller than that of mainstream classes (such as "welds"). If a normal loss function (such as cross-entropy) is used, the model will tend to "please" the majority class (because the majority class contributes more to the loss), resulting in the minority class being ignored (high miss rate). Weighted loss assigns higher weights to minority classes, increasing their proportion in the total loss and forcing the model to pay more attention to the learning of minority classes. In object detection, there are a large number of easy samples (such as clear and large-sized targets) and difficult samples (such as blurred, occluded, and small-sized targets). In a normal loss function, the loss value of easy samples is usually small, but because there are many of them, the total contribution may mask the loss of difficult samples, causing the model to "waste effort" on simple samples and making it difficult to improve the detection ability of difficult samples. Weighted loss assigns higher weights to difficult samples to amplify their loss. To make YOLO-AttCrossW focus more on small defects that are difficult to detect or defects in complex backgrounds, a weighted loss function is designed to dynamically adjust the weights of different loss terms. The core idea of the weighted loss function is to automatically adjust the weights of each term of the loss function according to the difficulty, size, and background interference of different targets. The specific steps are as follows:

[0112] 1) Loss function composition: The loss function of YOLO-AttCrossW includes bounding box loss, classification loss, and confidence loss.

[0113] Bounding box loss: measures the difference between the predicted box and the ground truth box, using smooth L1 loss.

[0114] Classification loss: Measures the difference between the predicted results and the true labels for each class, using cross entropy loss.

[0115] Confidence loss: Measures the model's confidence in the existence of the target, using binary cross entropy loss.

[0116] 2) Weighting coefficients: Dynamically calculate the weighting coefficients of the loss terms according to the size and morphology of the target. For example: For small defects, give higher positioning loss weight to improve the positioning accuracy of small targets. For areas with complex background, reduce the weight of confidence loss to reduce background interference.

[0117] 3) Weighted loss calculation: The final total loss is obtained by weighting each loss term, the specific formula is:

[0118] L total = σ·L box + β·L cls + γ·L conf (13)

[0119] Where α, β and γ are dynamically adjusted weight coefficients according to the target type. The advantage of this loss function is that it can adjust the focus in the training process according to the characteristics of each sample (such as size and background complexity), thereby significantly improving the detection accuracy of small defects and complex backgrounds.

Claims

1. A weld seam infrared thermal imaging image defect detection method based on target detection, characterized in that: The following steps are involved: The filtering algorithm is used to pre-process the infrared thermal imaging image of the weld, and the optimal filtering method is selected from median filtering, Gaussian filtering, mean filtering and wavelet filtering by comparing the peak signal-to-noise ratio. Build the YOLO-AttCrossW target detection model, which contains: Backbone network: Introduces a spanning auxiliary connection module to concatenate and fuse the input features with the features output by the spatial pyramid pooling fast layer, and then adds them to the output features of the bottleneck operation of the serial connection; Neck network: Connects to the deformable channel attention module and dynamically adjusts the feature sampling position through the deformable convolution kernel; Detection head: uses a weighted loss function to dynamically adjust the weight coefficients of positioning loss, classification loss, and confidence loss; The preprocessed images are used to train the target detection model to achieve defect recognition and location.

2. The method for detecting weld defects using infrared thermal imaging images based on target detection according to claim 1, wherein: The execution of the cross-auxiliary connection module includes: Perform spatial pyramid pooling fast operations and serial bottleneck operations on input features in parallel; Concatenate the output features of the spatial pyramid pooling fast layer with the original input features along the channel dimension; The concatenated result is fused with the serial bottleneck operation output features by element-wise addition.

3. The method for detecting weld defects by infrared thermal imaging images based on target detection according to claim 1, wherein: The execution of the deformable channel attention module includes: Perform batch normalization and nonlinear activation function processing on the input features in sequence; A dynamic sampling grid is generated by learnable offset parameters, so that the convolution kernel sampling position is adaptively offset along the defect geometric contour.

4. The method for detecting weld defects by infrared thermal imaging images based on target detection according to claim 1, wherein: The execution of the weighted loss function includes: The smooth L1 loss function is used for positioning loss, the cross entropy loss function is used for classification loss, and the binary cross entropy loss function is used for confidence loss; The weight coefficient of positioning loss is increased for small defects, and the weight coefficient of confidence loss is reduced for complex background areas.

5. The method for detecting weld defects by infrared thermal imaging images based on target detection according to claim 1, wherein: The selection of the filtering algorithm includes: Randomly select sample images containing cracks, incomplete penetration, lack of fusion, pores, and no defects; Calculate the peak signal-to-noise ratio after each filtering algorithm is processed; Select the filtering algorithm with the highest peak signal-to-noise ratio value.

6. The method for detecting weld defects by infrared thermal imaging images based on target detection according to claim 1, wherein: The optimization of the target detection model includes: Replace the standard convolutional layer with a lightweight convolutional module; An attention-guided feature fusion module is embedded in the neck network to suppress background noise features.

7. The method for detecting weld defects by infrared thermal imaging images based on target detection according to claim 1, wherein: Also includes: Denoising the dataset and image enhancement based on deep learning models; Balance computational cost and recognition accuracy by adjusting the number of network layers and channels.

Images