Anti-shielding, multi-scale and self-adaptive deformation welding spot defect detection method

Abstract

The invention discloses an anti-shielding, multi-scale and self-adaptive deformation welding spot defect detection method and system, and aims to detect a welding spot defect through a diagnostic and collaborative detection methodology. The key technical problems of poor robustness, easy missing detection, easy false detection and the like caused by concurrence of three major problems of defect shielding, small size and irregular shape in industrial welding detection are systematically solved. The method comprises the following steps: acquiring a welding spot image, and inputting the welding spot image into an improved YOLOv11 model; according to the model, on the basis of a YOLOv11 architecture, a C2PSALSKA module is fused at the tail end of a backbone network so as to enhance the reasoning ability of an occluded target through a big kernel attention mechanism; dynamicConv dynamic convolution is adopted in a lower sampling layer so as to adapt to geometrical shape changes of the defects in a self-adaptive mode; and a C3k2ECA module is introduced into the detection head part, so that the feature sensitivity to tiny defects is improved through high-efficiency channel attention. According to the invention, through collaborative optimization of multiple modules, the detection precision is obviously improved on the premise that the real-time performance is not sacrificed.

Description

Technical Field

[0001] This invention belongs to the field of industrial visual inspection, specifically relating to a method for detecting weld and weld point defects based on deep learning, and in particular, a real-time, high-precision detection method and system designed to systematically solve multiple challenges in industrial welding, such as physical obstruction, minute size and irregular shape. Background Technology

[0002] Welding, an indispensable key process in modern manufacturing, is widely used in core fields such as aerospace, shipbuilding, automotive, and energy equipment. Welds and weld points are critical components in welded structures, bearing loads and providing connection functions; their quality directly determines the safety and reliability of the entire structure. However, the welding process is subject to complex influences from various factors, including heat source characteristics, material composition, and process parameters, making it prone to defects such as porosity, cracks, and slag inclusions. Unlike conventional component assembly defects, welding defects typically exhibit irregular shapes, large size ranges, and are often accompanied by complex textures of metal oxide scale and heat-affected zones, posing unique challenges for detection. If these defects are not detected and addressed in a timely manner, they may expand during service life, leading to structural failures or even major safety accidents. Therefore, researching efficient and accurate weld defect detection technologies has significant engineering value and practical implications.

[0003] Currently, mainstream non-destructive testing (NDT) technologies for weld defects in industry include visual inspection, ultrasonic testing, X-ray inspection, and magnetic particle inspection. While visual inspection is intuitive, it is inefficient, highly subjective, and prone to missed detections and misjudgments. Ultrasonic testing uses the echo signal of high-frequency sound waves to identify internal defects, but it requires high equipment precision and workpiece surface roughness, and is easily affected by coupling media. X-ray inspection can image internal defects at high resolution, but the equipment is expensive, radiation protection requirements are stringent, and it is difficult to deploy flexibly in complex industrial environments. Magnetic particle inspection displays defects by applying a magnetic field to the weld surface, but its detection sensitivity is significantly affected by the magnetic field gradient. Furthermore, these traditional methods mostly rely on manually designed features for judgment, making it difficult to guarantee robustness and versatility when faced with diverse weld morphologies and complex lighting conditions.

[0004] In recent years, deep learning algorithms, represented by the YOLO (You Only Look Once) series, have demonstrated great potential in industrial inspection. YOLOv11, as a cutting-edge version of this series, adopts an advanced "Backbone-Feature Fusion-Detection Head" structure, achieving a good balance between accuracy and speed. However, when applied to solder joint defect detection, YOLOv11 still faces challenges: 1) In complex backgrounds, occlusion caused by oxide scale, spatter, or overlapping defects limits the receptive field of traditional convolutions, resulting in poor robustness to occluded defects and a high false negative rate; 2) Solder joint defects are small in size, and features are easily lost in deep networks, leading to false negatives; 3) Irregular solder joints or deformable defects such as multi-shaped weld slag are difficult to adapt to morphological changes due to the fixed convolution kernel, resulting in significant false positives.

[0005] In summary, although existing technologies have made some progress in the field of defect detection, there are still obvious limitations. Existing solutions can be roughly divided into several categories: (1) Lightweight solutions: Their main goal is to compress the model and apply it to scenarios such as PCB, but this is not the same as the goal of industrial weld inspection to pursue extreme robustness. (2) Single-point enhancement solutions: They improve the general accuracy of the model by introducing some novel attention or feature fusion module, but do not systematically solve the complex problem of multiple challenges occurring simultaneously in industrial welds. (3) Complex architecture solutions: They increase the complexity of the system to exchange for accuracy, but this sacrifices the simplicity and efficiency of the single model.

[0006] Therefore, there is still an urgent need in the field for a comprehensive technical solution that can solve the three concurrent problems of physical obstruction, minute size and irregular shape in industrial welds in a unified and efficient framework in a coordinated and targeted manner. This is the core technical problem that this invention aims to solve. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a solder joint defect detection method based on the improved YOLOv11, which aims to specifically solve the problems of poor robustness, easy to miss or falsely detect solder joint defects caused by occlusion, small target size and irregular shape.

[0008] The core inventive idea of ​​this invention lies not in generalizing the detection model, but in proposing a pioneering collaborative enhancement strategy. This invention first identifies three critical and concurrent challenges in industrial weld joint inspection: physical occlusion, minute defects, and irregular deformation. Subsequently, addressing the underlying mechanisms of each challenge, it matches diverse but complementary solutions from numerous technologies and integrates them through a unified YOLOv11 framework, thereby achieving robust detection of complex defect scenarios.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is: a solder joint defect detection method based on an improved YOLOv11, comprising the following steps:

[0010] Step 1: Data Acquisition and Preprocessing. Images of workpieces such as motor welds are acquired using industrial cameras to construct a dataset covering typical defects of large, medium, and small sizes. The images are then labeled and preprocessed.

[0011] Step 2: Build and train the improved YOLOv11 model. Three innovative modules are specifically integrated into the YOLOv11 baseline architecture, and the preprocessed dataset is input into the model for training, continuously optimizing the model parameters.

[0012] Step 3: Defect Detection and Deployment. Deploy the trained model on an industrial computer or embedded device to detect weld point images acquired in real time, and output the defect category, location, and confidence level.

[0013] Furthermore, in step 2, the basic architecture of the YOLOv11 model includes a backbone, a neck, and a head:

[0014] The backbone network is responsible for extracting multi-level features from the input image, and the feature output of each layer can be represented as: Among them, the function It typically consists of convolution operations and non-linear activations (such as leaky ReLU). Indicates the first Layer weights. The network uses batch normalization (BN) to standardize the data. The specific normalization process is as follows: in, , These are the mean and variance of the current batch, respectively. and For learnable parameters, and To prevent constants with a denominator of 0, a parameter update formula based on gradient descent is used: in, This represents the learning rate, ensuring that the network continuously moves towards the optimal solution during training.

[0015] The Neck module integrates low-level details with high-level semantic information by upsampling or downsampling feature maps from different layers, constructing a feature representation at a unified scale. Assuming we have feature maps F1, F2…Fn from different layers, after appropriate scale adjustments… The fusion result can then be expressed as: function It can be either an upsampling operation (such as bilinear interpolation) or a downsampling operation (such as max pooling) to unify the size of the various feature maps. This fusion method is similar to the common Feature Pyramid Network (FPN) structure and can fully supplement multi-scale information.

[0016] The detection head receives the fused feature map and performs target prediction for each grid cell, including category, confidence level, and bounding box position. This represents the predicted probability of the target category. This indicates the confidence level that the target exists. The coordinates of the center box of the boundary are, These represent the width and height of the bounding box, respectively. The specific prediction output can be described as: To enable the model to accurately predict the target's location and category, an overall loss function is introduced, which includes classification loss, target presence loss, and localization loss. Typically, the balance coefficients are set to [values ​​to be filled in]. = 0.5, = 1.0 and =0.05, its formula is: For classification and target existence determination, the network uses a binary cross-entropy loss function, where y∈{0,1} is the true label and p is the probability predicted by the model: To simultaneously consider the overlap between bounding boxes, the center distance, and the consistency of aspect ratio, YOLOv11 introduces the CIoU loss, the detailed expression of which is as follows: In the above formula, For prediction boxes With real frame Euclidean distance between the centers It is the diagonal length of the smallest closed rectangle containing two boxes.

[0017] The formula for IOU (Intersection over Union) is: The formula used to measure the difference in width-to-height ratio is: The equilibrium parameter is defined as follows:

[0018] Furthermore, in step 2, the improvement to the YOLOv11 model lies in the incorporation of three innovative modules designed to solve the aforementioned specific welding defect problems:

[0019] To address the poor robustness of defect detection caused by oxide scale and spatter, an improved C2PSA_LSKA module is introduced at the end of the backbone network. Unlike using LSKA as a general attention module to improve PCB inspection accuracy, this invention creatively applies it to solve the problem of physical occlusion in industrial welds. We found that weld occlusion is essentially a lack of contextual information, and LSKA's large receptive field can supplement and reconstruct the features of the occluded area from the surrounding broad texture. This innovative application scenario is unprecedented in existing technologies. Its core computational process is as follows: For the input feature map, For horizontal one-dimensional convolution, For one-dimensional convolution in the vertical direction, It is the sigmoid activation function.

[0020] To address the issue of missed detections due to the weak features of minute solder joint defects being easily overlooked in deep networks, an improved C3k2_ECA module is introduced in the detection head. To tackle the challenge of insignificant feature channels for minute targets, this invention employs efficient channel attention (ECA), which captures cross-channel information without dimensionality reduction through adaptively sized one-dimensional convolutions. This accurately amplifies the weights of key channels containing minute defect information, suppresses background noise interference, and thus enhances the detection sensitivity for minute targets. The attention calculation process is as follows: For global average pooling, This is a one-dimensional convolution with an adaptive kernel size (k is adaptively determined by the number of channels). It is the sigmoid activation function. Broadcast multiplication for channel dimensions.

[0021] To address the false detection of deformation defects caused by irregular weld joint shapes and varied weld slag shapes, a DynamicConv dynamic convolution module is used in the downsampling layer of the network. Unlike the use of DynamicConv for feature calibration in multimodal data fusion, this invention utilizes it to solve the problem of irregular defect shapes caused by unstable welding processes in single-modal vision. Leveraging its dynamic convolution kernel's ability to adaptively match the target geometry, it captures the diverse features of defects more flexibly and accurately. The convolution process can be represented as follows: in As candidate convolution kernels, For the weight generation network, N is the number of parallel convolution kernels (N=4 in this paper), and * represents the convolution operation.

[0022] in As candidate convolution kernels, For the weight generation network, N is the number of parallel convolution kernels (N=4 in this paper), and * represents the convolution operation.

[0023] The selection of each module in this invention has profound internal logic and irreplaceability: while LSKA alone can handle occlusion, it may smooth out minor defects without occlusion; while ECA alone can enhance minor features, it cannot cope with large-scale physical occlusion and shape distortion; while DynamicConv alone can adapt to deformation, it is powerless against occluded targets with weak features. Therefore, only by synergistically combining LSKA, ECA, and DynamicConv can the complex challenges of industrial weld joint detection be fully solved. This is the core innovation of this invention. While maintaining high inference speed, it significantly improves the accuracy and robustness of the model in detecting various weld joint defects in complex industrial scenarios, providing a reliable and efficient solution for intelligent monitoring of weld joint quality. Attached Figure Description

[0024] Figure 1 This is a flowchart of the weld joint defect detection method in this invention.

[0025] Figure 2 This is a comparison chart showing the detection performance of the method of this invention and the original YOLOv11 model on different types of defects. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, this embodiment takes the detection of defects in motor solder joints as an example to fully demonstrate the application process of the method of this invention.

[0027] Step 1: Dataset Construction and Preprocessing

[0028] A visual inspection system equipped with an industrial camera (model: KEYENCE LJ-X8080) was used to collect data on three typical solder joint defect samples. A dataset of 605 high-resolution images was constructed, covering large, medium, and micro solder joint defects. Stratified random sampling was used to divide the dataset into a training set of 424 images, a validation set of 160 images, and a test set of 21 images. The LabelImg tool was used for precise pixel-level annotation of defect regions.

[0029] Step 2: Construction and Training of the Improved YOLOv11 Model

[0030] Experimental environment:

[0031] Hardware: CPU is AMD Ryzen 57500F, GPU is NVIDIA GeForce RTX 3070.

[0032] Software: Operating system is Windows 10, deep learning framework is PyTorch 2.0, programming language is Python 3.10.

[0033] Model structure:

[0034] The core improvement in the overall network structure of the model lies in:

[0035] Backbone: In the deeper part of the network (layer 10), the C2PSA_LSKA module is used to replace the standard module to expand the receptive field and enhance the feature extraction capability of occluded targets.

[0036] Neck and Head: DynamicConv modules are used as downsampling convolutional layers in the upsampling and downsampling paths to adapt to defects of different shapes. Before the final detection head output (layer 23), a C3k2_ECA module is added to enhance the representation of channel features for small targets and improve detection sensitivity.

[0037] Training parameters:

[0038] The Stochastic Gradient Descent (SGD) optimizer was employed. The input image size was 640x640. The batch size was 16. The initial learning rate was 0.01. To enhance the model's generalization ability, techniques such as mosaic data augmentation, random rotation (±10°), and scaling (±20%) were used. Mixed Precision Training (AMP) was enabled during training to save GPU memory and accelerate computation.

[0039] Step 3: Experimental Verification and Result Analysis

[0040] ablation experiment

[0041] To verify the effectiveness of each improved module, an ablation experiment was designed, and the results are shown in Table 1.

[0042] Table 1: Ablation Experimental Studies of Various Improvements in the Improved YOLO Model Methods <![CDATA[mAP 50 (%)]]> FPS YOLOv11 74.3% 168.3 YOLOv11+LKSA 75.5% 166.4 YOLOv11+ECA 76.7% 166.7 YOLOv11+DynamicConv 75.2% 154.5 YOLOv11+all improvements 77.2% 153.1

[0043] As shown in Table 1, the baseline YOLOv11 model has an mAP50 of 74.3%. Adding the LKSA, ECA, and DynamicConv modules individually improves the mAP50 to 75.5%, 76.7%, and 75.2%, respectively, validating the independent effectiveness of each module. When all modules are integrated (using the method of this invention), the model achieves an optimal mAP50 of 77.2%, a 2.9 percentage point improvement over the baseline, while maintaining an inference speed of 153.1 FPS, meeting the requirements for real-time industrial detection.

[0044] Comparison of detection results for different defect types:

[0045] As shown in Table 2, the method of the present invention outperforms the original model in detecting different types of defects. In particular, the mAP50 is improved by 0.6% and 2.9% for the detection of medium-sized defects (Type 2) and small defects (Type 3), respectively. This indicates that the method of the present invention significantly enhances the ability to identify medium-sized and small defects, effectively reducing the false negative rate.

[0046] Table 2: Comparison of mAP50 (%) on three different types of defects Methods Type1 Type2 Type3 YOLOv11 80.2% 98% 74.3% Ours 81.8% 98.6% 77.2%

[0047] Comparison with other object detection algorithms:

[0048] The method of this invention was compared with mainstream algorithms such as Faster R-CNN, EfficientDet, and YOLOv8, and the results are shown in Table 3. The method of this invention achieved 77.2% mAP50, the highest among all compared methods. Although its speed (153.1 FPS) is slightly lower than YOLOv8 (170.8 FPS), it leads YOLOv8 in accuracy by nearly 6 percentage points, demonstrating the best overall performance balance and making it more suitable for industrial quality inspection scenarios with stringent accuracy requirements.

[0049] Table 3: Performance Comparison of Different Methods Methods <![CDATA[mAP 50 (%)]]> FPS Faster R-CNN 49.4% 30.7 EfficientDet 53.4% 38.2 CenterNet 55.1% 28.2 Cascade R-CNN 57.7% 41.6 YOLOv8 71.5% 170.8 Ours 77.2% 153.1

[0050] In summary, this invention, through innovative improvements to the structure of the YOLOv11 model, effectively solves key technical challenges in industrial weld joint defect detection, achieving a dual improvement in accuracy and speed, and possesses significant practical value and broad application prospects.

Claims

1. A solder joint defect detection method based on an improved YOLOv11, characterized in that, Includes the following steps: Step 1: Obtain the weld point image of the workpiece to be inspected; Step 2: Input the solder joint image into an improved YOLOv11 model for processing. The model consists of three parts: backbone, neck, and head. Step 3: In the backbone network of the model, a C2PSA_LSKA module is used. This module is configured to use a large kernel receptive field to capture the contextual texture information around defects that are physically obscured by weld slag or overlapping welds in order to reconstruct incomplete defect features. Step 4: In the downsampling path of the model, one or more DynamicConv dynamic convolution modules are used. These modules are configured to dynamically adjust the convolution kernel according to the irregular geometric shape of the solder joint defect in order to adaptively extract the features of multi-morphological defects. Step 5: In the detection head of the model, a C3k2_ECA module is used, which is configured to amplify the weights of weak feature channels containing information about small-sized defects through cross-channel interaction; Step 6: The model outputs the defect category, location, and confidence level information of the weld point image.

2. The method according to claim 1, characterized in that, The C2PSA_LSKA module incorporates a large kernel attention mechanism (LSKAttention), which decomposes a large two-dimensional convolutional kernel into a sequence of depthwise convolution, a depthwise dilated convolution, and a pointwise convolution. This reduces computational complexity while approximating the effect of large kernel convolution, thereby improving the ability to capture residual edge and texture features in occluded areas.

3. The method according to claim 1, characterized in that, The DynamicConv module contains multiple parallel convolution kernels. It dynamically generates the weights of each convolution kernel based on the input features through a lightweight attention mechanism, and aggregates the weighted convolution kernels so that the model can flexibly adapt to the variability of weld joint defects in shape and texture.

4. The method according to claim 1, characterized in that, The C3k2_ECA module incorporates an efficient channel attention (ECA) mechanism. This mechanism captures local cross-channel interaction information through a one-dimensional convolution operation with an adaptive kernel size without performing channel dimensionality reduction. This effectively improves the model's ability to express weak feature channels containing minute defect information while maintaining lightweight design.

5. The method according to claim 1, characterized in that, The backbone network ends with the C2PSA_LSKA module as described in claim 2; the downsampling layer in the neck network uses the DynamicConv module as described in claim 3; and the output layer of the detection head uses the C3k2_ECA module as described in claim 4.

Images