A steel structure weld defect identification method and system

Abstract

This invention provides a method and system for identifying weld defects in steel structures, relating to the field of weld defect detection technology. The method includes: acquiring weld images and inputting them into an improved lightweight target detection network; embedding a C2PSA_DCBAM orientation-aware attention module into the backbone of the network to enhance linear defect feature extraction; using GSConv lightweight convolutions in the neck region to reduce computational load; and using the Focal-EIoU loss function to optimize bounding box regression during training. This invention solves the problem of real-time deployment of high-precision detection models on low-computing-power devices by collaboratively optimizing feature extraction, network structure, and training objectives, achieving a significant reduction in model complexity while improving the detection accuracy and recall rate of weld defects, especially small linear defects.

Description

Technical Field

[0001] This invention relates to the field of weld defect detection technology, and in particular to a method and system for identifying weld defects in steel structures. Background Technology

[0002] In the field of modern construction engineering, steel structures are increasingly widely used due to their advantages such as high strength, large span, and convenient construction. Welding, as a key connection method in steel structures, directly affects the safety and reliability of the overall structure. Therefore, efficient and accurate automated detection of weld defects is particularly important.

[0003] In recent years, deep learning-based target detection models, especially the YOLO series algorithms, have provided powerful technical means for the automatic identification of weld defects. These models, through end-to-end learning, can directly locate and classify defects from images, significantly surpassing the efficiency and consistency of traditional manual inspection and traditional image processing methods. However, deploying such high-precision models in actual industrial inspection sites faces a prominent technical contradiction: while the latest models, such as YOLOv11, which are designed to pursue high accuracy (e.g., high mAP, high recall), have improved accuracy, their computational complexity is not conducive to deployment. They typically have a huge number of parameters and high computational overhead, making it difficult to meet the stringent requirements of real-time detection on embedded devices or industrial edge computing platforms with limited computing power.

[0004] Existing technologies typically require trade-offs between accuracy and efficiency. This leads to situations in resource-constrained and complex industrial environments where either simplified models miss critical minute defects, posing safety risks, or bulky models fail to respond in real-time, impacting the efficiency of the inspection pipeline. Therefore, developing a lightweight intelligent identification method that can maintain or even improve the accuracy of minute defect detection while significantly reducing model complexity and computational costs, thus truly adapting to industrial deployments, has become a critical technical challenge that urgently needs to be addressed in this field. Summary of the Invention

[0005] To address the core technical challenges of computationally complex high-precision models for weld defect detection, which are difficult to deploy in real-time on low-computing-power devices, this invention introduces three collaboratively improved technical features: a lightweight convolutional module (GSConv), a direction-aware attention mechanism (C2PSA_DCBAM), and a focused bounding box loss function (Focal-EIoU). This provides a method and system for identifying weld defects in steel structures, aiming to overcome the inherent contradiction between lightweight design and high precision. The goal is to significantly reduce model parameters and computational load while ensuring and improving the detection accuracy and recall rate for weld defects (especially small-sized and linear defects).

[0006] To achieve the above objectives, a first aspect of the present invention provides a method for identifying defects in steel structure welds, comprising:

[0007] Acquire images of the welds in the steel structure to be inspected;

[0008] The weld image is input into a trained weld defect recognition network, which includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network performs hierarchical feature extraction on the input image. The feature extraction includes directional feature enhancement of the feature map by embedding a direction-aware attention module, and sequentially extracting pixel-level detail features containing defect edge contours and surface textures, mid-level features that can distinguish different basic defect categories, and high-level semantic features containing defect direction attributes and large-size defect category attributes.

[0009] The neck network receives the extracted features from each layer, performs cross-scale feature fusion through upsampling, downsampling, and feature concatenation, and outputs a fused multi-scale feature map.

[0010] The detection head outputs the type, location, and confidence information of weld defects based on the fused multi-scale feature map.

[0011] Furthermore, the pixel-level detail features include at least one of the following: grayscale gradient features of weld fusion lines, edge contour features of cracks and pores, and fish scale and scratch texture features of weld surface.

[0012] The intermediate layer features include at least one of porosity, cracks, and inclusions;

[0013] The high-level semantic features include at least one of crack direction, incomplete penetration defect, and incomplete fusion defect.

[0014] Furthermore, the direction-aware attention module is a C2PSA_DCBAM module, and its specific steps for enhancing the direction awareness of the feature map include:

[0015] The input feature map is divided into two parts: one part is passed directly, and the other part is processed by the DCBAM attention module.

[0016] The DCBAM module includes a direction enhancer, a channel attention submodule, and a spatial attention submodule. It extracts directional features through depthwise separable convolution and applies attention weights to the channel and spatial dimensions respectively.

[0017] The two processed feature maps are then spliced ​​and fused to obtain the enhanced feature map.

[0018] Furthermore, the neck network employs a lightweight convolutional structure GSConv for feature fusion, specifically including:

[0019] The feature map is input into the GSConv module, which combines standard convolution and depthwise separable convolution, and performs feature information permeation through channel shuffling.

[0020] Output the fused feature map for subsequent upsampling, downsampling, or stitching operations.

[0021] Furthermore, the structure of the weld defect recognition network includes: an input layer, a Conv-1 convolutional layer, a Conv-2 convolutional layer, a C3k2-1 module, a Conv-3 convolutional layer, a C3k2-2 module, a Conv-4 convolutional layer, a C3k2-3 module, a spatial pyramid pooling module, a C2PSA_DCBAM attention module, a lightweight FPN+PAN fusion module, a detection head, and an output layer, all connected in sequence.

[0022] The working principle of the weld defect identification network includes:

[0023] The Conv-1 and Conv-2 convolutional layers perform preliminary feature extraction and downsampling on the input image to obtain an initial feature map. The C3k2-1, C3k2-2, and C3k2-3 modules extract and fuse pixel-level detail features containing defect edge contours and surface textures with mid-level features that can distinguish different basic defect categories through multi-branch residual learning. The spatial pyramid pooling module performs multi-scale pooling on the feature map to expand the receptive field. The C2PSA_DCBAM attention module enhances high-level semantic features containing defect direction attributes and large-size defect category attributes through a direction-aware mechanism. The lightweight FPN+PAN fusion module achieves multi-scale fusion between the pixel-level detail features, mid-level features, and high-level semantic features through upsampling, downsampling, and feature concatenation. The Detect head predicts the category, location, and confidence level of weld defects based on the fused feature map.

[0024] Furthermore, the recognition network uses Focal-EIoU as the bounding box regression loss function for target localization optimization during training. The Focal-EIoU loss function is expressed as:

[0025] ;

[0026] Where IoU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and γ is the focus suppression coefficient. The EIoU loss is calculated using the following formula:

[0027] ;

[0028] in, For prediction boxes Center point of the real frame The square of the Euclidean distance between them To determine the diagonal length of the minimum closed region covering both the predicted and ground truth bounding boxes, , These are the width and height of the predicted bounding box, respectively. , These are the width and height of the actual bounding box, respectively. These are the width and height of the minimum closed region covering the predicted bounding box and the ground truth bounding box, respectively.

[0029] A second aspect of the present invention provides a steel structure weld defect identification system, comprising:

[0030] Image acquisition unit, used to acquire images of the weld seams of the steel structure to be inspected;

[0031] The feature extraction unit includes a backbone network for performing hierarchical feature extraction on the input image. The feature extraction includes directional feature enhancement of the feature map by embedding a direction-aware attention module, and sequentially extracting pixel-level detail features containing defect edge contours and surface textures, mid-level features that can distinguish different basic defect categories, and high-level semantic features containing defect direction attributes and large-size defect category attributes.

[0032] The feature fusion module, including the neck network, is used to receive the extracted features from each layer, perform cross-scale feature fusion through upsampling, downsampling and feature concatenation, and output the fused multi-scale feature map.

[0033] The defect detection unit includes a detection head, which is used to output the category, location, and confidence information of weld defects based on the fused multi-scale feature map.

[0034] A third aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps in the steel structure weld defect identification method as described in the first aspect of the present invention.

[0035] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon that, when executed by a processor, implements the steps in the steel structure weld defect identification method as described in the first aspect of the present invention.

[0036] A fifth aspect of the present invention provides a computer program product including software code, wherein the program in the software code performs the steps of the steel structure weld defect identification method as described in the first aspect of the present invention.

[0037] Compared with the prior art, the steel structure weld defect identification method and system provided by the present invention has the following advantages:

[0038] (1) In this invention, GSConv lightweight convolutional layers are used in at least one branch of the feature pyramid structure of the neck network. GSConv is used to replace the standard convolution. By channel shuffling, the rich features of the standard convolution and the lightweight advantages of the depth-separable convolution are fused. This allows the model to significantly reduce the number of parameters and floating-point operations in the neck part without significantly losing the ability to transmit feature information, thereby significantly reducing the overall computational complexity of the model.

[0039] (2) In this invention, a DCBAM directional convolutional block attention module is embedded in the C2PSA backbone network module. DCBAM captures linear features through its directional enhancer (depth-separable convolution) and combines channel and spatial attention mechanisms to enable the network to focus on the directional texture details of the weld area. By embedding it into the cross-stage connection structure of C2PSA, this module can guide the network to focus on the detailed direction of the weld area, effectively enhancing the feature extraction and representation capabilities of small, linear defects, thereby significantly reducing the false negative rate of such high-risk defects from the algorithm level.

[0040] (3) This invention uses the Focal-EIoU loss function to replace the common CIoU loss function. Focal-EIoU introduces a focus adjustment mechanism (IoUγ) on the basis of EIoU. By directly and independently optimizing the center point distance, width, and height difference between the predicted box and the ground truth box through EIoU, the fuzzy optimization problem of the aspect ratio term in CIoU is avoided; at the same time, the gradient weight of low-quality (low IoU) predicted samples is dynamically reduced through the focus mechanism, so that the model training focuses more on optimizing high-quality, learning-value predicted samples. The two work together to not only greatly accelerate the convergence speed of the model, especially the bounding box regression branch, but also significantly improve the localization accuracy of small-sized defect bounding boxes, so that the detection box is closer to the actual contour of the defect. Attached Figure Description

[0041] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0042] Figure 1 This is a flowchart of a method for identifying weld defects in steel structures provided in Embodiment 1 of the present invention;

[0043] Figure 2 This is a schematic diagram of the weld defect identification network structure provided in Embodiment 1 of the present invention;

[0044] Figure 3 This is a schematic diagram of the C2PSA-DCBAM module structure provided in Embodiment 1 of the present invention;

[0045] Figure 4 This is a diagram of the PSABlock-DCBAM architecture provided in Embodiment 1 of the present invention;

[0046] Figure 5 This is a DCBAM architecture diagram provided in Embodiment 1 of the present invention;

[0047] Figure 6 The first confusion matrix of the original model provided in Embodiment 1 of the present invention;

[0048] Figure 7 The second confusion matrix of the original model provided in Embodiment 1 of the present invention;

[0049] Figure 8 The first confusion matrix of the improved model provided in Embodiment 1 of the present invention;

[0050] Figure 9 The second confusion matrix of the improved model provided in Embodiment 1 of the present invention;

[0051] Figure 10 This is an architecture diagram of the steel structure weld defect identification system provided in Embodiment 2 of the present invention. Detailed Implementation

[0052] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0053] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0054] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0055] All data acquisition in this embodiment is carried out in accordance with laws and regulations and with user consent, and the data is used legally.

[0056] Example 1

[0057] like Figure 1This embodiment provides a method for identifying defects in steel structure welds, including:

[0058] Acquire images of the welds in the steel structure to be inspected;

[0059] The weld image is input into a trained weld defect recognition network, which includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network performs hierarchical feature extraction on the input image. The feature extraction includes directional feature enhancement of the feature map by embedding a direction-aware attention module, and sequentially extracting pixel-level detail features containing defect edge contours and surface textures, mid-level features that can distinguish different basic defect categories, and high-level semantic features containing defect direction attributes and large-size defect category attributes.

[0060] The neck network receives the extracted features from each layer, performs cross-scale feature fusion through upsampling, downsampling, and feature concatenation, and outputs a fused multi-scale feature map.

[0061] The detection head outputs the type, location, and confidence information of weld defects based on the fused multi-scale feature map.

[0062] This invention constructs a collaborative detection process comprising four core stages: an input layer, a backbone network (feature extraction), a neck network (feature fusion), and a detection head (target prediction). First, the backbone network performs hierarchical feature extraction, enhancing directional features by embedding a direction-aware attention module. This strengthens the network's sensitivity to linear weld defects from the source. Next, the neck network receives features from different levels, fusing shallow features containing details with deep features containing semantics through upsampling, downsampling, and feature concatenation. This ensures that subsequent detection possesses both detail localization and classification capabilities. Finally, the detection head uses the fused multi-scale feature map for prediction. This process design ensures at the system level that the model can capture pixel-level defect traces and understand global defect categories, thereby systematically improving the overall recognition capability for multi-scale, multi-type weld defects.

[0063] Specifically, the pixel-level detail features include at least one of the following: grayscale gradient features of weld fusion lines, edge contour features of cracks and pores, and fish scale and scratch texture features of weld surface.

[0064] The intermediate layer features include at least one of porosity, cracks, and inclusions;

[0065] The high-level semantic features include at least one of crack direction, incomplete penetration defect, and incomplete fusion defect.

[0066] Specifically, the direction-aware attention module is a C2PSA_DCBAM module, and its structure is as follows: Figure 3 As shown, the specific steps for orientation-aware enhancement of feature maps include:

[0067] The input feature map is divided into two parts: one part is passed directly, and the other part is processed by the DCBAM attention module.

[0068] The structural diagram of the DCBAM module is as follows: Figure 5 As shown, it includes a directional enhancer, a channel attention submodule, and a spatial attention submodule. It extracts directional features through depthwise separable convolution and applies attention weights to the channel and spatial dimensions respectively.

[0069] The two processed feature maps are then spliced ​​and fused to obtain the enhanced feature map.

[0070] The module employs a cross-stage part (CSP) structure, dividing the input feature map into two branches. One branch retains the original information to ensure gradient flow; the other branch is fed into the DCBAM attention module for enhancement. DCBAM uses depthwise separable convolutions, whose spatial convolution kernels spontaneously develop sensitivity to edges in different directions (e.g., 0°, 45°, 90°) during training. This allows the module to actively enhance the linear response in the corresponding direction of the input feature map. Subsequently, the channel attention submodule evaluates the importance of directional features in each channel, while the spatial attention submodule evaluates which directional features in the feature map are more critical. The two are then weighted and fused with the original branch features. This mechanism enables the weld defect recognition network to selectively highlight traces of linear defects such as cracks and lack of fusion from complex weld background noise, thereby significantly improving the model's feature extraction capability and detection robustness for such defects.

[0071] Specifically, the neck network employs a lightweight convolutional structure GSConv for feature fusion, including:

[0072] The feature map is input into the GSConv module, which combines standard convolution and depthwise separable convolution, and performs feature information permeation through channel shuffling.

[0073] Output the fused feature map for subsequent upsampling, downsampling, or stitching operations.

[0074] Standard convolutions can fully integrate cross-channel and spatial information, but they are computationally expensive. Depthwise separable convolutions separate channel and spatial computations, offering lightweight performance but insufficient feature interaction. This invention, based on GSConv, first performs a set of lightweight depthwise separable convolutions, while simultaneously performing a set of standard convolutions with half the number of channels in parallel. Then, the feature maps output by the standard convolutions are uniformly inserted into the feature maps output by the depthwise separable convolutions through a channel shuffling operation. Ultimately, GSConv achieves feature representation capabilities close to those of standard convolutions at a computational cost similar to depthwise separable convolutions. Applying it to neck networks with intensive feature fusion requirements can significantly reduce the computational load in this part without almost affecting the fusion effect.

[0075] Specifically, the structure of the weld defect identification network is as follows: Figure 2 As shown, it includes: an input layer, a Conv-1 convolutional layer, a Conv-2 convolutional layer, a C3k2-1 module, a Conv-3 convolutional layer, a C3k2-2 module, a Conv-4 convolutional layer, a C3k2-3 module, a spatial pyramid pooling module, a C2PSA_DCBAM attention module, a lightweight FPN+PAN fusion module, a detection head, and an output layer, all connected in sequence.

[0076] The working principle of the weld defect identification network includes:

[0077] The Conv-1 and Conv-2 convolutional layers perform preliminary feature extraction and downsampling on the input image to obtain an initial feature map. The C3k2-1, C3k2-2, and C3k2-3 modules extract and fuse pixel-level detail features containing defect edge contours and surface textures with mid-level features that can distinguish different basic defect categories through multi-branch residual learning. The spatial pyramid pooling module performs multi-scale pooling on the feature map to expand the receptive field. The C2PSA_DCBAM attention module enhances high-level semantic features containing defect direction attributes and large-size defect category attributes through a direction-aware mechanism. The lightweight FPN+PAN fusion module achieves multi-scale fusion between the pixel-level detail features, mid-level features, and high-level semantic features through upsampling, downsampling, and feature concatenation. The Detect head predicts the category, location, and confidence level of weld defects based on the fused feature map.

[0078] The network structure provided by this invention strictly follows a feature processing logic from fine to coarse and then from coarse to fine. Specifically, after the original image enters the input layer, it undergoes two convolutional operations, Conv-1 and Conv-2, to complete preliminary feature extraction and downsampling, resulting in an initial feature map. This feature map then sequentially enters the C3k2-1 module, the Conv-3 convolutional layer, the C3k2-2 module, the Conv-4 convolutional layer, and the C3k2-3 module. Among them, the C3k2 series modules efficiently extract and fuse local features through their internal multi-branch residual learning structure: the C3k2-1 module mainly produces high-resolution, small receptive field pixel-level detail feature maps (e.g., corresponding to 160×160 resolution), which can accurately capture details such as microcrack edges and micropores; the C3k2-2 and C3k2-3 modules gradually abstract, producing mid-level feature maps with medium receptive fields and reduced resolution (e.g., 80×80), which initially have the ability to distinguish basic defect types such as pores, cracks, and inclusions.

[0079] Subsequently, the feature map enters the SPPF spatial pyramid pooling module, which significantly expands the receptive field through multi-scale pooling, enhancing the ability to represent defects of different sizes. Next, the feature map is processed by the C2PSA_DCBAM attention module, which further refines high-level semantic feature maps (e.g., corresponding to 40×40 resolution) by enhancing orientation awareness and channel spatial attention weighting.

[0080] The three-layer feature maps generated by the backbone network are then fed into the neck network (i.e., the lightweight FPN+PAN fusion module) for cross-scale fusion. The specific fusion path is as follows: First, the deep (P5) high-level semantic feature map is upsampled and concatenated with the mid-layer (P4) feature map, then fused through the C3k2 module to generate a mid-layer fused feature map. Second, this mid-layer fused feature map is upsampled and concatenated with the shallow (P3) pixel-level detail feature map, then fused again through the C3k2 module to generate a shallow fused feature map. Next, this shallow fused feature map is downsampled and concatenated with the mid-layer fused feature map to generate the final mid-layer feature map. Simultaneously, the shallow fused feature map is downsampled again and concatenated with the deep feature map to generate the final deep feature map. The neck network ultimately outputs optimized shallow, mid, and deep three-layer feature maps.

[0081] The detection head makes predictions based on these three layers of final feature maps: the shallow feature map (high resolution) is responsible for detecting small defects (small targets), the medium-sized feature map is responsible for detecting medium-sized defects (medium targets), and the deep feature map (large receptive field) is responsible for detecting large defects (large targets). Each detection branch outputs the target's bounding box coordinates, class probability, and confidence score in parallel through a decoupled head structure. This structural design ensures that feature information is progressively refined, fully exchanged, and rationally utilized from input to output, achieving accurate localization and identification of multi-scale weld defects.

[0082] Regarding the feature extraction mechanism of the weld defect recognition network, its backbone network performs hierarchical feature extraction on the input image. Specifically, in the shallow layer of the network, it focuses on capturing pixel-level low-level visual features, including the gray-scale gradient features of the weld and base material fusion line region to define the weld area; the edge contour features of defects such as cracks and pores to provide a basis for defect localization; and also includes inherent periodic textures such as fish scale patterns on the weld surface and possible noise textures such as scratches. The network needs to learn to distinguish between valid defects and background noise at this stage.

[0083] Subsequently, the feature maps are fed into a deep Spatial Pyramid Pooling (SPPF) module. This module performs parallel multi-scale pooling on the feature maps to achieve multi-receptive field processing of the low-level features, thereby enhancing the network's ability to represent weld defects of different sizes (from micro-pores to large incomplete penetrations) at multiple scales, ensuring that a response is obtained at an appropriate feature scale regardless of the defect size.

[0084] Finally, high-level feature abstraction and enhancement are completed via the C2PSA_DCBAM module. The core DPSABlock call of this module integrates the DCBAM attention mechanism of the orientation enhancer, and its internal PSABlock-DCBAM architecture is as follows: Figure 4 As shown, orientation-aware enhancement is applied to the feature maps after multi-scale pooling to specifically capture the directional structural features of defects such as linear cracks. Key feature channels and spatial locations are selected through dual attention weighting of DCBAM (Distributed Concentration Baseline Amplifier). Then, through nonlinear fusion using the DPSABlock feedforward network and a feature fusion strategy of C2PSA_DCBAM (residual branch and attention branch), the low-level features containing details and the mid-level enhanced features containing context are successfully integrated and further abstracted. This process ultimately enables the network to extract discriminative local semantic features of weld defects (such as defect category) and global semantic information (such as the overall morphology and positional relationship of defects within the weld), laying a solid foundation for subsequent accurate classification and localization.

[0085] Specifically, the recognition network uses Focal-EIoU as the bounding box regression loss function for target localization optimization during training. The Focal-EIoU loss function is expressed as follows:

[0086] ;

[0087] Where IoU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and γ is the focus suppression coefficient. The EIoU loss is calculated using the following formula:

[0088] ;

[0089] in, For prediction boxes Center point of the real frame The square of the Euclidean distance between them To determine the diagonal length of the minimum closed region covering both the predicted and ground truth bounding boxes, , These are the width and height of the predicted bounding box, respectively. , These are the width and height of the actual bounding box, respectively. These are the width and height of the minimum closed region covering the predicted bounding box and the ground truth bounding box, respectively.

[0090] During model training, the EIoU component explicitly decomposes the bounding box regression loss into three terms: center point distance loss, width loss, and height loss. This decompositional optimization directly targets the geometric parameters of the bounding box, especially minimizing the width-to-height difference. This makes the model more directional and convergent when optimizing small target boxes, resolving the optimization ambiguity that may arise from the aspect ratio term in traditional CIoU loss. Secondly, the Focal focusing mechanism reweights the loss through the IoU γ term. For samples with high prediction quality (large IoU), the loss weight is relatively amplified, allowing the model to learn effective localization knowledge. For anomalous samples with low prediction quality (small IoU), the loss weight is suppressed, preventing training from being dominated by a large number of difficult or noisy samples. The combination of these two aspects makes the model training process more stable and efficient, driving the model to produce more accurate, especially tighter, predicted bounding boxes for small defects.

[0091] In one specific embodiment, the dataset of the present invention comes from a publicly available online dataset, covering three common defects: cracks, pores, and spatter. The resolution of the images is uniformly 640*640 pixels, and the training set, validation set, and test set are divided in a 75:12.5:12.5 ratio.

[0092] YOLOv11, released by Ultralytics in 2024, is an object detection model consisting of a backbone, neck, and head. The inventors discovered that directly applying YOLOv11 to weld defect detection suffers from problems such as small target size, imbalanced defect categories, and complex detection environments. This invention modifies the YOLOv11 bounding box loss function to Focal-EIoU, replaces the PSABlock attention mechanism in C2PSA with DCBAM in the backbone, and replaces Conv with GSConv in the neck.

[0093] In object detection, bounding box regression is a crucial step in determining object localization performance. YOLOv11's default bounding box loss function is CIoU, which calculates the loss based on the overlap area between the predicted and ground truth bounding boxes, the distance between their center points, and their aspect ratio. The formula is as follows:

[0094] Total loss: ;

[0095] Center point distance: ;

[0096] in , The coordinates of the center of the predicted bounding box and the true bounding box; It is the diagonal length of the smallest closure box that contains both boxes.

[0097] Items with consistent aspect ratio: ;

[0098] Weighting coefficients: ;

[0099] Although CIoU performs well in most cases, the aspect ratio of CIoU loss is significant for weld defect detection. Measuring only the aspect ratio of the predicted and ground truth bounding boxes without considering their actual width and height leads to unclear optimization direction and slow convergence. Therefore, this invention replaces this with Focal-EIoU. Focal-EIoU introduces the concept of Focal Loss into EIoU, reweighting the original EIoU loss using IoU to ensure that high IoU (high-quality) samples contribute a larger gradient, thus accelerating convergence and improving localization accuracy. EIoU decomposes the width and height loss terms into two independent parts, directly optimizing the width and height differences between the predicted and ground truth bounding boxes. Simultaneously, by introducing a dynamic weighting mechanism, it enhances the gradient contribution of high-quality samples (high IoU values) and suppresses noise from low-quality samples (low IoU values), allowing the model to focus more efficiently on valuable information during training. This not only accelerates convergence but also significantly improves the final localization accuracy.

[0100] ;

[0101] in: ;

[0102] : for prediction boxes Center point of the real frame The square of the Euclidean distance between them.

[0103] : The diagonal length of the smallest closed region covering the predicted bounding box and the ground truth bounding box.

[0104] , These are the width and height of the predicted bounding box, respectively. , These are the width and height of the actual bounding box, respectively.

[0105] These are the width and height of the minimum closed region covering the predicted bounding box and the ground truth bounding box, respectively.

[0106] This is the focus suppression coefficient, which is usually set to 0.5.

[0107] GSConv is a lightweight convolutional architecture that combines standard convolution and depthwise separable convolution. Through channel shuffling, it infuses information generated by standard convolution into the output of depthwise separable convolution. Standard convolution extracts rich features, while depthwise separable convolution reduces the number of parameters and computational complexity. This convolutional structure also avoids the feature representation degradation problem caused by the complete isolation between channels when directly using depthwise separable convolution. GSConv significantly maintains detection accuracy while reducing the number of parameters. Replacing the Conv in the neck region with GSConv achieves the goal of lightweight architecture.

[0108] The C2PSA architecture in the YOLOv11 backbone consists of convolutional layers and an attention module. It achieves efficient feature extraction by combining a cross-stage part (CSP) architecture with a pyramid slice attention mechanism (PSA). C2PSA divides the feature map into two parts: one part is directly passed, and the other part is processed by the PSA attention module, and finally, they are concatenated and fused, balancing shallow details and deep semantics. The module uses multiple PASBlocks for multi-scale feature enhancement. Each PASBlock employs an attention branch and residual connections for addition. While the PASBlock uses a self-attention mechanism that captures global context, its O(n²) complexity is disadvantageous when deployed on edge devices. Therefore, this invention replaces the PASBlock attention mechanism with DCBAM, a CBAM with directional filters. CBAM is a lightweight attention mechanism that combines spatial and channel attention mechanisms, simultaneously adjusting the feature importance of channels and spatial locations, improving the model's perceptual ability and enhancing performance without increasing network complexity. The channel attention module performs global max pooling and global average pooling on the input feature map to extract channel-level feature description vectors. These two modules learn inter-channel dependencies through a shared fully connected layer. The outputs are summed element-wise and then normalized with a sigmoid function to obtain channel attention weights. Finally, the weights are multiplied channel-wise by the original feature map to obtain the attention-weighted channel feature map. The spatial attention module performs max pooling and average pooling along the channel dimension and concatenates the results. After fusion through a convolutional layer, a spatial attention weight map is generated. This weight map is normalized with a sigmoid function and multiplied spatially by the original feature map to achieve spatial feature enhancement. The output features of the channel attention module and the spatial attention module are multiplied element-wise to obtain the final attention-enhanced features. In this invention, a direction enhancer is embedded before the channel attention module in DCBAM. This unit uses depthwise convolution with batch normalization and ReLU. Depthwise convolution decomposes standard convolution into two independent operations: depthwise convolution and pointwise convolution. In directional feature extraction, depthwise convolution employs a channel-independent 3×3 spatial filtering mechanism, enabling each convolutional kernel to autonomously select specific directional sensitivities during training. By avoiding feature mixing between channels, each filter can focus on developing specialized responses to spatial patterns such as horizontal, vertical, or diagonal directions, thus achieving efficient capture of directional features. Batch normalization ensures stable training of directionally selective channels, preventing them from failing due to gradient instability in the early stages of training. The ReLU activation function enhances directional features and suppresses noise through its non-linear properties.It can accurately capture directional features in images with almost zero parameter growth, making it more sensitive to tasks that focus on linear structures, such as weld defect detection.

[0109] The operating system used in this experiment was WSL2. The hardware platform consisted of an i7-13650HX processor, an NVIDIA GeForce RTX 4060 Laptop GPU with 8 GB of video memory, and 11 GB of RAM. The architecture was CUDA 12.1, the framework was PyTorch 2.3.0, the programming tool was PyCharm 2024.3.5, and the language was Python 3.11.13. The SGD optimizer was used with an initial learning rate of 0.01 and cosine annealing, with a weight decay of 5 × 10⁻⁶. -4 The input images were uniformly adjusted to a size of 640×640 pixels, with a batch size of 16, and a total training duration of 200 epochs. As shown in Table 1, the ablation experiment results indicate that the network of this invention improved recall by 2.8% and mAP by 1.1% compared to the original YOLOv11, while reducing the number of parameters by 5.3%. The experimental results demonstrate that the model of this invention achieves lightweight design, improving recall and reducing false negatives without compromising accuracy. It also exhibits good real-time performance and is suitable for deployment in resource-constrained environments.

[0110]

[0111] The visualization results of the confusion matrix of the ablation experiment are as follows: Figures 6 to 9 As shown. Figure 6 and Figure 7 The diagram shows the first and second confusion matrices of the original model under different conditions, as follows: Figure 8 and Figure 9 The diagram shows the first and second confusion matrices of the improved model under different conditions. The confusion matrix is ​​used to represent the classification accuracy of a network structure model and is a common method for visualizing detection results. In the figure, the horizontal axis represents the predicted label, the vertical axis represents the true label, the numbers on the diagonal represent the number of correctly predicted images, and the numbers off the diagonal represent the number of incorrectly predicted images and their corresponding true labels. The comparison clearly shows that the improved model has increased classification accuracy.

[0112] Example 2

[0113] like Figure 10 This embodiment provides a steel structure weld defect identification system, including:

[0114] Image acquisition unit, used to acquire images of the weld seams of the steel structure to be inspected;

[0115] The feature extraction unit includes a backbone network for performing hierarchical feature extraction on the input image. The feature extraction includes directional feature enhancement of the feature map by embedding a direction-aware attention module, and sequentially extracting pixel-level detail features containing defect edge contours and surface textures, mid-level features that can distinguish different basic defect categories, and high-level semantic features containing defect direction attributes and large-size defect category attributes.

[0116] The feature fusion module, including the neck network, is used to receive the extracted features from each layer, perform cross-scale feature fusion through upsampling, downsampling and feature concatenation, and output the fused multi-scale feature map.

[0117] The defect detection unit includes a detection head, which is used to output the category, location, and confidence information of weld defects based on the fused multi-scale feature map.

[0118] Example 3

[0119] Embodiment 3 of the present invention provides an electronic device.

[0120] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the steel structure weld defect identification method as described in Embodiment 1 of the present invention.

[0121] The detailed steps are the same as those for the steel structure weld defect identification method provided in Example 1, and will not be repeated here.

[0122] Example 4

[0123] Embodiment 4 of the present invention provides a computer-readable storage medium.

[0124] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the steel structure weld defect identification method as described in Embodiment 1 of the present invention.

[0125] The detailed steps are the same as those for the steel structure weld defect identification method provided in Example 1, and will not be repeated here.

[0126] Example 5

[0127] Embodiment 5 of the present invention provides a computer program product.

[0128] A computer program product includes software code, wherein the program in the software code performs the steps of the steel structure weld defect identification method as described in Embodiment 1 of the present invention.

[0129] The detailed steps are the same as those for the steel structure weld defect identification method provided in Example 1, and will not be repeated here.

[0130] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0131] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for identifying weld defects in steel structures, characterized in that, include: Acquire images of the welds in the steel structure to be inspected; The weld image is input into a trained weld defect recognition network, which includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network performs hierarchical feature extraction on the input image. The feature extraction includes directional feature enhancement of the feature map by embedding a direction-aware attention module, and sequentially extracting pixel-level detail features containing defect edge contours and surface textures, mid-level features that can distinguish different basic defect categories, and high-level semantic features containing defect direction attributes and large-size defect category attributes. The direction-aware attention module is a C2PSA_DCBAM module, and its specific steps for enhancing the direction awareness of the feature map include: The input feature map is divided into two parts: one part is passed directly, and the other part is processed by the DCBAM attention module. The DCBAM module includes a direction enhancer, a channel attention submodule, and a spatial attention submodule. It extracts directional features through depthwise separable convolution and applies attention weights to the channel and spatial dimensions respectively. The two processed feature maps are then concatenated and merged to obtain the enhanced feature map. The neck network receives and extracts features from each layer, and performs feature fusion using a lightweight convolutional structure GSConv, specifically including: The feature map is input into the GSConv module, which combines standard convolution and depthwise separable convolution, and performs feature information permeation through channel shuffling. Output the fused feature map for subsequent upsampling, downsampling, or stitching operations; Cross-scale feature fusion is performed through upsampling, downsampling, and feature concatenation, and the fused multi-scale feature map is output. The detection head outputs the type, location, and confidence information of weld defects based on the fused multi-scale feature map; The recognition network uses Focal-EIoU as the bounding box regression loss function for target localization optimization during training. The Focal-EIoU loss function is expressed as: ； Where IoU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and γ is the focus suppression coefficient. The EIoU loss is calculated using the following formula: ； in, For prediction boxes Center point of the real frame The square of the Euclidean distance between them To determine the diagonal length of the minimum closed region covering both the predicted and ground truth bounding boxes, , These are the width and height of the predicted bounding box, respectively. , These are the width and height of the actual bounding box, respectively. These are the width and height of the minimum closed region covering the predicted bounding box and the ground truth bounding box, respectively.

2. The method as described in claim 1, characterized in that, The pixel-level detail features include at least one of the following: grayscale gradient features of weld fusion lines, edge contour features of cracks and pores, and fish scale and scratch texture features of weld surface. The intermediate layer features include at least one of porosity, cracks, and inclusions; The high-level semantic features include at least one of crack direction, incomplete penetration defect, and incomplete fusion defect.

3. The method as described in claim 1, characterized in that, The structure of the weld defect recognition network includes: an input layer, a Conv-1 convolutional layer, a Conv-2 convolutional layer, a C3k2-1 module, a Conv-3 convolutional layer, a C3k2-2 module, a Conv-4 convolutional layer, a C3k2-3 module, a spatial pyramid pooling module, a C2PSA_DCBAM attention module, a lightweight FPN+PAN fusion module, a detection head, and an output layer, all connected in sequence. The working principle of the weld defect identification network includes: The Conv-1 and Conv-2 convolutional layers perform preliminary feature extraction and downsampling on the input image to obtain an initial feature map. The C3k2-1, C3k2-2, and C3k2-3 modules extract and fuse pixel-level detail features containing defect edge contours and surface textures with mid-level features that can distinguish different basic defect categories through multi-branch residual learning. The spatial pyramid pooling module performs multi-scale pooling on the feature map to expand the receptive field. The C2PSA_DCBAM attention module enhances high-level semantic features containing defect direction attributes and large-size defect category attributes through a direction-aware mechanism. The lightweight FPN+PAN fusion module achieves multi-scale fusion between the pixel-level detail features, mid-level features, and high-level semantic features through upsampling, downsampling, and feature concatenation. The Detect head predicts the category, location, and confidence level of weld defects based on the fused feature map.

4. A steel structure weld defect identification system, characterized in that, include: Image acquisition unit, used to acquire images of the weld seams of the steel structure to be inspected; The feature extraction unit includes a backbone network for performing hierarchical feature extraction on the input image. The feature extraction includes directional feature enhancement of the feature map by embedding a direction-aware attention module, and sequentially extracting pixel-level detail features containing defect edge contours and surface textures, mid-level features that can distinguish different basic defect categories, and high-level semantic features containing defect direction attributes and large-size defect category attributes. The direction-aware attention module is a C2PSA_DCBAM module, and its specific steps for enhancing the direction awareness of the feature map include: The input feature map is divided into two parts: one part is passed directly, and the other part is processed by the DCBAM attention module. The DCBAM module includes a direction enhancer, a channel attention submodule, and a spatial attention submodule. It extracts directional features through depthwise separable convolution and applies attention weights to the channel and spatial dimensions respectively. The two processed feature maps are then concatenated and merged to obtain the enhanced feature map. The feature fusion module, including the neck network, receives the extracted features from each layer and uses a lightweight convolutional structure GSConv for feature fusion. Specifically, it includes: The feature map is input into the GSConv module, which combines standard convolution and depthwise separable convolution, and performs feature information permeation through channel shuffling. Output the fused feature map for subsequent upsampling, downsampling, or stitching operations; Cross-scale feature fusion is performed through upsampling, downsampling, and feature concatenation, and the fused multi-scale feature map is output. The defect detection unit includes a detection head, which is used to output the type, location, and confidence information of weld defects based on the fused multi-scale feature map. The system uses Focal-EIoU as the bounding box regression loss function for target localization optimization during training. The Focal-EIoU loss function is expressed as follows: ； Where IoU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and γ is the focus suppression coefficient. The EIoU loss is calculated using the following formula: ； in, For prediction boxes Center point of the real frame The square of the Euclidean distance between them To determine the diagonal length of the minimum closed region covering both the predicted and ground truth bounding boxes, , These are the width and height of the predicted bounding box, respectively. , These are the width and height of the actual bounding box, respectively. These are the width and height of the minimum closed region covering the predicted bounding box and the ground truth bounding box, respectively.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the steel structure weld defect identification method as described in any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the steel structure weld defect identification method as described in any one of claims 1 to 3.

7. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of the steel structure weld defect identification method as described in any one of claims 1 to 3.

Images