Weld surface defect detection method and system

Abstract

The application discloses a weld surface defect detection method and system, and relates to the technical field of image recognition. The method comprises the following steps: acquiring a weld image to be detected; inputting the weld image to be detected into a preset weld defect detection model to obtain a defect detection result. The preset weld defect detection model comprises a preset backbone network, a preset neck network and a preset head network. The preset backbone network and the preset neck network each comprise a plurality of lightweight C2f modules that replace C2f modules in an original YOLOv8n model. Therefore, the model parameter quantity and the calculation quantity can be reduced to reduce the storage and calculation resource requirements of the model. In addition, the lightweight C2f modules are constructed based on a lightweight convolution layer and a double-channel attention mechanism. Therefore, the model detection accuracy for small target defects can be improved, and the missed detection rate can be effectively reduced.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method and system for detecting surface defects in welds. Background Technology

[0002] In modern industrial production, automated welding technology is widely used, and weld quality directly affects product safety and reliability. Weld surface defect detection, as a key aspect of welding quality control, is crucial for ensuring product quality and production efficiency. Traditional manual inspection methods are not only inefficient but also susceptible to subjective factors, making them unsuitable for large-scale production. Therefore, computer vision-based automated inspection technology is gradually becoming the mainstream approach to solving this problem.

[0003] However, existing weld surface defect detection technologies still face many challenges in practical applications: On the one hand, high-precision detection models typically require a large amount of computing resources and storage space, making them difficult to deploy and run on resource-constrained embedded devices, thus limiting their widespread application in industrial settings. On the other hand, defects generated during the welding process, especially small-target defects such as porosity, are easily ignored by the detection model due to their low pixel proportion in the image, resulting in a high false negative rate. Summary of the Invention

[0004] The main purpose of this application is to provide a method and system for detecting surface defects in welds, which aims to solve the technical problems of large computational load and low detection accuracy in existing weld surface defect detection technologies.

[0005] To achieve the above objectives, this application proposes a method for detecting surface defects in welds, the method comprising: Acquire images of the weld to be inspected; The image of the weld to be detected is input into a preset weld defect detection model to obtain the defect detection result. The preset weld defect detection model includes a preset backbone network, a preset neck network, and a preset head network. The preset backbone network and the preset neck network each include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

[0006] In one embodiment, the step of inputting the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result includes: The preset backbone network is used to extract features from the weld seam image to be detected to obtain a scale feature map. The scale feature map is fused using the preset neck network to obtain a fused feature map; The fused features are mapped using the preset head network to obtain defect detection results.

[0007] In one embodiment, the preset backbone network further includes a lightweight convolutional layer module, wherein the lightweight C2f module includes a CBS layer and a lightweight bottleneck layer; The step of extracting features from the weld seam image to be detected through the preset backbone network to obtain a scale feature map includes: The lightweight convolutional layer module is used to extract features and perform adaptive feature filtering on the weld seam image to be detected, thereby obtaining an input feature map; A shallow feature map is obtained by performing shallow feature extraction on the input feature map through the CBS layer; Deep feature maps are obtained by performing deep feature extraction on the shallow feature map through the lightweight bottleneck layer. The shallow feature map and the deep feature map are spliced ​​together to obtain a scale feature map.

[0008] In one embodiment, the step of extracting features from the weld seam image to be detected and adaptively filtering features through the lightweight convolutional layer module to obtain an input feature map includes: The image of the weld to be detected is subjected to feature extraction based on the original convolution to obtain an initial feature map; Based on cosine similarity, the initial feature map is filtered to obtain a difference feature map, and a linear operation is performed on the difference feature map; The initial feature map and the differential feature map after linear operation are concatenated to obtain the input feature map.

[0009] In one embodiment, the lightweight bottleneck layer includes: a first lightweight convolutional layer, a dual-channel attention layer, and a second lightweight convolutional layer; The step of extracting deep features from the shallow feature map through the lightweight bottleneck layer to obtain a deep feature map includes: The shallow feature map is up-dimensioned through a first lightweight convolutional layer to obtain a high-dimensional feature map. The high-dimensional feature map is weighted by dual-channel attention layer to obtain a weighted feature map. The weighted feature map is reduced in dimension by channel through the second lightweight convolutional layer to obtain a dimension-reduced feature map. The dimension-reduced feature map is then added to the shallow feature map to obtain a deep feature map.

[0010] In one embodiment, the step of performing dual-channel attention weighting on the high-dimensional feature map through the dual-channel attention layer to obtain a weighted feature map includes: Channel dimension features are extracted from the high-dimensional feature map to obtain key channel feature information, and channel attention weights are determined based on the key channel feature information. Spatial dimension features are extracted from the high-dimensional feature map to obtain key spatial feature information, and a spatial attention map is generated based on the key spatial feature information. The channel attention map and the spatial attention map are multiplied element-wise with the high-dimensional feature map to obtain a weighted feature map.

[0011] In one embodiment, before the step of inputting the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result, the following steps are included: A set of weld surface defect images is obtained, and weld defects are annotated on the set of weld surface defect images to obtain a weld surface defect dataset. Based on the weld surface defect dataset, the initialized weld defect detection model is trained according to the Mosaic data augmentation strategy to obtain the trained weld defect detection model as the preset weld defect detection model.

[0012] In one embodiment, the step of training the initialized weld defect detection model based on the weld surface defect dataset according to the Mosaic data augmentation strategy includes: In each training batch, several sample images are extracted from the weld surface defect dataset according to the Mosaic data augmentation strategy and subjected to random transformation processing to obtain the sample images of the current batch. Defect detection is performed on the current round of sample images using an initialized weld defect detection model to obtain a prediction box; The loss is calculated for the predicted bounding box and the labeled bounding box corresponding to the current round sample image according to the preset loss function to obtain the loss value of the current round; The model parameters of the initialized weld defect detection model are updated based on the current loss value.

[0013] In one embodiment, the preset loss function is an adaptive loss function that replaces the standard loss function in the original YOLOv8n model. The adaptive loss function is obtained by adjusting the weights of the standard loss function according to a dynamic adjustment factor.

[0014] Furthermore, to achieve the above objectives, this application also proposes a weld surface defect detection system, the system comprising: The image acquisition module is used to acquire images of the weld seam to be inspected. The defect detection module is used to input the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result. The preset weld defect detection model includes a preset backbone network, a preset neck network, and a preset head network. The preset backbone network and the preset neck network each include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

[0015] This application discloses a method for detecting surface defects in welds, comprising: acquiring an image of a weld to be detected; inputting the image of the weld to be detected into a preset weld defect detection model to obtain defect detection results. The preset weld defect detection model includes: a preset backbone network, a preset neck network, and a preset head network. Both the preset backbone network and the preset neck network include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

[0016] Because this application's pre-defined weld defect detection model replaces the original YOLOv8n model's C2f module with a lightweight C2f module, this module, through the combination of lightweight convolutional layers and a dual-channel attention mechanism, effectively enhances the ability to extract surface defect features of welds while reducing the number of model parameters and computational load. This not only reduces the model's storage and computational resource requirements but also improves the model's detection accuracy for small target defects through the dual-channel attention mechanism, reducing the false negative rate. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the first embodiment of the weld surface defect detection method of this application; Figure 2 This is a schematic diagram of the model structure of the pre-set weld defect detection model in this application; Figure 3 This is a schematic diagram of the lightweight C2f module. Figure 4 This is a flowchart illustrating the second embodiment of the weld surface defect detection method of this application; Figure 5 This is a schematic diagram illustrating the AGF module feature map generation process. Figure 6 This is a schematic diagram of the CSAM attention module. Figure 7 This is a flowchart illustrating the third embodiment of the weld surface defect detection method of this application; Figure 8 A performance comparison table of other detection models and YOLOv8n-AGF on the weld surface defect dataset; Figure 9 This is a schematic diagram of the module structure of the weld surface defect detection system of this application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] This application provides a method for detecting surface defects in welds, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the weld surface defect detection method of this application. In this embodiment, the method includes: steps S10~S20: Step S10: Obtain the image of the weld to be inspected.

[0024] It should be noted that the execution subject of the method in this embodiment can be a computing service device with image acquisition, data processing, model calling, network communication, and program execution functions, such as a mobile phone, personal computer, weld defect detector, etc., or other electronic devices equipped with an embedded AI development board (such as Jetson Orin Nano), such as a weld defect detection robot, a central processing module in a weld defect detection platform, etc. This embodiment uses a weld surface defect detection device (hereinafter referred to as "device") as an example to illustrate this embodiment and the following embodiments.

[0025] It should be understood that the weld image to be inspected can be a real-time captured weld image, or a historical image that needs to be captured for defect detection and obtained from the network or the device cache. This embodiment does not limit this.

[0026] It should also be noted that in industrial scenarios, industrial cameras can be used to capture raw weld seam images of the scene, and algorithms such as Global Histogram Equalization (GHE) can be used to perform grayscale preprocessing on the captured raw weld seam images to obtain the weld seam images to be inspected.

[0027] The grayscale preprocessing steps include: The original weld seam image captured by the industrial camera is an RGB image. Therefore, when processing with GHE, the color image is first converted to the YUV color space. The YUV color space can clearly separate luminance information (Y channel) from chrominance information (U and V channels). This characteristic is of great significance for the detection of surface defects in weld seams, because common defects in weld seam surfaces, such as cracks and porosity, are closely related to luminance changes and show obvious differences in luminance compared to normal areas.

[0028] After color space conversion, GHE processing was performed only on the Y channel. This significantly enhanced the contrast of the Y channel, and markedly improved the distinguishability of weld edges, textures, and other features in the brightness dimension. Previously difficult-to-identify subtle features, such as the edges of tiny cracks and the outlines of minute pores, became much clearer, providing a superior image foundation for subsequent high-precision weld surface defect detection.

[0029] Step S20: Input the image of the weld to be detected into the preset weld defect detection model to obtain the defect detection result. The preset weld defect detection model includes: a preset backbone network, a preset neck network, and a preset head network. The preset backbone network and the preset neck network each include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

[0030] It should be noted that this preset weld defect detection model is an improvement upon the original YOLOv8n model. Specifically, the improvements are as follows: Lightweight C2f (C2f-AGF) modules replace the original C2f modules in the backbone network and neck feature extraction network, and lightweight convolutional layer (AGF) modules replace the standard convolutional layer modules. Hereafter, YOLOv8n-AGF can be used to refer to this preset weld defect detection model. See here for reference. Figure 2 This demonstrates the YOLOv8n-AGF. Figure 2 This is a schematic diagram of the model structure for a preset weld defect detection model.

[0031] It should be understood that the AGF module is built based on the Adaptive Ghost Filter (AGF). By setting a dynamic scaling factor and a cosine similarity-based feature selection strategy, it can effectively reduce redundant parameters while retaining key feature information, thereby reducing the computational burden and storage requirements of the model. Compared to standard convolutional layer modules, it can significantly reduce the number of model parameters and computational cost while maintaining feature extraction performance.

[0032] The C2f-AGF module is also built based on AGF. It features a lightweight bottleneck layer (AGF-Bottleneck) structure designed based on AGF. This structure is mainly composed of two stacked AGF modules, with a lightweight dual attention (CSAM) mechanism embedded between the two AGF modules to guide the model to focus more on the effective features of the target during feature extraction.

[0033] Furthermore, to illustrate the defect detection process implemented by YOLOv8n-AGF, step S20 specifically includes: steps S201~S203: Step S201: Extract features from the weld seam image to be detected using the preset backbone network to obtain a scale feature map.

[0034] Step S202: Perform feature fusion on the scale feature map through the preset neck network to obtain a fused feature map.

[0035] Step S203: Perform feature mapping on the fused features through the preset head network to obtain defect detection results.

[0036] It should be understood that, based on the above... Figure 2 It can be seen that the weld defect detection process of YOLOv8n-AGF is consistent with that of the original YOLOv8n model: the input weld image is first processed by the backbone network to extract features and generate feature maps of three different scales; then, the feature maps of different scales are fused across layers in the neck network to enhance the transmission of deep semantic information and positional information; finally, the feature maps generated by the neck network are fed into the detection head to complete target prediction and regression, and output the final detection result.

[0037] The detection result can be the corresponding probability of different defect types, or it can output only the defect type with the highest probability.

[0038] To illustrate the roles of the C2f-AGF module and the AGF module in YOLOv8n-AGF, we can use the C2f-AGF module and the AGF module in the preset backbone network as an example. The roles of the C2f-AGF module and the AGF module in the preset neck network can be found in the description of the preset backbone network. This embodiment will not elaborate further on these details. Therefore, step S201 specifically includes: steps S2011~S2014: Step S2011: The lightweight convolutional layer module is used to extract features and perform adaptive feature filtering on the weld seam image to be detected to obtain an input feature map.

[0039] It should be noted that the AGF module can adaptively adjust the feature map generation method by setting the channel scaling factor and using the cosine similarity-based feature map selection (FMS) strategy.

[0040] It should be understood that the AGF module first extracts features from the original input using standard convolutional layers to generate feature maps; then, it uses a cosine similarity-based feature map selection strategy to filter the generated feature maps, retaining those with greater differences; finally, it concatenates the feature maps with greater differences to the initially generated feature maps to obtain the input feature map. This effectively reduces the number of parameters while maintaining the model's feature extraction capabilities, and retains more differential feature information.

[0041] Step S2012: Perform shallow feature extraction on the input feature map through the CBS layer to obtain a shallow feature map.

[0042] It should be noted that the C2f-AGF module can be constructed based on two CBS layers and several AGF-Bottlenecks located between the two CBS layers. Each AGF-Bottleneck is composed of two AGF modules and a lightweight CSAM attention mechanism embedded between the two AGF modules. The specific module structure of the C2f-AGF module is as follows: Figure 3 As shown, Figure 3 This is a schematic diagram of the lightweight C2f module.

[0043] It should be understood that the CBS layer is a component of the C2f module in the original YOLOv8n model. It typically consists of a convolutional layer (Conv2d), a batch normalization layer (BatchNorm2d), and an activation function (such as SiLU) for convolutional transformation and channel adjustment of features.

[0044] Depend on Figure 3As can be seen, the C2f-AGF module can first use the CBS layer to process the input feature map and extract shallow feature information (such as contour information). The generated feature map is divided into two branches: the gradient flow main branch and the secondary branch.

[0045] Step S2013: Extract deep features from the shallow feature map using the lightweight bottleneck layer to obtain a deep feature map.

[0046] Step S2014: The shallow feature map and the deep feature map are spliced ​​together to obtain a scale feature map.

[0047] Also by Figure 3 As can be seen, in the main branch of gradient flow, multiple AGF-Bottlenecks work sequentially to extract deep feature information (such as texture and local shape) at multiple scales and in multiple directions. Compared with the C2f module in the original YOLOv8n model, the lightweight AGF-Bottleneck in this C2f-AGF module can effectively reduce the number of model parameters and computational cost.

[0048] In the sub-branch, the feature maps of all sub-branch can be directly passed to the final concatenation block (Concat) in the C2f-AGF module for concatenation. This part of the feature map retains the relatively original feature information (shallow feature map), ensuring that the basic features are not lost. Finally, a CBS layer can be used to further fuse the shallow feature map and the deep feature map to refine and enrich the feature expression, forming a feature map with rich feature information, thereby improving the model's ability to detect defects at different scales.

[0049] In its specific implementation, the preset backbone network and the preset neck network in YOLOv8n-AGF implement a feature extraction and feature fusion process with low computational cost, high efficiency and high precision based on the C2f-AGF module and the AGF module. The fused features are obtained and feature mapping is performed in the preset neck network. Finally, the detection head outputs the target prediction result, which is the probability of the defect type of the weld image to be detected. The defect type can include: weld bead, incomplete weld, weld with incorrect size, arc crater, surface porosity of weld and normal weld, etc.

[0050] This embodiment proposes a pre-defined weld defect detection model (YOLOv8n-AGF) improved from the original YOLOv8n model. YOLOv8n-AGF replaces the C2f module in the original YOLOv8n model with a C2f-AGF module. The C2f-AGF module, through the combination of lightweight convolutional layers (AGF) and dual-channel attention mechanism (CSAM), can effectively enhance the ability to extract surface defect features of welds while reducing the number of model parameters and computational cost, thereby reducing the false negative rate.

[0051] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the weld surface defect detection method of this application.

[0052] In this embodiment, to specifically illustrate the feature selection process of the AGF module, step S2011 includes: steps S20111~S20113: Step S20111: Perform feature extraction based on the original convolution on the weld seam image to be detected to obtain an initial feature map.

[0053] It should be understood that the original convolution is the standard convolutional layer. The ratio of the number of feature maps generated by the AGF module to the number of feature maps generated based on the original convolution can be expressed as the channel scaling factor s, which is calculated by the following formula:

[0054] Where m represents the number of feature maps generated by the standard convolutional layer, and n is the number of feature maps finally output by the AGF module.

[0055] Historical experimental data shows that the model achieves good results when s is 1 or 2. To avoid the impact of random selection on model performance stability, this embodiment uses the FMS strategy in the AGF module, which adaptively selects x feature maps with the lowest similarity from the feature maps generated by m standard convolutional layers.

[0056] It should be noted that this can be used as a reference. Figure 5 The method for generating feature maps for the AGF module is explained. Figure 5 This is a schematic diagram of the AGF module feature map generation process.

[0057] Depend on Figure 5 As can be seen, for the input feature map (input). First, a standard convolutional layer can be used for feature extraction to obtain an initial feature map. The initial feature map can be represented as follows:

[0058] in, Indicates the convolution kernel used. The size of the convolution kernel. The number of input channels for a regular convolution. This represents the number of convolutional kernels, i.e., the number of feature maps generated by a standard convolutional layer. The value of m varies according to a preset value of s. ).

[0059] Step S20112: Filter the initial feature map based on cosine similarity to obtain a difference feature map, and perform a linear operation on the difference feature map.

[0060] Step S20113: Concatenate the initial feature map and the differential feature map after linear operation to obtain the input feature map.

[0061] It should be understood that, reference Figure 5 Generating using standard convolutional layers Then, a cosine similarity-based feature map selection strategy (FMS) can be used to filter the m feature maps, retaining more feature maps with high dissimilarity to determine the subset of feature maps used for subsequent linear operations. Assuming there are K rounds of filtering, each round removes... Among the feature maps with the highest similarity, The calculation formula is as follows:

[0062] The selection process begins with an initial set of m input feature maps. In the i-th round of selection, the number of feature maps... Calculate using the following formula:

[0063] The specific screening process is as follows: First, from... Randomly select a baseline feature map A from the feature maps. For any other feature map besides A... (j is the feature map number, and) The cosine similarity between the two can be calculated using the following formula.

[0064]

[0065] In the formula, This represents the dot product operation of two feature map vectors. The product of the norms of two feature map vectors is used to measure the similarity between the feature maps. After ranking by similarity, the t most similar feature maps are removed. After K iterations, the remaining feature maps are... The feature map participates in linear operations. The final output of the AGF module is... As shown in the following formula:

[0066] In the formula, This represents the x-th feature map selected from a set of m input feature maps. This indicates performing a linear operation on the x-th feature map; Concat indicates converting the feature map... With feature map The pieces are stitched together along the channel dimension.

[0067] It should be noted that the AGF module can adaptively adjust the feature map generation method by setting the channel scaling factor and the cosine similarity-based feature map selection strategy (FMS). This effectively reduces the number of parameters while ensuring the model's feature extraction capabilities, which helps to reduce the model's computational burden and storage requirements. It also obtains input feature maps that retain more differential feature information, enabling efficient feature extraction and processing, and improving the model's running efficiency and detection performance.

[0068] Furthermore, the lightweight bottleneck layer can be further divided into: a first lightweight convolutional layer, a dual-channel attention layer, and a second lightweight convolutional layer; wherein the first and second lightweight convolutional layers have the same structure as the aforementioned lightweight convolutional layer module, differing only in name and location within the structure. And based on the aforementioned... Figure 3 It can be seen that the first and second lightweight convolutional layers in AGF-Bottleneck can still be represented as AGF. Therefore, to specifically illustrate the feature extraction process of the lightweight bottleneck layer (AGF-Bottleneck) in the main branch of gradient flow, step S2013 specifically includes: steps S20131~S20133: Step S20131: Upscale the shallow feature map through a first lightweight convolutional layer to obtain a high-dimensional feature map.

[0069] It should be understood that the first lightweight convolutional layer, namely the first AGF module in AGF-Bottleneck, can increase the channel dimension of the input features. By increasing the number of channels, richer feature information can be extracted, such as subtle texture changes on the weld surface and the contour features of tiny pores.

[0070] Step S20132: Apply dual-channel attention weighting to the high-dimensional feature map through the dual-channel attention layer to obtain the weighted feature map.

[0071] It should be understood that the dual-channel attention layer is constructed based on the CSAM attention module structure, and the CSAM attention module can be used to refer to this dual-channel attention layer in the following text. The CSAM attention mechanism can weight features, enhancing the model's feature representation capability by jointly generating channel attention and spatial attention. Within the vast amount of information contained in high-dimensional feature maps, this mechanism helps identify and enhance important features closely related to weld surface defects, while suppressing redundant information and guiding the model to focus on key features.

[0072] To illustrate the application of the CSAM attention mechanism in detail, please refer to [link / reference needed]. Figure 6To explain, Figure 6 This is a schematic diagram of the CSAM attention module.

[0073] Depend on Figure 6 It can be seen that the CSAM attention module can perform parallel processing of the input feature map in both channel and spatial dimensions to effectively extract and enhance the surface defect features of the weld, thereby improving the model's detection performance.

[0074] Here, SMP (Spatial Maximum Pooling) and SAP (Spatial Average Pooling) represent maximum pooling and average pooling in the spatial dimension, respectively; CMP (Channel Maximum Pooling) and CAP (Channel Average Pooling) represent maximum pooling and average pooling in the channel dimension, respectively.

[0075] In channel dimension processing, the size is... The high-dimensional feature maps were processed by SMP and SAP respectively to generate two 1 tensor and SMP can keenly capture salient features in the spatial location of weld images. In critical areas prone to defects, such as weld edges and corners, defects often exhibit unique characteristic responses. SMP can enhance these features, making them more prominent in subsequent processing stages. SAP, on the other hand, smooths these features to suppress local fluctuations caused by factors such as equipment noise and environmental interference.

[0076] In obtaining and Afterwards, and Element-wise addition is performed, and the spatial information extracted by the two pooling methods is combined to obtain a tensor. , It includes key features of the spatial dimension and overall spatial information of the weld. Then... Applying adaptive one-dimensional convolution offers significant advantages over ordinary two-dimensional convolution in terms of parameter count and computational complexity: it performs local cross-channel interactions only along the channel dimension, greatly reducing the number of parameters and computational complexity. In this way, while reducing computational burden, it effectively captures the dependencies between channels, achieving local cross-channel interactions across k channels and generating tensors containing rich channel information. .

[0077] Next, Inputting the sigmoid activation function yields the channel attention weights. This weight reflects the importance of each channel in the detection task. The original input feature map is then compared with the channel attention weights. Element-wise multiplication outputs feature maps This helps the model perceive potential defect information on the weld surface. For example, specific channels may contain feature information closely related to surface defects such as weld porosity and weld beads. This information can be obtained through channel attention weights. This can significantly enhance the signals of these defect-related channels while suppressing irrelevant channels, guiding the model to focus on information valuable for surface defect detection.

[0078] Regarding spatial dimension processing, CMP and CAP can be used to process high-dimensional feature maps and generate two two-dimensional features. and CMP highlights key features of a channel by selecting the maximum value at each spatial location in the channel dimension. Defect features may manifest as obvious grayscale changes or texture features in a specific channel. CMP operations can enhance the features of the spatial location where these defects are located. CAP, on the other hand, smooths the channel information by calculating the average value of the channel dimension to reduce the interference of noise and outliers.

[0079] and After dimensionality reduction by convolutional layers, the following is obtained: The dimensionality reduction operation retains key spatial information while significantly reducing the number of channels, thus avoiding the complex calculations caused by too many channels and highlighting important spatial features related to weld surface defects.

[0080] Subsequently, Spatial attention map generated using sigmoid activation function This figure quantifies the importance of each spatial location for surface defect detection and enhances the characteristic response of weld surface defect areas (such as weld beads and porosity).

[0081] Ultimately, the CSAM attention module will... and Element-wise multiplication is performed to form the final feature map with dual attention weighting. This process enables the model to extract comprehensive information from the input feature map. This allows the model to not only accurately identify weld surface defects but also precisely locate the defects, effectively improving the accuracy and reliability of weld surface defect detection.

[0082] In the specific implementation, the CSAM attention module can extract channel dimension features from the high-dimensional feature map to obtain key channel feature information, and determine the channel attention weights based on the key channel feature information; at the same time, it can extract spatial dimension features from the high-dimensional feature map to obtain key spatial feature information, and generate a spatial attention map based on the key spatial feature information; finally, it can multiply the channel attention map and the spatial attention map with the high-dimensional feature map element by element to obtain the weighted feature map.

[0083] Step S20133: Perform channel dimensionality reduction on the weighted feature map through the second lightweight convolutional layer to obtain a dimensionality-reduced feature map, and add the dimensionality-reduced feature map to the shallow feature map to obtain a deep feature map.

[0084] It should be understood that the second lightweight convolutional layer, namely the second AGF module in AGF-Bottleneck, can reduce the dimensionality of the channel count, thus simplifying the data dimension while preserving key features.

[0085] Finally, the AGF-Bottleneck structure can use residual connections to add the input features of the first AGF module to the output features of the second AGF module, forming the final output feature map, i.e., the deep feature map. This ensures the effective transmission of gradient information, avoids the vanishing or exploding gradient problem, and enables the model to learn deep feature information more efficiently.

[0086] It should be noted that AGF-Bottleneck uses a "dimensionality increase-attention weighting-dimensionality reduction" design strategy. It achieves adaptive feature selection through the attention mechanism, while using residual connections to ensure the effective transmission of gradient information, enabling the model to learn deep feature information more efficiently and improve the model's expressive power.

[0087] In this embodiment, the model employs the AGF module for feature extraction. By setting a dynamic scaling factor and a cosine similarity-based feature selection strategy, it can effectively reduce redundant parameters while retaining key feature information, thereby reducing the computational burden and storage requirements of the model. This achieves efficient feature extraction and processing, improving the model's running efficiency and detection performance. Furthermore, the AGF module and the CSAM module are introduced into the original Bottleneck structure to obtain the AGF-Bottleneck. During feature extraction based on this, the first AGF module increases the number of channels to extract richer features; the CSAM module performs weighted processing on the features, enhancing key features and suppressing redundant information; the second AGF module reduces the number of channels, simplifying the data dimensions; and finally, residual connections are used to ensure the effective transmission of gradient information. This approach maintains the model's detection performance while maintaining a lightweight design, improving the model's ability to detect small target defects.

[0088] Based on the first and second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 7 , Figure 7 This is a flowchart illustrating the third embodiment of the weld surface defect detection method of this application.

[0089] In this embodiment, to specifically illustrate how to construct the preset weld defect detection model, before step S20, the following steps are included: S01~S02: Step S01: Obtain a set of weld surface defect images and annotate the weld surface defect images to obtain a weld surface defect dataset.

[0090] It should be noted that this collection of weld surface defect images can come from open-source public datasets for weld surface defect detection and classification, such as LoHi-WELD. LoHi-WELD covers four common weld surface defect types: porosity, weld beads, discontinuous welds, and blemishes, and is divided into two subsets based on differences in exposure conditions, image resolution, and camera configuration: lweld (low-resolution welds) and hweld (high-resolution welds).

[0091] In addition, the collection of weld surface defect images can also include pipe weld surface defect images collected and constructed by the user. The user can collect six different types of weld morphologies according to the defect identification requirements, such as weld beads, incomplete welds, welds with incorrect dimensions, arc craters, surface porosity of welds, and normal welds.

[0092] It should also be noted that the acquired set of weld surface defect images can be labeled with bounding boxes according to their weld morphology or weld surface defect type. Finally, the labeled set of weld surface defect images is organized into a weld surface defect dataset. Furthermore, the dataset can be divided into training, validation, and test sets according to a preset ratio (e.g., 7:2:1).

[0093] For example, the Labelme annotation tool can be used to annotate weld defects. The weld surface defect dataset includes sample images corresponding to the following six labels: good (normal weld), surface_pores, wrong_size (size mismatch), crater, weld bead, and not_fully_soldered (not fully welded).

[0094] Step S02: Based on the weld surface defect dataset, train the initialized weld defect detection model according to the Mosaic data augmentation strategy to obtain the trained weld defect detection model as the preset weld defect detection model.

[0095] It should be understood that in each training batch, several sample images can be extracted from the weld surface defect dataset according to the Mosaic data augmentation strategy and subjected to random transformation processing to obtain the sample images for the current batch.

[0096] Specifically, each training batch can randomly select, for example, four images from the dataset, and perform various random transformation operations on each image, including geometric transformations (random flipping, scaling) and color space transformations (brightness, contrast, saturation adjustment). Subsequently, the four processed images are stitched together at random positions to form a synthetic image, which serves as the sample image for the current round, i.e., the model input.

[0097] Next, the initialized weld defect detection model can be used to detect defects in the current round of sample images to obtain predicted bounding boxes; then, loss calculations can be performed on the predicted bounding boxes and the corresponding labeled boxes of the current round of sample images according to a preset loss function to obtain the current round loss value, and the model parameters of the initialized weld defect detection model can be updated based on the current loss value.

[0098] It should be noted that the preset loss function is an adaptive loss function, namely the AIoU (Adaptive Intersection over Union) loss function, which replaces the standard loss function in the original YOLOv8n model. The AIoU loss function is obtained by adjusting the weights of the standard loss function according to the dynamic adjustment factor.

[0099] Specifically, the AIoU loss function introduces a dynamic adjustment factor. and This method can dynamically adjust the weight ratios of each component of the loss function during training, enabling small object defects to more accurately regress to the ground truth bounding boxes, thereby effectively improving the detection performance of small objects. AIoU loss function As shown in the following formula:

[0100]

[0101]

[0102]

[0103]

[0104] In the formula, ( , (x, y) represents the center coordinates of the ground truth bounding box in the IoU, and (x, y) represents the center coordinates of the predicted bounding box. and This represents the height and width of the smallest bounding rectangle formed by the combination of the predicted bounding box and the ground truth bounding box. This represents the IoU loss between the predicted bounding box and the ground truth bounding box; Calculate the square of the Euclidean distance between the center point of the predicted box and the center point of the ground truth box to quantify the degree of difference between the center positions of the two detection boxes; Represents the location loss term, with coefficients The weights used to control the loss term.

[0105] v is used to evaluate the consistency of the aspect ratio between the predicted bounding box and the ground truth bounding box. The principle is to map the aspect ratio to an angle value. When the two aspect ratios are the same, their arctangent values ​​are equal and the difference is 0. These are the weighting coefficients used to balance aspect ratio consistency. t represents the current training iteration step, and T represents the total number of iteration steps.

[0106] It should also be noted that in the early stages of training, the predicted bounding boxes and the ground truth bounding boxes typically exhibit a large center-to-center distance and a small Intersection of Values ​​(IoU). At this time, a larger IoU loss function will result in a smaller IoU distance. This results in a loss of IoU ( ) in total loss ( It plays a dominant role in the model training process. During training, the predicted bounding box and the ground truth bounding box are calculated through forward propagation. The values ​​are then used, followed by backpropagation, which, based on the chain rule, calculates the partial derivatives (gradients) of each parameter with respect to the loss function layer by layer. These gradients quantify the degree to which parameter changes affect the loss function. Since IoU loss is dominant, the model prioritizes optimization. The direction of reduction (i.e., increasing the overlap area between the predicted bounding box and the ground truth bounding box) is used to quickly adjust the position and size parameters of the predicted bounding box so that it converges to the ground truth bounding box.

[0107] For example, when the initial position of the predicted bounding box is located to the upper left of the ground truth bounding box and the overlap rate is low, the model will adjust the coordinates of the predicted bounding box according to the gradient information, so that it moves to the lower right to increase the overlap area with the ground truth bounding box.

[0108] As training progresses, the IoU between the predicted bounding boxes and the ground truth bounding boxes gradually increases and then stabilizes, while the distance between their center points gradually decreases. At this point, the focus of model optimization shifts to fine-tuning the center point position and aspect ratio of the predicted bounding boxes. Building upon the high IoU values ​​achieved in the early stages, the model can further refine the position and shape of the predicted bounding boxes, making them more accurately match the ground truth bounding boxes.

[0109] Based on the AIOU loss function mentioned above, by introducing a dynamic adjustment factor, the loss weight can be automatically adjusted according to the training stage to improve the model's bounding box regression accuracy for small target defects, significantly reduce the false negative rate and improve the overall detection performance.

[0110] In the specific implementation, the initial weld defect detection model is trained based on the AIOU loss function mentioned above, and continuously optimized on the validation set and test set, finally obtaining the trained weld defect detection model, namely YOLOv8n-AGF.

[0111] Furthermore, to deploy the trained weld defect detection model on devices, it can be deployed on an embedded AI development board (such as Jetson Orin Nano) based on the TensorRT framework, and the model can be quantized with FP16 precision to accelerate the model's inference process. After YOLOv8n-AGF is trained using the PyTorch framework, a weight file named YOLOv8n-AGF.pt can be generated. However, the PyTorch framework has low inference efficiency on GPUs. Therefore, ONNX can be used as an intermediate layer to convert the trained YOLOv8n-AGF model from PyTorch format to ONNX format, and then further to TensorRT format, thereby accelerating the model inference process and improving inference efficiency.

[0112] In the model building stage, the trained YOLOv8n-AGF can be converted into ONNX format and further serialized into an engine file (.engine file). Then, in the model inference stage, the engine file is deserialized into memory and a runtime engine is created. Subsequently, the input data is loaded into the GPU for inference computation, and finally the inference results are transferred from the GPU to the CPU for further processing.

[0113] This embodiment acquires a set of weld surface defect images and annotates them to obtain a weld surface defect dataset. Based on this dataset, an initialized weld defect detection model is trained using the Mosaic data augmentation strategy to obtain a trained model that serves as the preset weld defect detection model. Because this embodiment employs an adaptive loss function AIoU for model training, it can automatically adjust the loss weights according to the training phase by introducing a dynamic adjustment factor to improve the model's bounding box regression accuracy for small target defects, thereby reducing the false negative rate and improving overall detection performance.

[0114] Furthermore, to verify the performance of the pre-defined weld defect detection model (YOLOv8n-AGF) in the weld surface defect detection task, other detection models can be selected for comparison and verification with YOLOv8n-AGF. Performance metrics of other detection models and YOLOv8n-AGF on the weld surface defect dataset were obtained, and the experimental results are as follows: Figure 8 As shown, Figure 8 The table shows a performance comparison between other detection models and YOLOv8n-AGF on the weld surface defect dataset.

[0115] Depend on Figure 8 As can be seen, YOLOv8n-AGF outperforms the lightweight model YOLOv6n (0.9) with a detection accuracy of 0.919 mAP50, and improves upon the benchmark YOLOv8n (0.906) by 0.013. Although it lags behind YOLOv7-tiny (0.936) by 0.017, its parameter count (1.28M) is only 21.2% of that of YOLOv7-tiny (6.03M), demonstrating superior parameter efficiency.

[0116] In terms of computational complexity, YOLOv8n-AGF requires only 3.0 GFLOPs, equivalent to 42.3% of YOLOv5n (7.1 GFLOPs) and 25.4% of YOLOv6n (11.8 GFLOPs), achieving a significant reduction in computational resources. Compared to computationally intensive models, YOLOv8n-AGF reduces computational requirements by an order of magnitude while maintaining high detection performance, with its computational cost being only 1.6% of Faster R-CNN (184.9 GFLOPs) and 4.1% of RetinaNet-res50 (73.8 GFLOPs).

[0117] Therefore, YOLOv8n-AGF demonstrates significant advantages in terms of memory usage (number of parameters) and computational cost (GFLOPs), making it more suitable for deployment on embedded development platforms. In practical industrial applications, this model can be efficiently integrated into industrial production lines to scan and analyze welding quality in real time, quickly and accurately provide feedback on defect information, thereby improving production efficiency and reducing scrap rates.

[0118] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the weld surface defect detection method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0119] This application also provides a weld surface defect detection system. Please refer to... Figure 9 , Figure 9This is a schematic diagram of the module structure of the weld surface defect detection system of this application. The system includes: The image acquisition module is used to acquire images of the weld seam to be inspected. The defect detection module is used to input the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result. The preset weld defect detection model includes a preset backbone network, a preset neck network, and a preset head network. The preset backbone network and the preset neck network each include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

[0120] The image acquisition module can be an industrial camera installed in an industrial setting or robotic arm, and the defect detection module can be equipped with an embedded AI development board (e.g., Figure 9 The device shown in the Jetson Orin Nano, after undergoing data acquisition and processing, can be used to train YOLOv8n-AGF, which can then be deployed on this embedded AI development board.

[0121] Furthermore, in order to achieve real-time positioning of the weld and real-time detection of weld surface defects, the system may also include: a central scheduling module, a mobile platform motion control module, and a robotic arm motion control module.

[0122] Remote communication between the central scheduling module and Jetson Orin Nano is achieved through Socket sockets based on the TCP / IP protocol.

[0123] During system operation, the image acquisition module can acquire video stream data of weld seams in industrial scenarios in real time and transmit the data to the Jetson Orin Nano device via the UVC protocol, providing input support for subsequent defect detection.

[0124] The weld location and defect detection module (Jetson Orin Nano device) is based on the YOLOv8n-AGF model and performs real-time weld location and defect detection.

[0125] The core of the mobile platform motion control module is the STM32 microcontroller. The Jetson Orin Nano device transmits motion commands from the central scheduling module to the STM32 precisely via the USART serial port. Subsequently, the STM32 uses the CAN bus to control the chassis motors and servos of the mobile platform, coordinating the steering angle of the servos and the speed of the motors, thereby achieving precise control over the motion state of the mobile platform.

[0126] The robotic arm motion control module is responsible for controlling the movement of the robotic arm to perform the function of detecting defects on the weld surface. The industrial control computer (STM32) built into the robotic arm communicates with the Jetson Orin Nano device through the USART serial port and adjusts the posture of the robotic arm according to the weld position information.

[0127] Specifically, the robotic arm has a built-in industrial control computer, which serves as the core of the robotic arm's motion control module. This computer is responsible for parsing control commands and generating corresponding control signals to drive the motors and achieve the robotic arm's movement. Simultaneously, the industrial control computer receives joint position information from the encoder in real time and dynamically adjusts the motor's motion state using a PID control algorithm, ensuring that the robotic arm can quickly and smoothly reach the target position to complete the weld surface defect detection task.

[0128] This embodiment of the system integrates an image acquisition module, a mobile platform motion control module, a weld seam positioning and defect detection module, a robotic arm motion control module, and a central scheduling module to achieve real-time weld seam positioning and real-time weld seam surface defect detection. Furthermore, because this embodiment uses a lightweight C2f module to replace the original C2f module in the YOLOv8n model for real-time positioning and defect detection, this module, through the combination of lightweight convolutional layers and a dual-channel attention mechanism, effectively enhances the ability to extract weld seam surface defect features while reducing the number of model parameters and computational load. This not only reduces the model's storage and computational resource requirements but also improves the model's detection accuracy for small target defects through the dual-channel attention mechanism, reducing the false negative rate.

[0129] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other elements in the process, method, article, or system that includes that element.

[0130] The sequence numbers of the above embodiments of the present invention are merely for description and do not represent the superiority or inferiority of the embodiments. They are only some embodiments of this application and are not intended to limit the scope of this application. All equivalent structural transformations made under the technical concept of this application and based on the content of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the protection scope of this application.

Claims

1. A method for detecting surface defects in welds, characterized in that, The method includes: Acquire images of the weld to be inspected; The image of the weld to be detected is input into a preset weld defect detection model to obtain the defect detection result. The preset weld defect detection model includes a preset backbone network, a preset neck network, and a preset head network. The preset backbone network and the preset neck network each include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

2. The method as described in claim 1, characterized in that, The step of inputting the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result includes: The preset backbone network is used to extract features from the weld seam image to be detected to obtain a scale feature map. The scale feature map is fused using the preset neck network to obtain a fused feature map; The fused features are mapped using the preset head network to obtain defect detection results.

3. The method as described in claim 2, characterized in that, The preset backbone network further includes: a lightweight convolutional layer module, and the lightweight C2f module includes: a CBS layer and a lightweight bottleneck layer; The step of extracting features from the weld seam image to be detected through the preset backbone network to obtain a scale feature map includes: The lightweight convolutional layer module is used to extract features and perform adaptive feature filtering on the weld seam image to be detected, thereby obtaining an input feature map; A shallow feature map is obtained by performing shallow feature extraction on the input feature map through the CBS layer; Deep feature maps are obtained by performing deep feature extraction on the shallow feature map through the lightweight bottleneck layer. The shallow feature map and the deep feature map are spliced ​​together to obtain a scale feature map.

4. The method as described in claim 3, characterized in that, The step of extracting features and adaptively filtering features from the weld seam image to be detected using the lightweight convolutional layer module to obtain the input feature map includes: The image of the weld to be detected is subjected to feature extraction based on the original convolution to obtain an initial feature map; Based on cosine similarity, the initial feature map is filtered to obtain a difference feature map, and a linear operation is performed on the difference feature map; The initial feature map and the differential feature map after linear operation are concatenated to obtain the input feature map.

5. The method as described in claim 3, characterized in that, The lightweight bottleneck layer includes: a first lightweight convolutional layer, a dual-channel attention layer, and a second lightweight convolutional layer; The step of extracting deep features from the shallow feature map through the lightweight bottleneck layer to obtain a deep feature map includes: The shallow feature map is up-dimensioned through a first lightweight convolutional layer to obtain a high-dimensional feature map. The high-dimensional feature map is weighted by dual-channel attention layer to obtain a weighted feature map. The weighted feature map is reduced in dimension by channel through the second lightweight convolutional layer to obtain a dimension-reduced feature map. The dimension-reduced feature map is then added to the shallow feature map to obtain a deep feature map.

6. The method as described in claim 5, characterized in that, The step of performing dual-channel attention weighting on the high-dimensional feature map through the dual-channel attention layer to obtain the weighted feature map includes: Channel dimension features are extracted from the high-dimensional feature map to obtain key channel feature information, and channel attention weights are determined based on the key channel feature information. Spatial dimension features are extracted from the high-dimensional feature map to obtain key spatial feature information, and a spatial attention map is generated based on the key spatial feature information. The channel attention map and the spatial attention map are multiplied element-wise with the high-dimensional feature map to obtain a weighted feature map.

7. The method as described in claim 1, characterized in that, Before the step of inputting the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result, the following steps are included: A set of weld surface defect images is obtained, and weld defects are annotated on the set of weld surface defect images to obtain a weld surface defect dataset. Based on the weld surface defect dataset, the initialized weld defect detection model is trained according to the Mosaic data augmentation strategy to obtain the trained weld defect detection model as the preset weld defect detection model.

8. The method as described in claim 7, characterized in that, The step of training the initialized weld defect detection model based on the weld surface defect dataset according to the Mosaic data augmentation strategy includes: In each training batch, several sample images are extracted from the weld surface defect dataset according to the Mosaic data augmentation strategy and subjected to random transformation processing to obtain the sample images of the current batch. Defect detection is performed on the current round of sample images using an initialized weld defect detection model to obtain a prediction box; The loss is calculated for the predicted bounding box and the labeled bounding box corresponding to the current round sample image according to the preset loss function to obtain the loss value of the current round; The model parameters of the initialized weld defect detection model are updated based on the current loss value.

9. The method as described in claim 8, characterized in that, The preset loss function is an adaptive loss function that replaces the standard loss function in the original YOLOv8n model. The adaptive loss function is obtained by adjusting the weights of the standard loss function according to a dynamic adjustment factor.

10. A weld surface defect detection system, characterized in that, The system includes: The image acquisition module is used to acquire images of the weld seam to be inspected. The defect detection module is used to input the image of the weld to be detected into a preset weld defect detection model to obtain the defect detection result. The preset weld defect detection model includes a preset backbone network, a preset neck network, and a preset head network. The preset backbone network and the preset neck network each include several lightweight C2f modules that replace the C2f modules in the original YOLOv8n model. The lightweight C2f modules are constructed based on lightweight convolutional layers and a dual-channel attention mechanism.

Images