Laser spot welding spot detection method and system based on improved YOLOv9

Abstract

The invention discloses a laser spot welding spot detection method and system based on improved YOLOv9, and relates to the technical field of welding spot detection. Comprising the following steps: acquiring a laser spot welding spot image data set in a real industrial environment; marking the laser spot welding spot image data set, and distinguishing different spot welding spot types; carrying out enhancement processing on the marked laser spot welding spot image data set; different scale features of the laser spot welding spot image are integrated through a multi-scale attention fusion module, and the module is added into an attention mechanism; a MidInerSIoU loss function is adopted, the middle IoU interval of the laser spot welding spot image is optimized, and the regression performance of the low IoU interval and the high IoU interval of the laser spot welding spot image is improved through nonlinear and linear growth strategies; and generating a low-cost ghost feature map by using a ghost feature fusion module, and obtaining a spot welding spot feature map based on the ghost feature map. According to the invention, not only is the detection precision improved, but also the number of parameters and GFLOPS are greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of solder joint detection, and in particular to a laser spot welding solder joint detection method and system based on improved YOLOv9. Background Art

[0002] In today's highly sophisticated industrial manufacturing field, laser welding technology has been widely used in aerospace, automobile manufacturing, electronic product production and other industries due to its high precision, high efficiency and high reliability. With the increasing popularity of this technology, the importance of solder joint quality inspection has become increasingly prominent, becoming a key link that directly affects product performance and safety.

[0003] Compared with traditional PCB solder joints, laser spot welding joints often show various shapes, large size differences and irregular contours, which undoubtedly increases the complexity and challenge of detection. At present, the detection algorithm for laser spot welding joints is still insufficient, which undoubtedly further increases the difficulty of detection.

[0004] In the field of solder joint inspection, in order to cope with the growing inspection demand, a variety of solder joint quality inspection technologies have been deployed, including but not limited to optical imaging, X-ray imaging, and ultrasonic imaging. In recent years, with the rapid development of deep learning technology, the inspection accuracy and efficiency of laser spot welding joints have been significantly improved, which has strongly promoted the innovation and progress of welding quality inspection technology. However, in the inspection process of laser spot welding joints, existing methods still face many challenges, such as the difficulty of multi-scale target recognition, the difficulty of processing complex texture information, and the inaccuracy of bounding box positioning. Summary of the invention

[0005] The purpose of the present invention is to propose a laser spot welding spot detection method and system based on improved YOLOv9, which enhances the detection capability under complex textures and multi-scale targets, improves the detection accuracy, and greatly reduces the parameter amount.

[0006] To achieve the above object, the present invention provides a laser spot welding spot detection method based on improved YOLOv9, comprising the following steps:

[0007] Obtain a dataset of laser spot welding images in a real industrial environment;

[0008] Annotating the laser spot welding spot image data set to distinguish different spot welding spot categories;

[0009] Perform enhancement processing on the labeled laser spot welding spot image dataset;

[0010] The different scale features of laser spot welding spot images are integrated through the Multi-Scale Attention Fusion Module (MSAFM), which adds an attention mechanism.

[0011] The MidInnerSIoU (MISIoU) loss function is used to optimize the mid-IoU interval of the laser spot welding spot image, and the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image is improved through nonlinear and linear growth strategies;

[0012] A low-cost ghost feature map is generated by using the Ghost Feature Fusion Module (GFFM), and a spot welding point feature map is obtained based on the ghost feature map.

[0013] In one embodiment, the multi-scale attention fusion module includes a scale sequence feature fusion module (Scale Sequence Feature Fusion, SSFF), a triple feature encoder (Triple Feature Encoder, TFE), and a hybrid local channel and position attention mechanism (Hybrid Local Channel and Position Attention Mechanism, MPAM);

[0014] The scale sequence feature fusion module divides the laser spot welding spot image into different levels, namely P1 to P5; integrates the information of the three levels of P3, P4 and P5 through stacking operation; after fusing the different scale features, further processing through 3D convolution, batch normalization (BN) and SiLU activation function can deeply understand the complex texture changes, grain interface and geometric shape of the laser spot welding spot;

[0015] The triple feature encoder is used to process feature maps of laser spot welding spots of different scales. For large-scale welds, redundant information is reduced by downsampling, and a combination of maximum pooling and average pooling is used to effectively extract important features and enhance anti-interference capabilities. Medium-scale welds are processed by convolution, batch normalization, and activation functions to fully capture feature information and improve training stability. For small-scale welds, the nearest neighbor interpolation method is used for upsampling to restore the spatial resolution of the feature map.

[0016] The hybrid local channel and position attention mechanism performs two-step pooling on the laser spot welding spot feature map to effectively extract important information.

[0017] In one embodiment, based on CIoU, a shape constraint factor is added to design a SIoU loss function to evaluate and enhance the similarity in shape between the predicted box and the true box of the laser spot welding spot:

[0018]

[0019] Where d is the distance between the center point of the predicted box and the real box, c is the diagonal distance of the minimum bounding rectangle, and ν is a correction factor used to further adjust the loss function, taking into account the shape and direction of the target box. (w G ,h G ) and (w P ,h P ) are the width and height of the predicted box and the real box respectively, S represents the shape constraint factor, so the loss function L SIoU for:

[0020]

[0021] In one of the embodiments, based on the adaptability of the auxiliary bounding box, a loss function MISIoU is designed, which dynamically adjusts the scale of the border by calculating the IoU loss of the auxiliary bounding box; the intersection area of ​​the predicted box and the true box is obtained based on the product of the intersection lengths in the horizontal and vertical directions:

[0022]

[0023] Among them, b t ,b r ,b l ,b b Respectively represent the top, right, left, and bottom boundary values ​​of the bounding box; Respectively represent the top, right, left, and bottom boundary values ​​of the real box;

[0024] The union area of ​​the predicted box and the true box considers the area of ​​the true box and the auxiliary bounding box and the scaling factor ratio:

[0025] union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter (7)

[0026] Among them, w gt and h gt are the width and height of the true box, w and h are the width and height of the auxiliary bounding box.

[0027] The IoU of the adaptive auxiliary bounding box is defined as:

[0028]

[0029] The final regression loss is:

[0030] L MISIoU =L SIoU +IoU-IoU innner -IoU Mid (9)

[0031] In one of the embodiments, the ghost feature fusion module uses 1x1 convolution to compress the number of channels of the laser spot welding spot feature map; then, more feature maps are generated through depthwise separable convolution (layer-by-layer convolution); then, different feature maps are concatenated (concat) to form a new output; in this process, GhostConv is used to replace traditional convolution to construct the Ghost Module module.

[0032] The input feature map of the GhostConv module is X∈R (h×w×c) , where h and w represent the height and width of the feature map, respectively, and c and m represent the number of input channels and output channels, respectively.

[0033] After the convolution operation f', the output feature map Y'∈R is obtained (h′×w′×m′) ,as follows:

[0034] Y'=X×f' (10)

[0035] Among them, X represents the input feature map, and f' represents the convolution operation;

[0036] In one embodiment, the GhostConv module performs a linear transformation Φ on the feature map Y' to generate more feature maps Y ij :

[0037] Y ij =Φ ij (y′ i ) (11)

[0038] Among them, y' i represents each feature map in Y', Φ ij Represents an operation that performs a linear transformation.

[0039] On this basis, two different GhostBlock modules are designed through two stacked Ghost Modules; the first GhostBlock consists of two Ghost Modules in series, where the first Ghost Module increases the number of channels, while the second Ghost Module reduces the number of channels to one with the input channels; the residual edge part is set the same as ResNet; the second GhostBlock adds a depthwise separable convolution with a stride of 2 between the two Ghost Modules, which can compress the height and width of the feature map to 1 / 2 of the input size; in the residual edge part, a depthwise separable convolution with a stride of 2x2 and a 1x1 ordinary convolution are also added to ensure that the Add operation can be aligned; since S=2, the height and width of the input feature layer will be compressed, the purpose of which is to change the shape of the input feature layer.

[0040] In one embodiment, all conventional convolutions in GhostModule are replaced by pointwise convolutions; next, a GHBCSP module is constructed according to the architecture of CSPNet, and the GHBCSP is constructed into a GFFM module based on the ELAN architecture.

[0041] According to a second aspect of an embodiment of the present disclosure, a laser spot welding spot detection system based on improved YOLOv9 is provided, comprising:

[0042] The dataset construction module obtains the laser spot welding spot image dataset in a real industrial environment;

[0043] A labeling module, which labels the laser spot welding spot image data set to distinguish different spot welding spot categories;

[0044] An enhancement module performs enhancement processing on the labeled laser spot welding spot image data set;

[0045] Multi-scale attention fusion module integrates the different scale features of laser spot welding spot images. This module adds attention mechanism;

[0046] The loss function design module uses the MidInnerSIoU loss function to optimize the mid-IoU interval of the laser spot welding spot image, and improves the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image through nonlinear and linear growth strategies;

[0047] By utilizing the ghost feature fusion module, a low-cost ghost feature map is generated, and the spot welding point feature map is obtained based on the ghost feature map.

[0048] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored and running on the memory, wherein when the processor executes the program, the laser spot welding spot detection method based on improved YOLOv9 is implemented.

[0049] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the laser spot welding spot detection method based on improved YOLOv9 is implemented.

[0050] Compared with the prior art, the above technical solution adopted by the present invention has the following advantages: the present invention introduces the MSAFM module, which significantly enhances the detection capability of complex texture features and multi-scale targets of laser spot welding welds. In addition, a MISIoU loss function is proposed, which effectively solves the problem of inaccurate bounding box positioning and inconsistent gradient changes in different IoU value intervals, thereby improving the detection accuracy of laser spot welding welds. Finally, by introducing the GFFM module, not only the improvement of detection accuracy is achieved, but also the number of parameters and GFLOPS are greatly reduced, which effectively meets the actual detection needs of the factory. This solution provides a more accurate and fast solution for the detection of laser spot welding welds. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0052] Figure 1 This is the YOLOv9 model architecture diagram;

[0053] Figure 2 This is a diagram of the detection network architecture of the present invention;

[0054] Figure 3 The overall module diagram of the MSAFM proposed by the present invention;

[0055] Figure 4 The present invention proposes a flow chart of a TFE module in a MSAFM module;

[0056] Figure 5 The MPAM attention flow chart in the MSAFM module is proposed for the present invention;

[0057] Figure 6 It is the structure diagram of GhostConv of the present invention;

[0058] Figure 7 This is the GFFM module architecture diagram proposed by the present invention;

[0059] Figure 8 A diagram of the types of data sets of the present invention;

[0060] Fig. 9 A distribution diagram and detailed label information diagram of the data set of the present invention;

[0061] Fig.10 It is a comparative experimental result diagram of the present invention;

[0062] Fig.11 is a comparison diagram of ablation experiments of the original data set of the present invention;

[0063] Fig.12 This is a comparison diagram of the ablation experiment of the enhanced data set of the present invention;

[0064] Fig.13 This is a comparison chart of the detection effects of YOLOv9 and the model of the present invention. DETAILED DESCRIPTION

[0065] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0066] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0067] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0068] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the methods and systems according to various embodiments of the present disclosure. It should be noted that each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, or they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the flowchart and / or block diagram, and the combination of boxes in the flowchart and / or block diagram can be implemented using a dedicated hardware-based system that performs a specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0069] Embodiment 1:

[0070] YOLO (You Only Look Once) target detection method is a representative one-stage detection method, with the characteristics of fast network operation speed and small memory usage. The YOLOv9 network model includes input part, feature extraction part, multi-scale fusion part and output part, such as Figure 1 The present invention improves the YOLOv9 network by adding an MSAFM module to its neck and changing the feature extraction module of YOLOv9 to a GFFM module, as shown in FIG. Figure 2 shown.

[0071] This embodiment provides a laser spot welding spot detection method based on improved YOLOv9, comprising the following steps:

[0072] S1: Acquire a laser spot welding spot image dataset in a real industrial environment;

[0073] Specifically, each image contains six solder joints. In order to effectively evaluate the quality of solder joints, they are divided into seven types according to the characteristics of solder joints: OK, GB, XH, DL, SLX, HC and WNL, as shown in Figure 3 shown.

[0074] S2: labeling the laser spot welding spot image data set to distinguish different spot welding spot categories;

[0075] S3: Enhance the labeled laser spot welding spot image dataset;

[0076] In order to enrich the sample data and simulate the scenarios that may occur in actual industrial laser spot welding spots, this embodiment performs three data enhancement processes on the original data set. Specific data enhancement methods include: 1) Rotation transformation: Rotate the laser spot welding spot image at random angles to simulate different changes in detection viewing angles. 2) Contrast transformation: Randomly adjust the saturation of the laser spot welding spot image to simulate the changes in the image under different lighting conditions. 3) Noise disturbance: Add random noise to the laser spot welding spot image to simulate the interference that may occur during the image acquisition process and enhance the model's anti-noise ability. The distribution and label information of the training data set are as follows: Figure 4 As shown, (a) represents the distribution of the center coordinates of the marking box, and (b) represents the distribution of the width and height of the marking box.

[0077] S4: Integrate different scale features of laser spot welding spot images through a multi-scale attention fusion module, which adds an attention mechanism;

[0078] like Figure 5 As shown, the module includes three key components: SSFF, TFE and MPAM;

[0079] SSFF focuses on improving the network's ability to extract and integrate multi-scale features. This module divides the laser spot welding spot image into five different levels of information from P1 to P5. On this basis, SSFF integrates the information of the three levels of P3, P4 and P5 through stacking operations. After fusing features of different scales, it is further processed through 3D convolution, batch normalization (BN) and SiLU activation function, so that the model can deeply understand the complex texture changes, grain interfaces and geometric shapes of laser spot welding spots.

[0080] TFE is a feature fusion mechanism in MSAFM, which is designed to process feature maps of laser spot welding spots of different scales. For larger-scale welds, TFE reduces redundant information by downsampling, and uses a combination of maximum pooling and average pooling to effectively extract important features and enhance anti-interference capabilities. Medium-scale welds are processed through convolution, batch normalization, and activation functions to fully capture feature information and improve training stability. For smaller-scale welds, the nearest neighbor interpolation method is used for upsampling to restore the spatial resolution of the feature map, which is simple and effective, ensuring the accuracy and response speed of small targets. Finally, the feature maps of different scales are merged into a unified representation through splicing operations, such as Figure 6 shown.

[0081] MPAM performs two-step pooling on the laser spot welding spot feature map (C×W×H) by mixing local channel and position attention mechanism to effectively extract important information, such as Figure 7As shown. First, the laser spot welding spot feature map is converted into a vector of 1×C×ks×ks by local average pooling (LAP) to capture local spatial features. LAP pays more attention to the details and texture changes in the laser spot welding spot image. Then global average pooling (GAP) is performed to provide overall context, thereby helping the model understand the overall structure of the image. After pooling, the laser spot welding spot feature map is flattened into a one-dimensional vector to prepare for the subsequent one-dimensional convolution. The role of 1D convolution is to compress the number of channels and redistribute weights, strengthen the features related to weld spot detection, and weaken irrelevant information. This process realizes the channel attention mechanism. At this time, the size of the convolution kernel k is proportional to the channel dimension C, ensuring that only the relationship between each channel and its k adjacent channels is considered when capturing local cross-channel interaction information. The selection of k is adjusted according to formula (1):

[0082]

[0083] Among them, C is the number of channels, k is the size of the convolution kernel, b and γ are both hyperparameters, the default value is 2, odd means k is an odd number, if it is an even number, add 1.

[0084] The local features restore the spatial structure through Rehape, while the global features are upsampled to the original size through unpooling (UNAP). Finally, the local features are fused with the global features to enhance the model's understanding of the solder joint structure.

[0085] The position attention mechanism focuses on the extraction of spatial information. The position attention input is formed by combining the output of the hybrid local and global attention mechanism with the features of SSFF, thereby providing complementary information, aiming to extract the key position information of each unit. Different from the hybrid local and global attention mechanism, the position attention mechanism first divides the input feature map into two parts by width and height, then encodes the features of these two parts respectively, and finally merges them to generate the output. Specifically, the input laser spot welding spot feature map is average pooled on the horizontal (pw) and vertical (ph) axes to retain the spatial structure information, see Equations (2) and (3):

[0086]

[0087]

[0088] Where W and H are the width and height of the input laser spot welding spot feature map, respectively, and E(i,j) is the value at position (i,j) of the input laser spot welding spot feature map.

[0089] When generating the position attention coordinates, the feature maps are first combined through the concatenation operation (Concat), and then 1×1 convolution and other operations are performed to finally output the position coordinates P(aw ,a h ).

[0090] P(a w ,a h )=Conv[Concat(p w ,p h )] (4)

[0091] When segmenting the attention features, a pair of feature maps that depend on position information will be generated, as shown in Equations (5) and (6), where sw and sh are the width and height of the segmentation output, respectively.

[0092] s w =Split(a w ) (5)

[0093] s h =Split(a h ) (6)

[0094] The final output of MPAM is defined as:

[0095] F MPAM =E*s w *s h #(7)

[0096] Where E represents the weight matrix of the channel and its position during redirection.

[0097] S5: The MidInnerSIoU loss function is used to optimize the mid-IoU interval of the laser spot welding spot image, and the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image is improved through nonlinear and linear growth strategies;

[0098] In order to evaluate the regression loss between the real box and the predicted box of the laser spot welding spot, the present invention proposes a MISIoU loss function. In order to ensure the stability of the laser spot welding spot data training and avoid excessive gradient fluctuations, the present invention introduces a nonlinear growth part in the low IoU interval to achieve a gradual increase in loss. At the same time, in the high IoU interval, a smooth gradient signal is provided by the linear growth part, which facilitates fine-tuning of the weld spot bounding box.

[0099] In addition, considering the characteristics of laser spot welding spots with diverse shapes and sizes, the present invention adds shape constraint factors based on CIoU and designs the SIoU loss function to evaluate and enhance the similarity in shape between the predicted box and the real box of the laser spot welding spots, thereby reducing the matching error caused by the difference in the shape of the target box.

[0100]

[0101]

[0102]

[0103]

[0104] Where d is the distance between the center point of the predicted box and the real box, c is the diagonal distance of the minimum bounding rectangle, and ν is a correction factor used to further adjust the loss function, taking into account the shape and direction of the target box. (w G ,h G ) and (w P ,h P ) are the width and height of the predicted box and the true box respectively, S represents the shape constraint factor, and its value is between 0 and 1. The closer it is to 1, the more similar the shape of the predicted box and the true box of the laser spot welding spot is:

[0105]

[0106] In order to further improve the regression performance of SIoU in laser spot welding, this paper introduces an auxiliary bounding box mechanism. Through the adaptability of the auxiliary bounding box, MISIoU is designed, which dynamically adjusts the scale of the border by calculating the IoU loss of the auxiliary auxiliary bounding box. The intersection area of ​​the predicted box and the true box is based on the product of the intersection length in the horizontal and vertical directions:

[0107]

[0108] Among them, b t ,b r ,b l ,b b Respectively represent the top, right, left, and bottom boundary values ​​of the bounding box; Respectively represent the top, right, left, and bottom boundary values ​​of the real box;

[0109] The union area of ​​the predicted box and the true box considers the area of ​​the true box and the auxiliary bounding box and the scaling factor ratio:

[0110] union=(w gt *h gt )*((ratio) 2 +(w*h)*(ratio) 2 -inter (14)

[0111] Among them, w gt and h gt are the width and height of the true box, w and h are the width and height of the auxiliary bounding box.

[0112] The IoU of the adaptive auxiliary box is defined as:

[0113]

[0114] The final regression loss is:

[0115] L MISIoU =L SIoU +IoU-IoU innner -IoU Mid (16)

[0116] S6: Generate a low-cost ghost feature map using the ghost feature fusion module, and obtain a spot welding point feature map based on the ghost feature map;

[0117] In order to improve the model's lightweight and inference speed in laser spot welding spot detection, a ghost feature fusion module (GFFM) is proposed. The core idea of ​​the GFFM module is to generate more "ghost" feature maps from existing feature maps through low-cost linear transformation, thereby improving the network's computational efficiency and the ability to express subtle features.

[0118] First, the input laser spot welding spot image is compressed using ordinary 1x1 convolution. Then, more laser spot welding spot feature maps are generated through depthwise separable convolution (layer-by-layer convolution). Then, different feature maps are concatenated (concat) to form a new output. In this process, GhostConv is used to replace traditional convolution to construct Ghost Module.

[0119] Specifically, the input feature map of the GhostConv module is X∈R (h×w×c) , where h and w represent the height and width of the feature map, respectively, and c and m represent the input channel and output

[0120] ”'

[0121] The number of output channels is obtained by convolution operation f' to obtain the output feature map Y'∈R (h×w×m) , the formula is as follows:

[0122] Y'=X×f'(17)

[0123] Among them, X represents the input feature map and f' represents the convolution operation.

[0124] Next, the GhostConv module performs a linear transformation Φ on the feature map Y' to generate more feature maps Y ij :

[0125] Y ij =Φ ij (y′i ) (18)

[0126] Among them, y' i represents each feature map in Y', Φ ij Represents the operation of performing a linear transformation. In this way, the Ghost module retains the key information in the input feature map while greatly reducing the computational complexity.

[0127] On this basis, two different GhostBlock modules are designed through two stacked Ghost Modules. The first GhostBlock consists of two Ghost Modules in series, where the first Ghost Module increases the number of channels, while the second Ghost Module reduces the number of channels to the same as the input channels. The residual edge part is set the same as ResNet. Since S = 1, the height and width of the input feature layer will not be compressed, the purpose is to deepen the depth of the network.

[0128] The second GhostBlock adds a depthwise separable convolution with a stride of 2 between the two Ghost Modules, which can compress the height and width of the feature map to 1 / 2 of the input size. In the residual edge part, a depthwise separable convolution with a stride of 2x2 and a 1x1 normal convolution are also added to ensure that the Add operation can be aligned. Since S=2, the height and width of the input feature layer are compressed, the purpose of which is to change the shape of the input feature layer.

[0129] To further improve efficiency, all conventional convolutions in GhostModule are replaced by pointwise convolution. Next, the GHBCSP module is built according to the CSPNet architecture, and the GHBCSP is built into a GFFM module based on the ELAN architecture. Finally, the model is lightweight in laser spot welding spot detection, which improves the inference speed and overall accuracy.

[0130] In this example, the performance evaluation was performed on a computing platform equipped with four NVIDIA GeForce RTX 2080Ti GPUs (each GPU is equipped with 11GB RAM), and the processor is Intel Xeon CPU E5-2680 v4 @ 2.40GHz. The experiment was conducted in a Linux environment using Python 3.8 and Torch 2.1.0. During the training process, the epoch was set to 500, the batch size was 12, and the resolution of the input image was 640×640 pixels.

[0131] The present invention uses the following indicators to evaluate the effectiveness of the algorithm: precision, recall, mean average precision (mAP) and F1 score. mAP is the average AP value of different laser weld categories and is defined as follows:

[0132]

[0133] Where N represents the number of laser weld types and AP is the area enclosed by the PR curve, as shown in the equation:

[0134]

[0135] Among them, P represents precision, that is, the probability of being correctly classified in the predicted positive samples, and the calculation formula is the ratio of correctly predicted positive instances to all instances marked as positive by the model. TP (True Positive) is the number of correctly identified positive samples, and FP (False Positive) is the number of negative samples incorrectly marked as positive.

[0136]

[0137] R stands for recall, which is a measure of the proportion of actual positive samples that are correctly identified by the model. FN (false negatives) is the number of positive samples that are incorrectly classified as negative.

[0138]

[0139] The F1 score is the harmonic mean of precision and recall and is used to assess the overall accuracy and completeness of the model.

[0140]

[0141] In order to verify the advantages of the network model proposed in this invention, this embodiment comprehensively compares it with various mainstream methods on the laser welding spot dataset. The experimental results are shown in Table 1. The comprehensive situation of mAP50 and parameter quantity of different models is shown in Table 1. Fig.10 shown.

[0142] Table 1 Comparative experimental results

[0143]

[0144] The experimental results show that the present invention exhibits excellent performance in the laser weld detection task and achieves significant improvements in multiple key indicators. In order to more comprehensively evaluate the impact of the improvement measures and diversified scenarios proposed in the present invention on the performance of laser weld detection, this embodiment gradually introduces the MSAFM module, GFFM module and MISIoU loss in the original data set and the enhanced data set. The ablation experimental results of the original data set and data enhancement are shown in Table 2 and Table 3 respectively, and the map50 result graphs are shown in Fig.11 and Fig.12middle.

[0145] Table 2. Results of the original data ablation experiment

[0146]

[0147]

[0148] Table 3 Enhanced data ablation experimental results

[0149]

[0150] The experimental results show that the original laser spot welding spot dataset exhibits low precision and high computational complexity in the baseline model. By introducing multiple improved modules, the model accuracy is significantly improved, and the number of parameters and computational complexity are effectively reduced. After combining different modules, the feature extraction and fusion capabilities of the model are enhanced. In addition, the implementation of data enhancement technology expands the experimental scenarios and simulates different application scenarios that may exist in actual industrial inspections, proving its effectiveness in increasing the number of image samples and the overall performance of the model, thereby enhancing the generalization ability of the model and verifying the stability and reliability of the improved module in a changing environment. Fig.13 The detection effect of YOLOv9 and the network model proposed in this invention is compared.

[0151] Embodiment 2:

[0152] This embodiment provides a laser spot welding spot detection system based on improved YOLOv9, including:

[0153] The dataset construction module obtains the laser spot welding spot image dataset in a real industrial environment;

[0154] A labeling module, which labels the laser spot welding spot image data set to distinguish different spot welding spot categories;

[0155] An enhancement module performs enhancement processing on the labeled laser spot welding spot image data set;

[0156] Multi-scale attention fusion module integrates the different scale features of laser spot welding spot images. This module adds attention mechanism;

[0157] The loss function design module uses the MidInnerSIoU loss function to optimize the mid-IoU interval of the laser spot welding spot image, and improves the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image through nonlinear and linear growth strategies;

[0158] By utilizing the ghost feature fusion module, a low-cost ghost feature map is generated, and the spot welding point feature map is obtained based on the ghost feature map.

[0159] Embodiment three:

[0160] An electronic device includes a memory, a processor, and a computer program stored and running on the memory, wherein the processor implements the above-mentioned laser spot welding spot detection method and system based on improved YOLOv9 when executing the program, including:

[0161] Obtain a dataset of laser spot welding images in a real industrial environment;

[0162] Annotating the laser spot welding spot image data set to distinguish different spot welding spot categories;

[0163] Perform enhancement processing on the labeled laser spot welding spot image dataset;

[0164] The different scale features of laser spot welding spot images are integrated through a multi-scale attention fusion module, which adds an attention mechanism;

[0165] The MidInnerSIoU loss function is used to optimize the mid-IoU interval of the laser spot welding spot image, and the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image is improved through nonlinear and linear growth strategies;

[0166] The ghost feature fusion module is used to generate a low-cost ghost feature map, and the spot welding point feature map is obtained based on the ghost feature map.

[0167] Embodiment 4:

[0168] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned laser spot welding spot detection method and system based on improved YOLOv9, including:

[0169] Obtain a dataset of laser spot welding images in a real industrial environment;

[0170] Annotating the laser spot welding spot image data set to distinguish different spot welding spot categories;

[0171] Perform enhancement processing on the labeled laser spot welding spot image dataset;

[0172] The different scale features of laser spot welding spot images are integrated through a multi-scale attention fusion module, which adds an attention mechanism;

[0173] The MidInnerSIoU loss function is used to optimize the mid-IoU interval of the laser spot welding spot image, and the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image is improved through nonlinear and linear growth strategies;

[0174] The ghost feature fusion module is used to generate a low-cost ghost feature map, and the spot welding point feature map is obtained based on the ghost feature map.

[0175] Those skilled in the art should understand that the modules or steps of the present disclosure can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present disclosure is not limited to any specific combination of hardware and software.

[0176] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0177] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A laser spot welding spot detection method based on improved YOLOv9, characterized in that: The following steps are involved: Obtain a dataset of laser spot welding images in a real industrial environment; Annotating the laser spot welding spot image data set to distinguish different spot welding spot categories; Perform enhancement processing on the labeled laser spot welding spot image dataset; The different scale features of laser spot welding spot images are integrated through a multi-scale attention fusion module, which adds an attention mechanism; The MidInnerSIoU loss function is used to optimize the mid-IoU interval of the laser spot welding spot image, and the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image is improved through nonlinear and linear growth strategies; The ghost feature fusion module is used to generate a low-cost ghost feature map, and the spot welding point feature map is obtained based on the ghost feature map.

2. According to the laser spot welding spot detection method based on improved YOLOv9 as claimed in claim 1, it is characterized in that: The multi-scale attention fusion module includes a scale sequence feature fusion module, a triple feature encoder, and a hybrid local channel and position attention mechanism; The scale sequence feature fusion module divides the laser spot welding spot image into different levels, namely P1 to P5; integrates the information of the three levels of P3, P4 and P5 through stacking operation; after fusing the different scale features, further 3D convolution, batch normalization and SiLU activation function processing are performed to deeply understand the complex texture changes, grain interface and geometric shape of the laser spot welding spot; The triple feature encoder is used to process feature maps of laser spot welding spots of different scales. For large-scale welds, redundant information is reduced by downsampling, and a combination of maximum pooling and average pooling is used to effectively extract important features and enhance anti-interference capabilities. Medium-scale welds are processed by convolution, batch normalization, and activation functions to fully capture feature information and improve training stability. For small-scale welds, the nearest neighbor interpolation method is used for upsampling to restore the spatial resolution of the feature map. The hybrid local channel and position attention mechanism performs two-step pooling on the laser spot welding spot feature map to effectively extract important information.

3. According to the laser spot welding spot detection method based on improved YOLOv9 as claimed in claim 1, it is characterized in that: Based on CIoU, a shape constraint factor is added and the SIoU loss function is designed to evaluate and enhance the similarity in shape between the predicted box and the real box of the laser spot welding spot: Where d is the distance between the center point of the predicted box and the real box, c is the diagonal distance of the minimum bounding rectangle, and ν is a correction factor used to further adjust the loss function, taking into account the shape and direction of the target box. (w G ,h G ) and (w P ,h P ) are the width and height of the predicted box and the real box respectively, S represents the shape constraint factor, so the loss function L SIoU for:

4. According to the laser spot welding spot detection method based on improved YOLOv9 as claimed in claim 3, it is characterized in that: Based on the adaptability of the auxiliary bounding box, a loss function MISIoU is designed, which dynamically adjusts the scale of the border by calculating the IoU loss of the auxiliary bounding box; the intersection area of ​​the predicted box and the real box is obtained based on the product of the intersection lengths in the horizontal and vertical directions: Among them, b t ,b r ,b l ,b b Represent the upper, right, left, and lower boundary values ​​of the bounding box respectively; Represent the upper, right, left, and lower boundary values ​​of the real box respectively; The union area of ​​the predicted box and the true box considers the area of ​​the true box and the auxiliary bounding box and the scaling factor ratio: union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter (7) Among them, w gt and h gt are the width and height of the ground-truth box, w and h are the width and height of the auxiliary bounding box. The IoU of the adaptive auxiliary bounding box is defined as: The final regression loss is: L MISIoU =L SIoU +IoU-IoU innner -IoU Mid (9)。 5. According to the laser spot welding spot detection method based on improved YOLOv9 as claimed in claim 1, it is characterized in that: The ghost feature fusion module uses 1x1 convolution to compress the number of channels of the laser spot welding spot feature map; then, more feature maps are generated through depthwise separable convolution; then, different feature maps are spliced ​​and combined into new outputs; in this process, GhostConv is used to replace traditional convolution to construct the Ghost Module module; The input feature map of the GhostConv module is X∈R (h×w×c) , where h and w represent the height and width of the feature map, respectively, and c and m represent the number of input channels and output channels, respectively,"' After the convolution operation f', the output feature map Y'∈R is obtained (h×w×m) ,as follows: Y'=X×f' (10) Among them, X represents the input feature map and f' represents the convolution operation.

6. According to the laser spot welding spot detection method based on improved YOLOv9 as claimed in claim 5, it is characterized in that: The GhostConv module performs a linear transformation Φ on the feature map Y' to generate more feature maps Y ij : AND ij =Φ ij (and' i ) (11) Among them, y' i represents each feature map in Y', Φ ij represents an operation that performs a linear transformation; On this basis, two different GhostBlock modules are designed through two stacked Ghost Module modules; the first GhostBlock consists of two Ghost Modules in series, where the first Ghost Module increases the number of channels, while the second Ghost Module reduces the number of channels to one with the input channels; the residual edge part is set the same as ResNet; the second GhostBlock adds a depthwise separable convolution with a stride of 2 between the two Ghost Modules, which can compress the height and width of the feature map to 1 / 2 of the input size; in the residual edge part, a depthwise separable convolution with a stride of 2x2 and a 1x1 ordinary convolution are also added to ensure that the Add operation can be aligned; since S=2, the height and width of the input feature layer will be compressed, the purpose of which is to change the shape of the input feature layer.

7. A laser spot welding spot detection method based on improved YOLOv9 according to claim 6, characterized in that: All conventional convolutions in GhostModule are replaced by pointwise convolutions. Next, the GHBCSP module is constructed according to the architecture of CSPNet, and GHBCSP is constructed into a ghost feature fusion module based on the ELAN architecture.

8. A laser spot welding spot detection system based on improved YOLOv9, characterized in that: include: The dataset construction module obtains the laser spot welding spot image dataset in a real industrial environment; A labeling module, which labels the laser spot welding spot image data set to distinguish different spot welding spot categories; An enhancement module is used to enhance the labeled laser spot welding spot image data set; Multi-scale attention fusion module integrates the different scale features of laser spot welding spot images. This module adds attention mechanism; The loss function design module uses the MidInnerSIoU loss function to optimize the mid-IoU interval of the laser spot welding spot image, and improves the regression performance of the low IoU interval and high IoU interval of the laser spot welding spot image through nonlinear and linear growth strategies; The ghost feature fusion module is used to generate a low-cost ghost feature map, and the spot welding point feature map is obtained based on the ghost feature map.

9. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, characterized in that: When the processor executes the program, the laser spot welding spot detection method based on improved YOLOv9 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a laser spot welding spot detection method based on improved YOLOv9 is implemented.

Images