Laser welding spot defect detection method and system based on YOLOv9 improved model

Abstract

The invention discloses a laser welding spot defect detection method and system based on a YOLOv9 improved model, and relates to the technical field of laser welding spot detection. Comprising the following steps that a laser welding spot image data set is obtained, and the data set is marked to distinguish different types of laser welding spots; by using a mixed local-global attention mechanism, global features are kept while complex texture details of laser welding spots are captured, and the feature loss risk is reduced by using jump connection; biFPN architecture is adopted, and bidirectional flow and fusion of multi-level laser welding spot features are achieved; a branch enhancement connection module is constructed, the laser welding spot detection capacity under the complex background is enhanced, multi-scale key information is captured, and the reasoning speed is optimized; iMIoU loss is designed, the difference between a prediction frame and a real frame is reflected, and the capability of processing laser welding spots with different scales and IoU values is improved. According to the method, the scale information, the texture features and the bounding box regression information of the laser welding spots can be more effectively captured, so that the detection accuracy of the model is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of laser solder joint detection, and particularly relates to a laser solder joint defect detection method and system based on an improved YOLOv9 model. Background Art

[0002] With the advantages of high precision and high efficiency, laser welding technology has been widely used in modern equipment manufacturing, such as radar and precision electronic equipment. The quality of laser solder joints is directly related to the reliability and safety of equipment. However, the complex heat conduction and material properties during the laser welding process pose higher requirements for the quality of solder joints. Compared with PCB solder joints, the detection complexity of laser solder joints increases significantly. The diversity of their sizes, shapes, and the complexity of texture information make traditional detection methods face many challenges. At the same time, the problem of inaccurate bounding box positioning is also a major difficulty in the research of this field.

[0003] In recent years, the rapid development of deep learning technology has provided broad prospects for the application of object detection algorithms in the field of industrial automation detection. Mainstream algorithms such as Faster R-CNN, CenterNet, Deformable DETR, and the YOLO series have all demonstrated excellent performance in different scenarios. Among them, the YOLO series algorithms, due to their single-stage detection architecture, have real-time detection capabilities while ensuring relatively high detection accuracy, and have been widely used in industrial detection.

[0004] Although object detection technology based on deep learning has been relatively mature, research in the field of solder joint detection mainly focuses on PCB solder joints, and detection algorithms specifically for laser solder joints are still very scarce. During the welding process, laser solder joints will exhibit characteristics such as inconsistent target sizes and complex texture information. These factors lead to problems such as difficulties in multi-scale target recognition, complex textures, and inaccurate bounding box positioning during the laser solder joint detection process. Summary of the Invention

[0005] The purpose of the present invention is to propose a laser solder joint defect detection method and system based on an improved YOLOv9 model, which can more effectively capture the scale information, texture features, and bounding box regression information of laser solder joints, thereby significantly improving the detection accuracy of the model.

[0006] According to the first aspect of the embodiments of the present disclosure, a laser solder joint defect detection method based on an improved YOLOv9 model is provided, including the following steps:

[0007] Obtain a laser solder joint image dataset, and annotate the dataset to distinguish different categories of laser solder joints;

[0008] By using the hybrid local-global attention (HLGA) mechanism, while capturing the complex texture details of laser solder joints, the global features are maintained, and skip connections are utilized to reduce the risk of feature loss;

[0009] The BiFPN (Bidirectional Feature Pyramid Network) architecture is adopted to achieve the bidirectional flow and fusion of multi-level laser solder joint features;

[0010] A diverse branch enhanced connection (DBCE) module is constructed to enhance the laser solder joint detection ability in complex backgrounds, capture key information at multiple scales, and optimize the inference speed;

[0011] The IMIoU (Inner-MPDIoU) loss is designed to reflect the difference between the predicted bounding box and the ground truth bounding box, and improve the ability to process laser solder joints with different scales and IoU values.

[0012] In one embodiment, to enrich the laser solder joint image data and simulate the scenarios that may be encountered in the actual detection environment, three data augmentation techniques are implemented on the labeled dataset, including rotation transformation, contrast transformation, and noise perturbation.

[0013] In one embodiment, the hybrid local-global attention mechanism first performs local average pooling and global average pooling on the laser solder joint image. After pooling, the features are flattened in the spatial dimension through a Reshape operation; the number of channels is compressed through 1D convolution, and the importance of the channels is learned to reallocate weights, strengthening the features related to laser solder joint detection while weakening irrelevant information, thereby implementing the channel attention mechanism; after the channel attention mechanism processes, the local features restore the spatial structure through a Reshape operation; while the global features are upsampled to the original size through unpooling (UNAP) and fused with the local features through an addition operation;

[0014] The fused feature information is unpooled again and combined with the original features through a multiplication operation to selectively strengthen the key features; the strengthened feature map then undergoes depthwise separable convolution (DW convolution) to further extract features.

[0015] In one embodiment, the BiFPN architecture introduces bidirectional feature flow. While the high-level semantic features are propagated top-down, the low-level detailed features are fed back bottom-up. When fusing features, the weight distribution of different layers of features can be dynamically adjusted according to the actual scenario.

[0016] In one embodiment, during training, the branch enhancement connection module uses different branches to learn diverse features, and during inference, these branches are simplified into a single convolutional layer; in addition, this module also introduces a combined structure of group convolution and 1*1-k*k convolution. By grouping the input features and using different convolutional kernels for each group, it can effectively capture feature information at different scales and enhance the learning ability of multi-scale features.

[0017] In one embodiment, in the IMIoU loss represents the boundary of the ground truth box, and (b l ,b r ,b t ,b b ) is the boundary of the auxiliary box; the intersection area is obtained by multiplying the intersection lengths in the horizontal and vertical directions:

[0018]

[0019] The union area takes into account the areas of the ground truth box and the auxiliary box and the scaling factor ratio:

[0020] union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter

[0021] The IoU of the adaptive auxiliary box is defined as:

[0022]

[0023] The final IMIoU loss function is defined as:

[0024] L IMoU =L MPDIOU +IoU-IoU inner

[0025] According to the second aspect of the embodiments of the present disclosure, a laser solder joint detection system improved based on YOLOv9 is provided, including:

[0026] An annotation module that obtains a laser solder joint image dataset and annotates the dataset to distinguish different types of laser solder joints;

[0027] An attention module that, by using a hybrid local-global attention mechanism (HLGA), captures complex texture details of laser solder joints while maintaining global features and uses skip connections to reduce the risk of feature loss;

[0028] The BiFPN (Bidirectional Feature Pyramid Network) architecture module realizes the bidirectional flow and fusion of multi-level laser solder joint features;

[0029] The diverse branch enhanced connection (DBCE) module enhances the laser solder joint detection ability in complex backgrounds, captures key information at multiple scales, and optimizes the inference speed;

[0030] The Inner-MPDIoU (IMIoU) loss module reflects the difference between the predicted box and the ground truth box, and improves the ability to process laser solder joints with different scales and IoU values.

[0031] According to the third aspect of the embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program running on the memory. When the processor executes the program, the laser solder joint defect detection method based on the improved YOLOv9 model is implemented.

[0032] According to the fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the laser solder joint defect detection method based on the improved YOLOv9 model is implemented.

[0033] The above technical solutions adopted by the present invention, compared with the prior art, have the following advantages: The present invention introduces a new HLGA attention mechanism, which can effectively capture the complex texture details of laser solder joints while maintaining global features, thus significantly reducing the risk of feature loss. In addition, by adopting the BiFPN architecture, the bidirectional flow and fusion of multi-level features are realized, enabling the full integration of laser solder joint information at different scales. To further improve the diversity of laser solder joint feature extraction and inference performance in complex backgrounds, the present invention proposes a DBCE module. At the same time, the designed IMIoU loss function can more accurately reflect the difference between the predicted box and the ground truth box, thereby enhancing the ability to process samples with different scales and IoU values. Description of the Drawings

[0034] The specification drawings constituting a part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation of this application.

[0035] Figure 1 It is the overall model architecture diagram of the present invention;

[0036] Figure 2 It is the dataset type diagram of the present invention;

[0037] Figure 3 It is a comparison chart of the original image and the enhanced image of the present invention;

[0038] Figure 4 It is a flowchart of HLGA of the present invention;

[0039] Figure 5 It is a comparison chart of the architectures of FPN and BiFPN of the present invention;

[0040] Figure 6 It is an architecture diagram of the DBCE module proposed by the present invention;

[0041] Figure 7 It is an architecture diagram of the DBB proposed by the present invention;

[0042] Figure 8 It is a comparison chart of the losses of the present invention;

[0043] Figure 9 It is a comparison chart of the comparative experiments of the present invention;

[0044] Figure 10 It is a comparison chart of the ablation experiments of the present invention;

[0045] Figure 11 It is a comparison chart of the detection effects of YOLOv9 and the network model proposed by the present invention. Detailed implementation manners

[0046] The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

[0047] It should be noted that the following detailed descriptions are all illustrative and are intended to provide further descriptions of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present application belongs.

[0048] It should be noted that the terms used herein are only for describing specific implementation manners and are not intended to limit the exemplary implementation manners according to the present application. As used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0049] Note that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of methods and systems according to various embodiments of the present disclosure. It should be noted that each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code may include one or more executable instructions for implementing the logical functions specified in each embodiment. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the flowchart and / or block diagram, and the combinations of blocks in the flowchart and / or block diagram, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.

[0050] Embodiment 1:

[0051] This embodiment provides a laser solder joint defect detection method based on an improved YOLOv9 model, which adds an HLGA attention mechanism module to the backbone network part of YOLOv9; modifies the FPN architecture to a BiFPN architecture in the neck of YOLOv9; and changes the feature extraction module of YOLOv9 to a DBCE module, as Figure 1 shown, including the following steps:

[0052] S1. Obtain a laser solder joint image dataset and label the dataset to distinguish different types of laser solder joints;

[0053] Specifically, the dataset used in this embodiment is sourced from laser solder joint images in actual industrial scenarios, and each image contains six solder joints. To effectively evaluate the quality of laser solder joints, technicians classify them into seven types according to the characteristics of the laser solder joints: OK, GB, XH, DL, SLX, HC, and WNL, as Figure 2 shown; according to the classification criteria proposed by the technicians, the laser solder joint image dataset is labeled.

[0054] To enrich the sample data and simulate the scenarios that may occur in actual industrial laser solder joints, the present invention performs three data augmentation processes on the original dataset. The specific data augmentation methods include: 1) Rotation transformation: Randomly rotate the image by an angle to simulate different detection perspective changes. 2) Contrast transformation: Randomly adjust the saturation of the image to simulate the changes in the image under different lighting conditions. 3) Noise perturbation: Add random noise to the image to simulate the interference that may occur during the image acquisition process and improve the anti-noise ability of the model. The comparison between the original image and the enhanced image is as Figure 3As shown

[0055] S2. By using the hybrid local-global attention (HLGA) mechanism, while capturing the complex texture details of laser solder joints, the global features are maintained, and skip connections are used to reduce the risk of feature loss;

[0056] The novel HLGA attention mechanism of the present invention copes with complex texture changes by simultaneously capturing local details and global background information in the laser solder joint image. Due to the complex texture of the laser solder joint image, a single local or global feature extraction method often ignores key details or global semantics. By simultaneously capturing local and global features, HLGA can not only enhance the focus on key areas of laser solder joints, but also effectively filter out irrelevant background interference, thereby improving the accuracy of laser solder joint detection. The HLGA process is as Figure 4 shown

[0057] HLGA first performs local average pooling (LAP) and global average pooling (GAP) on the input laser solder joint image (C, W, H), where C represents the number of channels, and W and H represent the width and height of the feature map respectively. Local pooling is used to extract minute details in the laser solder joint image and capture local changes in the texture, while global pooling is used to obtain overall context information to help the model understand the global structure and background of the laser solder joint image. Through this dual pooling, HLGA can balance the weights of local and global features and comprehensively capture the information of the laser solder joint image.

[0058] Furthermore, the pooled features undergo a Reshape operation to flatten the spatial dimensions in preparation for subsequent 1D convolution. 1D convolution can not only compress the number of channels, but also redistribute weights by learning the importance of channels, strengthening features related to laser solder joint detection while weakening irrelevant information, thereby implementing the channel attention mechanism. After the channel attention mechanism processes, the local features are restored to the spatial structure through Reshape, preparing for attention processing in the spatial dimension; while the global features are upsampled to the original size through unpooling (UNAP) and fused with the local features through an addition operation, enabling the model to retain details while also using global information to enhance the understanding of the laser solder joint structure.

[0059] Furthermore, the fused feature information undergoes unpooling again and is combined with the original features through a multiplication operation to selectively strengthen key features, thereby improving the detection accuracy. The fused feature map then undergoes depthwise separable convolution (DW convolution) to further extract features and reduce the computational complexity.

[0060] Furthermore, to ensure the stability of the model in the deep network, HLGA adopts skip connections to fuse the original features with the features after pooling and convolution in the channel dimension.

[0061] S3. Adopt the BiFPN architecture to achieve two-way flow and fusion of multi-level laser solder joint features;

[0062] The BiFPN architecture is an improvement on the original FPN structure in Yolov9, aiming to enhance the fusion ability of multi-level laser solder joint feature information. The traditional FPN adopts unidirectional feature flow, where high-level semantic features are propagated from top to bottom, and low-level detailed features cannot be effectively fed back to the high level, which may lead to the loss of small-scale key detail information when dealing with multi-scale problems. Especially in complex backgrounds, small or blurred solder joints in laser solder joint detection may be difficult to be accurately recognized. The architecture comparison diagram of FPN and BiFPN is as Figure 5 shown.

[0063] Furthermore, by introducing two-way feature flow, BiFPN allows high-level semantic features to be propagated from top to bottom while low-level detailed features can be fed back from bottom to top. This two-way flow method ensures that laser solder joint features at different scales can be fully fused. Especially in the detection of small-scale laser solder joints, the detail information of low-level features and the semantic information of high-level features are well fused, thus enhancing the model's perception ability of subtle features.

[0064] Furthermore, BiFPN introduces a learnable weighting mechanism, which can dynamically adjust the weight distribution of features at different layers during feature fusion. This mechanism ensures that the model can adaptively adjust the importance of each layer of features, thereby reducing information redundancy while maximizing the retention of key information. Compared with the simple feature splicing method of traditional FPN, BiFPN not only improves the efficiency of feature fusion but also enhances the model's detection performance in dealing with complex and variable laser solder joint images.

[0065] S4. Construct a diverse branch enhanced connection (DBCE) module to enhance the laser solder joint detection ability in complex backgrounds, capture multi-scale key information and optimize the inference speed;

[0066] This module refers to the concept of the decoupled branch block DBB, which can enhance the laser solder joint detection ability of the model in complex backgrounds, capture multi-scale key information and optimize the inference speed. The overall process of the DBCE module is as Figure 6 shown.

[0067] By introducing diverse branch structures, DBB enables the network to capture multi-scale laser solder joint features more fully during the training phase. Specifically, DBB uses multiple convolutional kernels of different sizes (such as 1*1 and k*k convolutions) and average pooling operations during the training phase, enabling the network to effectively extract and fuse information at various scales. In addition, DBB also designs a structure that separates training and inference, such as Figure 7 shown. During training, complex branches are used to learn diverse features, while during inference, these branches are simplified to a single convolutional layer to ensure efficiency in inferring laser solder joints.

[0068] Furthermore, the DBCE module also adopts this idea, enhancing the diversity of features through multiple branches and merging them into a simplified computational path during the inference phase to ensure inference efficiency. In addition, DBCE also introduces a combined structure of group convolution and 1*1-k*k convolution. By grouping the input features and using different convolutional kernel operations for each group, it effectively captures feature information at different scales, enhancing the multi-scale feature learning ability. Especially in the laser solder joint detection task under complex backgrounds and variable scenarios, it further improves the accuracy and robustness of the model.

[0069] S5. Designed the IMIoU loss, which reflects the difference between the predicted bounding box and the ground truth box, improving the ability to handle laser solder joints with different scales and IoU values.

[0070] In object detection and instance segmentation, traditional bounding box regression loss functions often fail to accurately reflect the difference between the predicted bounding box and the ground truth box when dealing with bounding boxes of the same aspect ratio but different sizes, especially in applications such as precisely regressing laser solder joints. To address this issue, a new bounding box similarity metric method - MPDIoU is proposed for the precise regression of laser solder joints. This method is calculated based on the minimum point distance of the bounding box, comprehensively considering the overlapping area, center point distance, and width-height deviation. In two different bounding box regression results (the green box is the ground truth bounding box, and the red box is the predicted bounding box), the loss values calculated by traditional loss functions (such as GIoU, DIoU, CIoU, and EIoU) are the same, but the loss values obtained by the MPDIoU method are different, as Figure 8 shown. This indicates that traditional methods have limitations in distinguishing bounding boxes with the same aspect ratio but different sizes or positions, while MPDIoU can more accurately reflect the difference between them, providing a more effective loss metric method.

[0071] During the training phase, by minimizing the MPDIoU loss function, the model can better align the predicted bounding box with the ground truth bounding box, thereby improving the detection accuracy and more effectively learning the features of different laser solder joints.

[0072] Bprd = [x prd , y prd , w prd , y prd T

[0073] B gt = [x gt , y gt , w gt , h gt T

[0074] Approximate its true bounding box by minimizing the following loss function;

[0075]

[0076] where B prd is the set of predicted bounding boxes, B gt is the set of true bounding boxes, and θ are the parameters of the regression depth model. represents the squared distance between the top-left corner points of the true box and the predicted box, represents the squared distance between the bottom-right corner points of the true box and the predicted box.

[0077] L MPDIoU = 1 - MPDIoU

[0078] Although MPDIoU performs well in the bounding box regression of laser solder joints, the problem of inconsistent gradient changes still exists when dealing with samples in different IoU value intervals. To address this challenge, this example proposes an adaptive auxiliary bounding box mechanism. This mechanism dynamically adjusts the scale of the bounding box according to the IoU loss of the auxiliary bounding box to better adapt to the characteristics of laser solder joints. For laser solder joint samples with high IoU values, a smaller-scale auxiliary bounding box is used for loss calculation to accelerate the convergence speed of the model; while for samples with low IoU values, a larger-scale auxiliary bounding box is adopted to improve the accuracy of regression. This adaptive adjustment mechanism not only enhances the generalization ability of the IMIoU (Inner-MPDIoU) function in laser solder joint detection but also enables it to be optimized according to different detection tasks, thereby further improving the overall model performance. represents the boundary position of the true box, (b l , b r , b t , b b ) are the boundaries of the auxiliary bounding box. The intersection area is calculated by the product of the intersection lengths in the horizontal and vertical directions:

[0079] The union area takes into account the areas of the true box and the auxiliary detection box and the scaling factor ratio:

[0080] ​​union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter

[0081] The IoU of the adaptive auxiliary box is defined as:

[0082]

[0083] The definition of the final IMIoU loss function is:

[0084] L IMoU =L MPDIOU +IoU - IoU miner

[0085] In this embodiment, the model performance is evaluated on a computing platform equipped with four NVIDIA GeForce RTX 2080Ti GPUs (each GPU is equipped with 11GB RAM). The processor is an Intel Xeon CPU E5-2680 v4 @ 2.40GHz. The experiment is carried out using Python 3.8 and Torch 2.1.0 in a Linux environment. During the training process, the epoch is set to 500, the batch size is 12, and the resolution of the input laser solder joint image is 640×640 pixels. The learning rate is 0.01, the weight decay is 0.0005, the momentum is 0.937, and the optimizer selected is SGD.

[0086] This embodiment uses the following metrics to evaluate the effectiveness of the algorithm: precision, recall, mean average precision (mAP), and F1 score. mAP is the average AP value of different laser solder joint categories and is defined as follows:

[0087]

[0088] where N represents the number of laser solder joint types, and AP is the area enclosed by the PR curve.

[0089]

[0090] P represents precision, that is, the probability of being correctly classified among the predicted positive samples. The calculation formula is the ratio of the correctly predicted positive instances to all instances marked as positive by the model. TP is the number of correctly identified positive samples, and FP is the number of negative samples mislabeled as positive.

[0091]

[0092] R represents the recall rate, which is the proportion of the actual positive samples correctly identified by the model. FN is the number of positive samples misclassified as negative.

[0093]

[0094] The F1 score is the harmonic mean of precision and recall, used to evaluate the overall accuracy and integrity of the model.

[0095]

[0096] To verify the advantages of the network model proposed in the present invention, a comparative experiment was conducted in this embodiment with the current mainstream models Yolov5l, Yolov8l, Yolov9, Yolov10l, RetinaNet, CenterNet, and Faster R-CNN on the enhanced dataset. The results of the comparative experiment are shown in Table 1, and the comparison results of mAP 50 are shown in Figure 9 it.

[0097] Table 1 Comparative Experiment Results

[0098]

[0099]

[0100] The experimental results show that the method proposed in the present invention is superior to other existing methods in all key indicators. To more comprehensively evaluate the impact of these improvement measures and diverse scenarios on the performance of laser solder joint detection, we conducted ablation experiments. By gradually introducing the HLGA attention mechanism, BiFPN structure, DBCE module, and IMIoU loss, we verified the specific contributions of each part to the performance improvement of the laser solder joint detection model. This experiment not only confirmed the effectiveness of each component in complex backgrounds but also enhanced the system's ability to recognize the features of laser solder joints. The results of the ablation experiment are shown in Table 2, and the comparison results of mAP 50 are shown in Figure 10 it.

[0101] Table 2 Ablation Experiment Results

[0102]

[0103] The experimental results show that the various improvement measures have significantly enhanced the performance of the laser solder joint detection model from different perspectives. Through the combined analysis of different modules, the results indicate that the combined effects are better than those of the improvement measures implemented individually. These remarkable improvement results demonstrate that the effective combination of multiple modules has significantly enhanced the model's ability to express laser solder joint features and detection accuracy. Especially when dealing with complex scenarios and multi-scale laser solder joints, the model has demonstrated excellent performance. This further validates the effectiveness and reliability of our method in practical applications. Figure 11 Comparison of the detection effects of YOLOv9 and the network model proposed in the present invention.

[0104] Example Two:

[0105] This example provides a laser solder joint detection system improved based on YOLOv9, including:

[0106] A labeling module, which acquires a laser solder joint image dataset and labels the dataset to distinguish different types of laser solder joints;

[0107] An attention module, by using a hybrid local-global attention (HLGA) mechanism, while capturing the complex texture details of laser solder joints, it maintains global features and uses skip connections to reduce the risk of feature loss;

[0108] A BiFPN architecture module, which realizes the two-way flow and fusion of multi-level laser solder joint features;

[0109] A diverse branch enhanced connection (DBCE) module, which enhances the laser solder joint detection ability in complex backgrounds, captures key information at multiple scales and optimizes the inference speed;

[0110] An IMIoU loss module, which reflects the difference between the predicted box and the ground truth box and improves the ability to process laser solder joints with different scales and IoU values.

[0111] Example Three:

[0112] An electronic device, including a memory, a processor, and a computer program running on the memory, wherein when the processor executes the program, it implements the above-mentioned method for detecting laser solder joint defects based on an improved YOLOv9 model, including:

[0113] Acquire a laser solder joint image dataset and label the dataset to distinguish different types of laser solder joints;

[0114] By using the hybrid local-global attention (HLGA) mechanism, while capturing the complex texture details of laser solder joints, the global features are maintained, and skip connections are utilized to reduce the risk of feature loss;

[0115] The BiFPN architecture is adopted to achieve the two-way flow and fusion of multi-level laser solder joint features;

[0116] A diverse branch enhanced connection (DBCE) module is constructed to enhance the laser solder joint detection ability in complex backgrounds, capture key information at multiple scales, and optimize the inference speed;

[0117] The IMIoU loss is designed to reflect the difference between the predicted box and the ground truth box, improving the ability to process laser solder joints with different scales and IoU values.

[0118] Example 4:

[0119] A computer-readable storage medium stores a computer program thereon, and when the program is executed by a processor, it implements the above-mentioned laser solder joint defect detection method based on the improved YOLOv9 model, including:

[0120] Obtain a laser solder joint image dataset and annotate the dataset to distinguish different categories of laser solder joints;

[0121] By using the hybrid local-global attention (HLGA) mechanism, while capturing the complex texture details of laser solder joints, the global features are maintained, and skip connections are utilized to reduce the risk of feature loss;

[0122] The BiFPN (Bidirectional Feature Pyramid Network) architecture is adopted to achieve the two-way flow and fusion of multi-level laser solder joint features;

[0123] A diverse branch enhanced connection (DBCE) module is constructed to enhance the laser solder joint detection ability in complex backgrounds, capture key information at multiple scales, and optimize the inference speed;

[0124] The IMIoU (Inner-MPDIoU) loss is designed to reflect the difference between the predicted box and the ground truth box, improving the ability to process laser solder joints with different scales and IoU values.

[0125] Those skilled in the art should understand that the above-mentioned modules or steps of the present disclosure can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present disclosure is not limited to any specific combination of hardware and software.

[0126] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, the present application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included within the protection scope of the present application.

[0127] Although the specific implementation manners of the present disclosure have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that, based on the technical solutions of the present disclosure, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present disclosure.

Claims

1. A laser solder joint defect detection method based on the improved YOLOv9 model, characterized in that: The following steps are involved: Obtain a laser weld spot image dataset and annotate the dataset to distinguish different types of laser weld spots; By using a hybrid local-global attention mechanism, the global features are maintained while capturing the complex texture details of the laser weld, and skip connections are used to reduce the risk of feature loss. Adopt BiFPN architecture to achieve bidirectional flow and fusion of multi-level laser welding point features; Build a branch-enhanced connection module to enhance the laser weld spot detection capability in complex backgrounds, capture multi-scale key information and optimize the inference speed; The IMIoU loss is designed to reflect the difference between the predicted box and the true box and improve the ability to handle laser welds of different scales and IoU values.

2. According to the laser welding spot defect detection method based on the improved YOLOv9 model of claim 1, it is characterized in that: Three data enhancement techniques are implemented on the annotated dataset, including rotation transformation, contrast transformation, and noise perturbation.

3. According to the laser welding spot defect detection method based on the improved YOLOv9 model of claim 1, it is characterized in that: The hybrid local-global attention mechanism first performs local average pooling and global average pooling on the laser weld image, and the pooled features are flattened by a Reshape operation; the number of channels is compressed by 1D convolution, and the importance of the channels is learned to redistribute weights, thereby strengthening the features related to laser weld detection and weakening irrelevant information, thereby realizing the channel attention mechanism; after the channel attention mechanism is processed, the local features are restored to the spatial structure through the Reshape operation; and The global features are upsampled to the original size through unpooling and fused with the local features through addition operations; The fused feature information is unpooled again and combined with the original features through multiplication operations to selectively strengthen key features; The enhanced feature map is then subjected to depthwise separable convolution to further extract features.

4. According to the laser welding spot defect detection method based on the YOLOv9 improved model of claim 1, it is characterized in that: The BiFPN architecture introduces bidirectional feature flow, where high-level semantic features are propagated from top to bottom while low-level detail features are fed back from bottom to top.

5. According to the laser welding spot defect detection method based on the improved YOLOv9 model of claim 1, it is characterized in that: The BiFPN architecture introduces a learnable weighting mechanism, which can dynamically adjust the weight distribution of features at different layers according to the actual scenario during feature fusion.

6. According to the laser welding spot defect detection method based on the improved YOLOv9 model of claim 1, it is characterized in that: The branch enhanced connection module uses different branches to learn diverse features during training, and simplifies these branches into a single convolution layer during inference; in addition, the module also introduces a group convolution and 1*1-k*k convolution combined structure. By grouping the input features and using different convolution kernel operations for each group, it effectively captures feature information of different scales and enhances the learning ability of multi-scale features.

7. According to the laser welding spot defect detection method based on the YOLOv9 improved model of claim 1, it is characterized in that: The IMIoU loss represents the boundary of the real box, (b l ,b r ,b t ,b b ) is the boundary of the auxiliary frame; the intersection area is obtained by multiplying the intersection lengths in the horizontal and vertical directions: The union area takes into account the area of ​​the real frame and the auxiliary frame and the scaling factor ratio: union=(w gt *h gt )*(ratio) 2 +(w*h)*(ratio) 2 -inter The IoU of the adaptive auxiliary box is defined as: The final IMIoU loss function is defined as: L IMoU =L MPDIOU +IoU-IoU inner .

8. A laser welding spot detection system based on YOLOv9 improvement, characterized in that: include: A labeling module obtains a laser welding spot image dataset and labels the dataset to distinguish different types of laser welding spots; The attention module uses a hybrid local-global attention mechanism to capture the complex texture details of laser welds while maintaining global features and uses skip connections to reduce the risk of feature loss; BiFPN architecture module realizes the bidirectional flow and fusion of multi-level laser welding point features; The branch-enhanced connection module enhances the laser weld detection capability in complex backgrounds, captures key information at multiple scales, and optimizes the inference speed; The IMIoU loss module reflects the difference between the predicted box and the real box, and improves the ability to process laser welds of different scales and IoU values.

9. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, characterized in that: When the processor executes the program, the laser welding spot defect detection method based on the improved YOLOv9 model is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, a laser welding spot defect detection method based on the improved YOLOv9 model is implemented.

Images