Automobile door panel welding spot defect detection method and system based on small object target detection

Abstract

The invention discloses an automobile door panel welding spot defect detection method and system based on small object target detection. The method comprises the following steps: establishing an automobile door panel welding spot data set; wherein the training set is divided into a training set of a target detection module model and a training set of a normal welding spot feature library; training a target detection model according to the data set to obtain a target detection model capable of accurately identifying the welding spots of the automobile door panel picture; based on the trained target detection model, target detection and feature extraction are carried out on the automobile door panel pictures only containing the normal welding spots, lightweight processing is carried out, and a normal welding spot feature library is formed; performing target detection on the to-be-detected automobile door panel picture by using a pre-trained target detection model, and extracting all features; the obtained features are calculated and compared with matched features in a normal welding spot feature library, the abnormal score of each pixel is obtained, and whether each pixel is abnormal or not is judged according to a preset threshold value; and if yes, determining that the current welding spot has defects and positioning the positions of the defects.

Description

Technical Field

[0001] The present invention relates to the technical field of welding detection, and in particular to a method and system for detecting automobile door panel weld defects based on feature library small object target detection. Background Art

[0002] In the automotive manufacturing process, the quality of welds directly affects the structural strength and safety of the entire vehicle, so the detection and evaluation of weld defects is crucial. Traditional weld defect detection methods mostly rely on image segmentation technology. These methods usually segment the weld area in the image separately and analyze it. Traditional image segmentation methods mainly process individual welds in a single image, identifying welds through techniques such as threshold segmentation and edge detection. They can usually detect the location of welds relatively accurately, but their performance in complex backgrounds is unsatisfactory. Because these methods mostly rely on manually designed features and rules, they have significant limitations, especially when dealing with the detection of multiple welds, the interference of complex backgrounds, or the detection of smaller welds, the accuracy decreases significantly.

[0003] In recent years, deep learning has demonstrated significant advantages in solder joint defect detection. Through end-to-end training, deep learning methods can automatically extract high-level image features and perform object classification and regression, adapting to more complex scenarios and diverse defect types. Compared with traditional methods, deep learning offers higher accuracy and greater robustness. However, deep learning methods also have limitations, particularly when dealing with large-scale, multi-target, and object-sized datasets, resulting in high computational resource consumption and prolonged training time.

[0004] To address this issue, object detection algorithms have emerged as a new direction for solder joint defect detection. Object detection algorithms possess strong capabilities for object detection and localization. They can identify multiple objects in an image and output bounding boxes, helping to quickly locate solder joint defects. However, they still have significant shortcomings when detecting small objects. In particular, when the solder joint occupies a small proportion of the image or when the image background is complex, the detection accuracy of object detection algorithms can drop significantly. This is fundamentally due to the poor adaptability of the object detection algorithm's gridding mechanism and fixed anchor boxes for small objects. When solder joint defects are small, the object detection algorithm's gridding often causes the tiny defect to be confused with the surrounding background or other objects, making accurate classification difficult. Furthermore, when the contrast between the target and the background is low, the object detection algorithm's feature extraction and classification processes cannot effectively distinguish the solder joint defect from the complex background, thus affecting detection accuracy.

[0005] Furthermore, while the object detection algorithm can accurately locate the bounding box of a solder joint, it cannot provide a detailed segmentation map of the defect. In other words, for each detected solder joint, the object detection algorithm can only provide a rough bounding box, unable to precisely calibrate the shape and location of the defect area. This limitation places higher demands on the accuracy and reliability of defect detection in practical applications, as understanding the details and shape of the defect is crucial for subsequent quality assessment and processing. Summary of the Invention

[0006] The present invention aims to solve the defects of the prior art and provide a method and system for detecting defects in automobile door panel welds based on small object target detection. The positioning and evaluation method is divided into two stages in the training phase. The first is the automobile door panel weld target detection training phase, in which the automobile door panel welds are semantically annotated so that the network can accurately find the weld positions in the automobile door panel image. Then, based on the deep learning target detection method, the automobile door panel weld area is located, the weld area image features are extracted using a pre-trained network, the features of normal weld images are learned, and a normal weld memory library is constructed. In the detection phase, the specific area of ​​the automobile door panel weld is located using a deep learning target detection method, the weld area image features are extracted using a pre-trained network, and the extracted features are compared with the constructed normal weld feature memory library to achieve more accurate automobile door panel weld defect judgment, determine the specific location of the weld defect, and realize positioning analysis and online evaluation of surface damage of automobile door panel welds.

[0007] The first aspect of the present invention is to disclose a method for detecting and locating automobile door panel weld defects based on small object target detection, comprising:

[0008] Step 1: Establish a dataset of automobile door panel welds; the established dataset includes a training set, a validation set, and a test set; the training set is divided into two parts: one is a training set for the target detection module model, which is used to train the target detection network; the other is a training set for the normal weld feature library, which is used to train the normal weld feature library, wherein the training set of the normal weld feature library is an image of the normal weld area within the normal weld mark box in the target detection training set;

[0009] Step 2: Train the target detection model using the training set of the target detection module model in the automobile door panel weld dataset established in step 1. The training of the network framework of the target detection model includes extracting features using a pre-trained ResNet50 feature extraction network, using GIOU as the loss function for coordinate regression, and adopting an optimization method of learning rate adjustment. The learning rate is initially set to 0.0001, and the Adam optimization algorithm is used to associate the learning rate with the current training number and continuously update it.

[0010] When the loss function stops decreasing, cross-validation is used to evaluate the model. The best model is evaluated using the validation set and then modified to complete the training of the network framework. A multi-level feature connection-based target detection model is obtained that can accurately identify welds in images of automobile door panels and generate prediction boxes containing areas of normal and abnormal welds.

[0011] Step 3: Use the training set of the normal solder joint feature library established in Step 1 to train the normal solder joint feature library. Manually mark the solder joint areas in the normal solder joint images in the training set. Use the object detection model trained in Step 2 to perform object detection and feature extraction on the car door panel images containing only normal solder joints, ensuring that each solder joint area is accurately extracted. Use the core subset algorithm to lightweight the features. The lightweight features constitute the normal solder joint feature library.

[0012] Step 4: Use the trained target detection model to perform target detection on the image of the car door panel to be inspected, and obtain the area where all (including normal welds and abnormal welds) are located; then use the pre-trained ResNet50 network to extract features in the area; calculate the geometric distance between it and the matching features in the normal weld feature library obtained in step 3, and use this distance as the anomaly score of the feature; then restore the feature to the image scale to obtain the anomaly score of each pixel, and then normalize the anomaly score to the range of 0 to 1, and judge whether each pixel has an anomaly based on the preset threshold; if it is judged that there is an anomaly, the current weld is considered to be defective and the defect position is located.

[0013] Furthermore, the step 1 specifically includes:

[0014] Construct a training set for the object detection module model: An industrial camera is used to capture multiple images of automobile door panels containing welds as training data. The images include images of automobile door panel welds from different angles, lighting conditions, and material conditions. 80% of these images are selected as the automobile door panel training set, 10% as the validation set, and 10% as the test set. The locations of the welds in each image are marked with rectangular boxes as training labels.

[0015] Construct a training set for the normal solder joint feature library: In the training set of the target detection module model, mark the boxes of all normal solder joints in the training set, crop the pictures according to the marked boxes, and use the cropped pictures inside the marked boxes as the training set for the normal solder joint feature library.

[0016] In step 1, only the target detection network is trained to locate all types of solder joints without classification. The positions of all solder joints (including normal and abnormal solder joints) in the image are marked as training labels. At this time, normal or abnormal solder joints are not distinguished, and solder joints are used as unified training labels.

[0017] The images in the training set in step 1 are processed as follows:

[0018] Perform random rotation, scaling, and translation operations on the training data to display the solder joint area at different angles;

[0019] Adjust the brightness, contrast, and hue of the image to simulate the solder joint area under different lighting environments;

[0020] Noise is added and occlusion is performed on the solder joint area to simulate the conditions of the solder joint area under different materials.

[0021] Furthermore, the step 2 specifically includes:

[0022] S21: Establish a deep learning target detection framework. Use the target detection algorithm to import the training labels from step 1 as a training dataset into the target detection model for model training. This obtains a target detection model that can accurately identify solder joint areas, thereby accurately identifying solder joint areas from images. The training labels are only used to identify solder joint areas and do not distinguish between normal and abnormal areas.

[0023] S22: Monitor the loss function value during training to determine whether the model is overfitting or underfitting and make appropriate adjustments. GIOU is used as the loss function for coordinate regression, which is defined as follows:

[0024]

[0025] Among them, A and B represent the true box and the regression box respectively, IOU is the intersection-over-union ratio between A and B, C is the minimum closed figure containing A and B, and C\(A∪B) represents the part of C that removes A∪B. As can be seen from the formula, the output range of GIOU is (-1,1]; GIOU is the lower bound of IOU; and when IOU is 0, GIOU is not 0. The farther the true box is from the regression box, the closer GIOU is to -1. When the true box is closer to the regression box, the closer GIOU is to 1.

[0026] When the loss function no longer decreases, use cross-validation to evaluate the accuracy of the model;

[0027] S23: Based on the evaluation results, the Adam optimization algorithm is used to update the parameters, adjust the model structure or training parameters, and thus complete the training of the network framework.

[0028] The step three specifically includes:

[0029] S31: Using the trained object detection module obtained in step 2, perform object detection on the image of the automobile door panel containing only normal welds, automatically identify and segment the weld area, and use the pre-trained convolutional neural network ResNet50 to extract local features of the welds within the weld area; and normalize the extracted features;

[0030] S32: Establish a normal solder joint feature library based on the geometric shape, surface condition and material type of the solder joint; and use the core subset lightweight method to remove redundant features in the feature library.

[0031] Furthermore, the local features in S31 include color, texture, shape and edge information.

[0032] The step 4 specifically includes:

[0033] S41: Use the object detection model trained in step 1 to perform object detection on the image of the automobile door panel to be detected, automatically identify and segment the welding point area, and obtain images of all the segmented welding point areas;

[0034] S42: using the convolutional neural network ResNet50 trained in step 2 to extract local features of the solder joint within the solder joint area, including color, texture, shape, and edge information;

[0035] S43: Based on the feature categories in the solder joint feature library obtained in step 3, preliminarily matching the solder joint feature information extracted in S42 with the features in the normal solder joint feature library obtained in step 3 to determine whether the solder joint has defects;

[0036] The Euclidean distance is then used to calculate the similarity between the extracted solder joint features and each solder joint feature in the feature library:

[0037]

[0038] Where m is the feature vector in the memory bank, m test is the feature vector of the test data;

[0039] Then use maximum and minimum normalization to process the abnormal scores so that the normalized score range is between [0,1];

[0040] After obtaining the normalized anomaly score, a threshold τ is set to perform defect segmentation; if the normalized anomaly score If the value is greater than the set threshold τ, the location is considered to be a defective area, otherwise it is considered to be a normal area;

[0041] The formula for segmentation is as follows:

[0042]

[0043] Among them, D i,j is the segmentation result, indicating whether the position (i, j) belongs to the defect area; D i,j =1 indicates that the location is a defect area; D i,j =0 means the location is a normal area;

[0044] The abnormal scores are converted into binary images to realize the judgment of solder joint defects and the extraction and segmentation of defective areas.

[0045] A second aspect of the present invention is to provide an automobile door panel weld defect detection system based on small object target detection, comprising a weld data set module, a training module, an image preprocessing module, a normal weld feature library module, and a weld identification and positioning module;

[0046] The solder joint dataset module includes a training set, a validation set, and a test set. The training set is divided into two parts: one is a training set for the target detection model, which is used for training the target detection network; the other is a training set for the normal solder joint feature library, which is used for training the normal solder joint feature library.

[0047] The training module includes constructing a target detection model based on multi-level feature connection, training the target detection model based on the training set of the target detection module model of the weld dataset module, and using cross-validation to perform model evaluation, thereby obtaining a target detection model based on multi-level feature connection that can accurately identify welds in automobile door panel images and providing a prediction box containing areas of normal / abnormal welds.

[0048] The image preprocessing module includes: using the manually annotated normal solder joint images in the training set of the solder joint dataset module, intercepting the image within the annotated frame, and extracting features from the intercepted image through the convolutional neural network ResNet50 feature extraction network pre-trained by the training module;

[0049] The normal solder joint feature library module includes: (this module is only run in the training phase) obtaining manually annotated normal solder joint images in the training set of the solder joint dataset module, intercepting the image within the annotated frame, using the features within the normal solder joint annotated frame extracted by the image preprocessing module, and lightweighting the features using the core subset algorithm. The lightweight features constitute the normal solder joint feature library;

[0050] The solder joint identification and positioning module includes (this module is only run in the testing phase): for the area where all solder joints (including normal solder joints and abnormal solder joints) are located given by the target detection network during the test, the image preprocessing module is used to extract features in the area, and the geometric distance between the area and the matching features in the normal solder joint feature library is calculated. This distance is used as the anomaly score of the feature, and the feature is then restored to the image scale to obtain the anomaly score of each pixel. The anomaly score is then normalized to the range of 0 to 1, and a preset threshold is used to determine whether each pixel has an anomaly.

[0051] The third aspect of the present invention is to provide an automobile door panel weld defect detection device based on small object target detection, including a processor and a memory for storing a computer program that can be run on the processor; wherein the processor is used to execute the steps of the above-mentioned detection method when running the computer program.

[0052] A fourth aspect of the present invention is to provide a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the above-mentioned detection method are implemented.

[0053] The beneficial effects of the present invention are:

[0054] The automobile door panel weld defect detection method based on small object target detection described in the present invention utilizes multi-scale feature fusion technology to process images at different scales, ensuring accurate identification even when the weld defects are small. It can accurately identify the automobile door panel weld area and determine whether the weld has defects, effectively improving the detection accuracy of weld damage, reducing human errors, and ensuring the reliability of welding quality.

[0055] Secondly, this method can not only accurately locate solder joints, but also achieve high-precision defect assessment through feature comparison and evaluation. Through data enhancement and diversified training, the model has strong adaptability and can accurately identify solder joints under different lighting, background and material conditions, without being affected by environmental factors, significantly improving the accuracy and reliability of detection.

[0056] Unlike traditional methods that can only detect single solder joints, this method replaces traditional manual inspection with an automated visual inspection system, which can simultaneously process multiple solder joint defects in the image, solving the problem of multi-target detection. It can also speed up the solder joint quality inspection cycle, improve overall production efficiency, and reduce production line downtime. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a network structure diagram of the target detection model constructed in the embodiment;

[0058] Figure 2This is a flow chart of the automobile door panel weld defect detection method based on small object target detection. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions, beneficial effects and significant improvements of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the drawings provided in the examples of the present invention. Obviously, all the described embodiments are only partial embodiments of the present invention, rather than all embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] In the description of this application, unless otherwise expressly specified or limited, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance; the term "plurality" refers to two or more; unless otherwise specified or explained, the terms "connected" and "fixed" should be understood in a broad sense. For example, "connected" can mean a fixed connection, a detachable connection, an integral connection, or an electrical connection; "connected" can mean a direct connection or an indirect connection through an intermediate medium. For those skilled in the art, the specific meanings of the above terms in this application can be understood according to the specific circumstances.

[0061] like Figure 2 As shown, a method for detecting and locating automobile door panel weld defects based on small object target detection in a feature library includes:

[0062] Step 1: Create a dataset of automobile door panel weld points. The dataset includes a training set, a validation set, and a test set. The training set is divided into two parts: one for training the target detection network and the other for training the normal weld point feature library. Specifically, it includes:

[0063] S11: Use an industrial camera to capture multiple images of automobile door panels containing welds as training data. The images include images of automobile door panel welds from different angles, lighting conditions, and material conditions. 80% of these images are selected as the training set, 10% as the validation set, and 10% as the test set. Use the labelimg annotation tool to annotate each image with a rectangular box to mark the location of the welds (including normal and abnormal welds) as training labels. This training set is used for the object detection model.

[0064] To increase the diversity of training data and prevent model overfitting, perform the following operations on the training data:

[0065] Perform random rotation, scaling, and translation on the training data to display the solder joint area at different angles to increase data diversity.

[0066] Adjust the brightness, contrast, and hue of the image to simulate the solder joint area under different lighting environments;

[0067] Noise addition and occlusion processing are performed on the solder joint area to simulate the conditions of the solder joint area under different materials and enhance the robustness of the model.

[0068] This ensures that the solder joint area on the image can cover different angles, lighting, and material conditions, so that the subsequently established model has good generalization capabilities.

[0069] S12: For the annotation boxes of all the welding points in the training set in step S11, the images are cropped according to the annotation boxes, and the cropped images inside the annotation boxes are used as the training set of the data set.

[0070] Step 2: Use the training set in the data set constructed in step 1 to train the deep learning target detection model; use the target detection model to perform target detection on the car door panel image to be inspected, automatically identify and segment all weld area, use the target detection model to extract features from the weld area, and compare them with the features in the normal weld feature library obtained in step 1 to determine whether there are defects in the weld and locate the defect position.

[0071] Specifically, the target detection model includes:

[0072] S21: Using the target detection algorithm to train the image annotated in step S11, a target detection model is obtained that can accurately identify the solder joint area, thereby accurately identifying the solder joint area from the image;

[0073] The target detection model is trained on the training labels of step S11 so that the model can accurately identify the solder joint area. In this stage, normal and abnormal solder joints are not classified. There is only one type of training label, namely solder joints, and only regional recognition of solder joints is achieved.

[0074] Specifically include:

[0075] S211: Establish a deep learning target detection framework, and import the training labels in step S11 as a training data set into the target detection model for model training. Figure 1As shown in the figure, the target detection model uses the deep convolutional neural network ResNet50, which consists of four stages: Stem (initial convolution and pooling layer) and Res2, Res3, Res4, and Res5. This method selects Res2, Res3, and Res4 as subsequent inputs. In the subsequent stages, the deep low-resolution feature map Res4 is gradually amplified through upsampling and concatenated with the shallow high-resolution feature maps Res3 and Res2 to enhance the feature representation capability of objects of different scales. The DBL structure (3×3 convolution + batch normalization + leaky ReLU) is widely used in this stage to ensure the stability and robustness of feature representation.

[0076] GIOU is used as the loss function for coordinate regression, which is defined as follows:

[0077]

[0078] Here, A and B represent the ground truth box and the regression box, respectively. IOU is the intersection-of-union ratio between A and B. C is the minimum closed shape containing A and B. C\(A∪B) represents the portion of C excluding A∪B. As can be seen from the formula, the output range of GIOU is (-1, 1]; GIOU is the lower bound of IOU; and when IOU is 0, GIOU is not 0. The farther the ground truth box is from the regression box, the closer GIOU is to -1. The closer the ground truth box is to the regression box, the closer GIOU is to 1. GIOU has these favorable properties and is directly related to the quality of the regression box, so it can be directly used as a loss function.

[0079] S212: The training dataset is divided into a training set and a validation set for model training and evaluation. The training dataset accounts for 80% of the total data, and the validation dataset accounts for 20%. The dataset contains only one type of training label, which is used only for identifying solder joint areas without differentiation. Training parameters such as the learning rate, batch size, and number of iterations are set, and model training is started.

[0080] S213: Input the training set obtained in step S11 into the target detection framework for training, and use the validation set to evaluate the best model; wherein, during training, all parameters are fully learned so that the network can identify the welding points of automobile door panels.

[0081] S214: Monitor the loss function value during training to determine whether the model is overfitting or underfitting and make appropriate adjustments. When the loss function stops decreasing, use cross-validation to evaluate the accuracy of the model and modify the model to complete the training of the network framework and obtain a network framework that can accurately identify welds in images of automobile door panels. S22: Adjust the model structure or training parameters based on the evaluation results to further optimize the model performance, including:

[0082] This training uses the Adam optimization algorithm to update the parameters. In order to make the training fast and sufficient, this article will manually adjust the most important hyperparameter - the learning rate. In order to maintain fast training of the network, the initial value of the learning rate is set to 0.0001, and then the learning rate is continuously updated according to the formula:

[0083]

[0084] Here, lr_init and lr_end are the initial and final learning rates, respectively, while global_step and train_step are the current and set number of training steps, respectively. Linking the learning rate to the current number of training steps allows it to adjust automatically based on the current training situation, allowing the network to be fully trained while maintaining a high rate and avoiding falling into a local optimum.

[0085] S23: Conduct final testing to verify the model’s performance in a real-world environment and ensure that it can accurately identify solder joints, thereby obtaining a trained object detection model.

[0086] Step 3: Use the trained object detection model obtained in step 2 to perform object detection on the car door panel image containing only normal welds, obtain the segmented image of all normal weld areas, extract features from the above images through the pre-trained network, perform core subset lightweighting on the features, and build a normal weld feature library. Specifically, it includes:

[0087] S31: Use the trained object detection model obtained in step 2 to perform object detection on the image of the car door panel containing only normal welds, automatically identify and segment the weld area, and ensure that each weld area is accurately extracted;

[0088] Use the convolutional neural network ResNet50 trained in step 2 to extract the local features of the solder joints within the solder joint area, including color, texture, shape, edge information, etc.

[0089] The extracted features are normalized to ensure the standardization of features of different solder joints, which is convenient for subsequent comparison.

[0090] The feature vectors obtained from the pre-trained convolutional neural network model are processed by adaptive average pooling. This step ensures that input images of different sizes can produce feature vectors with consistent dimensions:

[0091]

[0092] Among them, i represents the first dimension of the feature, that is, the i-th long feature image, h and w are the pixel coordinates in the feature image, j represents the j-th layer of the ResNet50 network used when extracting features, and fagg is an average pooling function.

[0093] S32: Based on the geometric shape, surface condition, and material type of the solder joints, the extracted features are sorted and a normal solder joint feature library is established, specifically including:

[0094] S321: Optimize the feature library to ensure that the features cover a variety of solder joint shapes and actual working conditions, and can provide a reliable comparison standard in the subsequent inspection stage.

[0095] S322: Redundant features are removed from the feature library to ensure the representativeness of each feature category, including:

[0096] Use the core subset lightweight method to lightweight the obtained feature vector and store it in the feature library:

[0097]

[0098] Among them, M is the feature vector set before sampling, and MC is the feature vector set after sampling.

[0099] Step 4: Detection phase

[0100] S41: Using the trained target detection model obtained in step 1, perform target detection on the image of the automobile door panel to be detected, automatically identify and segment the welding point areas, and obtain images of all the segmented welding point areas;

[0101] S42: Similar to step S32, using the convolutional neural network ResNet50 trained in step 2 to extract local features of the solder joints within the solder joint area, including color, texture, shape, edge information, etc.;

[0102] S43: Based on the feature categories (such as color, texture, shape, etc.) in the solder joint feature library, the solder joint feature information extracted in S42 is preliminarily matched with the features in the normal solder joint feature library obtained in step 3 to determine whether the solder joint has defects;

[0103] The Euclidean distance is then used to calculate the similarity between the extracted solder joint features and each solder joint feature in the feature library:

[0104]

[0105] s * =||m test,* -m * ||2

[0106] Where m is the feature vector in the memory bank, m test is the feature vector of the test data.

[0107] In order to normalize the pixel-level anomaly scores to a uniform range, this method uses maximum and minimum normalization to process the anomaly scores. The formula for maximum and minimum normalization is as follows:

[0108]

[0109] Among them, S i,j is the raw anomaly score, the anomaly value at position (i, j). min(S) and max(S) represent the minimum and maximum anomaly scores of the image, respectively. is the normalized anomaly score, and the normalized score range is between [0,1].

[0110] After obtaining the normalized anomaly score, defect segmentation is performed by setting a threshold τ. Specifically, for each pixel, if its normalized anomaly score If the value is greater than the set threshold τ, the position is considered to be a defect area, otherwise it is considered to be a normal area. In this method, τ is set to 0.5. The segmentation formula is as follows:

[0111]

[0112] Among them, D i,j Is the segmentation result, indicating whether the position (i, j) belongs to the defect area. i,j =1 indicates that the location is a defect area. i,j =0 indicates that the position is a normal area. The abnormal score is converted into a binary image to realize the judgment of solder joint defects and the extraction and segmentation of defective areas.

[0113] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in the embodiments can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for detecting automobile door panel weld defects based on small object target detection, characterized in that: include: Step 1: Establish a dataset of automobile door panel welds; the established dataset includes a training set, a validation set, and a test set; the training set is divided into two parts: one is a training set for the target detection module model, which is used to train the target detection network; the other is a training set for the normal weld feature library, which is used to train the normal weld feature library, wherein the training set of the normal weld feature library is an image of the normal weld area within the normal weld mark box in the target detection training set; Step 2: Train the target detection model using the training set of the target detection module model in the automobile door panel weld dataset established in step 1. The training of the network framework of the target detection model includes pre-training the convolutional neural network ResNet50 to extract features, using GIOU as the loss function for coordinate regression, and adopting an optimization method of learning rate adjustment. The learning rate is initially set to 0.0001, and the Adam optimization algorithm is used to associate the learning rate with the current training number and continuously update it. When the loss function stops decreasing, cross-validation is used to evaluate the model. The best model is evaluated using the validation set and then modified to complete the training of the network framework. A multi-level feature connection-based target detection model is obtained that can accurately identify welds in images of automobile door panels and generate prediction boxes containing areas of normal and abnormal welds. Step 3: Use the training set of the normal solder joint feature library established in Step 1 to train the normal solder joint feature library. Manually mark the solder joint areas in the normal solder joint images in the training set. Use the object detection model trained in Step 2 to perform object detection and feature extraction on the car door panel images containing only normal solder joints, ensuring that each solder joint area is accurately extracted. Use the core subset algorithm to lightweight the features. The lightweight features constitute the normal solder joint feature library. Step 4: Use the trained target detection model to perform target detection on the image of the car door panel to be inspected to obtain the area where all the welds are located; then use the pre-trained ResNet50 network to extract features in the area; calculate the geometric distance between the features and the matching features in the normal weld feature library obtained in step 3, and use this distance as the anomaly score of the feature; then restore the features to the image scale to obtain the anomaly score of each pixel, and then normalize the anomaly score to the range of 0 to 1. According to the preset threshold, it is judged whether each pixel has an anomaly; if it is judged that an anomaly exists, the current weld is considered to be defective and the defect position is located.

2. The automobile door panel weld defect detection method based on small object target detection according to claim 1 is characterized in that: The step 1 specifically includes: Construct a training set for the object detection module model: An industrial camera is used to capture multiple images of automobile door panels containing welds as training data. The images include images of automobile door panel welds from different angles, lighting conditions, and material conditions. 80% of these images are selected as the automobile door panel training set, 10% as the validation set, and 10% as the test set. The locations of the welds in each image are marked with rectangular boxes as training labels. Construct a training set for the normal solder joint feature library: In the training set of the target detection module model, mark the boxes of all normal solder joints in the training set, crop the pictures according to the marked boxes, and use the cropped pictures inside the marked boxes as the training set for the normal solder joint feature library.

3. The automobile door panel weld defect detection method based on small object target detection according to claim 1 is characterized in that: The images in the training set in step 1 are processed as follows: Perform random rotation, scaling, and translation operations on the training data to display the solder joint area at different angles; Adjust the brightness, contrast, and hue of the image to simulate the solder joint area under different lighting environments; Noise is added and occlusion is performed on the solder joint area to simulate the conditions of the solder joint area under different materials.

4. The automobile door panel weld defect detection method based on small object target detection according to claim 1, characterized in that: The second step specifically includes: S21: Establish a deep learning target detection framework. Use the target detection algorithm to import the training labels from step 1 as a training dataset into the target detection model for model training. This obtains a target detection model that can accurately identify solder joint areas, thereby accurately identifying solder joint areas from images. The training labels are only used to identify solder joint areas and do not distinguish between normal and abnormal areas. S22: Monitor the loss function value during training to determine whether the model is overfitting or underfitting and make appropriate adjustments. GIOU is used as the loss function for coordinate regression, which is defined as follows: Among them, A and B represent the true box and the regression box respectively, IOU is the intersection-over-union ratio between A and B, C is the minimum closed figure containing A and B, and C\(A∪B) represents the part of C that removes A∪B. As can be seen from the formula, the output range of GIOU is (-1,1]; GIOU is the lower bound of IOU; and when IOU is 0, GIOU is not 0. The farther the true box is from the regression box, the closer GIOU is to -1. When the true box is closer to the regression box, the closer GIOU is to 1. When the loss function no longer decreases, use cross-validation to evaluate the accuracy of the model; S23: Based on the evaluation results, the Adam optimization algorithm is used to update the parameters, adjust the model structure or training parameters, and thus complete the training of the network framework.

5. The automobile door panel weld defect detection method based on small object target detection according to claim 1 is characterized in that: The step three specifically includes: S31: Using the trained object detection module obtained in step 2, perform object detection on the image of the automobile door panel containing only normal welds, automatically identify and segment the weld area, and use the pre-trained convolutional neural network ResNet50 to extract local features of the welds within the weld area; and normalize the extracted features; S32: Establish a normal solder joint feature library based on the geometric shape, surface condition and material type of the solder joint; and use the core subset lightweight method to remove redundant features in the feature library.

6. The automobile door panel weld defect detection method based on small object target detection according to claim 5 is characterized in that: The local features in S31 include color, texture, shape and edge information.

7. The automobile door panel weld defect detection method based on small object target detection according to claim 1, characterized in that: The step 4 specifically includes: S41: Use the object detection model trained in step 1 to perform object detection on the image of the automobile door panel to be detected, automatically identify and segment the welding point area, and obtain images of all the segmented welding point areas; S42: using the convolutional neural network ResNet50 trained in step 2 to extract local features of the solder joint within the solder joint area, including color, texture, shape, and edge information; S43: Based on the feature categories in the solder joint feature library obtained in step 3, preliminarily matching the solder joint feature information extracted in S42 with the features in the normal solder joint feature library obtained in step 3 to determine whether the solder joint has defects; The Euclidean distance is then used to calculate the similarity between the extracted solder joint features and each solder joint feature in the feature library: s * =||m test,* -m * ||2 Where m is the feature vector in the memory bank, m test,* is the feature vector of the test data; Then use maximum and minimum normalization to process the abnormal scores so that the normalized score range is between [0,1]; After obtaining the normalized anomaly score, a preset threshold τ is set to perform defect segmentation; if the normalized anomaly score If the value is greater than the set threshold τ, the location is considered to be a defective area, otherwise it is considered to be a normal area; The formula for segmentation is as follows: Among them, D i,j is the segmentation result, indicating whether the position (i, j) belongs to the defect area; D i,j =1 indicates that the location is a defect area; D i,j =0 means the location is a normal area; The abnormal scores are converted into binary images to realize the judgment of solder joint defects and the extraction and segmentation of defective areas.

8. A car door panel weld defect detection system based on small object target detection, characterized in that: It includes solder joint data set module, training module, image preprocessing module, normal solder joint feature library module and solder joint recognition and positioning module; The solder joint dataset module includes a training set, a validation set, and a test set. The training set is divided into two parts: one is a training set for the target detection model, which is used for training the target detection network; the other is a training set for the normal solder joint feature library, which is used for training the normal solder joint feature library. The training module includes constructing a target detection model based on multi-level feature connection, training the target detection model based on the training set of the target detection module model of the weld dataset module, and using cross-validation to perform model evaluation, thereby obtaining a target detection model based on multi-level feature connection that can accurately identify welds in automobile door panel images and providing a prediction box containing areas of normal / abnormal welds. The image preprocessing module includes: using the manually annotated normal solder joint images in the training set of the solder joint dataset module, intercepting the image within the annotated frame, and extracting features from the intercepted image through the convolutional neural network ResNet50 feature extraction network pre-trained by the training module; The normal solder joint feature library module includes: obtaining manually annotated normal solder joint images in the training set of the solder joint data set module, intercepting the image within the annotated frame, using the features within the normal solder joint annotated frame extracted by the image preprocessing module, and lightweighting the features using the core subset algorithm, so that the lightweight features constitute the normal solder joint feature library; The solder joint identification and positioning module includes: for the area where all solder joints, including normal solder joints and abnormal solder joints, are located, given by the target detection network during the test process, using the image preprocessing module to extract features in the area, calculating the geometric distance between the area and the matching features in the normal solder joint feature library, using this distance as the anomaly score of the feature, then restoring the feature to the image scale to obtain the anomaly score of each pixel, and then normalizing the anomaly score to the range of 0 to 1. Based on a preset threshold, it is determined whether each pixel has an anomaly.

9. A car door panel weld defect detection device based on small object target detection, characterized in that It comprises a processor and a memory for storing a computer program that can be run on the processor; wherein, when the processor is used to run the computer program, it executes the steps of the automobile door panel weld defect detection method according to any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the automobile door panel weld defect detection method according to any one of claims 1 to 7 are implemented.

Images