Ultrasonic radar welding spot defect detection and classification method

Abstract

The invention provides an ultrasonic radar welding spot defect detection and classification method. The method comprises the following steps: S1, detecting the position and the center point coordinate of a welding spot and the width and the height of the welding spot by using a trained welding spot detection model; s2, whether the welding spot is located in a preset area or not is judged based on the coordinates of the center point of the welding spot, and defect detection is conducted according to the width, height, area and aspect ratio of the welding spot and the area ratio of the left welding spot and the right welding spot; s3, the area where the welding spot is located is expanded, and the expanded welding spot area is classified through the trained welding spot classification model; and S4, segmenting the silver wire, and judging the deviation condition of the positions of the silver wire and the welding spot by acquiring the coordinates of the junction point of the silver wire and the welding spot. The automatic detection efficiency in the welding process can be improved, errors of manual detection are reduced, and comprehensive detection and classification of welding spots are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of welding quality inspection in industrial automation and manufacturing, and particularly relates to a method for detecting and classifying solder joint defects by an ultrasonic radar, which is applicable to the scenario of automated welding quality inspection, and has wide applications especially in fields such as electronic components, automotive manufacturing, and aerospace with high precision requirements. Background Art

[0002] Under the background of the rapid development of industrial automation and manufacturing, welding, as a key link in the manufacturing process, its quality has a crucial impact on the overall performance and service life of products. Solder joints, as the core part of the welding process, their quality directly determines the strength and conductivity of the welding. Especially in industries such as electronics, automotive, and aerospace, due to the extremely high requirements for precision and reliability of products, any flaw in welding quality may lead to equipment failures, performance degradation, and even serious safety hazards.

[0003] In the manufacturing process of ultrasonic radars, the quality of solder joints is a key link to ensure the performance and reliability of the equipment. Solder joint defects may cause unstable signal transmission, increased resistance, and even equipment failures, seriously affecting the overall performance of ultrasonic radars. Therefore, product quality control through solder joint defect detection and classification is an important means to ensure the quality of ultrasonic radars.

[0004] Currently, the detection of solder joint defects mainly relies on manual inspection. However, this detection method has many deficiencies. Firstly, manual inspection is inefficient and difficult to meet the requirements of high - efficiency production in modern manufacturing. Secondly, manual inspection is easily affected by subjective factors, resulting in errors in detection results and unable to ensure the consistency and accuracy of detection. In addition, traditional detection methods often only focus on surface defects of solder joints and ignore the detection of internal or structural defects of solder joints, making it difficult to comprehensively evaluate welding quality.

[0005] With the progress of technology, especially the development of deep learning technology, solder joint detection methods based on image processing have gradually emerged. Such methods train models to automatically identify parameters such as the center position, size, and morphology of solder joints, and accordingly conduct defect determination and classification. Compared with traditional manual detection methods, the detection methods based on image processing have significantly improved in terms of efficiency and accuracy.

[0006] However, existing solder joint detection methods based on image processing still have some limitations. Most methods are only limited to the detection of the solder joints themselves, and lack effective detection means for the bonding defects between the solder joints and silver wires. In industries such as electronics, automotive, and aerospace, the matching quality of silver wires and solder joints is also an important part of the welding quality. Traditional methods are difficult to effectively detect and analyze the bonding situation between silver wires and solder joints, which limits the comprehensiveness and accuracy of welding quality detection.

[0007] Therefore, in order to meet the requirements of high-precision welding detection, it is necessary to provide a detection method that integrates solder joint defect detection, classification, and silver wire segmentation. This method should be able to comprehensively and accurately detect various defects of solder joints, including surface defects, internal defects, and structural defects, and at the same time should be able to effectively evaluate the bonding quality between silver wires and solder joints. Through such a detection method, the efficiency and accuracy of solder joint detection can be comprehensively improved, providing strong quality assurance for the manufacturing processes in industries such as electronics, automotive, and aerospace. Summary of the Invention

[0008] In order to solve the above problems existing in the prior art, the purpose of the present invention is to provide an ultrasonic radar solder joint defect detection and classification method, which aims to improve the automation detection efficiency in the welding process, reduce the error of manual detection, and achieve comprehensive detection and classification of solder joints, especially achieving good results in the detection of the bonding quality between silver wires and solder joints.

[0009] The present invention achieves the above purpose through the following technical solutions:

[0010] An ultrasonic radar solder joint defect detection and classification method, which includes the following steps:

[0011] Step S1: Use a trained solder joint detection model to detect the position, center point coordinates, width, and height of the solder joint.

[0012] Step S2: Determine whether the solder joint is located within a predetermined area based on the center point coordinates of the solder joint, and perform defect detection according to the width, height, area, width-to-height ratio, and area ratio of the left and right solder joints.

[0013] Step S3: Expand the area where the solder joint is located, and use a trained solder joint classification model to classify the expanded solder joint area.

[0014] Step S4: Perform silver wire segmentation, and judge the offset situation between the silver wire and the solder joint by obtaining the intersection point coordinates of the silver wire and the solder joint; wherein, the silver wire segmentation result is used to calculate the distance from the center point of the solder joint to the extension line of the silver wire, and the position offset degree between the silver wire and the solder joint is calculated based on this distance.

[0015] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, in the image acquisition stage before step S1, a high-resolution camera is used to acquire images of the welded workpiece, and the lighting conditions and imaging angles are selected to ensure that the details of the solder joints and silver wires can be accurately captured.

[0016] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, the solder joint detection model is a deep learning detection model based on a convolutional neural network. Among them, the specific steps for constructing the solder joint detection model include:

[0017] Design the basic architecture of the convolutional neural network, including determining the number of layers of the network, i.e., the model depth, the type of each layer, the number of neurons in each layer, i.e., the model width, and the selection of the activation function;

[0018] Initialize the network parameters, including weights and biases, and use random initialization or pre-trained model parameters as the initial values;

[0019] Prepare a large number of labeled solder joint image datasets. The datasets include positive samples, i.e., images containing solder joints, and negative samples, i.e., images that do not contain solder joints or have defective solder joints, and preprocess the images;

[0020] Use the labeled solder joint image datasets to train the convolutional neural network. Calculate the output through forward propagation, calculate the difference between the predicted value and the true value through the loss function, and update the network parameters through the backpropagation algorithm. Iteratively optimize the model until the preset number of training rounds or the convergence condition of the loss function is reached;

[0021] After the training is completed, a solder joint detection model is obtained. This model can automatically extract the features of the solder joints and output at least the center coordinates, width, and height dimensions of the solder joints, which are used for subsequent solder joint defect detection and classification tasks; among them, the center point of the solder joint is used to determine the welding position of the solder joint, and the width and height of the solder joint are used to further calculate the solder joint parameters, which at least include the area and aspect ratio parameters.

[0022] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, the defect detection specifically includes:

[0023] Extract the position, width, height, area, and center point coordinates of the solder joints from the solder joint detection model. Based on the area of the solder joints, set the area threshold range, and compare the detected solder joint area with the threshold range to determine whether the solder joint is too small or too large;

[0024] Moreover, based on the aspect ratio of the solder joints, set the standard aspect ratio range, and compare the detected solder joint aspect ratio with the standard range to determine whether the roundness of the solder joint meets the requirements;

[0025] Moreover, based on the area ratio of the left and right solder joints on the same detection object, a tolerance range for the area ratio is set, and the detected area ratio of the left and right solder joints is compared with the tolerance range to determine whether there is a situation of one large and one small.

[0026] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, after solder joint detection, the solder joint area is extracted, and the solder joint area is expanded to ensure that the expanded area includes the silver wire area around the solder joint;

[0027] The expanded area is used as an input and input into the solder joint classification model;

[0028] Wherein, the solder joint classification model is a deep learning classification model based on a convolutional neural network.

[0029] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, the steps of extracting the solder joint area and expanding the solder joint area specifically include:

[0030] According to the solder joint position information and size information output by the solder joint detection model, the boundary of the solder joint is determined;

[0031] Then, based on the center point of the solder joint, a predetermined pixel distance is expanded around to ensure that the expanded area not only includes the complete solder joint, but also includes the silver wire area around the solder joint;

[0032] Wherein, the predetermined pixel distance is determined according to the average size of the solder joint, the width of the silver wire, and the requirement of the classification task for the surrounding area information to ensure that the expanded area can provide sufficient context information for the solder joint classification model.

[0033] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, when evaluating the silver wire offset, the boundary information and the center line position of the silver wire are obtained from the silver wire segmentation model, and the intersection point coordinates of the silver wire and the solder joint edge are determined;

[0034] According to the linear characteristic of the silver wire, at the intersection point, a linear fitting algorithm or a geometric method is used to fit the extended line equation of the silver wire; wherein, the extended line equation of the silver wire is a straight line equation, representing the extension direction of the silver wire at the intersection point; the part where the silver wire contacts the solder joint is straight, and the intersection point coordinates of the silver wire and the solder joint edge are obtained through silver wire segmentation, and the straight line is tangent to the silver wire at this intersection point;

[0035] Calculate the perpendicular distance from the center point of the solder joint to the extended line of the silver wire as the offset degree of the combination of the silver wire and the solder joint; if the offset distance exceeds the set threshold, it is determined as a poor combination defect.

[0036] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, the calculation of the vertical distance from the center point of the solder joint to the extended line of the silver wire includes:

[0037] Determine the center point coordinates of the solder joint;

[0038] Using the fitted extended line equation of the silver wire, calculate the vertical projection point of the center point of the solder joint to the extended line;

[0039] According to the center point coordinates of the solder joint and the coordinates of the vertical projection point, apply the geometric distance formula to calculate the vertical distance from the center point of the solder joint to the extended line of the silver wire;

[0040] The vertical distance is used as a quantitative index for the degree of offset between the silver wire and the solder joint, and is used to evaluate the quality or bonding stability of the solder joint.

[0041] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, by analyzing the morphological characteristics of the solder joint, the solder joints are divided into the following three categories:

[0042] The first category is OK solder joints, which have good morphology, the silver wire is tightly wrapped, and the welding quality meets the requirements;

[0043] The second category is PASS solder joints, which have good silver wire wrapping, but slightly flawed appearance, and the flaw does not affect the welding quality;

[0044] The third category is NG solder joints, which have obvious defects, and the defects include at least one of the solder joint being seriously off - center, porcelain burning, irregular shape, yellowing of the solder pad, silver wire floating, and solder joint blackening.

[0045] According to an ultrasonic radar solder joint defect detection and classification method provided by the present invention, it also performs:

[0046] Extract multiple initial features from the image or signal data of the ultrasonic radar solder joint, including at least the width, height, area, aspect ratio, texture feature, shape feature of the solder joint, and the relative position relationship between the solder joint and the surrounding environment;

[0047] Adopt the chi - square test algorithm to perform statistical analysis on the correlation between the extracted initial features and the solder joint defects, calculate the chi - square value of each feature, and evaluate its contribution degree to defect detection;

[0048] Alternatively, adopt the information gain algorithm to calculate the amount of information provided by each feature in the defect detection task, that is, the influence degree of the presence or absence of this feature on the defect classification result;

[0049] According to the magnitude of the chi - square value or information gain, sort the initial features, and screen out the feature subset that contributes the most to defect detection;

[0050] The selected feature subset is used as the input for subsequent defect detection and classification, ignoring features with less contribution, thereby reducing the feature dimension and computational complexity to improve the computational efficiency of solder joint defect detection and classification.

[0051] It can be seen that, compared with the prior art, the present invention has significant beneficial effects in the field of welding quality detection, which are specifically manifested in the following aspects:

[0052] 1. Improve detection efficiency and accuracy: The present invention completes the defect detection and classification of solder joints through automated means, significantly improving the detection efficiency compared with traditional manual detection methods. Automated detection can quickly and continuously detect a large number of solder joints, greatly shortening the detection cycle.

[0053] 2. Combining deep learning algorithms, the present invention can accurately identify the position, shape, and size of solder joints and precisely classify solder joints based on these features. The powerful learning ability of deep learning enables the detection model to be continuously optimized, improving the detection accuracy.

[0054] 3. Achieve comprehensive detection of solder joints: The present invention not only focuses on the surface defects of solder joints but also can detect internal defects and structural defects of solder joints. Through a multi-step detection process, it ensures that every aspect of the solder joint is comprehensively evaluated. At the same time, the present invention can also classify solder joints according to the bonding situation between the solder joint and the silver wire, which is difficult to achieve by traditional detection methods. This comprehensive detection method makes the evaluation of welding quality more accurate and comprehensive.

[0055] 4. Ensure that the quality of solder joints meets the requirements: Through the detection and classification method of the present invention, defects in solder joints can be promptly discovered, providing a strong basis for subsequent repair or rework, which helps to ensure that the welding quality of the final product meets the requirements and improves the reliability and performance of the product. The method of the present invention can also provide data support for the optimization of the welding process. By analyzing the detection data, problems existing in the welding process can be found, and then the welding process can be improved to enhance the overall welding quality.

[0056] 5. Improve the intelligent level of the manufacturing industry: The ultrasonic radar solder joint defect detection and classification method of the present invention reflects the intelligent development trend of the manufacturing industry. By introducing automated and intelligent technologies, the welding quality detection becomes more efficient and accurate, providing strong support for the transformation and upgrading of the manufacturing industry. The application of this intelligent detection method can also promote the technological progress and innovation of related industries and facilitate the coordinated development of the industrial chain.

[0057] In summary, the ultrasonic radar solder joint defect detection and classification method of the present invention has significant beneficial effects in improving detection efficiency and accuracy, achieving comprehensive detection of solder joints, ensuring that the quality of solder joints meets requirements, and enhancing the intelligent level of the manufacturing industry. The application of this method will greatly improve the level of welding quality detection and provide strong quality assurance for the manufacturing processes of industries such as electronics, automotive, and aerospace.

[0058] The following further elaborates on the present invention in detail in conjunction with the accompanying drawings and specific embodiments. Description of the Drawings

[0059] Figure 1 is a flowchart of an embodiment of the ultrasonic radar solder joint defect detection and classification method of the present invention. Specific Embodiments

[0060] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without making creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0061] Reference to "embodiment" herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0062] See Figure 1 , this embodiment provides an ultrasonic radar solder joint defect detection and classification method. During the manufacturing process of ultrasonic radar, product quality control is carried out through solder joint defect detection and classification. If there are defects in the solder joints, it may affect the overall performance of the ultrasonic radar. This method includes the following steps:

[0063] Step S1: Use a trained solder joint detection model to detect the position, center point coordinates, width, and height of the solder joints;

[0064] Step S2: Determine whether the solder joints are located within a predetermined area based on the center point coordinates of the solder joints, and perform defect detection according to the width, height, area, width-to-height ratio, and area ratio of the left and right solder joints;

[0065] Step S3: Expand the area where the solder joints are located, and use a trained solder joint classification model to classify the expanded solder joint area;

[0066] Step S4: Perform silver wire segmentation. By obtaining the intersection point coordinates of the silver wire and the solder joint, the offset situation between the silver wire and the solder joint is judged. Among them, the silver wire segmentation result is used to calculate the distance from the center point of the solder joint to the extension line of the silver wire, and the position offset degree between the silver wire and the solder joint is calculated based on this distance.

[0067] In the image acquisition stage before step S1, a high-resolution camera is used to acquire images of the welded workpiece, and the lighting conditions and imaging angles are selected to ensure that the details of the solder joints and silver wires can be accurately captured.

[0068] Specifically, in this embodiment, the labeled solder joint images are used to train the solder joint detection model, the classified solder joint images are used to train the solder joint classification model, and the segmented silver wire images are used to train the silver wire segmentation model. During the process of training these models, image enhancement techniques are adopted to improve the generalization ability of the models.

[0069] First, the training images are randomly changed in hue, saturation, and brightness within a small range to simulate the subtle changes in lighting in the production line. Such processing can enable the model to maintain stable performance under different lighting conditions. Second, the training images are also randomly offset horizontally and vertically within a small range to simulate the mechanical jitter in the production line. This simulation can enhance the robustness of the model to small displacements of the images. Moreover, the training images also undergo random changes in magnification and reduction within a small range to simulate the subtle height differences between the camera and the fixture in the production line. This processing method helps the model to accurately identify the solder joints under different focal lengths and shooting distances.

[0070] In addition, other image enhancement techniques are also adopted to further improve the generalization ability of model training. These enhancement techniques may include, but are not limited to, image rotation, flipping, adding noise, etc., aiming to make the model perform excellently in various complex scenarios.

[0071] By using the above image enhancement techniques, the adaptability and accuracy of the model in the actual production line can be significantly improved, ensuring the reliability and efficiency of solder joint defect detection and classification.

[0072] In this embodiment, the solder joint detection model is a deep learning detection model based on a convolutional neural network. In the experimental stage, the multiple models are trained using the labeled solder joint image dataset; the trained models are experimentally tested to evaluate the accuracy, recall rate, and inference speed of each model in the solder joint detection task. According to the experimental data, comprehensively considering the accuracy, recall rate, inference speed of the model, as well as the heartbeat rhythm of the production line operation and the demand targets of different tasks, the depth and width of the model after trade-off optimization are selected. The purpose of this step is to find the best balance between the model inference speed and accuracy to meet the heartbeat rhythm of the production line operation and the demand targets of different tasks. The model is trained using a large number of labeled solder joint images, enabling it to automatically locate the center coordinates of the solder joints and detect the width and height dimensions of the solder joints. The center point of the solder joint is crucial for determining the accurate welding position of the solder joint, while the width and height of the solder joint are used to further calculate parameters such as the area and aspect ratio of the solder joint.

[0073] Among them, the detection results include the position, width and height, and center point coordinates of the solder joint, and these parameters will be used for subsequent defect determination. If the model fails to detect the solder joint information, it is directly determined that there is no solder joint defect.

[0074] Specifically, the specific steps for constructing the solder joint detection model include:

[0075] Design the basic architecture of the convolutional neural network, including determining the number of network layers (i.e., the depth of the model), the type of each layer (such as convolutional layer, pooling layer, fully connected layer, etc.), the number of neurons in each layer (i.e., the width of the model), and the selection of activation functions;

[0076] Initialize the network parameters, including weights and biases, using random initialization or the parameters of a pre-trained model as the initial values.

[0077] Prepare a large number of labeled solder joint image datasets, where the dataset includes positive samples (i.e., images containing solder joints) and negative samples (i.e., images without solder joints or with defective solder joints), and preprocess the images, such as normalization, cropping, rotation, etc., to enhance the generalization ability of the model.

[0078] Use the labeled solder joint image dataset to train the convolutional neural network, calculate the output through forward propagation, calculate the difference between the predicted value and the true value through the loss function, and update the network parameters through the backpropagation algorithm, iteratively optimizing the model until the preset number of training epochs or the convergence condition of the loss function is reached.

[0079] After training is completed, a solder joint detection model is obtained. This model can automatically extract the features of solder joints and output at least the center coordinates, width, and height dimensions of the solder joints, which are used for subsequent solder joint defect detection and classification tasks. Among them, the center point of the solder joint is used to determine the welding position of the solder joint, and the width and height of the solder joint are used to further calculate the solder joint parameters, which at least include the area and aspect ratio parameters.

[0080] The solder joint detection model constructed through the above steps has high detection accuracy and robustness, and can adapt to the solder joint detection requirements in different scenarios.

[0081] In this embodiment, the specific steps for determining the number of network layers (model depth), the type of each layer, the number of neurons in each layer (model width), and the selection of activation functions include:

[0082] Determine the number of network layers (model depth): Based on the complexity of the solder joint image and the required feature extraction ability, set an initial range of network layers through experiments and experience; by gradually increasing or decreasing the number of layers and observing the performance changes of the model on the validation set, determine the optimal number of network layers to balance the model's expressive ability and computational complexity.

[0083] Select the type of each layer: According to the characteristics of the solder joint detection task, select a suitable combination of layer types, including but not limited to convolutional layers for extracting local features, pooling layers for reducing the dimension of the feature map, and fully connected layers for integrating features and performing classification or regression; at the same time, auxiliary layers such as batch normalization layers and dropout layers can be considered to improve the training stability and generalization ability of the model.

[0084] Set the number of neurons in each layer (model width): According to the size of the input image and the desired feature representation ability, set parameters such as the number of convolutional kernels in the convolutional layer and the number of neurons in the fully connected layer; by experimentally comparing the model performance under different width settings, select the number of neurons setting that can minimize the computational amount while ensuring the model accuracy.

[0085] Select the activation function: According to the non-linear mapping requirements of the network and the characteristics of gradient propagation, select a suitable activation function, such as ReLU (Rectified Linear Unit) for increasing the non-linearity of the network, and Sigmoid or Tanh for classification or regression tasks in the output layer; at the same time, parameterized activation functions or adaptive activation functions can be considered to further improve the expressive ability and training efficiency of the model.

[0086] In this embodiment, the defect detection specifically includes:

[0087] Extract the position, width, height, area, and center point coordinates of the solder joints from the solder joint detection model. Based on the area of the solder joints, set an area threshold range, and compare the detected area of the solder joints with the threshold range to determine whether the solder joints are too small or too large.

[0088] Moreover, based on the aspect ratio of the solder joints, set an aspect ratio standard range, and compare the detected aspect ratio of the solder joints with the standard range to determine whether the roundness of the solder joints meets the requirements.

[0089] Furthermore, based on the area ratio of the left and right solder joints on the same detection object, set an area ratio tolerance range, and compare the detected area ratio of the left and right solder joints with the tolerance range to determine whether there is a situation of one large and one small.

[0090] After solder joint detection, extract the solder joint area and expand this solder joint area to ensure that the expanded area includes the silver wire area around the solder joint;

[0091] Take the expanded area as the input and input it into the solder joint classification model;

[0092] Among them, the solder joint classification model is a deep learning classification model based on a convolutional neural network. Determine the model depth (i.e., the number of layers of the neural network) and model width (i.e., the number of neurons in each layer) suitable for the solder joint classification task through experimental tests; according to the experimental data, use the appropriate model depth and model width to conduct a trade-off optimization of the model inference speed and accuracy to meet the heartbeat rhythm of the production line operation and the demand goals of different tasks.

[0093] Among them, the steps of extracting the solder joint area and expanding this solder joint area specifically include:

[0094] Determine the boundary of the solder joint according to the solder joint position information and size information output by the solder joint detection model;

[0095] Then, taking the center point of the solder joint as the benchmark, expand a predetermined pixel distance around to ensure that the expanded area not only includes the complete solder joint but also includes the silver wire area around the solder joint;

[0096] Among them, the predetermined pixel distance is determined according to the average size of the solder joint, the width of the silver wire, and the demand for surrounding area information in the classification task to ensure that the expanded area can provide sufficient context information for the solder joint classification model.

[0097] To more accurately judge the bonding situation between the solder joint and the silver wire, this embodiment introduces a silver wire segmentation model, which focuses on the fine segmentation of the silver wire part in the solder joint area.

[0098] First, crop the image region containing the solder joints, retaining the solder joints and the surrounding silver wire parts, which can reduce the interference of irrelevant information and improve the accuracy of silver wire segmentation. The silver wire segmentation model in this embodiment is also a deep learning segmentation model based on a convolutional neural network. During the experimental stage, by testing model configurations with different depths and widths, the most suitable model structure for the silver wire segmentation task was found; according to the experimental data, the model depth and width that achieve the best balance between inference speed and accuracy were selected to meet the requirements of high efficiency and accuracy in the production line operation.

[0099] Use the optimized silver wire segmentation model for inference to obtain the preliminary segmentation results. Filter the confidence of the inference result data to remove the low-confidence segmentation regions and reduce misjudgments. Apply the non-maximum suppression technique to further refine the segmentation results and ensure the continuity and accuracy of the silver wire. Through a series of processes such as matrix operations, generate the final activation map. In the activation map, the activated part represents the silver wire.

[0100] Extract the boundary information and center line position of the silver wire from the activation map, and calculate the distance between the silver wire boundary points and the center of the solder joint to judge the degree of silver wire floating. Excessive floating may mean poor welding quality. By calculating the distance between the extension line of the silver wire and the center point of the solder joint, judge the offset degree of the silver wire from the solder joint. Excessive offset may also affect the welding quality.

[0101] Through the above steps, the bonding situation between the solder joint and the silver wire can be accurately judged, ensuring good contact between the solder joint and the silver wire, thereby guaranteeing the welding quality.

[0102] In this embodiment, when evaluating the silver wire offset, obtain the boundary information and center line position of the silver wire from the silver wire segmentation model to determine the coordinates of the intersection points between the silver wire and the edge of the solder joint.

[0103] According to the linear characteristics of the silver wire, at the intersection point, use a linear fitting algorithm or a geometric method to fit the extension line equation of the silver wire; among them, the extension line equation of the silver wire is a straight line equation, representing the extension direction of the silver wire at the intersection point; the part where the silver wire contacts the solder joint is straight. Obtain the coordinates of the intersection points between the silver wire and the edge of the solder joint through silver wire segmentation, and the straight line is tangent to the silver wire at this intersection point.

[0104] Calculate the perpendicular distance from the center point of the solder joint to the extension line of the silver wire as the offset degree of the combination between the silver wire and the solder joint; if the offset distance exceeds the set threshold, it is determined as a bonding defect.

[0105] Among them, calculating the perpendicular distance from the center point of the solder joint to the extension line of the silver wire includes:

[0106] Determine the central point coordinates of the solder joint; using the fitted extension line equation of the silver wire, calculate the vertical projection point of the central point of the solder joint to this extension line; according to the central point coordinates of the solder joint and the vertical projection point coordinates, apply the geometric distance formula to calculate the vertical distance from the central point of the solder joint to the extension line of the silver wire, and the vertical distance is used as a quantitative index for the deviation degree of the combination of the silver wire and the solder joint, which is used to evaluate the quality or combination stability of the solder joint.

[0107] In this embodiment, by analyzing the morphological characteristics of the solder joint, the solder joints are classified into the following three categories:

[0108] The first category is the OK type solder joint, which has a good morphology, the silver wire is tightly wrapped, and the welding quality meets the requirements;

[0109] The second category is the PASS type solder joint, which has a good silver wire wrapping, but has slight defects in appearance, such as slight deviation, and this defect does not affect the welding quality;

[0110] The third category is the NG type solder joint, which has obvious defects, and the defects include but are not limited to at least one of the solder joint being severely close to the edge, porcelain burning, irregular shape, the solder pad turning yellow, the silver wire floating up, and the solder joint turning black.

[0111] In this embodiment, the following is also executed:

[0112] Extract multiple initial features from the image or signal data of the ultrasonic radar solder joint, including at least the width, height, area, aspect ratio, texture feature, shape feature of the solder joint, and the relative position relationship between the solder joint and the surrounding environment;

[0113] Adopt the chi-square test algorithm to conduct statistical analysis on the correlation between the extracted initial features and the solder joint defects, calculate the chi-square value of each feature, and evaluate its contribution degree to defect detection;

[0114] Alternatively, adopt the information gain algorithm to calculate the amount of information provided by each feature in the defect detection task, that is, the influence degree of the presence or absence of this feature on the defect classification result;

[0115] Sort the initial features according to the size of the chi-square value or information gain, and screen out the feature subset with the greatest contribution to defect detection;

[0116] Use the screened feature subset as the input for subsequent defect detection and classification, ignore the features with less contribution, thereby reducing the feature dimension and reducing the computational complexity to improve the computational efficiency of solder joint defect detection and classification.

[0117] In practical applications, based on solder joint detection and silver wire segmentation, comprehensively analyze the parameters of the solder joint and its bonding area to complete the judgment and evaluation of solder joint defects, specifically including:

[0118] Parameter extraction: Extract the position, width, height, area, and center point coordinates of the solder joints from the solder joint detection model; obtain the boundary information and center line position of the silver wires from the silver wire segmentation model.

[0119] Solder joint defect determination:

[0120] Area judgment:

[0121] Calculate the area of the solder joint and compare it with the preset minimum and maximum area thresholds.

[0122] If the area is less than the minimum value, it is determined as a defect of too small solder joint.

[0123] If the area is greater than the maximum value, it is determined as a defect of too large solder joint.

[0124] For example, the preset minimum and maximum area thresholds are 100000 pixels and 180000 pixels respectively. Compare the calculated area with the preset minimum area threshold of 100000 pixels and the maximum area threshold of 180000 pixels.

[0125] If the area is less than 100000 pixels, it is determined as a defect of too small solder joint.

[0126] If the area is greater than 180000 pixels, it is determined as a defect of too large solder joint.

[0127] Aspect ratio judgment:

[0128] Calculate the aspect ratio of the solder joint to evaluate the roundness of the solder joint; if the aspect ratio exceeds the set range (such as deviating too much from the ideal circle), it is determined as a morphological defect.

[0129] For example, the preset minimum and maximum aspect ratio thresholds are 0.95 and 1.2 respectively. If the aspect ratio is less than the preset minimum aspect ratio threshold of 0.95 or greater than the preset maximum aspect ratio threshold of 1.2, it is determined as a morphological defect.

[0130] Area ratio judgment of left and right solder joints:

[0131] For the left and right solder joints, calculate the area ratio of the left and right solder joints; if the area ratio deviates from the set range, it is determined that there is a "one large and one small" defect.

[0132] For example, the preset minimum and maximum area ratio thresholds are 0.55 and 1.8 respectively. If the area ratio is less than the preset minimum area ratio threshold of 0.55 or greater than the preset maximum area ratio threshold of 1.8, it is determined that there is a "one large and one small" defect.

[0133] Silver wire offset evaluation: Silver wire extension line calculation: Based on the silver wire segmentation result, fit the extension line equation of the silver wire; The part where the silver wire contacts the solder joint is straight. By segmenting the silver wire, obtain the intersection point coordinates of the silver wire and the edge of the solder joint. This straight line is tangent to the silver wire at this intersection point.

[0134] Offset distance calculation: Calculate the perpendicular distance from the center point of the solder joint to the extension line of the silver wire as the offset degree of the combination of the silver wire and the solder joint.

[0135] Bonding quality determination: If the offset distance exceeds the set threshold, it is determined as a bonding defect, such as the silver wire floating or excessive offset.

[0136] Defect classification and output:

[0137] According to the above parameter analysis results, classify the solder joint defects into the following categories:

[0138] OK (qualified): The area, aspect ratio, left - right symmetry, and silver wire bonding situation all meet the standards.

[0139] PASS (better): There are minor defects, but they do not affect the welding performance, such as minor offset or slightly irregular shape.

[0140] NG (unqualified): There are serious defects, such as abnormal area, excessive aspect ratio, imbalance in the left - right area ratio, or severe silver wire offset.

[0141] Output the defect classification results for subsequent processing or quality control.

[0142] In summary, the ultrasonic radar solder joint defect detection and classification method provided in this embodiment has significant beneficial effects in improving the detection efficiency and accuracy, achieving comprehensive detection of solder joints, ensuring that the solder joint quality meets the requirements, and enhancing the intelligent level of the manufacturing industry. The application of this method will greatly improve the level of welding quality detection and provide strong quality assurance for the manufacturing processes of industries such as electronics, automotive, and aerospace.

[0143] Furthermore, this embodiment completes the defect detection and classification of solder joints through automated means. Compared with traditional manual detection methods, the detection efficiency is significantly improved. Automated detection can quickly and continuously detect a large number of solder joints, greatly shortening the detection cycle. This embodiment can accurately identify the position, shape, and size of solder joints and accurately classify solder joints based on these features. The powerful learning ability of deep learning enables the detection model to be continuously optimized, improving the detection accuracy.

[0144] Furthermore, this embodiment not only focuses on the surface defects of solder joints, but also can detect internal defects and structural defects of solder joints. Through a multi-step detection process, it ensures that every aspect of the solder joint is comprehensively evaluated. At the same time, this embodiment can also classify solder joints according to the bonding situation between the solder joint and the silver wire, which is difficult to achieve by traditional detection methods. This comprehensive detection method makes the evaluation of welding quality more accurate and comprehensive.

[0145] Furthermore, through the detection and classification method of this embodiment, defects existing in solder joints can be timely discovered, providing a strong basis for subsequent repair or rework. This helps to ensure that the welding quality of the final product meets the requirements and improves the reliability and performance of the product. The method of this embodiment can also provide data support for the optimization of the welding process. By analyzing the detection data, problems existing in the welding process can be found, and then the welding process can be improved to enhance the overall welding quality.

[0146] Furthermore, the ultrasonic radar solder joint defect detection and classification method of this embodiment reflects the intelligent development trend of the manufacturing industry. By introducing automation and intelligent technologies, the welding quality detection becomes more efficient and accurate, providing strong support for the transformation and upgrading of the manufacturing industry. The application of this intelligent detection method can also promote the technological progress and innovation of related industries and facilitate the coordinated development of the industrial chain.

[0147] The above embodiments are only the preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantive changes and substitutions made by those skilled in the art based on the present invention fall within the scope of protection required by the present invention.

Claims

1. An ultrasonic radar weld defect detection and classification method, characterized in that: The method comprises the following steps: Step S1: using a trained solder joint detection model to detect the position, center point coordinates, and width and height of the solder joint; Step S2: determining whether the solder joint is located in a predetermined area based on the center point coordinates of the solder joint, and performing defect detection based on the width, height, area, aspect ratio, and area ratio of the solder joint to the left and right solder joints; Step S3: expanding the area where the solder joint is located, and classifying the expanded solder joint area using the trained solder joint classification model; Step S4: perform silver wire segmentation, and determine the offset of the silver wire and the solder point by obtaining the coordinates of the intersection point between the silver wire and the solder point; wherein the silver wire segmentation result is used to calculate the distance from the center point of the solder point to the extension line of the silver wire, and based on this distance, calculate the degree of position offset of the silver wire and the solder point.

2. The method according to claim 1, characterized in that: In the image acquisition stage before step S1, a high-resolution camera is used to acquire images of the welded workpiece, and the lighting conditions and imaging angles are selected to ensure that the details of the solder joints and silver wires can be accurately captured.

3. The method according to claim 1, characterized in that: The solder joint detection model is a deep learning detection model based on a convolutional neural network, wherein the specific steps of constructing the solder joint detection model include: Design the basic architecture of the convolutional neural network, including determining the number of layers in the network (i.e., model depth), the type of each layer, the number of neurons in each layer (i.e., model width), and the choice of activation function; Initialize network parameters, including weights and biases, using random initialization or pre-trained model parameters as initial values; Prepare a large number of annotated solder joint image datasets, which include positive samples, i.e. images containing solder joints, and negative samples, i.e. images that do not contain solder joints or have defects in solder joints, and preprocess the images; The convolutional neural network is trained using the labeled solder joint image dataset, the output is calculated through forward propagation, the difference between the predicted value and the true value is calculated through the loss function, and the network parameters are updated through the back-propagation algorithm. The model is iteratively optimized until the preset training rounds or the convergence condition of the loss function is reached; After the training is completed, a solder joint detection model is obtained, which can automatically extract the features of the solder joint and output at least the center coordinates, width and height dimensions of the solder joint for subsequent solder joint defect detection and classification tasks; wherein the center point of the solder joint is used to determine the welding position of the solder joint, and the width and height of the solder joint are used to further calculate the solder joint parameters, which include at least the area and aspect ratio parameters.

4. The method according to claim 3, characterized in that The defect detection specifically includes: Extracting the position, width, height, area and center point coordinates of the solder joint from the solder joint detection model, setting an area threshold range based on the area of ​​the solder joint, and comparing the detected solder joint area with the threshold range to determine whether the solder joint is too small or too large; Furthermore, based on the aspect ratio of the solder joint, a standard range of the aspect ratio is set, and the detected aspect ratio of the solder joint is compared with the standard range to determine whether the roundness of the solder joint meets the requirements; Furthermore, based on the area ratio of the left and right solder joints on the same inspection object, an area ratio tolerance range is set, and the detected area ratio of the left and right solder joints is compared with the tolerance range to determine whether there is a situation where one is larger and the other is smaller.

5. The method according to claim 4, characterized in that: After the solder joint is inspected, the solder joint area is extracted and expanded to ensure that the expanded area includes the silver line area around the solder joint; The expanded region is used as input into the solder joint classification model; Among them, the solder joint classification model is a deep learning classification model based on convolutional neural network.

6. The method according to claim 5, characterized in that The steps of extracting the solder joint area and expanding the solder joint area specifically include: Determine the boundary of the solder joint according to the solder joint position information and size information output by the solder joint detection model; Then, the center point of the solder joint is used as a reference, and a predetermined pixel distance is extended in all directions to ensure that the expanded area includes not only the complete solder joint but also the silver line area around the solder joint; The predetermined pixel distance is determined based on the average size of the solder joint, the width of the silver line, and the classification task's requirement for surrounding area information, to ensure that the expanded area can provide sufficient contextual information for the solder joint classification model.

7. The method according to claim 1, characterized in that: When evaluating the silver wire offset, the boundary information and the centerline position of the silver wire are obtained from the silver wire segmentation model, and the coordinates of the intersection point between the silver wire and the edge of the solder joint are determined; According to the straight line characteristics of the silver wire, at the intersection point, a linear fitting algorithm or a geometric method is used to fit the equation of the extension line of the silver wire; wherein the equation of the extension line of the silver wire is a straight line equation, which represents the extension direction of the silver wire at the intersection point; the part where the silver wire contacts the solder point is straight, and the coordinates of the intersection point between the silver wire and the edge of the solder point are obtained by segmenting the silver wire, and the straight line is tangent to the silver wire at the intersection point; The vertical distance from the center of the solder joint to the extension line of the silver wire is calculated as the offset degree of the combination of the silver wire and the solder joint; if the offset distance exceeds the set threshold, it is judged as a poor combination defect.

8. The method according to claim 7, wherein the calculating the vertical distance from the center of the solder joint to the extension line of the silver wire comprises: Determine the center point coordinates of the welding point; Using the fitted silver wire extension line equation, calculate the vertical projection point from the center point of the solder joint to the extension line; According to the coordinates of the center point of the solder joint and the coordinates of the vertical projection point, the vertical distance from the center point of the solder joint to the extension line of the silver wire is calculated using the geometric distance formula; The vertical distance is used as a quantitative indicator of the degree of deviation between the silver wire and the solder joint, and is used to evaluate the quality or bonding stability of the solder joint.

9. The method according to claim 1, characterized in that: By analyzing the morphological characteristics of solder joints, solder joints are divided into the following three categories: The first category is OK solder joints, which have good shape, the silver wire is tightly wrapped, and the welding quality meets the requirements; The second category is PASS solder joints, which have good wrapped silver wires but slight defects in appearance, and the defects do not affect the welding quality; The third category is NG solder joints, which have obvious defects, including at least one of the following: severe edge proximity, porcelain burning, irregular shape, yellowing of pads, floating silver wires, and blackening of solder joints.

10. The method according to claim 1, characterized in that Also execute: Extracting multiple initial features from the image or signal data of the ultrasonic radar weld, including at least the width, height, area, aspect ratio, texture features, shape features of the weld, and the relative position relationship between the weld and the surrounding environment; The chi-square test algorithm is used to perform statistical analysis on the correlation between the extracted initial features and solder joint defects, and the chi-square value of each feature is calculated to evaluate its contribution to defect detection; Alternatively, the information gain algorithm is used to calculate the amount of information provided by each feature in the defect detection task, that is, the degree of influence of the presence or absence of the feature on the defect classification result; According to the chi-square value or information gain, the initial features are sorted to select the feature subset that contributes most to defect detection; The screened feature subset is used as the input for subsequent defect detection and classification, and the features with smaller contributions are ignored, thereby reducing the feature dimension and computational complexity to improve the computational efficiency of solder joint defect detection and classification.

Images