A welding guidance system based on keypoint detection

Abstract

The application relates to the field of industrial automation and machine vision, and in particular to a welding guide system based on key point detection. The welding guide system comprises an image acquisition unit configured to acquire a top image of a motor stator to be welded; a calculation processing unit configured to execute an algorithm model, the algorithm model comprising a key point detection neural network model, and simultaneously performing target frame prediction and key point prediction, and outputting a detection frame and an angle point coordinate of a PIN pin of the motor stator to be welded; the algorithm model further comprises a coordinate conversion model, the angle point coordinates of the PIN pins belonging to the same welding instance are calculated to obtain welding guide point coordinates, the welding guide point coordinates are input into the coordinate conversion model, the welding guide coordinates are converted into world coordinate system coordinates of a motion mechanism according to a calibration matrix, and the welding gun is controlled to move to the converted coordinates to perform welding. The welding guide system can quickly adapt to production model change requirements of different types of electrical appliances, reduces algorithm debugging time, and greatly improves the flexibility, robustness and production efficiency of an automatic welding system.

Description

Technical Field

[0001] This application relates to the fields of industrial automation and machine vision, and in particular to a welding guidance system based on key point detection. Background Technology

[0002] With the rapid development of industries such as new energy vehicles and industrial servo drives, the performance requirements for motors, as core power sources, are increasing. Flat wire motors, due to their advantages such as high power density, high efficiency, and excellent thermal conductivity, have gradually become the mainstream choice in the market. However, the manufacturing process of flat wire motors, especially the welding process of the stator pins, faces significant challenges.

[0003] Traditional automated welding workstations typically employ rule-based machine vision systems for weld point positioning. The technical approach is roughly as follows: an industrial camera captures images of the workpiece, and pre-defined algorithms (such as edge detection, template matching, and grayscale analysis) are used to identify the location of feature points, thereby guiding the welding robot to perform operations.

[0004] However, this technical approach, which relies on manually setting rules and feature engineering, has revealed significant limitations in actual large-scale production. Because the arrangement, shape, and even background environment of the pins vary between different models and batches of motors, vision engineers must spend a considerable amount of time re-adjusting algorithms and parameters whenever a production line needs to be changed. This results in a lengthy debugging cycle, severely restricting production flexibility and failing to adapt to the modern manufacturing trend of multi-variety, small-batch production. Even more challenging is the complex welding environment. Pins are often partially obscured by surrounding wiring harnesses and have diverse shapes. Traditional algorithms struggle to reliably and accurately extract effective feature point coordinates from complex image backgrounds. Their inherent poor generalization ability and weak anti-interference capabilities make it difficult to guarantee the accuracy and stability of welding positioning, becoming a key bottleneck for improving production quality and efficiency. Summary of the Invention

[0005] This application provides a welding guidance system based on key point detection, comprising: An image acquisition unit is installed in front of the welding station to acquire images of the top of the motor stator to be welded. A computational processing unit is used to execute an algorithm model, which includes a keypoint detection neural network model. The computational processing unit is configured to input the acquired image into the neural network model during operation. The keypoint detection neural network model is configured to simultaneously perform target box prediction and keypoint prediction, and output the detection box and corner coordinates of the PIN pins of the motor stator to be welded. The detection box is used to determine whether the PIN pin is qualified. When the keypoint detection neural network model determines whether the PIN pin is qualified, it also determines the completeness and accuracy of the coordinates of the vertices of the detection box. The algorithm model also includes a coordinate transformation model. The calculation processing unit is configured to calculate the welding guide point coordinates based on the corner coordinates of the PINs belonging to the same welding instance for qualified PINs during operation, and input the welding guide point coordinates into the coordinate transformation model. The coordinate transformation model is configured to convert the welding guide coordinates into the world coordinate system coordinates of the motion mechanism according to the calibration matrix, and control the welding torch to move to the transformed coordinates for welding.

[0006] Specifically, the detection box output by the key point detection neural network model is the minimum bounding rectangle of the PIN pin. The key point detection neural network model outputs the coordinates of the four vertices of the detection box and the label of the detection box. The label includes qualified and unqualified. When the label is unqualified, the corresponding instance does not participate in the calculation of the welding guide point coordinates.

[0007] Specifically, the corner coordinates include the coordinates of eight key points in each PIN group, each PIN group including an upper PIN and a lower PIN, and the key point detection neural network model is configured to use the four corner coordinates of the upper PIN and the four corner coordinates of the lower PIN as regression targets simultaneously.

[0008] Specifically, the calculation processing unit is configured to obtain the coordinates of the welding guide point based on the center point of the coordinates of the four corner points of the upper PIN pin and the center point of the coordinates of the four corner points of the lower PIN pin belonging to the same welding instance, wherein the coordinates of the welding guide point are the midpoint of the two center points.

[0009] Specifically, during training, the training set of the key point detection neural network model includes high-resolution images of at least 20 different types of motors and 10 different shapes of PIN pins. The shapes of the PIN pins include at least square, rectangular, trapezoidal, and circular. The high-resolution images are labeled with key points.

[0010] Specifically, the key point detection neural network model includes shallow feature maps for small target detection.

[0011] Specifically, the keypoint detection neural network model includes a sliding segmentation module, which is configured to dynamically calculate the segmentation size based on the area of ​​the circumscribed rectangle of the PIN group and execute multi-level judgment logic to retain sub-images for training. The multi-level judgment logic includes: Level 1 judgment: All eight key points of the entire PIN pin group are located within the sub-diagram; Second-level judgment: Based on the geometric relationship of the eight key points, calculate the relative distance and angle between the key points of the upper PIN and the lower PIN. Only when the relative distance and angle are within the preset tolerance range will the second-level judgment be passed. Third-level judgment: Based on the label of the PIN group corresponding to the sub-image in the training data, the third-level judgment is passed only if the label is qualified; The sliding block module is configured to retain a subgraph for training only when the subgraph passes the first, second, and third level of judgment simultaneously, so that the key point detection neural network model can judge the accuracy of the vertices of the detection box when it is working.

[0012] Specifically, during training, the key point detection neural network model uses random pruning and mosaic augmentation in the cold start phase, and stops using data augmentation for incomplete targets after more than 30 training cycles.

[0013] Specifically, the training loss function of the key point detection neural network model consists of three parts: bounding box regression loss, key point regression loss, and classification loss. The bounding box regression loss adopts CIoU Loss, the key point regression loss adopts Wing Loss or mean squared error loss, and the classification loss adopts Focal Loss. The training of the model is supervised by weighted summation.

[0014] Specifically, the keypoint detection neural network model is customized based on the YOLO series network architecture to coordinate object detection and keypoint regression tasks.

[0015] This application has the following technical advantages: This invention effectively overcomes the inherent defects of traditional machine vision methods by adopting key point detection technology based on deep learning. It can quickly adapt to the production changeover needs of different models of electrical appliances, significantly reduce algorithm debugging time, and achieve stable and high-precision weld point positioning when facing on-site occlusion, shape changes and interference factors, thanks to the powerful understanding of complex features by neural networks. This greatly improves the flexibility, robustness and production efficiency of automated welding systems. Attached Figure Description

[0016] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts.

[0017] Figure 1 This is a schematic diagram of the architecture of the welding guidance system based on key point detection in the embodiments of this application; Figure 2 This is a schematic diagram of data annotation in an embodiment of this application; Figure 3 This is a schematic diagram of the data annotation of a special-shaped PIN pin in an embodiment of this application. Detailed Implementation

[0018] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0019] The automated welding bottleneck encountered in the mass production of flat wire motors and similar precision motors stems from the inherent, long-standing, unresolved defects of existing welding point guidance methods based on traditional rule-based machine vision, such as poor generalization ability, weak anti-interference capability, and difficulty in recognizing complex features. The core of this problem lies in the fact that static, well-trained, and fixed models struggle to adapt to dynamic and uncertain real-world scenarios: the lighting conditions at the welding site can change due to the electric arc, ambient light, and even equipment reflections; the workpieces to be welded themselves contain different types of pins, whose geometry, surface material, color, and the interwoven enameled wire background constitute a high-dimensional, complex feature space; and unavoidable random noise interference such as oil stains and splatter occurs during production. Manually maintained, fixed methods for extracting edges and grayscale cannot adapt to the complex changes in real-world scenarios; once the imaging conditions or workpiece appearance exceed the description range of the preset rules, the system will fail.

[0020] Based on the above analysis, the fundamental objective of this application is to provide a novel solder joint guidance solution to overcome the limitations of traditional machine vision. This application provides a solder joint guidance system based on key point detection, such as... Figure 1 As shown, it includes: An image acquisition unit is set in front of the welding station to acquire images of the top of the motor stator to be welded; The computational processing unit is used to execute the algorithm model, which includes a keypoint detection neural network model. The computational processing unit is configured to input the acquired image into the neural network model during operation. The keypoint detection neural network model is configured to simultaneously perform target box prediction and keypoint prediction, and output the detection box and corner coordinates of the PIN pins of the motor stator to be welded. The detection box is used to determine whether the PIN pin is qualified. When the keypoint detection neural network model determines whether the PIN pin is qualified, it also determines the completeness and accuracy of the coordinates of the vertices of the detection box. The algorithm model also includes a coordinate transformation model. The computational processing unit is configured to calculate the coordinates of the welding guide point for qualified PIN pins based on the corner coordinates of PIN pins belonging to the same welding instance during operation, and input the welding guide point coordinates into the coordinate transformation model. The coordinate transformation model is configured to convert the welding guide coordinates into the world coordinate system coordinates of the motion mechanism according to the calibration matrix, and control the welding torch to move to the transformed coordinates for welding.

[0021] In one embodiment of the invention, the system hardware mainly includes a high-resolution industrial camera deployed above the welding station and a high-performance industrial computer responsible for computation. The high-resolution industrial camera, combined with a structured light or coaxial light source of a specific wavelength, ensures clear imaging and moderate contrast in the PIN area, reducing interference such as overexposure or shadows. The industrial computer has a built-in high-performance graphics processor, providing computational support for real-time inference of the deep learning model. The core of the system lies in the algorithm software running on the industrial computer. The workflow begins with the image acquisition unit acquiring a digital image of the top of the motor stator to be welded. The image is then fed into a pre-trained keypoint detection neural network model. The model, trained on massive amounts of diverse data, no longer relies on manually defined shallow image features but can directly understand the deep features of what constitutes a PIN and what constitutes a qualified PIN.

[0022] When the keypoint detection neural network model is working, the output detection box is the minimum bounding rectangle of the PIN pin. The model outputs the coordinates of the four vertices of the detection box and the box label, which includes "pass" and "fail". When the label is "fail", the corresponding instance is not included in the calculation of the welding guide point coordinates. Specifically, the model first locates and performs a preliminary quality assessment on each PIN pin instance in the image, outputting its minimum bounding rectangle and assigning a classification label. If a PIN pin is severely bent, damaged, or excessively obscured by foreign objects, its representation in the feature space will differ significantly from that of a pass sample, thus being identified and marked as "fail" by the model. This pre-welding detection mechanism, embedded in the model's forward propagation process, greatly simplifies the system process and improves response speed.

[0023] For the PIN pin instances determined to be qualified, the model will synchronously regress the coordinates of their precise geometric key points. In the implementation scheme for the stator of a flat wire motor, each welding unit is usually composed of an upper PIN pin and a lower PIN pin. Therefore, the model needs to regress the coordinates of a total of eight key points, namely the four corner points of each of the upper and lower PIN pins. During the learning process, the model must understand the relative spatial position relationship between the upper and lower PIN pins, and this geometric constraint enhances the stability and accuracy of the prediction.

[0024] The model's ability to understand the geometric form of the PIN pins comes from the sliding slicing module adopted in the training stage. If simple random cropping or a sliding window of a fixed size is used, a large number of incomplete training samples will be generated. In this embodiment, a sliding slicing module with a multi-level judgment logic is introduced in the model training stage. The sliding slicing module first dynamically calculates the slicing size based on the area distribution of the bounding rectangles of the PIN pin groups in the training set, ensuring that the slicing operation specifically enhances the representation of small target areas rather than processing the entire image indiscriminately. Subsequently, during the sliding slicing process, a strict three-level judgment is performed on each candidate sub-image.

[0025] The first-level judgment is a basic spatial integrity check, that is, to check whether all eight key points of a PIN pin group are within the boundaries of the current sub-image. The second-level judgment delves deeper into the rationality of the geometric relationship. The module will calculate the relative spatial relationship between the upper PIN pin and the lower PIN pin based on the coordinates of these eight key points. For example, it will calculate the relative distance and the angle of the connection line between the center of the minimum bounding rectangle formed by the four corner points of the upper PIN pin and the corresponding rectangle center of the lower PIN pin. If the calculated distance or angle exceeds the preset tolerance range statistically obtained from qualified samples, even if all eight points are within the sub-image, this sample will be filtered out. The third-level judgment is related to the quality label. Only when this PIN pin group in the original training data is labeled as "qualified", its corresponding sub-image will be retained. After these three strict screenings, the retained training samples each represent a PIN pin group with a complete structure, reasonable geometry, and qualified quality. When the model is trained and put into actual operation, when the model processes the image to be detected, the "accuracy" of the four vertex coordinates of the output detection box does not only refer to the precision of the regression of these points themselves, but more deeply refers to whether the overall geometric form formed by these eight key points conforms to the "complete and reasonable" pattern learned during training and guaranteed by the sliding slicing module. If a PIN pin group appears severely bent or distorted in the imaging, or the relative position of the upper and lower PIN pins shows abnormal deviation, then the geometric relationship presented by its eight key points will deviate significantly from the "qualified" pattern internalized by the model. At this time, even if the model can barely regress the coordinates of the eight points, its internal judgment logic will tend to classify this instance as "unqualified" due to this geometric mismatch.

[0026] After model inference is completed, the system enters the post-processing stage. The computational processing unit is configured to obtain the coordinates of the welding guide point based on the center points of the four corner points of the upper PIN and the four corner points of the lower PIN belonging to the same welding instance. The welding guide point coordinates are the midpoint of the two center points. Specifically, for qualified instances, the system first groups the eight key points of the upper and lower PINs belonging to the same welding unit according to the instance identifier output by the model. Subsequently, the computational unit calculates the center point coordinates of the four corner points of the upper PIN and the four corner points of the lower PIN respectively. Finally, the midpoint of these two center points is taken as the accurate welding guide point of the welding unit. Compared with directly calculating the average center of the eight points, this effectively mitigates the possible deviations in the prediction of a single corner point, demonstrating better robustness. After obtaining the welding guide point coordinates in the image coordinate system, the coordinate transformation model begins to work. The coordinate transformation model is essentially a perspective transformation matrix obtained through a high-precision hand-eye calibration process. Using this calibration matrix, the system can seamlessly convert the two-dimensional image coordinates into three-dimensional world coordinates that the welding robot or motion mechanism can understand. Control commands are then issued, driving the welding torch to precisely move to the calculated world coordinates, completing the welding operation. The entire process, from image acquisition to the welding torch's arrival, can be completed quickly with computing power, meeting the demands of high-speed production lines.

[0027] Furthermore, to further improve system performance, especially in terms of accuracy and robustness when dealing with the detection of small target pins of various shapes, in this embodiment, the training set covers more than 20 different types of motor stators and more than 10 different shapes of pins, including square, rectangular, trapezoidal, circular, or more complex irregular structures. The total number of images reaches hundreds of thousands, and this extreme diversity ensures that the model can cover a sufficiently broad feature distribution during the learning process, thereby achieving strong generalization ability. Regarding data annotation, unlike conventional annotation methods that require constructing skeletal connections between points, in this embodiment, for each welding unit, the coordinates of the eight corner points of the upper and lower pins are precisely annotated sequentially using labelme or x-anylabeling software, such as... Figure 2 As shown. For non-standard rectangles, such as trapezoids or PIN pins with rounded corners, their corner points are defined as the virtual intersection of the extended lines of the outline or the corner points of the smallest bounding rectangle to ensure the clarity and consistency of the key point definition, such as... Figure 3 As shown.

[0028] In other training phases, besides the sliding block module, this embodiment also includes data augmentation strategies. During the initial cold start phase, various data augmentation techniques, including random cropping and mosaic augmentation, are employed to quickly expand data diversity and help the model establish initial generalization capabilities. However, when the model parameters begin to converge to a better solution after more than thirty training epochs, data augmentation methods that might produce incomplete targets, such as overly aggressive random cropping, are discontinued to ensure the model can subsequently perform refined learning in a complete target context. Data augmentation methods include photometric distortion (adjusting brightness, contrast, saturation, and hue), geometric distortion (random rotation, scaling, and cropping), noise addition (Gaussian noise, simulating solder splatter), and random pasting (pasting pins from other images into different backgrounds). This significantly expands the distribution of training data, simulating various complex working conditions, and is key to achieving "Few-Shot Learning" capability—that is, when faced with a completely new model of motor, only a small number (even single-digit) of labeled samples are needed for fine-tuning, and the model can quickly adapt.

[0029] In this embodiment, the loss function consists of three parts: the bounding box regression loss uses CIoU Loss, which fully considers the overlapping area, center point distance, and aspect ratio; the keypoint regression loss uses WingLoss, which is more sensitive to small and medium errors; and the traditional mean squared error loss and classification loss use Focal Loss, which can balance the difficulty of positive and negative samples. These three losses are weighted and summed to jointly supervise the model training, enabling the network to optimize its ability to perform target localization, keypoint regression, and classification tasks in a balanced manner. The organic combination of all these training details enables the final deployed model to quickly adapt to new motor stators with only a very small number of new samples for fine-tuning, demonstrating highly efficient transfer learning capabilities and solving the core pain point of long debugging cycles during model changeovers.

[0030] The model's backbone network is custom-developed based on current advanced single-stage detection architectures such as the YOLO series. Its network design facilitates the coordination of the two subtasks: object detection and keypoint regression. Addressing the challenge that PINs occupy a small pixel area in high-resolution images, the model retains feature maps from shallow layers of the network. While these shallow feature maps lack rich semantic information, they preserve finer spatial details. Through feature pyramid networks or multi-scale prediction mechanisms, the model can fuse deep and shallow features, thereby enabling rapid target localization over a large area while achieving accurate regression of small PIN corner points.

[0031] Obviously, the embodiments described above are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0032] It should be understood that when the terms "first," "second," etc., are used in the claims, description, and drawings of this application, they are only used to distinguish different objects and not to describe a specific order. The terms "comprising" and "including" used in the description and claims of this application indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

Claims

1. A welding guidance system based on key point detection, characterized in that, include: An image acquisition unit is installed in front of the welding station to acquire images of the top of the motor stator to be welded. A computational processing unit is used to execute an algorithm model, which includes a keypoint detection neural network model. The computational processing unit is configured to input the acquired image into the neural network model during operation. The keypoint detection neural network model is configured to simultaneously perform target box prediction and keypoint prediction, and output the detection box and corner coordinates of the PIN pins of the motor stator to be welded. The detection box is used to determine whether the PIN pin is qualified. When the keypoint detection neural network model determines whether the PIN pin is qualified, it also determines the completeness and accuracy of the coordinates of the vertices of the detection box. The algorithm model also includes a coordinate transformation model. The calculation processing unit is configured to calculate the welding guide point coordinates based on the corner coordinates of the PINs belonging to the same welding instance for qualified PINs during operation, and input the welding guide point coordinates into the coordinate transformation model. The coordinate transformation model is configured to convert the welding guide point coordinates into the world coordinate system coordinates of the motion mechanism according to the calibration matrix, and control the welding torch to move to the transformed coordinates for welding. The detection box output by the key point detection neural network model is the minimum bounding rectangle of the PIN pin; the key point detection neural network model outputs the coordinates of the four vertices of the detection box and the label of the detection box, the label includes qualified and unqualified, when the label is unqualified, the corresponding instance does not participate in the calculation of the welding guide point coordinates; The keypoint detection neural network model includes a sliding segmentation module, which is configured to dynamically calculate the segmentation size based on the area of ​​the circumscribed rectangle of the PIN group and execute multi-level judgment logic to retain sub-images for training. The multi-level judgment logic includes: Level 1 judgment: All eight key points of the entire PIN pin group are located within the sub-diagram; Second-level judgment: Based on the geometric relationship of the eight key points, calculate the relative distance and angle between the key points of the upper PIN and the lower PIN. Only when the relative distance and angle are within the preset tolerance range will the second-level judgment be passed. Third-level judgment: Based on the label of the PIN group corresponding to the sub-image in the training data, the third-level judgment is passed only if the label is qualified; The sliding block module is configured to retain a subgraph for training only when the subgraph passes the first, second, and third level of judgment simultaneously, so that the key point detection neural network model can judge the accuracy of the vertices of the detection box when it is working.

2. The system according to claim 1, characterized in that, The corner coordinates include the coordinates of eight key points in each PIN group, each PIN group including an upper PIN and a lower PIN. The key point detection neural network model is configured to use the four corner coordinates of the upper PIN and the four corner coordinates of the lower PIN as regression targets simultaneously.

3. The system according to claim 2, characterized in that, The calculation processing unit is configured to obtain the coordinates of the welding guide point based on the center point of the coordinates of the four corner points of the upper PIN and the center point of the coordinates of the four corner points of the lower PIN belonging to the same welding instance, wherein the coordinates of the welding guide point are the midpoint of the two center points.

4. The system according to claim 1, characterized in that, During training, the key point detection neural network model uses a training set that includes high-resolution images of at least 20 different types of motors and 10 different shapes of pin groups. The shapes of the pins include at least square, trapezoidal, and circular. The high-resolution images are labeled with key points.

5. The system according to claim 1, characterized in that, The keypoint detection neural network model includes a shallow feature map for small target detection.

6. The system according to claim 1, characterized in that, During training, the keypoint detection neural network model uses random pruning and mosaic augmentation in the cold start phase, and stops using data augmentation for incomplete targets after more than 30 training cycles.

7. The system according to claim 1, characterized in that, The training loss function of the keypoint detection neural network model consists of three parts: bounding box regression loss, keypoint regression loss, and classification loss. The bounding box regression loss uses CIoU Loss, the keypoint regression loss uses Wing Loss or mean squared error loss, and the classification loss uses FocalLoss. The training of the model is supervised by weighted summation.

8. The system according to claim 1, characterized in that, The keypoint detection neural network model is customized based on the YOLO series network architecture to coordinate object detection and keypoint regression tasks.

Images