Space curve welding seam identifying and tracking system and method for welding robot

Through the combination of laser vision sensors and neural network algorithms, automatic identification and tracking of the welds of the intersecting space curve between the docking pipe and the container is achieved, solving the problems of welding automation and intelligence in the existing technology, and improving welding efficiency and quality.

CN118951529APending Publication Date: 2024-11-15WUHAN INST OF TECH +1
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411216654.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-02
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing robot welding technology is difficult to achieve complete automation and intelligence, especially when dealing with space curve welds where the takeover and the container, it is impossible to effectively identify and track the weld characteristics, resulting in low welding efficiency and quality.

Method used

Using laser vision sensors and improved neural network algorithms, weld images are obtained through laser vision sensors, data optimization is used to optimize data processing, pixel coordinates of weld feature points are extracted and converted into spatial pose information, combined with robot motion control The system adjusts the welding gun path in real time to realize automatic identification and tracking of welds.

Benefits of technology

It realizes high-precision weld recognition and tracking, improves the stability and efficiency of welding, can accurately and quickly extract weld feature points, reduces the number of model parameters and overfitting risks, and improves the robustness and prediction of the network. Accuracy of the results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118951529A_ABST
    Figure CN118951529A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of automatic welding, and particularly relates to a welding robot space curve welding seam identifying and tracking system which comprises a laser vision sensing system, a laser positioning system, a laser positioning system and a control system. The welding seam recognition and tracking module is used for processing the welding seam image, recognizing welding seam features, extracting pixel coordinates of welding seam feature points and converting the pixel coordinates into space pose information; and the robot motion control system is used for adjusting the path of the welding gun in real time according to the spatial position information. The method is used for automatically identifying and tracking the space curve welding seam where the connecting pipe and the container intersect. According to the system, the laser vision sensor and an improved neural network algorithm are researched and applied, and high-precision welding seam recognition and tracking are achieved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of automated welding, and in particular relates to a spatial curve weld recognition and tracking system and method for a welding robot. Background Art

[0004] The application of industrial robot welding in manufacturing has significantly improved the efficiency, quality and consistency of welding compared to manual welding technology. However, at present, most robot welding uses teaching programming, which cannot fully realize the requirements of automated and intelligent welding. Therefore, automated welding is the focus of current robot welding research. Summary of the invention

[0005] The purpose of the present invention is to provide a welding robot space curve weld recognition and tracking system and method, which is used for automatic recognition and tracking of space curve welds that intersect with a nozzle and a container. The system achieves high-precision weld recognition and tracking by studying and applying laser vision sensors and improved neural network algorithms.

[0006] The technical solution adopted by the present invention is as follows: A welding robot spatial curve weld recognition and tracking system, comprising: A laser vision sensor system, wherein the laser vision sensor system is used to obtain weld information, wherein the weld information is a weld image; A weld recognition and tracking module, which is used to process the weld image, identify weld features, extract pixel coordinates of weld feature points, and convert them into spatial pose information, wherein the laser stripes emitted by the laser vision sensor system will have an inflection point at the spatial curve weld where the nozzle intersects, and the midpoint of the inflection point is the weld feature point; A robot motion control system, wherein the robot motion control system is used to adjust the welding gun path in real time according to the spatial position information.

[0007] Furthermore, the laser vision sensing system is installed on the welding gun, and the laser vision sensing system includes an industrial camera and a laser emitter. The line laser emitted by the laser emitter irradiates the surface of the workpiece to form laser stripes. After the laser stripes are reflected by the surface of the workpiece, a weld image is presented on the industrial camera.

[0008] Furthermore, the weld image includes a plurality of original weld images with spatter and arc interference.

[0009] Furthermore, in the step of processing the weld image and identifying weld features, it is first necessary to perform data optimization processing on the interference existing in the weld image through an improved U-Net model. The optimization processing includes image denoising, enhancement and labeling. The U-Net model includes a feature recognition module, a decoder module, a skip connection module and a segmentation prediction module.

[0010] Furthermore, the step of processing the weld image and identifying weld features includes: Use image annotation software to annotate the outline of the laser stripe point by point and generate an annotated json file; Convert the json file into a VOC format dataset that is easy for network recognition, and generate a picture of the marked area, where the laser stripes are marked in red; The midpoint of the inflection point of the laser stripes at the space curve weld where the nozzle intersects is the weld feature point.

[0011] Furthermore, the step of extracting pixel coordinates of weld feature points includes: The feature recognition algorithm based on neural network improves the noise resistance of images; The edge of the laser stripe is extracted using the LOG operator, and the center line of the laser stripe edge is fitted; The coordinates of the inflection point are found by using a particle matching method. The step of adjusting the welding gun path in real time according to the spatial position information includes: By calling the inverse kinematics solution algorithm, the spatial position of the weld feature points is calculated, converted into the robot's joint angle information, and transmitted to the robot controller through Socket communication; the robot controller accurately controls the movement of the robot end according to the received joint angle information to perform the weld tracking task.

[0012] The second object of the present invention is to provide a method for identifying and tracking a spatial curved weld of a welding robot, comprising: Acquire weld information, wherein the weld information is a weld image; Processing the weld image, identifying weld features and extracting pixel coordinates of weld feature points, and converting them into spatial pose information; The welding gun path is adjusted in real time according to the spatial position information.

[0013] Furthermore, the method further comprises: Smoothing out the welding gun path; The moving speed of the welding gun is automatically adjusted according to the weld characteristics.

[0014] Furthermore, the method also includes: identifying weld defects and calibrating defect areas; Determine the defect type according to the defect area, wherein the defect type includes pores or slag inclusions; Adjust the welding gun action in real time according to the defect type.

[0015] The technical effects achieved by the present invention are as follows: the software part of the robot weld tracking system is integrated, and the call to the API interface of the OpenCV algorithm library and the neural network model is realized, so that the pixel coordinates of the weld feature points can be accurately and quickly extracted, and the spatial coordinate conversion is performed to obtain the spatial pose information of the weld feature points. The host computer of the robot weld tracking platform calculates the spatial pose of the weld feature points by calling the inverse kinematics solution algorithm, converts it into the robot's joint angle information, and transmits it to the robot controller through Socket communication. The controller accurately controls the movement of the robot end according to the received angle information to perform the weld tracking task.

[0016] The present invention adopts the Mobile-Net backbone feature extraction network to replace the original Encode module, which reduces the number of model parameters and the occurrence of overfitting. The ECA attention mechanism is integrated into the jump connection between the codec, which improves the ability to extract network detail features. The network parameter standard is optimized using a hybrid loss function, which improves the prediction results and the robustness of the network. This improvement makes the extraction of weld feature points more accurate and reliable.

[0017] The present invention identifies welding images and formulates welding strategies in real time according to welding conditions, thereby making the welding operation more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a system flow chart of the present invention.

[0019] Figure 2 This is a structural diagram of the welding robot weld tracking system of the present invention.

[0020] Figure 3 This is the structure diagram of the U-Net model of the present invention.

[0021] Figure 4 This is a flow chart of weld feature point extraction of the present invention.

[0022] Figure 5 It is the weld tracking flow chart of the present invention.

[0023] Figure 6 This is a block diagram of the laser vision system calibration algorithm of the present invention.

[0024] Figure 7 It is the software block diagram of the robot weld seam tracking system of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose and advantages of the present invention more clearly understood, the present invention is specifically described below in conjunction with embodiments. It should be understood that the following text is only used to describe one or several specific embodiments of the present invention, and does not strictly limit the scope of protection of the specific claims of the present invention.

[0026] See also Figure 1 , a spatial curve weld recognition and tracking system for a welding robot, comprising: A laser vision sensor system, wherein the laser vision sensor system is used to obtain weld information, wherein the weld information is a weld image; A weld recognition and tracking module, which is used to process the weld image, identify weld features, extract pixel coordinates of weld feature points, and convert them into spatial pose information, wherein the laser stripes emitted by the laser vision sensor system will have an inflection point at the spatial curve weld where the nozzle intersects, and the midpoint of the inflection point is the weld feature point; A robot motion control system, wherein the robot motion control system is used to adjust the welding gun path in real time according to the spatial position information.

[0027] The system is mainly composed of a robot motion control system, a laser vision sensor system, and a weld recognition and tracking module. It integrates visual detection, host computer control, and motion execution to complete the automatic tracking task of welds. The specific laser vision sensor system irradiates the workpiece surface and receives the reflected laser stripe image, collects a clear weld image and transmits it to the industrial computer. The weld recognition and tracking module performs image processing after receiving the image, identifies the weld features and extracts the pixel coordinates of the feature points, converts them into spatial posture information and transmits them to the robot controller. The robot motion control system adjusts the welding gun path in real time according to the spatial position information to achieve automatic tracking of the weld, thereby improving the welding quality and efficiency. For its specific working principle, please refer to Figure 2 as well as Figure 6 shown.

[0028] Specifically, the laser vision sensing system is installed on the welding gun. The laser vision sensing system includes an industrial camera and a laser emitter. The line laser emitted by the laser emitter is irradiated onto the surface of the workpiece to form laser stripes. After the laser stripes are reflected by the surface of the workpiece, a weld image is presented on the industrial camera. The weld image includes several original weld images with spatter and arc interference. The measurement principle of the laser emitter is based on the displacement and deformation of the laser stripes to calculate the position and shape of the weld.

[0029] Based on the above, the laser vision sensing system also includes a lens, a lens, a baffle and a housing. The baffle at the connection has protection functions such as splash protection and heat resistance. The laser vision sensing system is installed on the welding gun as a whole. The internal industrial camera is parallel to the front surface of the housing, and the laser emitter is placed at a fixed angle. When the welding task is performed, the laser vision sensing system maintains a certain height between the workpiece to be welded. The line laser emitted by the internal laser emitter irradiates the surface of the workpiece, forming a laser stripe. After being reflected by the surface of the workpiece, the corresponding image is presented on the industrial camera through the lens.

[0030] It should be further explained that the laser transmitter was calibrated on the experimental platform, and the precise parameters of the camera were obtained using the multi-point calibration method. The robot's tool coordinate system calibration was completed using the six-point method, and a hand-eye calibration model between the robot and the laser vision sensor was established, realizing the spatial posture conversion between multiple coordinate systems. The specific calibration steps are as follows: The laser transmitter is calibrated using Zhang's calibration method, and the checkerboard calibration plate is used as the calibration tool. First, the actual size of the checkerboard is measured, and the calibration plate is fixed in an unchanged position. Then, by changing the robot's posture multiple times, the laser transmitter at the end of the robot is moved, and the calibration plate image is collected from different angles. According to the size of the checkerboard and the data of the origin of the world coordinate system, the physical coordinates of the corner position of the calibration plate are solved, and finally the intrinsic parameter matrix and distortion parameters of the camera in the laser vision sensor are obtained. For the specific calibration algorithm block diagram, please refer to Figure 6 At the same time, the six-point method was used to complete the robot tool calibration task, the robot hand-eye calibration model was established, and the hand-eye calibration of the welding robot and laser vision sensor was completed.

[0031] In addition, the workpiece was welded on the welding robot weld tracking experimental platform to collect weld images during the welding process. The equipment used was the CXZK-JXB300LS laser vision sensor of Chuangxiang Intelligent Control. The laser vision sensor was installed 235 mm from the end of the welding gun and emitted a red laser line with a wavelength of 850 nanometers. The image was collected using a CMOS camera with a resolution of 1280 pixels × 960 pixels. During the welding process, a total of 410 original weld images with spatter and arc interference were collected.

[0032] The weld recognition and tracking system includes a host computer system of a laser vision sensor, an OpenCV algorithm library, an API interface of a neural network model, a calibration algorithm of a laser vision system, and a weld tracking algorithm. The above together constitute the software part of the weld recognition and tracking system. The weld tracking system includes a host computer system of a laser vision sensor, an OpenCV algorithm library, and an API interface of a neural network model. The weld tracking accuracy is closely related to the calibration accuracy of the laser vision system, and the calibration of the laser vision sensor, the robot tool coordinate system, and the robot hand-eye calibration are completed.

[0033] In the step of processing the weld image and identifying weld features, it is first necessary to perform data optimization processing on the interference existing in the weld image through an improved U-Net model. The optimization processing includes image denoising, enhancement and labeling. The U-Net model includes a feature recognition module, a decoder module, a skip connection module and a segmentation prediction module.

[0034] Finally, a high-quality weld feature dataset was established, and the improved U-Net algorithm used Mobile-Net as the backbone feature extraction network.

[0035] Secondly, use the image annotation software Labelme to annotate the outline of the laser stripe point by point and generate an annotated json file; Convert the json file into a VOC format dataset that is easy for network recognition, and generate a picture of the marked area, where the laser stripes are marked in red; The midpoint of the inflection point of the laser stripes at the space curve weld where the nozzle intersects is the weld feature point.

[0036] Aiming at the problem that traditional image processing algorithms cannot accurately identify weld features in welding interference environments, an improved U-Net algorithm is proposed. The improved U-Net algorithm uses Mobile-Net as the backbone feature extraction network to replace the original encoding module, reducing the number of model parameters, reducing the risk of overfitting, and improving the network's operating efficiency. In the skip connection between encoding and decoding, an efficient channel attention machine (EfficientChannelAttention) is integrated to achieve feature weighted fusion, which significantly improves the network's ability to recognize detail features. A hybrid loss function (such as a combination of cross entropy loss and Dice loss) is used to optimize the network parameter standard, improving the accuracy and robustness of the prediction results. Specifically, the improved U-Net model is as follows: Figure 3As shown in the figure, it consists of a feature recognition module, a decoder module, a skip connection module and a segmentation prediction module. The feature recognition module uses advanced convolutional neural network (CNN) technology to gradually abstract more advanced and representative feature representations through multi-layer convolution and pooling operations, laying a solid foundation for subsequent processing. Then, after receiving the features output by the feature recognition module, the decoder module decodes these abstract features into understandable and actionable information to maintain the integrity and accuracy of the information. The decoder module usually uses technologies such as deconvolution or upsampling to gradually restore the feature map to its original size or close to its original size, and in the process, fine-tune and optimize the features. The skip connection module connects feature representations at different levels, so that shallow detail information and deep semantic information can be effectively integrated. The jump connection module achieves fast fusion and transmission of feature maps through simple addition or concatenation operations, which brings significant performance improvement to the system. Finally, the segmentation prediction module accurately segments and predicts the input data based on the output results of the previous module. This module usually uses a fully connected layer (FullyConnectedLayer) or a convolutional layer as a classifier to determine the category of each pixel or region. In order to improve the accuracy and robustness of the prediction, the segmentation prediction module will also introduce some post-processing techniques, such as conditional random field (CRF) and non-maximum suppression (NMS), to further optimize the segmentation results.

[0037] Furthermore, to realize the automatic tracking of the weld by the welding robot, it is necessary to know the coordinates and poses of the weld points during welding, and the steps of extracting the pixel coordinates of the weld feature points include: The feature recognition algorithm based on neural network improves the noise resistance of images; The edge of the laser stripe is extracted using the LOG operator, and the center line of the laser stripe edge is fitted; Use the particle matching method to find the coordinates of the inflection point. Through edge detection algorithm and particle matching algorithm, the pixel coordinates of weld feature points can be extracted quickly and accurately. These algorithms have good noise resistance and can effectively deal with interference such as spatter and arc light during welding. For specific implementation steps, please refer to Figure 4 shown.

[0038] By introducing the convolutional neural network (CNN) in deep learning to deepen the ability of feature extraction, CNN can automatically learn and extract multi-level abstract features. These features are not limited to edge information, but also include complex patterns such as texture and shape, thereby more comprehensively describing the characteristics of the weld. The front end of the CNN architecture can use a combination of multiple convolutional layers and pooling layers to extract local features in the image and perform downsampling, reducing the amount of calculation while retaining important information. Subsequently, several fully connected layers are introduced to integrate global information, and finally the predicted coordinates of the weld feature points are output. In order to improve the model's resistance to noise, simulated noise data such as spatter and arc light can be added during the training process, and data enhancement methods can be used to enhance the generalization ability of the model. Furthermore, in order to locate the inflection point of the weld more accurately, the candidate inflection points obtained after particle matching can be further screened and optimized by combining Hough Transform or Random Sampling Consensus Algorithm (RANSAC). These algorithms can effectively fit the optimal geometric model (such as straight line, circle, etc.) from data containing noise and outliers, so as to locate the inflection point of the weld more accurately. In real-time application scenarios, considering the limitations of computing resources and time, compression technologies such as model pruning and quantization can be used to reduce the size of the CNN model and speed up the inference speed, ensuring that the algorithm can run efficiently on welding robots or intelligent detection equipment.

[0039] The step of adjusting the welding gun path in real time according to the spatial position information comprises: By calling the inverse kinematics solution algorithm, the spatial position of the weld feature point is calculated, converted into the robot's joint angle information, and transmitted to the robot controller through Socket communication; the robot controller accurately controls the movement of the robot end according to the received joint angle information to perform the weld tracking task. For specific welding tasks, please refer to Figure 5 shown.

[0040] The present invention also discloses a method for identifying and tracking a space curve weld of a welding robot, comprising: Acquire weld information, wherein the weld information is a weld image; Processing the weld image, identifying weld features and extracting pixel coordinates of weld feature points, and converting them into spatial pose information; The welding gun path is adjusted in real time according to the spatial position information.

[0041] The specific implementation process and steps of this method are consistent with those of the welding robot spatial curve weld seam recognition and tracking system. For details, please refer to the welding robot spatial curve weld seam recognition and tracking system.

[0042] Furthermore, the welding robot spatial curve weld recognition and tracking method further includes smoothing the welding gun path; The moving speed of the welding gun is automatically adjusted according to the weld characteristics.

[0043] In the above steps, in order to avoid excessive acceleration or deceleration of the welding gun during movement, the system will smooth the path to reduce sharp turns and sudden speed changes. At the same time, according to the characteristics of the weld, such as changes in width or material differences, the system can automatically adjust the movement speed of the welding gun. In areas where the weld is wide or the material has good thermal conductivity, the welding speed is appropriately increased to improve efficiency; in areas where the weld is narrow or requires fine welding, the speed is reduced to ensure welding quality.

[0044] In addition, the welding robot spatial curve weld recognition and tracking method also includes identifying weld defects and calibrating defect areas; Determine the defect type according to the defect area, wherein the defect type includes pores or slag inclusions; Adjust the welding gun action in real time according to the defect type.

[0045] By adopting the above steps, a welding strategy can be formulated according to the defects of the weld. Specifically for different defects, the system will choose to increase / decrease the welding current, dwell time, or use multiple welding methods to ensure that the welding area can be fully welded. At the same time, corresponding welding strategies can also be formulated for the detailed characteristics of the weld (appearance, penetration depth, penetration width and other parameters). The specific method is the same as the above defect formulation method, which is also achieved by changing the path of the welding gun and welding parameters.

[0046] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. A welding robot spatial curve weld recognition and tracking system, characterized in that: include: A laser vision sensor system, wherein the laser vision sensor system is used to obtain weld information, wherein the weld information is a weld image; A weld recognition and tracking module, which is used to process the weld image, identify weld features, extract pixel coordinates of weld feature points, and convert them into spatial pose information, wherein the laser stripes emitted by the laser vision sensor system will have an inflection point at the spatial curve weld where the nozzle intersects, and the midpoint of the inflection point is the weld feature point; A robot motion control system, wherein the robot motion control system is used to adjust the welding gun path in real time according to the spatial position information.

2. The spatial curve weld recognition and tracking system of the welding robot according to claim 1, characterized in that: The laser vision sensing system is installed on the welding gun. The laser vision sensing system includes an industrial camera and a laser emitter. The line laser emitted by the laser emitter irradiates the surface of the workpiece to form laser stripes. After the laser stripes are reflected by the surface of the workpiece, a weld image is presented on the industrial camera.

3. The spatial curve weld recognition and tracking system of the welding robot according to claim 1, characterized in that: The weld image includes a plurality of original weld images with spatter and arc interference.

4. The spatial curve weld recognition and tracking system of the welding robot according to claim 1, characterized in that: In the step of processing the weld image and identifying weld features, it is first necessary to perform data optimization processing on the interference existing in the weld image through an improved U-Net model. The optimization processing includes image denoising, enhancement and labeling. The U-Net model includes a feature recognition module, a decoder module, a skip connection module and a segmentation prediction module.

5. The spatial curve weld recognition and tracking system of the welding robot according to claim 3 is characterized in that: The step of processing the weld image and identifying weld features comprises: Use image annotation software to annotate the outline of the laser stripe point by point and generate an annotated json file; Convert the json file into a VOC format dataset that is easy for network recognition, and generate a picture of the marked area, where the laser stripes are marked in red; The midpoint of the inflection point of the laser stripes at the space curve weld where the nozzle intersects is the weld feature point.

6. The spatial curve weld recognition and tracking system of the welding robot according to claim 1, characterized in that: The step of extracting pixel coordinates of weld feature points comprises: The feature recognition algorithm based on neural network improves the noise resistance of images; The edge of the laser stripe is extracted using the LOG operator, and the center line of the laser stripe edge is fitted; Use the particle matching method to find the coordinates of the inflection point.

7. The spatial curve weld recognition and tracking system of the welding robot according to claim 1, characterized in that: The step of adjusting the welding gun path in real time according to the spatial position information comprises: By calling the inverse kinematics solution algorithm, the spatial position of the weld feature points is calculated, converted into the robot's joint angle information, and transmitted to the robot controller through Socket communication; the robot controller accurately controls the movement of the robot end according to the received joint angle information to perform the weld tracking task.

8. A method for spatial curve weld recognition and tracking of a welding robot, characterized in that: include: Acquire weld information, wherein the weld information is a weld image; Processing the weld image, identifying weld features and extracting pixel coordinates of weld feature points, and converting them into spatial pose information; The welding gun path is adjusted in real time according to the spatial position information.

9. The method for spatial curve weld recognition and tracking of a welding robot according to claim 1, characterized in that: Also includes: Smoothing out the welding gun path; The moving speed of the welding gun is automatically adjusted according to the weld characteristics.

10. The method for spatial curve weld recognition and tracking of a welding robot according to claim 1, characterized in that: Also includes: Identify weld defects and mark defect areas; Determine the defect type according to the defect area, wherein the defect type includes pores or slag inclusions; Adjust the welding gun action in real time according to the defect type.

Citation Information

Cited By

  • Pipeline all-position multi-layer multi-channel TIG welding bead morphology measuring device and method

    CN120598899A

  • Stainless steel air pipe welding spot automatic identification method and system based on infrared thermal imaging

    CN121223207A

  • Volute weld joint positioning auxiliary device and volute welding device

    CN224359602U