Space discontinuous welding seam real-time tracking method and system based on laser vision

Abstract

The invention discloses a space discontinuous welding seam real-time tracking method and system based on laser vision, and relates to the technical field of robot intelligent welding, and the method comprises the steps that a laser stripe image of a to-be-welded area is collected in real time; performing denoising processing on the laser stripe image to obtain a denoised stripe image, extracting an image region of interest in the denoised stripe image, and extracting a stripe image center point in the image region of interest by adopting a gray centroid algorithm; performing segmentation fitting based on the central point of the stripe image to obtain a fitting straight line segment I and a fitting straight line segment II; and based on the calibration mapping relation, the image intersection points are mapped into a robot coordinate system through the image coordinate system, the pre-positioning image welding points are recognized as welding jump points, and the continuous welding line center fitted by the welding line feature image is recognized as continuous welding points. The method provided by the invention has the beneficial effects that the pre-positioning welding point and the continuous welding point are recognized in advance to guide the welding arm to carry out welding operation, the automatic operation of welding is facilitated, and the welding quality is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent welding, and particularly to a real-time tracking method and system for spatially discontinuous weld seams based on laser vision. Background Art

[0002] Welding, as a basic structural connection technology, plays a crucial role in many fields such as automobile manufacturing, construction transportation, and aerospace. With the rapid development of automation technology, welding robots have been more and more widely used, greatly improving the efficiency and consistency of the welding process.

[0003] In practical applications, the operation modes of welding robots are still mainly "teaching playback" and "offline programming". The "teaching playback" mode relies on manual teaching. The operator needs to manually guide the robot to complete the setting of the welding path at the welding site. This method will produce errors in complex workpieces or changing environments and lacks flexibility. At the same time, the "offline programming" mode requires pre-generation of the welding path. Although it can improve efficiency, for complex welding tasks, the process of creating and adjusting the model is often cumbersome and time-consuming.

[0004] For complex spatial three-dimensional structures, it is necessary to pre-spot weld and fix them before welding. During the subsequent spot welding process, if welding is not carried out across the pre-spot welds, quality problems such as bulging are likely to occur. Currently, common real-time weld tracking methods usually rely on sensors, mainly divided into laser sensor sensing, arc sensor sensing, contact sensor sensing, and vision sensor sensing. First, the most widely used laser sensors have the advantage of strong anti-interference ability, but they are complex to debug and have high technical requirements for operators; at the same time, limited by the working distance and the working mode of the laser triangulation method, it is difficult to carry out large-scale on-site applications. Second, arc sensors are insensitive to arcs, temperature, and dust, and there is no signal leading problem; however, they cannot sense the weld feature points in advance, which limits their application in workpieces with spatially discontinuous welds. Third, contact sensors cannot meet the requirements of the industrial manufacturing field with high precision requirements. Finally, vision sensors can accurately obtain data in real time and have the advantage of obtaining leading data, which is crucial for tracking spatially discontinuous welds in complex working scenarios. In the prior art, the real-time weld tracking method based on laser vision sensors is only applicable to simple welds and is difficult to meet the real-time tracking requirements of spatially discontinuous welds in complex scenarios; the existing technology based on line fitting can be used for feature point extraction of V-shaped welds and fillet welds, but most of the prior art uses traditional image processing technology to denoise welding images with complex backgrounds and strong noise, otherwise high-quality images cannot be obtained for subsequent algorithm analysis, which limits its further development. Therefore, it is particularly important to improve the leading detection and identification of pre-positioned weld spots by welding robots in complex environments. It is necessary to propose a real-time tracking method and system for spatially discontinuous welds based on laser vision to solve or at least alleviate some of the above defects. Summary of the Invention

[0005] The main object of the present invention is to provide a real-time tracking method and system for spatially discontinuous welds based on laser vision, aiming to solve the technical problem in the prior art that it is difficult to lead-identify pre-positioned weld spots based on machine vision, resulting in repeated welding of pre-positioned weld spots and poor welding quality.

[0006] To achieve the above object, the present invention provides a real-time tracking method for spatially discontinuous welds based on laser vision, which uses a front laser emitter and a front camera disposed on a welding arm to perform weld tracking on a spatial three-dimensional structure. The front laser emitter is upstream of the welding arm, and the front camera is upstream of the front laser emitter. The front camera is arranged at an inclination angle of α degrees in the direction towards the front laser emitter. The spatial three-dimensional structure includes a first plate and a second plate arranged at an included angle of β, where β is between 0 degrees and 180 degrees. The first plate and the second plate are pre-positioned for welding through pre-positioned weld spots. The camera coordinate system corresponding to the front camera has a calibration mapping relationship with the robot coordinate system of the welding arm, and the method includes the following steps:

[0007] S10, Collect laser stripe images in real time through a forward-looking camera;

[0008] S20, Denoise the laser stripe images to obtain denoised stripe images, extract the regions of interest in the denoised stripe images, and use the gray centroid algorithm to extract the center points of the stripe images in the regions of interest;

[0009] S30, Based on the center points of the stripe images, use the RANSAC line fitting algorithm for segmented fitting to obtain the first fitted line segment and the second fitted line segment;

[0010] S40, Establish an image coordinate system with the upper left corner endpoint of the denoised stripe image as the origin, the vertical extension direction as the x-axis, and the horizontal extension direction as the y-axis. Based on the image coordinate system, determine whether the current slope K1 of the first fitted line segment is greater than 0, and based on the image coordinate system, determine whether the current slope K2 of the second fitted line segment is less than 0;

[0011] S50, If the current slope K1 is greater than 0 and the current slope K2 is less than 0, then obtain the intersection point of the first fitted line segment and the second fitted line segment as the center point of the weld feature image; if the current slope K1 is not greater than 0 and the current slope K2 is less than 0, then determine the intersection point of the first fitted line segment and the second fitted line segment as the pre-positioned image solder joint;

[0012] S60, Based on the calibration mapping relationship, map the image intersection point from the image coordinate system to the robot coordinate system, identify the pre-positioned image solder joint as a welding jump point, and identify the center point of the weld feature image as a continuous welding point.

[0013] Furthermore, the expression for mapping the pixel coordinate point (u, ν) in the image coordinate system to the camera coordinate point (μ, ν0) in the camera coordinate system is

[0014] where f x is the radial focal length of the camera lens, f y is the tangential focal length of the camera lens, (x c , y c , z c ) is the laser stripe center coordinate point in the camera coordinate system obtained by camera shooting, and z c is the camera optical center height coordinate;

[0015] Use the formula to calculate and obtain the laser stripe center coordinate point (x c , y c , z c ), where A, B, and C are the laser plane equation in the camera coordinate system z c = Ax c + Byc The normal vector of the plane of +C;

[0016] Using the formula Obtain the calibration mapping relationship between the camera coordinate system and the robot coordinate system (for hand-eye calibration), where H T is the hand-eye transformation matrix, H B is the transformation matrix of the robot coordinate system, (x B , y B , z B , 1) is the homogeneous coordinate of the laser stripe in the robot coordinate system, (x C , y C , z C , 1) is the homogeneous coordinate of the laser stripe in the camera coordinate system, R 3×3 is the rotation angle of the x, y, and z axes from the world coordinate to the camera coordinate, t 3×1 is the translation distance required for the x, y, and z axes from the world coordinate to the camera coordinate is.

[0017] Furthermore, step S20 specifically includes:

[0018] Use the U-NET network model to perform strong light image segmentation and firework image segmentation on the laser stripe image to obtain a denoised stripe image;

[0019] Determine the region of interest (ROI) in the denoised stripe image through the gray mean processing method;

[0020] Use the gray centroid algorithm to extract the center point of the stripe image in the region of interest.

[0021] Furthermore, use the formula to determine the region of interest, where ROWc and COLc are the row and column coordinate sizes of the point obtained according to the gray center of gravity value in the image coordinate system.

[0022] Furthermore, use the formula G th =(G max +G mean ) / 2 to determine the gray threshold, G max is the maximum gray value of the image in the region of interest, G mean is the gray mean value of the image in the region of interest, G th is the gray threshold;

[0023] Use the formula to screen the pixel points g(u i , v i ) in the search area of the i-th column in the image coordinate system, where g(u i , v i) is the center point candidate, and m is the pixel value at the corresponding g(u i ,v i ) point;

[0024] The formula is used to obtain the coordinates g c (i) of the center point of the i-th column of stripes.

[0025] Further, in step S30, the center point of the stripe image is divided into two fitting coordinate point groups by a preset cutting block line, where the preset cutting block line is parallel to the x-axis and is located at the horizontal middle line position of the image region of interest.

[0026] Further, the value range of a is 40° to 50°.

[0027] Further, β is 90°.

[0028] The present invention also provides a real-time tracking system for spatially discontinuous welds based on laser vision,

[0029] including a welding robot, a front laser emitter, and a front camera. The welding robot has a base and a robotic arm, and both the front laser emitter and the front camera are arranged on the robotic arm.

[0030] A processing device is arranged on the welding robot, and the processing device is used to implement the steps of the above-mentioned real-time tracking method for spatially discontinuous welds based on laser vision.

[0031] Compared with the prior art, the real-time tracking method for spatially discontinuous welds based on laser vision provided by the present invention has the following beneficial effects:

[0032] A real-time tracking method for spatially discontinuous welds based on laser vision provided by the present invention collects laser stripe images of the area to be welded in real time through a pre-mounted camera upstream of the welding arm; performs denoising processing on the laser stripe images to obtain denoised stripe images, extracts the region of interest in the denoised stripe images, and uses the gray centroid algorithm to extract the center points of the stripe images in the region of interest; after performing piecewise fitting based on the center points of the stripe images to obtain the first fitted straight line segment and the second fitted straight line segment, determines the intersection point of the images as the center point of the weld feature image or the pre-positioned image solder joint based on the slopes of the first fitted straight line segment and the second fitted straight line segment in the image coordinate system; finally, maps the intersection point of the images from the image coordinate system to the robot coordinate system based on the calibration mapping relationship, identifies the pre-positioned image solder joint as a welding jump point, and identifies the center point of the weld feature image as a continuous welding point. The method of the present invention, based on the combination of a pre-mounted laser emitter and a pre-mounted camera to form a 2.5D laser vision sensor, can obtain the depth information of the laser stripe, can accurately calibrate the mapping relationship, and is convenient for segmenting the laser stripe images. Based on the calibration mapping relationship and analyzing the laser stripe images, it can identify the pre-positioned solder joints and continuous welding points in advance to guide the welding arm to perform welding operations, which is beneficial to the automated operation of welding and improves the welding quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0034] Figure 1 It is a schematic flowchart of a real-time tracking method for spatially discontinuous welds based on laser vision in an embodiment of the present invention;

[0035] Figure 2 It is a schematic diagram of the mapping characteristics of laser stripes in an embodiment of the present invention. Among them, 2a is a schematic diagram of the first corresponding relationship between two laser projection lines in the case of continuous welds, and 2b is a schematic diagram of the second corresponding relationship between two laser projection lines during pre-positioned solder joint interference;

[0036] Figure 3 It is a schematic diagram of denoising laser stripe images using a U-NET network model in an embodiment of the present invention. Among them, 3a is a schematic diagram of strong light image segmentation, and 3b is a schematic diagram of spark image segmentation;

[0037] Figure 4 It is a schematic diagram of the principle for determining the region of interest in an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the principle for fitting the first and second fitting straight line segments in an embodiment of the present invention. Among them, 5a is a schematic diagram of the principle for obtaining continuous welding points, 5b is one of the schematic diagrams of the principle for obtaining welding jump points, and 5c is the other schematic diagram of the principle for obtaining welding jump points.

[0039] The realization of the object, functional features and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings. Detailed implementation manners

[0040] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0042] It should be noted that all the directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0043] In addition, the descriptions involving "first", "second", etc. in the present invention are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0044] Please refer to the appended Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 and Figure 5, the present invention provides a real-time tracking method for spatially discontinuous weld seams based on laser vision, which uses a front laser emitter and a front camera arranged on a welding arm to perform weld seam tracking on a spatial three-dimensional structure. The front laser emitter is located upstream of the welding arm, and the front camera is located upstream of the front laser emitter. The front camera is arranged at an inclination angle of α in the direction towards the front laser emitter. The spatial three-dimensional structure includes a first plate and a second plate arranged at an included angle β, where β is between 0 degrees and 180 degrees. The first plate and the second plate are pre-positioned for welding through pre-positioning solder joints. The camera coordinate system corresponding to the front camera has a calibration mapping relationship with the robot coordinate system of the welding arm, and the method includes the following steps:

[0045] S10, collect laser stripe images in real time through the front camera;

[0046] S20, perform denoising processing on the laser stripe images to obtain denoised stripe images, extract the region of interest in the denoised stripe images, and use the gray centroid algorithm to extract the center points of the stripe images in the region of interest;

[0047] S30, perform segmented fitting on the basis of the center points of the stripe images using the RANSAC line fitting algorithm to obtain a first fitted line segment and a second fitted line segment;

[0048] S40, establish an image coordinate system with the upper left corner endpoint of the denoised stripe image as the origin, the vertical extension direction as the x-axis, and the horizontal extension direction as the y-axis. Based on the image coordinate system, determine whether the current slope K1 of the first fitted line segment is greater than 0, and based on the image coordinate system, determine whether the current slope K2 of the second fitted line segment is less than 0;

[0049] S50, if the current slope K1 is greater than 0 and the current slope K2 is less than 0, then obtain the image intersection point of the first fitted line segment and the second fitted line segment as the center point of the weld seam feature image (the center point of the feature image of the continuous weld seam); if the current slope K1 is not greater than 0 and the current slope K2 is less than 0, then determine the image intersection point of the first fitted line segment and the second fitted line segment as the pre-positioning image solder joint;

[0050] S60, map the image intersection point from the image coordinate system to the robot coordinate system based on the calibration mapping relationship, identify the pre-positioning image solder joint as a welding jump point, and identify the center point of the weld seam feature image as a continuous welding point.

[0051] The real-time tracking method for spatially discontinuous weld seams based on laser vision provided by the present invention collects laser stripe images of the area to be welded in real time through a pre-positioned camera upstream of the welding arm; performs denoising processing on the laser stripe images to obtain denoised stripe images, extracts the region of interest in the denoised stripe images, and uses the gray centroid algorithm to extract the center point of the stripe image in the region of interest; after performing piecewise fitting based on the center point of the stripe image to obtain a first fitted straight line segment and a second fitted straight line segment, determines the intersection point of the images as the center point of the weld feature image or the pre-positioned image solder joint based on the slopes of the first fitted straight line segment and the second fitted straight line segment in the image coordinate system; finally, maps the intersection point of the images from the image coordinate system to the robot coordinate system based on the calibration mapping relationship, identifies the pre-positioned image solder joint as a welding jump point, and identifies the center point of the weld feature image as a continuous welding point. The method of the present invention, based on the combination of a pre-positioned laser emitter and a pre-positioned camera to form a 2.5D laser vision sensor, can obtain the depth information of the laser stripe, can accurately calibrate the mapping relationship, and is convenient for segmenting and processing the laser stripe images. Based on the calibration mapping relationship and analyzing the laser stripe images, it can identify pre-positioned solder joints and continuous welding points in advance to guide the welding arm to perform welding operations, which is beneficial to the automated operation of welding and improves the welding quality.

[0052] Specifically, the formula is used for judgment.

[0053] Please refer to Figure 2 and Figure 3 , and through research, it is found that after the first laser projection line and the second laser projection line are captured by the 2.5D vision sensor; if there is a continuous weld seam between the first plate and the second plate, the first laser projection line and the second laser projection line have a first corresponding relationship; if there is a pre-positioned solder joint between the first plate and the second plate, under the interference of the pre-positioned solder joint, the first laser projection line and the second laser projection line have a second corresponding relationship.

[0054] Furthermore, the expression for mapping the pixel coordinate point (u, ν) in the image coordinate system to the camera coordinate point (μ, ν0) in the camera coordinate system is

[0055] where f x is the radial focal length of the camera lens, f y is the tangential focal length of the camera lens, (x c , y c , z c ) is the camera coordinate point of the center of the laser stripe in the camera coordinate system obtained by camera shooting (i.e., the coordinate point of the center of the laser stripe mapped in the camera coordinate system), and z c is the height coordinate of the camera optical center;

[0056] The formula Calculate and obtain the central coordinate points (x c , y c , z c ) of the laser stripe, where A, B, and C are the plane normal vectors of the laser plane equation z c = Ax c + By c + C in the camera coordinate system;

[0057] Use the formula to obtain the calibration mapping relationship between the camera coordinate system and the robot coordinate system (perform hand-eye calibration), where H T is the hand-eye transformation matrix, H B is the transformation matrix of the robot coordinate system, (x B , y B , z B , 1) is the homogeneous coordinate of the laser stripe in the robot coordinate system, (x C , y C , z C , 1) is the homogeneous coordinate of the laser stripe in the camera coordinate system, R 3×3 is the rotation angle of the x, y, and z axes when the world coordinate is transformed to the camera coordinate, and t 3×1 is the distance that needs to be translated for the x, y, and z axes when the world coordinate is transformed to the camera coordinate.

[0058] Understandably, in a specific embodiment of the present invention, first, convert the pixel coordinate points of the laser stripe image captured by the front camera into coordinate points in the camera coordinate system; second, obtain the laser plane equation emitted by the laser in the camera coordinate system; finally, use the optical vision model to solve the coordinates. The specific principle includes: obtaining the laser plane equation in the camera coordinate system through a calibration board. The captured laser stripe image contains the stripe features of the laser, so the weld coordinates need to satisfy both the plane equation and the condition of being captured by the camera optical center; in robot vision, the core task of hand-eye calibration (calibration mapping relationship) is to convert the coordinate points obtained in the camera coordinate system into coordinate points in the robot base coordinate system. The hand-eye transformation matrix can be solved by quaternion or Rodriguez rotation formula, and the transformation matrix of the robot base coordinate is solved by the basic formula of robot kinematics.

[0059] Further, step S20 specifically includes: using the U-NET network model to perform strong light image segmentation and spark image segmentation on the laser stripe image to obtain a denoised stripe image; determining the region of interest (ROI) in the denoised stripe image through the gray mean processing method; using the gray centroid algorithm to extract the center point of the stripe image in the region of interest.

[0060] In a specific embodiment of the present invention, a pre-segmentation network model is established for image semantic segmentation, and the pre-segmentation network model is trained for glare image segmentation and fireworks image segmentation using training data to obtain a U-NET network model; the U-NET network model only needs to have the ability to segment laser stripes, avoiding the problem of insufficient generalization ability of complex models.

[0061] Further, the formula is used to determine the region of interest of the image, where ROWc and COLc are the row and column coordinate sizes of the points obtained according to the gray center-of-gravity value in the image coordinate system.

[0062] Further, the formula G th =(G max +G mean ) / 2 is used to determine the gray threshold, where G max is the maximum gray value of the image within the region of interest of the image, G mean is the average gray value of the image within the region of interest of the image, and G th is the gray threshold;

[0063] The formula is used to screen the pixel points g(u i , v i ) within the search region of the i-th column in the image coordinate system, where g(u i , v i ) is the candidate point of the center point, and m is the pixel value at the point corresponding to g(u i , v i );

[0064] The formula is used to obtain the coordinate g c (i) of the center point of the i-th column stripe.

[0065] Specifically, the true coordinates of the weld are included in the stripe features. The extraction of weld feature values mainly includes: pre-determining the region of interest ROI (Region of Interest, ROI) in the image through the average gray value, obtaining the center point of the stripe image using the gray center extraction algorithm, and fitting the center point p s of the stripe image. The coordinates of the center point p s of the stripe image are extracted, and the slope K of the fitted straight line is judged by a threshold to identify whether there is a need for skip welding.

[0066] In actual welding operations, strong noise interferences such as sparks and arc lights are usually accompanied. In the denoised fringe image after denoising processing, most of the strong noise has been effectively filtered. I(x,y) represents the denoised fringe image after denoising. Since the gray value of the laser fringe in the denoised fringe image is relatively large, the position of the laser fringe region of interest (image region of interest Laserroi) can be estimated by the gray centroid in the row direction and the column direction. Within the obtained Laserroi, based on the gray threshold G th Obtain the center line of the stripe by the gray centroid method.

[0067] Specifically, in step S30, after obtaining the center point of the stripe image, the straight-line fitting algorithm based on RANSAC is used to remove the center point noise to obtain the first fitting straight-line segment and the second fitting straight-line segment. First, randomly select two different points (x1,y1) and (x2,y2) from the extracted center point set (center point of the stripe image) to form a straight line. Subsequently, check whether the coordinate values of other center points satisfy the distance threshold condition (d i (u,v)<d th ). Save the points that meet the conditions as candidate points for estimating the center line equation, and record the number of points that meet the conditions. Finally, repeat the first step N times, and take the group of candidate points with the largest number for fitting the straight-line equation. Among them, the formula is used to determine the center point of the stripe image. In the formula, d i (u,v) is the distance from the candidate point to be determined as the center to the fitting straight line, (u i ,v i ) is the point to be determined as the center. For the finally determined candidate points, the least squares method is used for straight-line fitting: In the formula, are the slope and intercept of the fitting straight-line equation. (x i ,y i )(i = 1,2,…n) are the candidate points. is the average value. n is the number of candidate points. Two straight lines are successfully fitted and displayed through the object detection algorithm.

[0068] Furthermore, the purpose of the invention is to obtain the first fitting straight-line segment and the second fitting straight-line segment by fitting based on the center point of the stripe image. In order to improve the fitting efficiency and the recognition accuracy of the pre-positioned solder joints, in step S30, the center points of the stripe image are divided into two fitting coordinate point groups by a preset cutting block line. Among them, the preset cutting block line is parallel to the x-axis, and the preset cutting block line is at the horizontal midline position of the image region of interest.

[0069] Furthermore, the laser projects onto the surfaces of the first plate and the second plate at an angle a, and the value range of a is 40° - 50°. Preferably, α is 45 degrees.

[0070] Further, β is 90°.

[0071] Further, obtain the continuous extension length of the welding jump point, and determine the welding arc extinguishing strategy according to the continuous extension length.

[0072] The real-time tracking method for spatially discontinuous welds based on laser vision of the present invention mainly includes the following technical means: designing and building a 2.5D laser vision sensor; pre-shooting a large number of weld images to construct a sufficiently rich data set, and training a U-NET neural network model capable of performing efficient segmentation in real time with this; inputting the captured laser stripe image into the trained neural network model for segmentation and noise reduction; based on the segmentation result, proposing a real-time target tracking algorithm for accurately obtaining the weld center point (the center point of the stripe image) in the region of interest of the image; using the RANSAC algorithm to perform linear fitting on the extracted weld center points; finding the intersection point of the two fitted weld lines as the welding feature point, and performing a threshold judgment on this point; based on the fact that the straight line slope K can reflect the welding characteristics of spatially discontinuous welds, providing a decision basis for obstacle jumping in the welding path.

[0073] The present invention also provides a real-time tracking system for spatially discontinuous welds based on laser vision, including a welding robot, a front laser emitter, and a front camera. The welding robot has a base and a robotic arm. The front laser emitter and the front camera are both arranged on the robotic arm. A processing device is arranged on the welding robot, and the processing device is used to implement the steps of the above-mentioned real-time tracking method for spatially discontinuous welds based on laser vision.

[0074] The above embodiments only represent several implementation manners of the present application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A real-time tracking method for spatially discontinuous weld seams based on laser vision, characterized in that a front laser emitter and a front camera disposed on a welding arm are used to perform weld seam tracking on a spatial three-dimensional structure. The front laser emitter is upstream of the welding arm, and the front camera is upstream of the front laser emitter. The front camera is arranged at an inclination angle of α in the direction towards the front laser emitter. The spatial three-dimensional structure includes a first plate and a second plate arranged at an included angle β, where β is between 0 degrees and 180 degrees. The first plate and the second plate are pre-positioned for welding through pre-positioned solder joints. There is a calibration mapping relationship between the camera coordinate system corresponding to the front camera and the robot coordinate system of the welding arm, and the method includes the following steps: S10, real-time collect laser stripe images through the front camera; S20, perform denoising processing on the laser stripe images to obtain denoised stripe images, extract the region of interest in the denoised stripe images, and use the gray centroid algorithm to extract the center point of the stripe image in the region of interest; S30, perform segmented fitting on the basis of the center point of the stripe image using the RANSAC line fitting algorithm to obtain a first fitted line segment and a second fitted line segment; S40, establish an image coordinate system with the upper left corner endpoint of the denoised stripe image as the origin, the vertical extension direction as the x-axis, and the horizontal extension direction as the y-axis. Based on the image coordinate system, determine whether the current slope K1 of the first fitted line segment is greater than 0, and based on the image coordinate system, determine whether the current slope K2 of the second fitted line segment is less than 0; S50, if the current slope K1 is greater than 0 and the current slope K2 is less than 0, then obtain the image intersection point of the first fitted line segment and the second fitted line segment as the center point of the weld seam feature image; if the current slope K1 is not greater than 0 and the current slope K2 is less than 0, then determine the image intersection point of the first fitted line segment and the second fitted line segment as the pre-positioned image solder joint; S60, based on the calibration mapping relationship, map the image intersection point from the image coordinate system to the robot coordinate system, identify the pre-positioned image solder joint as a welding jump point, and identify the center point of the weld seam feature image as a continuous welding point.

2. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to claim 1, characterized in that the expression for mapping the pixel coordinate point (u, ν) in the image coordinate system to the camera coordinate point (μ, ν0) in the camera coordinate system is Among them, f x is the radial focal length of the camera lens, and f y is the tangential focal length of the camera lens. (x c , y c , z c ) is the center coordinate point of the laser stripe in the camera coordinate system obtained by camera shooting, and z xc is the height coordinate of the camera optical center; Using the formula to calculate and obtain the center coordinate points (x c , y c , z c ) of the laser stripe. Among them, A, B, and C are the plane normal vectors of the laser plane equation z c = Ax c + By c + C in the camera coordinate system; Using the formula to obtain the calibration mapping relationship between the camera coordinate system and the robot coordinate system (for hand-eye calibration), where H T is the hand-eye transformation matrix, H B is the transformation matrix of the robot coordinate system, (x B , y B , z B , 1) is the homogeneous coordinate of the laser stripe in the robot coordinate system, (x C , y C , z C , 1) is the homogeneous coordinate of the laser stripe in the camera coordinate system, R 3×3 is the rotation angle of the x, y, and z axes when the world coordinate is transformed into the camera coordinate, and t 3×1 is the translation distance required for the x, y, and z axes when the world coordinate is transformed into the camera coordinate.

3. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to claim 1, characterized in that Step S20 specifically includes: using a U-NET network model to perform strong light image segmentation and spark image segmentation on the laser stripe images to obtain denoised stripe images; determine the region of interest in the denoised stripe images through a gray mean processing method; use the gray centroid algorithm to extract the center point of the stripe image in the region of interest.

4. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to claim 3, characterized in that Use the formula to determine the region of interest of the image, where ROW c , COL c are the row and column coordinate sizes of the points obtained according to the gray center-of-gravity value in the image coordinate system.

5. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to claim 4, characterized in that Use the formula G th =(G max +G mean ) / 2 to determine the gray threshold. G max is the maximum gray value of the image within the region of interest of the image, G mean is the average gray value of the image within the region of interest of the image, and G th is the gray threshold; Use the formula to screen the pixel points g(u i , v i ) within the search area of the i-th column in the image coordinate system, where g(u i , v i ) is the candidate point of the center point, and m is the pixel value at the point corresponding to g(u i , v i ); Using the formula to obtain the coordinates g of the center point of the i-th column of fringes c (i).

6. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to any one of claims 1 to 5, characterized in that In step S30, the center point of the stripe image is divided into two fitting coordinate point groups by a preset cutting block line, wherein the preset cutting block line is parallel to the x-axis and is located at the horizontal midline position of the image region of interest.

7. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to claim 1, characterized in that The value range of a is 40° to 50°.

8. The real-time tracking method for spatially discontinuous weld seams based on laser vision according to claim 1, characterized in that β is 90°.

9. A real-time tracking system for spatially discontinuous weld seams based on laser vision, characterized in that it includes a welding robot, a front laser emitter and a front camera. The welding robot has a base and a robotic arm. The front laser emitter and the front camera are both arranged on the robotic arm. A processing device is arranged on the welding robot, and the processing device is used to implement the steps of the real-time tracking method for spatially discontinuous weld seams based on laser vision according to any one of claims 1 to 8.

Images