Welding robot system based on visual positioning

Abstract

The invention provides a welding robot system based on visual positioning, which comprises a robot body and a control host, the robot body comprises a multi-degree-of-freedom mechanical arm, the tail end of the mechanical arm is provided with a welding gun, a monocular camera and a tracking laser, and the control host is respectively connected and communicated with the monocular camera and the tracking laser; the tracking laser emits linear laser towards the linear welding seam of the workpiece; circularly executing the following steps until the welding operation is finished; judging whether the position of the workpiece is deviated or not based on the workpiece image; if the position of the workpiece is not deviated, carrying out the welding operation according to the welding path of the previous cycle; and if it is determined that the position of the workpiece deviates, the current position coordinates of the linear weld joint in the robot coordinate system are obtained through the current coordinates of the standard ball in the robot coordinate system and the last cycle coordinates, a welding path is planned again, and welding operation is conducted according to the welding path planned again. The welding seam is accurately positioned based on monocular vision, whether the workpiece deviates or not is monitored in real time, and the welding quality is improved.

Description

Technical Field

[0001] The invention relates to the technical field of welding robots, and in particular to a welding robot system based on visual positioning. Background Art

[0002] Manual welding is difficult to maintain the stability and consistency of welding work for a long time. Moreover, due to the poor working conditions of welding, the number of people willing to engage in manual welding is decreasing, and there is a trend of shortage of skilled technical workers. Welding robots have largely met the requirements of welding automation. Traditional welding robots use fixtures to fix the welding workpiece, and are programmed in advance through the teaching pendant to achieve automatic welding. They are highly dependent on fixtures and have high costs. With the rapid development of visual positioning technology, automated welding technology based on machine vision no longer relies on fixtures and has become a research hotspot.

[0003] Existing visual positioning technologies mainly include binocular vision and monocular vision. Although the positioning technology based on binocular vision has higher accuracy, the image information it processes has too much computational effort, the application object is single, the cost is too high, and it is not easy to integrate into embedded systems. Although the positioning method based on monocular vision has difficulty in obtaining depth information, resulting in low accuracy, it has moderate computational effort, simple structure, flexible movement, easy calibration, and high real-time performance. Therefore, the existing welding robots based on monocular vision have low positioning accuracy and welding quality that needs to be improved. Summary of the invention

[0004] The present invention aims to at least solve the technical problems that the existing welding robots based on monocular vision positioning have low positioning accuracy and welding quality needs to be improved, and provide a welding robot system based on vision positioning.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a welding robot system based on visual positioning, including a robot body and a control host, wherein the robot body includes a multi-degree-of-freedom mechanical arm, a welding gun, a monocular camera and a tracking laser are installed at the end of the mechanical arm, and the control host is connected and communicated with the monocular camera and the tracking laser respectively;

[0006] The tracking laser emits a line laser toward a straight weld of the workpiece, and the line laser is located within the field of view of the monocular camera;

[0007] During welding, the following steps are performed repeatedly until the welding operation is completed:

[0008] The control host reads the workpiece image captured by the monocular camera and determines whether the workpiece position is offset based on the workpiece image:

[0009] If the straight weld in the workpiece image intersects perpendicularly at the midpoint of the line laser, it is considered that the workpiece position is not offset, and the robot arm is controlled to drive the welding gun to perform welding operations on the straight weld according to the welding path of the previous cycle;

[0010] If the straight weld in the workpiece image does not intersect the midpoint of the line laser vertically, it is considered that the workpiece position is offset. The current coordinates of three or more standard balls pre-installed on the workpiece in the robot coordinate system are obtained based on the workpiece image. The current coordinates of the standard balls in the robot coordinate system and the coordinates of the previous cycle are used to obtain the current position coordinates of the straight weld in the robot coordinate system. The welding path is re-planned according to the current position coordinates of the straight weld in the robot coordinate system, and the robotic arm is controlled to drive the welding gun to perform welding operations on the straight weld according to the re-planned welding path.

[0011] Preferably, judging whether the workpiece position is offset based on the workpiece image includes:

[0012] Extracting weld seam boundaries of linear weld seams from workpiece images;

[0013] Pick up multiple point pairs along the length direction of the weld on the weld boundary, where the two points of each point pair are located at the upper and lower edges of the weld boundary respectively and the line connecting the two points of each point pair is parallel to the width direction of the weld;

[0014] Obtain the distance between the upper point in each set of point pairs and the upper end point of the centerline laser of the workpiece image, recorded as the first distance, obtain the distance between the lower point in each set of point pairs and the lower end point of the centerline laser of the workpiece image, recorded as the second distance, and obtain the deviation between the first distance and the second distance;

[0015] The deviation mean of multiple groups of point pairs is calculated. If the deviation mean is less than the preset deviation threshold, it is considered that the straight weld in the workpiece image intersects perpendicularly at the midpoint of the line laser. If the deviation mean is greater than or equal to the preset deviation threshold, it is considered that the straight weld in the workpiece image does not intersect perpendicularly at the midpoint of the line laser.

[0016] Preferably, the three or more standard balls have different colors and / or radii.

[0017] Preferably, three standard balls of different colors are arranged on the workpiece.

[0018] Preferably, the step of acquiring the current coordinates of three or more standard balls pre-installed on the workpiece in the robot coordinate system based on the workpiece image includes:

[0019] Based on the workpiece image, the center coordinates and radius of each standard sphere in the pixel coordinate system are obtained;

[0020] Substitute the center coordinates and radius of each standard sphere in the pixel coordinate system into the preset coordinate system transformation formula to obtain the center coordinates of each standard sphere in the camera coordinate system;

[0021] According to the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system obtained by hand-eye calibration, the center coordinates of each standard ball in the camera coordinate system are transformed to obtain the center coordinates of each standard ball in the robot coordinate system, that is, the current coordinates of each standard ball in the robot coordinate system.

[0022] Preferably, acquiring the center coordinates and radius of each standard sphere in a pixel coordinate system based on the workpiece image includes:

[0023] performing a first morphological processing on the workpiece image;

[0024] Performing a first bilateral filtering process on the image after the first morphological processing to obtain a first filtered image;

[0025] Obtaining a segmented image of each standard ball from the first filtered image based on a threshold segmentation method;

[0026] The segmented image of each standard sphere is subjected to circle recognition to obtain the center coordinates and radius of the standard sphere.

[0027] Preferably, after performing circle recognition on the segmented image of each standard sphere to obtain the center coordinates and radius of the standard sphere, the method further includes:

[0028] The center coordinates of the standard sphere are taken as the initial center coordinates, and the radius of the standard circle is taken as the initial radius;

[0029] Extracting the original image of each standard sphere from the workpiece image using the initial center coordinates and initial radius of each standard sphere;

[0030] Performing the second morphological processing and the second bilateral filtering processing on the original image of each standard ball in sequence to obtain a filtered image of each standard ball;

[0031] Performing grayscale processing on the filtered image of each standard ball to obtain a grayscale image of each standard ball;

[0032] Using an edge operator to extract the edge of the grayscale image of each standard ball to obtain the edge image of each standard ball;

[0033] The edge image of each standard ball is subjected to circle recognition to obtain the final center coordinates and radius of the standard ball.

[0034] Preferably, the process of obtaining the rotation matrix and the translation matrix between the robot coordinate system and the camera coordinate system through hand-eye calibration includes:

[0035] Step 1: Place the standard ball at the end of the robotic arm and use a monocular camera to capture the image of the standard ball;

[0036] Step 2, obtaining the center coordinates and radius of the standard sphere in the pixel coordinate system based on the standard sphere image;

[0037] Step 3, substituting the center coordinates and radius of the standard sphere in the pixel coordinate system into a preset coordinate system transformation formula to obtain the center coordinates of the standard sphere in the camera coordinate system;

[0038] Step 4, read the spatial coordinates of the end of the robotic arm in the robot coordinate system through the teaching pendant of the welding robot and use them as the spatial coordinates of the center point of the standard ball in the robot coordinate system;

[0039] Step 5, the spatial coordinates of the center point of the standard ball in the robot coordinate system and the center coordinates of the standard ball in the camera coordinate system are combined into a set of corresponding point coordinates;

[0040] Step 6, move the end of the robot arm through the teach pendant multiple times, repeat the above steps 1-5 after each movement, obtain multiple sets of corresponding point coordinates, and obtain the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system through the singular value decomposition method based on the multiple sets of corresponding point coordinates.

[0041] Preferably, the method of obtaining the current position coordinates of the straight weld in the robot coordinate system by using the current coordinates of the standard ball in the robot coordinate system and the previous cycle coordinates includes:

[0042] The current coordinates of the multiple standard balls in the robot coordinate system are combined into a first coordinate matrix, and the previous cycle coordinates of the multiple standard balls in the robot coordinate system are combined into a second coordinate matrix;

[0043] Constructing an expression equation for expressing the conversion relationship between the first coordinate matrix and the second coordinate matrix by using the workpiece rotation matrix and the workpiece translation matrix;

[0044] The singular value decomposition method is used to solve the expression equation to obtain the workpiece rotation matrix and the workpiece translation matrix;

[0045] The workpiece rotation matrix and the workpiece translation matrix are used to transform the previous cycle position coordinates of the linear weld in the robot coordinate system to obtain the current position coordinates of the linear weld in the robot coordinate system.

[0046] Preferably, it also includes a wire feeder, a wire feeding reel and a wire guide tube; the wire guide tube is used to transport the welding wire of the wire feeder to the wire feeding reel, and the wire feeding reel is used to provide welding wire to the weld seam of the workpiece.

[0047] The welding robot system provided by the present invention is provided with a tracking laser, which performs real-time tracking of the weld by emitting a line laser. During the welding operation, it is regularly judged whether the workpiece is offset according to whether the straight weld in the workpiece image intersects vertically with the midpoint of the line laser. If it is not offset, the welding operation is performed according to the welding path of the previous cycle. If it is offset, the current coordinates of three or more standard balls pre-installed on the workpiece in the robot coordinate system and the coordinates of the previous cycle are required to obtain the current position coordinates of the weld in the robot coordinate system, and the welding path is re-planned for welding operation. The present invention uses more than three standard balls to realize the three-dimensional spatial coordinate calculation of the workpiece, can realize accurate positioning of the weld based on monocular vision, and monitor in real time whether the workpiece is offset, thereby improving the welding quality, eliminating fixed tooling and fixtures, and the position of the welding robot and the workpiece to be welded is no longer fixed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of equipment connection of a welding robot system in a preferred embodiment of the present invention;

[0049] Figure 2 is a control flow chart of welding operation in a preferred embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the operation of a tracking laser in a preferred embodiment of the present invention;

[0051] Figure 4 is a schematic diagram of determining workpiece offset in a preferred embodiment of the present invention;

[0052] Figure 5 The edge image extracted by Sobel edge operator for a standard ball;

[0053] Figure numerals: 1 robot arm; 2 control host 2; 3 welding power source; 4 water cooler; 5 welding cable; 6 control host and a part of the welding robot connecting cable; 7 control host and another part of the welding robot connecting cable; 8 wire feeder; 9 wire guide tube; 10 wire feed reel; 11 welding gun; 12 tracking laser; 13 welding wire; 14 straight weld; 15 workpiece. DETAILED DESCRIPTION

[0054] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0055] In the description of the present invention, it is necessary to understand that the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0056] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installed", "connected" and "connected" should be understood in a broad sense. For example, it can be a mechanical connection or an electrical connection, or it can be the internal connection between two components. It can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to the specific circumstances.

[0057] The present invention provides a welding robot system based on visual positioning. In a preferred embodiment, referring to Figure 1 As shown, the welding robot system includes a robot body and a control host 2. The robot body includes a multi-degree-of-freedom mechanical arm 1. A welding gun 11, a monocular camera and a tracking laser 12 are installed at the end of the mechanical arm 1. The control host 2 is connected and communicated with the monocular camera and the tracking laser 12 respectively. Figure 1 As shown, the control host 2 is connected and communicated with the electric control terminal, the monocular camera and the tracking laser 12 of the welding robot by using cables (6, 7). Preferably, the robot arm 1 is a six-degree-of-freedom robot arm 1. Figure 3 As shown, the tracking laser 12 emits a line laser toward the straight weld 14 of the workpiece 15, and the line laser is located in the field of view of the monocular camera. The tracking laser 12 emits the line laser vertically toward the workpiece 15, forming a straight line spot on the workpiece 15, and the weld of the workpiece 15 is perpendicular to the midpoint of the straight line spot. When the welding robot system provided by the present invention performs welding operations, refer to Figure 2 As shown, the following steps are executed cyclically until the welding operation is completed:

[0058] Step S1, controlling the host 2 to read the workpiece image captured by the monocular camera, and judging whether the position of the workpiece 15 is offset based on the workpiece image:

[0059] If the straight weld 14 in the workpiece image intersects perpendicularly with the midpoint of the line laser, it is considered that the position of the workpiece 15 is not offset, and step S2 is executed. Step S2: control the robot arm 1 to drive the welding gun 11 to perform welding operation on the straight weld 14 according to the welding path of the previous cycle;

[0060] If the straight weld 14 in the workpiece image does not intersect the midpoint of the line laser perpendicularly, it is considered that the workpiece 15 is offset and the following is executed:

[0061] Step S3, obtaining the current coordinates of three or more standard balls pre-installed on the workpiece 15 in the robot coordinate system based on the workpiece image; the workpiece image includes the line laser spot, the weld of the workpiece 15 and the three or more standard balls.

[0062] Step S4, using the current coordinates of the standard ball in the robot coordinate system and the previous cycle coordinates to obtain the current position coordinates of the linear weld 14 in the robot coordinate system.

[0063] Step S5, replanning the welding path according to the current position coordinates of the straight weld 14 in the robot coordinate system.

[0064] Step S6, controlling the robot arm 1 to drive the welding gun 11 to perform welding operations on the linear weld 14 according to the re-planned welding path.

[0065] In this embodiment, in order to distinguish each standard ball, the colors and / or radii of the three or more standard balls on the workpiece 15 are inconsistent. In order to more accurately distinguish each standard ball, three standard balls of different colors are arranged on the workpiece 15, and the colors of the three standard balls can be blue, green, and red, respectively.

[0066] In this embodiment, the method for judging whether the straight weld 14 intersects perpendicularly at the midpoint of the line laser based on the workpiece image can be: generating a minimum circumscribed rectangular frame of the line laser spot in the workpiece image, drawing the minimum circumscribed rectangular frame parallel to the center line of the length direction of the line laser, recorded as the first center line; generating a minimum circumscribed rectangular frame of the weld (which may include unwelded gaps and welded weld spots) in the workpiece image, drawing the center of the minimum circumscribed rectangular frame parallel to the length direction of the weld, recorded as the second center line; calculating the angle between the second center line and the first center line in the image plane, calculating the ratio of the distance between the intersection of the first center line and the second center line and the two ends of the first center line, if the angle is 90 degrees or close to 90 degrees and the distance ratio is 1 or close to 1, it is considered that the straight weld 14 intersects perpendicularly at the midpoint of the line laser, otherwise it is considered that the straight weld 14 does not intersect perpendicularly at the midpoint of the line laser.

[0067] In this embodiment, the welding path is a set of motion control instructions for the robot arm 1 that controls the host 2 to drive the robot arm 1 to drive the welding gun 11 to move along the spatial position coordinates of the weld.

[0068] In a preferred embodiment, in order to improve the accuracy of determining whether the position of the workpiece 15 is offset, in the above step S1, reference is made to Figure 4 As shown, judging whether the position of the workpiece 15 is offset based on the workpiece image includes:

[0069] Step S11, extracting the weld boundary of the linear weld 14 from the workpiece image. Specifically, the weld pixel threshold range can be set based on experience, and the pixel points in the workpiece image are screened according to the weld pixel threshold range, so as to extract the weld boundary. Figure 4 In the schematic diagram, the weld boundary is equivalent to a rectangle.

[0070] Step S12, picking multiple groups of point pairs along the length direction of the weld on the weld boundary, the two points of each group of point pairs are respectively located at the upper and lower edges of the weld boundary and the line connecting the two points of each group of point pairs is parallel to the width direction of the weld. Figure 4 , the line connecting each pair of points is perpendicular to the long side of the rectangle bounding the weld.

[0071] Step S13, obtaining the distance between the upper point in each point pair and the upper end point of the line laser in the workpiece image, recorded as the first distance d1, obtaining the distance between the lower point in each point pair and the lower end point of the line laser in the workpiece image, recorded as the second distance d2, and obtaining the deviation between the first distance and the second distance;

[0072] Step S14, calculate the deviation mean of multiple groups of point pairs. If the deviation mean is less than the preset deviation threshold, it is considered that the straight weld 14 in the workpiece image intersects vertically at the midpoint of the line laser. If the deviation mean is greater than or equal to the preset deviation threshold, it is considered that the straight weld 14 in the workpiece image does not intersect vertically at the midpoint of the line laser.

[0073] In a preferred embodiment, in order to more accurately obtain the current coordinates of the center of the standard ball in the robot coordinate system, in step S3, the current coordinates of three or more standard balls pre-installed on the workpiece 15 in the robot coordinate system are obtained based on the workpiece image, including:

[0074] Step S31, obtaining the center coordinates and radius of each standard sphere in the pixel coordinate system based on the workpiece image.

[0075] Step S32, substitute the center coordinates and radius of each standard sphere in the pixel coordinate system into the preset coordinate system transformation formula to obtain the center coordinates of each standard sphere in the camera coordinate system. Specifically, the preset coordinate system transformation formula is as follows:

[0076]

[0077] Among them, J x , J y , J z They represent the center coordinates (center coordinates) of a standard sphere in the camera coordinate system (x-axis, y-axis, and z-axis); (U, V) are the center coordinates of the standard sphere in the pixel coordinate system, r is the radius pixel length of the standard sphere in the pixel coordinate system, and U 0 With V 0is the origin of the image coordinate system in the pixel coordinate system, both in pixels, and dx and dy are the pixel sizes, both in mm / pixel. 2 is the actual radius of the standard sphere. f is the focal length of the monocular camera.

[0078] First intermediate quantity:

[0079] Second intermediate quantity:

[0080] Step S33, according to the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system obtained by hand-eye calibration, the center coordinates of each standard ball in the camera coordinate system are transformed to obtain the center coordinates of each standard ball in the robot coordinate system, that is, the current coordinates of each standard ball in the robot coordinate system.

[0081] In this embodiment, in step S31, the center coordinates and radius of each standard sphere in the pixel coordinate system are obtained based on the workpiece image, including:

[0082] Step S311, perform the first morphological processing on the workpiece image. The first morphological processing includes sequentially performing opening and closing operations on the workpiece image, and using the morphologyEx function in the OpenCV library to complete the first morphological processing, which can reduce the noise influence in the process of camera image acquisition, reduce the bright spots and gaps in the image, and restore the edge part as much as possible.

[0083] Step S312, performing a first bilateral filtering process on the image after the first morphological processing to obtain a first filtered image, which can further eliminate the noise mixed in the target image and extract useful features in the image.

[0084] Step S313, based on the threshold segmentation method, the segmented image of each standard ball is obtained from the first filtered image. In one example, the red, green and blue areas of the workpiece image are extracted using the threshold segmentation method. The segmented images of the three standard balls of red, green and blue are obtained. Specifically, a mask layer is set for each standard ball to extract the segmented image, and the mask layer is the HSV color range corresponding to the standard ball. The cvtColor function in OpenCV can also be used to convert the RGB space into the HSV space, and then the appropriate threshold is set by the inRange function to segment a more ideal processing result. The segmented image of each standard ball only contains the binary image of the standard ball and the background, so that only the target area image is retained, which greatly reduces the influence of image noise and improves the stability of the algorithm.

[0085] Step S314, circle recognition is performed on the segmented image of each standard sphere to obtain the center coordinates and radius of the standard sphere. Specifically, the existing Hough circle detection method can be used to perform circle recognition, such as using the HoughCircles() function in OpenCV to perform circle recognition to obtain the center coordinates and radius of the standard sphere.

[0086] In this embodiment, in order to obtain more accurate coordinates of the center and radius of the standard sphere, preferably, after executing step S314, the following is further included:

[0087] Step S315, taking the center coordinates of the standard sphere obtained in step S314 as the initial center coordinates, and taking the radius of the standard circle as the initial radius.

[0088] Step S316, extracting the original image of each standard sphere from the workpiece image using the initial center coordinates and initial radius of the standard sphere;

[0089] Step S317, the original image of each standard ball is sequentially subjected to second morphological processing and second bilateral filtering processing to obtain a filtered image of each standard ball. The second morphological processing includes sequentially performing opening and closing operations on the workpiece image, and the second morphological processing is completed using the morphologyEx function in the OpenCV library.

[0090] Step S318, grayscale processing is performed on the filtered image of each standard ball to obtain a grayscale image of each standard ball. Grayscale processing can be implemented by the cvtColor function in OpenCV.

[0091] Step S319: Use an edge operator to extract the edge of each standard ball grayscale image to obtain an edge image of each standard ball. The edge operator can be Roberts, Prewitt, Canny and Sobel operators, preferably Sobel edge operator. Figure 5 The figure shows the edge image of a standard ball extracted using the Sobel edge operator.

[0092] Step S320, circle recognition is performed on the edge image of each standard ball to obtain the final center coordinates and radius of the standard ball. Specifically, the existing Hough circle detection method can be used to perform circle recognition, such as using the HoughCircles() function in OpenCV to perform circle recognition to obtain the final center coordinates and radius of the standard ball.

[0093] In a preferred embodiment, in step S33, the process of obtaining the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system through hand-eye calibration includes:

[0094] Step 1, place a standard ball at the end of the robot arm 1, and use a monocular camera to capture an image of the standard ball;

[0095] Step 2, obtaining the center coordinates and radius of the standard sphere in the pixel coordinate system based on the standard sphere image;

[0096] Step 3, substituting the center coordinates and radius of the standard sphere in the pixel coordinate system into a preset coordinate system transformation formula to obtain the center coordinates of the standard sphere in the camera coordinate system;

[0097] Step 4, read the spatial coordinates of the end of the robot arm 1 in the robot coordinate system through the teaching pendant of the welding robot and use them as the spatial coordinates of the center point of the standard ball in the robot coordinate system;

[0098] Step 5, the spatial coordinates of the center point of the standard ball in the robot coordinate system and the center coordinates of the standard ball in the camera coordinate system are combined into a set of corresponding point coordinates;

[0099] Step 6: Move the end of the robot arm 1 through the teaching pendant for multiple times, and repeat the above steps 1-5 after each movement to obtain multiple sets of corresponding point coordinates. According to the multiple sets of corresponding point coordinates, the rotation matrix R between the robot coordinate system and the camera coordinate system is obtained by the singular value decomposition method. B and the translation matrix T B .

[0100] In this embodiment, the rotation matrix R B and the translation matrix T B The relationship matrix B between the robot coordinate system and the camera coordinate system is composed of:

[0101] In this embodiment, in step 6, the rotation matrix R between the robot coordinate system and the camera coordinate system is obtained by singular value decomposition method according to the coordinates of multiple sets of corresponding points. B and the translation matrix T B The process includes:

[0102] Assume that the homogeneous coordinates of the standard ball in the robot coordinate system are (X D ,Y D ,Z D ), the homogeneous coordinates in the camera coordinate system are (X C ,Y C ,Z C ), then we can get formula 1.

[0103]

[0104] Where r ij (i=1,2,3;j=1,2,3) is the rotation matrix R B The constituent elements of i(i=1,2,3) is the translation vector T B If the matrix composed of the coordinates of the target point in the camera coordinate system is set as matrix C, and the matrix composed of the coordinates in the robot coordinate system is set as D, the above formula can be expressed as formula 2.

[0105] CX=D (2)

[0106] It can also be expressed by formula 3:

[0107] D=R*C+t (3)

[0108] Divide multiple groups of corresponding points into two groups of coordinates, one group is the spatial coordinates of the center of the standard sphere in the machine coordinate system and the robot coordinate system, recorded as the target point set B; the other group is the spatial coordinates of the center of the standard sphere in the camera coordinate system, recorded as the source point set A; solve the rotation matrix R between the two groups of coordinates by singular value decomposition method B and the translation matrix T B , usually requires the following three steps:

[0109] (1) Calculate the center point

[0110] That is to find the centroid positions of the two groups of points respectively, and the solution formula is shown in Formula 4.

[0111]

[0112] P i A is the coordinate of the i-th point in the source point set A, P i B is the coordinate of the i-th point in the target point set B, N is the number of points in the point set, P is the three-dimensional coordinate point set, μ A With μ B are the center point coordinates of the source point set and the target point set respectively.

[0113] (2) Move the point set to the origin and calculate the optimal rotation matrix R

[0114] After eliminating the influence of the t matrix in advance, the result of solving the rotation matrix is ​​more accurate. Therefore, consider centralizing the points in each point set, and subtracting the coordinate values ​​of each point set from the center point of its point set to generate new point sets A' and B', and finally solve the covariance matrix between the two point sets.

[0115] The formula for solving A' and B' is shown in Equation 5.

[0116]

[0117] Where A' i With B i' is the i-th centralized point of the source point set and the target point set.

[0118] The covariance matrix H between the point sets is calculated using Formula 4-13, and the solution process is shown in Formula 6.

[0119]

[0120] By using the singular value decomposition method to obtain the matrices U, S, and V, we can calculate the rotation matrix R between the point sets, that is, the rotation matrix R B , as shown in Formula 7:

[0121]

[0122] (3) Calculate the translation matrix t

[0123] After obtaining the rotation matrix R, the translation matrix t can be obtained according to the formula shown in formula 8, that is, the translation matrix T B :

[0124] t=-R×μ A +μ B (8)

[0125] In a preferred embodiment, in step S4, the current position coordinates of the linear weld 14 in the robot coordinate system are obtained using the current coordinates of the standard ball in the robot coordinate system and the previous cycle coordinates, including:

[0126] Step S41, the current coordinates of the multiple standard balls in the robot coordinate system are combined into a first coordinate matrix, and the previous cycle coordinates of the multiple standard balls in the robot coordinate system are combined into a second coordinate matrix.

[0127] In one example, the initial coordinates of the weld are read by the teaching pendant at the initial position of the workpiece 15, and are set as P(X, Y, Z). Three red, green, and blue standard balls are set on the workpiece 15. Assume that at the initial time (before the position of the workpiece 15 is offset), the workpiece image is captured by the camera, and the center coordinates of the red standard ball, green standard ball, and blue standard ball in the robot coordinate system are obtained according to steps S3 and S4, and are set as Rf(X rf , Y rf , Z rf )、Gf(X gf , Y gf , Z gf )、Bf(X bf , Y bf , Z bf ), through Rf(X rf , Y rf , Z rf )、Gf(X gf , Y gf , Z gf)、Bf(X bf , Y bf , Z bf ) to form the first coordinate matrix E. Assume that after the position of the workpiece 15 is offset, the camera is used to collect the image of the workpiece 15 after it is moved, and the three-dimensional space coordinates of the three primary color standard ball of the workpiece 15 after the offset in the robot coordinate system are calculated according to steps S3 and S4, which are recorded as Ra(X ra , Y ra , Z ra )、Gf(X ga , Y ga , Z ga )、Bf(X ba , Y ba , Z ba ), through Ra(X ra , Y ra , Z ra )、Gf(X ga , Y ga , Z ga )、Bf(X ba , Y ba , Z ba ) form the second coordinate matrix F.

[0128] Step S42, constructing the workpiece rotation matrix R ef and the workpiece translation matrix T ef An expression equation expressing the conversion relationship between the first coordinate matrix and the second coordinate matrix.

[0129] In the above example, the expression equation is:

[0130] F=R ef *E+T ef (9)

[0131] Get the rotation matrix R ef and the translation matrix T ef Afterwards, the weld seam coordinates P(X, Y, Z) before the workpiece 15 is offset are substituted into Formula 4-19 to calculate the weld seam coordinates P'(X', Y', Z') in the robot coordinate system after the workpiece 15 is offset:

[0132] P'=R ef *P+T ef (10)

[0133] Step S43, using a singular value decomposition method to solve the expression equation to obtain a workpiece rotation matrix and a workpiece translation matrix;

[0134] Step S44, using the workpiece rotation matrix and the workpiece translation matrix to transform the previous cycle position coordinates of the linear weld 14 in the robot coordinate system, to obtain the current position coordinates of the linear weld 14 in the robot coordinate system.

[0135] In a preferred embodiment, referring to Figure 1 As shown, the welding robot system also includes a wire feeder 8, a wire feeding reel 10 and a wire guide tube 9; the wire guide tube 9 is used to transport the welding wire 13 of the wire feeder 8 to the wire feeding reel 10, and the wire feeding reel 10 is used to provide the welding wire 13 to the weld seam of the workpiece 15.

[0136] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0137] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A welding robot system based on visual positioning, characterized in that: It includes a robot body and a control host, wherein the robot body includes a multi-degree-of-freedom mechanical arm, a welding gun, a monocular camera and a tracking laser are installed at the end of the mechanical arm, and the control host is connected and communicated with the monocular camera and the tracking laser respectively; The tracking laser emits a line laser toward a straight weld of the workpiece, and the line laser is located within the field of view of the monocular camera; During welding, the following steps are performed repeatedly until the welding operation is completed: The control host reads the workpiece image captured by the monocular camera and determines whether the workpiece position is offset based on the workpiece image: If the straight weld in the workpiece image intersects perpendicularly at the midpoint of the line laser, it is considered that the workpiece position is not offset, and the robot arm is controlled to drive the welding gun to perform welding operations on the straight weld according to the welding path of the previous cycle; If the straight weld in the workpiece image does not intersect the midpoint of the line laser vertically, it is considered that the workpiece position is offset. The current coordinates of three or more standard balls pre-installed on the workpiece in the robot coordinate system are obtained based on the workpiece image. The current coordinates of the standard balls in the robot coordinate system and the coordinates of the previous cycle are used to obtain the current position coordinates of the straight weld in the robot coordinate system. The welding path is re-planned according to the current position coordinates of the straight weld in the robot coordinate system, and the robotic arm is controlled to drive the welding gun to perform welding operations on the straight weld according to the re-planned welding path.

2. A welding robot system based on visual positioning as claimed in claim 1, characterized in that: The step of judging whether the position of the workpiece is offset based on the workpiece image includes: Extracting weld seam boundaries of linear weld seams from workpiece images; Pick up multiple point pairs along the length direction of the weld on the weld boundary, where the two points of each point pair are located at the upper and lower edges of the weld boundary respectively and the line connecting the two points of each point pair is parallel to the width direction of the weld; Obtain the distance between the upper point in each set of point pairs and the upper end point of the centerline laser of the workpiece image, recorded as the first distance, obtain the distance between the lower point in each set of point pairs and the lower end point of the centerline laser of the workpiece image, recorded as the second distance, and obtain the deviation between the first distance and the second distance; The deviation mean of multiple groups of point pairs is calculated. If the deviation mean is less than the preset deviation threshold, it is considered that the straight weld in the workpiece image intersects perpendicularly at the midpoint of the line laser. If the deviation mean is greater than or equal to the preset deviation threshold, it is considered that the straight weld in the workpiece image does not intersect perpendicularly at the midpoint of the line laser.

3. A welding robot system based on visual positioning as described in claim 1 or 2, characterized in that: The three or more standard balls have different colors and / or radii.

4. A welding robot system based on visual positioning as described in claim 3, characterized in that: Three standard balls of different colors are arranged on the workpiece.

5. A welding robot system based on visual positioning as claimed in claim 4, characterized in that: The method of acquiring the current coordinates of three or more standard balls pre-installed on the workpiece in the robot coordinate system based on the workpiece image includes: Based on the workpiece image, the center coordinates and radius of each standard sphere in the pixel coordinate system are obtained; Substitute the center coordinates and radius of each standard sphere in the pixel coordinate system into the preset coordinate system transformation formula to obtain the center coordinates of each standard sphere in the camera coordinate system; According to the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system obtained by hand-eye calibration, the center coordinates of each standard ball in the camera coordinate system are transformed to obtain the center coordinates of each standard ball in the robot coordinate system, that is, the current coordinates of each standard ball in the robot coordinate system.

6. A welding robot system based on visual positioning as claimed in claim 5, characterized in that: The step of obtaining the center coordinates and radius of each standard sphere in a pixel coordinate system based on the workpiece image includes: performing a first morphological processing on the workpiece image; Performing a first bilateral filtering process on the image after the first morphological processing to obtain a first filtered image; Obtaining a segmented image of each standard ball from the first filtered image based on a threshold segmentation method; The segmented image of each standard sphere is subjected to circle recognition to obtain the center coordinates and radius of the standard sphere.

7. A welding robot system based on visual positioning as claimed in claim 6, characterized in that: After performing circle recognition on the segmented image of each standard sphere to obtain the center coordinates and radius of the standard sphere, the method further includes: The center coordinates of the standard sphere are taken as the initial center coordinates, and the radius of the standard circle is taken as the initial radius; Extracting the original image of each standard sphere from the workpiece image using the initial center coordinates and initial radius of each standard sphere; Performing the second morphological processing and the second bilateral filtering processing on the original image of each standard ball in sequence to obtain a filtered image of each standard ball; Performing grayscale processing on the filtered image of each standard ball to obtain a grayscale image of each standard ball; Using an edge operator to extract the edge of the grayscale image of each standard ball to obtain the edge image of each standard ball; The edge image of each standard ball is subjected to circle recognition to obtain the final center coordinates and radius of the standard ball.

8. A welding robot system based on visual positioning as described in claim 5, 6 or 7, characterized in that: The process of hand-eye calibration to obtain the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system includes: Step 1: Place the standard ball at the end of the robotic arm and use a monocular camera to capture the image of the standard ball; Step 2, obtaining the center coordinates and radius of the standard sphere in the pixel coordinate system based on the standard sphere image; Step 3, substituting the center coordinates and radius of the standard sphere in the pixel coordinate system into a preset coordinate system transformation formula to obtain the center coordinates of the standard sphere in the camera coordinate system; Step 4, read the spatial coordinates of the end of the robotic arm in the robot coordinate system through the teaching pendant of the welding robot and use them as the spatial coordinates of the center point of the standard ball in the robot coordinate system; Step 5, the spatial coordinates of the center point of the standard ball in the robot coordinate system and the center coordinates of the standard ball in the camera coordinate system are combined into a set of corresponding point coordinates; Step 6, move the end of the robot arm through the teach pendant multiple times, repeat the above steps 1-5 after each movement, obtain multiple sets of corresponding point coordinates, and obtain the rotation matrix and translation matrix between the robot coordinate system and the camera coordinate system through the singular value decomposition method based on the multiple sets of corresponding point coordinates.

9. A welding robot system based on visual positioning as described in claim 1 or 2 or 4 or 5 or 6 or 7, characterized in that: The method of obtaining the current position coordinates of the straight weld in the robot coordinate system by using the current coordinates of the standard ball in the robot coordinate system and the previous cycle coordinates includes: The current coordinates of the multiple standard balls in the robot coordinate system are combined into a first coordinate matrix, and the previous cycle coordinates of the multiple standard balls in the robot coordinate system are combined into a second coordinate matrix; Constructing an expression equation for expressing the conversion relationship between the first coordinate matrix and the second coordinate matrix by using the workpiece rotation matrix and the workpiece translation matrix; The singular value decomposition method is used to solve the expression equation to obtain the workpiece rotation matrix and the workpiece translation matrix; The workpiece rotation matrix and the workpiece translation matrix are used to transform the previous cycle position coordinates of the linear weld in the robot coordinate system to obtain the current position coordinates of the linear weld in the robot coordinate system.

10. A welding robot system based on vision positioning as described in claim 1 or 2 or 4 or 5 or 6 or 7, characterized in that: It also includes wire feeder, wire feed reel and wire guide tube; The wire guide tube is used to transport the welding wire from the wire feeder to the wire feed reel, and the wire feed reel is used to provide welding wire to the weld seam of the workpiece.

Images