An intelligent welding method for a humanoid welding robot

Abstract

The present application relates to the technical field of automatic welding, and specifically provides a humanoid welding robot intelligent welding method, which comprises the following steps: recording welding workbench area image data, running a three-dimensional measurement algorithm, determining the to-be-welded area position and contour three-dimensional point set of a to-be-welded workpiece; controlling the humanoid welding robot to move to the welding workbench along a set walking route; determining process parameters according to the material of the to-be-welded workpiece based on a welding process parameter library; planning the motion trajectory of the two-side welding mechanical arms of the humanoid welding robot respectively according to the to-be-welded area position, the contour three-dimensional point set and the position of the humanoid welding robot; and controlling the two-side welding mechanical arms of the humanoid welding robot to move along the planned trajectory to perform welding work. The method can realize high-precision identification and three-dimensional measurement of the to-be-welded area, can significantly improve the automation and intelligence level of the welding system, can greatly improve the intelligent level of welding, can reduce the time cost of welding, and thus can improve the welding output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic welding, in particular to an intelligent welding method of a humanoid welding robot. BACKGROUND

[0002] The current mainstream welding robot lacks flexibility, and the welding path and welding parameters must be set in advance according to the actual working conditions, which has obvious shortcomings in work and cannot adapt to the characteristics of novel and unique building steel structure modeling and various forms of welded joints, so it is urgent to develop a new type of intelligent welding robot system with self-monitoring and self-trajectory planning functions. SUMMARY

[0003] In order to overcome the shortcomings and deficiencies in the prior art, the purpose of the present application is to provide an intelligent welding method of a humanoid welding robot; the method can realize high-precision identification and three-dimensional measurement of the welding area, significantly improve the automation and intelligence of the robot welding system, greatly improve the intelligent degree of welding, reduce the time cost of welding, and thus improve the welding output.

[0004] In order to achieve the above purpose, the present application realizes the technical scheme as follows: an intelligent welding method of a humanoid welding robot, realized by a robot welding system; the robot welding system comprises a humanoid welding robot and a welding workbench for fixing a workpiece to be welded; the humanoid welding robot is equipped with a robot binocular camera and two side welding mechanical arms; the distal end of each of the two side welding mechanical arms is provided with a welding gun;

[0005] The intelligent welding method of the humanoid welding robot comprises the following steps:

[0006] S1, using an RGB-D camera fixed above the welding workbench to record image data of the welding workbench area, running a three-dimensional measurement algorithm to determine the position and contour three-dimensional point set P=(p1 p2 p3…p n-1 p n ) of the welding area of the workpiece to be welded;

[0007] S2, controlling the humanoid welding robot to move along the set walking route to the welding workbench;

[0008] S3, determining the process parameters according to the material of the workpiece to be welded based on a welding process parameter library;

[0009] S4, planning the movement trajectories of the two side welding mechanical arms respectively according to the position of the welding area, the contour three-dimensional point set P and the position of the humanoid welding robot;

[0010] S5, controlling the two side welding mechanical arms to move along the planned trajectories to perform welding work;

[0011] S6, the welding work is finished, and the two welding mechanical arms are controlled to return to the original positions.

[0012] Preferably, in the step S1, the recording of the image data of the welding workbench area by the RGB-D camera fixed above the welding workbench refers to recording the corrected RGB-D camera image data for three-dimensional measurement of the to-be-welded area, including a two-dimensional RGB image and a depth image.

[0013] The three-dimensional measurement algorithm refers to identifying and high-precision three-dimensionally measuring the to-be-welded area by the image data recorded by the RGB-D camera, including to-be-welded area identification based on a two-dimensional RGB image, three-dimensional point cloud generation based on a two-dimensional RGB image and a depth image, and to-be-welded area contour output based on the three-dimensional point cloud.

[0014] The to-be-welded area identification based on a two-dimensional RGB image refers to the following steps: performing grayscale processing on the RGB color image to extract black and gray pixel points; performing binaryzation processing on the grayscale-processed image to separate the to-be-welded area by using the binaryzation processing based on the color difference between the to-be-welded area and the welding workbench; performing a region growing algorithm to select a starting pixel point from the binaryzation-processed image as a seed of a region, then expand the pixel points around the seed until a preset similarity value or a boundary is reached; and performing processing on the expansion result to obtain the to-be-welded area contour, optimize the to-be-welded area contour by morphological operation to eliminate noise points and holes, and save the size and position information of the to-be-welded area on the two-dimensional RGB image.

[0015] The three-dimensional point cloud generation based on a two-dimensional RGB image and a depth image refers to obtaining the two-dimensional RGB image and the depth image of the RGB-D camera respectively, calibrating the two-dimensional RGB image and the depth image to be in the same coordinate system, reading the depth value of the to-be-welded area contour pixel value in the depth image by using the size and position information of the to-be-welded area on the two-dimensional RGB image obtained by the to-be-welded area identification based on a two-dimensional RGB image, and generating a three-dimensional point set with color information by using the position information and the depth value and relying on the internal parameter information of the RGB-D camera.

[0016]

[0017] wherein, [X Y Z] is the position coordinate of the three-dimensional point, s d is a depth factor, d is a depth, [u v] is a pixel point in the two-dimensional RGB image, f x ,f yThese represent the focal lengths of the pixel along the X and Y axes, respectively, and (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system. The complete information of the three-dimensional point is [XYZRGB], where [RGB] represents the color information of the red, green, and blue channels of the three-dimensional point.

[0018] Outputting the contour of the area to be soldered based on 3D point cloud refers to outputting a 3D point set of the contour of the area to be soldered, based on the size and position information of the area to be soldered on a 2D RGB image and the 3D point cloud.

[0019] P = (p1 p2 p3…p) n-1 p n ), p i =[X i Y i Z i R i G i B i ];

[0020] The output 3D point set of the outline of the area to be soldered is located in the RGB-D camera coordinate system.

[0021] Preferably, in step S2, the humanoid welding robot moving along a set walking route to the welding workbench means planning the walking route of the humanoid welding robot in a factory environment and laying it out on the ground with yellow lines; the robot moves to the welding workbench by recognizing the yellow lines through a binocular camera. The binocular camera recognizing the yellow lines includes the following steps: binarizing the binocular camera from a bird's-eye view; converting the image BGR color channel to an HSV color channel and setting upper and lower limits for yellow; setting the region of interest (ROI) and detecting the yellow line pixels using a sliding window method; fitting a curve and outputting the position of the yellow line.

[0022] Connecting to the welding power source, wire feeding system, and gas cylinder via cables means that after the humanoid welding robot moves to the welding worktable, the welding power source, wire feeding system, and gas cylinder are connected to the humanoid welding robot using cables.

[0023] Preferably, in step S3, the process parameters include welding current, wire feed speed, gas intake speed, welding speed, and welding angle; the welding process parameter library refers to establishing the correspondence between the workpiece material to be welded and the welding process parameters through preliminary process experiments, changing the welding current, wire feed speed, gas intake speed, welding speed, and welding angle respectively by controlling the variable method, and determining the optimal process parameters through welding quality and performance, thereby establishing the mapping relationship between the workpiece material to be welded and the optimal process parameters.

[0024] Preferably, in step S4, the three-dimensional point set P of the location and contour of the area to be welded refers to the three-dimensional point set P of the location and contour of the area to be welded output in step S1.

[0025] The human-shaped welding robot position is obtained by an RGB-D camera fixed above the welding workbench; comprising the following steps: controlling the left and right welding robot arms to move into the field of view of the RGB-D camera; performing grayscale processing on the two-dimensional RGB image collected by the RGB-D camera, and extracting black and gray pixel points; setting a region of interest (ROI); performing binaryzation processing on the grayscale-processed image, separating the welding torches of the left and right welding robot arms by using binaryzation processing due to the obvious color difference between the welding torches and the welding workbench; performing a region growing algorithm to expand the pixel points around the welding torch until a preset similarity value or a boundary is reached; optimizing the welding torch contour by morphological operation to eliminate noise points and holes, and saving the size and position information of the welding torches of the left and right welding robot arms on the two-dimensional RGB image; converting the position information of the welding torch of the left welding robot arm into a three-dimensional point T l = [X l Y l Z l ], and converting the position information of the welding torch of the right welding robot arm into a three-dimensional point T r = [X r Y r Z r ];

[0026] The separate planning of the motion trajectories of the two welding robot arms refers to decomposing the three-dimensional point set P = (p1p2 p3…p n-1 p n ) of the contour of the to-be-welded region into two segments of uniform and continuous three-dimensional point sets:

[0027] P l = (p1 p2 p3…p m-1 p m ), and P r = (p m+1 p m+2 p m+3 …p n-1 p n );

[0028] The path of the left welding robot arm is set as follows: taking T l as the starting position, p1 of the three-dimensional point set P l as the end point, p1 as the optimal welding torch angle, and passing through the three-dimensional point set P l = (p1 p2 p3…p m-1 p m ); and the path of the right welding robot arm is set as follows: taking T r as the starting position, p m+1 of the three-dimensional point set P r as the end point, p m+1 as the optimal welding torch angle, and passing through the three-dimensional point set Pr = (p m+1 p m+2 p m+3 …p n-1 p n ) of the path; wherein the path planning method of the three-dimensional point set P l and P r is that the two welding mechanical arms are enveloped by cylinders respectively, and the obstacles in the space are enveloped by cuboids; the obstacle envelope is mapped into the obstacle avoidance space of the left welding mechanical arm; the obstacle envelope and the left welding mechanical arm envelope are mapped into the obstacle avoidance space of the right welding mechanical arm;

[0029] The starting position is taken as the root node of the random tree, and the first three-dimensional point of the three-dimensional point set is taken as the termination point to generate the random tree; the generation method of the random tree is that a new node is generated in the space by random sampling; if the new node does not collide with the obstacle avoidance space, the new node is added to the random tree as a node of the random tree; the random tree is expanded in the space, the corresponding parent node of each node is recorded, and the cost from the starting point to each node is calculated until the termination point and the optimal welding torch angle are found; in the random sampling process, the nodes in the random tree are rechecked and reconnected to find the minimum cost path;

[0030] Then, the three-dimensional point of the current three-dimensional point set is taken as the starting point, and the next three-dimensional point of the three-dimensional point set is taken as the termination point to generate the random tree; the execution is repeated until the last three-dimensional point of the three-dimensional point set is reached; the planned path is obtained by tracing back from the termination point to the starting position.

[0031] Preferably, in the step S5, the control of the movement of the two-sided welding mechanical arms along the planned trajectory means that the two-sided welding mechanical arms are controlled to move along the movement trajectory planned in the step S4 at the same time, so as to complete the welding work.

[0032] Preferably, in the step S6, the control of the two-sided welding mechanical arms to return to the original point means that after the end of the step S5, the left welding mechanical arm is located at the termination point p l of the three-dimensional point set P m , the right welding mechanical arm is located at the termination point p r of the three-dimensional point set P n , and the two-sided welding mechanical arms are controlled to move along straight lines in the three-dimensional space to the initial positions T l ,T r .

[0033] Preferably, after the step S6, there is further a step S7 of determining the welding quality by using an RGB-D camera fixed above the welding workbench.

[0034] Preferably, the step S7 comprises the following sub-steps: taking a post-welding RGB image using an RGB-D camera; establishing a two-dimensional contour band ROI region on the RGB image according to the post-welding region contour three-dimensional point set P=(p1 p2 p3…p n-1 p n ), using a region growing algorithm to take the post-welding region as a segmentation target, and by similarity comparison with surrounding pixel points, similar pixel points in the RGB image are aggregated into a region, so as to realize segmentation of the post-welding region; saving the size and position information of the post-welding region contour on the two-dimensional RGB image; converting the post-welding region contour into a three-dimensional point cloud, and subtracting the contour three-dimensional point cloud set in step S1 to output a three-dimensional measurement result of the weld; judging whether there is a welding quality problem for the three-dimensional measurement result of the weld.

[0035] Preferably, the robot welding system further comprises a guarantee device; the guarantee device comprises a wire feeding system, a welding power supply and a gas cylinder; and the humanoid welding robot is connected with the guarantee device.

[0036] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0037] 1. The method and device proposed by the present application can effectively replace welders to complete welding work, greatly improve the intelligent degree of welding, save labor costs, and solve the industry problem of welder shortage;

[0038] 2. The workbench RGB-D camera image high-precision three-dimensional measurement method proposed by the present application can realize high-precision identification and three-dimensional measurement of the to-be-welded region. This method first generates a to-be-welded region contour under a two-dimensional RGB image by using a series of image processing methods, and then generates a to-be-welded region three-dimensional contour point cloud by using a corresponding depth image. This scheme has low cost, high precision and fast running speed, and can also be used for post-welding workpiece quality monitoring, which can significantly improve the automation and intelligence of the robot welding system;

[0039] 3. The left-right dual-welding robot arm welding method and trajectory planning algorithm proposed by the present application take the initial positions of the two welding robot arms as starting points, use the to-be-welded region three-dimensional contour point cloud as a target point, and plan a dual-arm collaborative welding path by taking obstacle avoidance, optimal welding angle and welding speed as constraints, which can improve the welding work efficiency of the humanoid welding robot by more than 54%, reduce the time cost of welding, and thus improve the welding output. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a flowchart of the intelligent welding method of the humanoid welding robot of the present application;

[0041] Figure 2 is a structural schematic diagram of the robot welding system of the present application;

[0042] Figure 3 This is a schematic diagram of the communication system of the robot welding system of the present invention;

[0043] In this diagram, 1 represents the wire feeding system; 2 represents the welding power source; 3 represents the gas cylinder; 4 represents the humanoid welding robot; 5 represents the robot's binocular camera; 6 represents the wireless network receiving module; 7 represents the left welding robotic arm; 8 represents the right welding robotic arm; 9 represents the welding torch; 10 represents the computing platform; 11 represents the welding worktable; 12 represents the workpiece to be welded; and 13 represents the RGB-D camera. Detailed Implementation

[0044] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0045] Example

[0046] This embodiment describes an intelligent welding method using a humanoid welding robot, such as... Figure 1 As shown, this is achieved through a robotic welding system. The robotic welding system includes a support device, a welding worktable 11, a humanoid welding robot 4, an RGB-D camera 13, and a computing platform 10. The welding worktable 11 is used to fix the workpiece 12 to be welded. The support device includes a wire feeding system 1, a welding power source 2, and a gas cylinder 3. The support device is used to optimize the welding current, wire feeding speed, and gas intake speed. The RGB-D camera 13 is used to acquire images of the area to be welded, the welding torches of the welding robotic arms on both sides, and the post-weld area.

[0047] The humanoid welding robot 4 is equipped with a robot binocular camera 5, a wireless network receiving module 6, a left welding robotic arm 7, and a right welding robotic arm 8. Each welding robotic arm on both sides has a welding torch 9 at its end. The humanoid welding robot 4 is connected to the support equipment via a cable. The binocular images from the robot binocular camera 5, the motion data of the welding robotic arms on both sides, and the communication data with the support equipment are wirelessly transmitted to the computing platform 10. The computing platform 10 is used for complex calculations such as three-dimensional measurement of the area to be welded, initial position detection of the welding torches of the welding robotic arms on both sides, quality inspection of the welded area, route recognition and visual obstacle avoidance, trajectory planning of the welding robotic arms on both sides, and control of the optimal welding process parameters. It is connected to the RGB-D camera and the humanoid welding robot via a wireless network.

[0048] The intelligent welding method of the humanoid welding robot includes the following steps:

[0049] S1. Use an RGB-D camera fixed above the welding workbench to record image data of the welding workbench area, run a 3D measurement algorithm, and determine the location and contour of the area to be welded using a 3D point set P = (p1 p2 p3…p ... n-1 p n ).

[0050] Specifically, in the step S1, recording the welding workbench area image data using the RGB-D camera fixed above the welding workbench refers to recording the RGB-D camera image data after correcting the distortion for three-dimensional measurement of the to-be-welded area, including a two-dimensional RGB image and a depth image;

[0051] The three-dimensional measurement algorithm refers to identifying and high-precision three-dimensional measuring the to-be-welded area through the image data recorded by the RGB-D camera, including to-be-welded area identification based on a two-dimensional RGB image, three-dimensional point cloud generation based on a two-dimensional RGB image and a depth image, and to-be-welded area contour output based on a three-dimensional point cloud;

[0052] The to-be-welded area identification based on a two-dimensional RGB image refers to using a region growing algorithm, taking the to-be-welded area as a segmentation target, and aggregating similar pixel points in the RGB image into a region through similarity comparison with surrounding pixel points, so as to realize segmentation of the to-be-welded area, and finally obtain the contour of the to-be-welded area; including the following steps: performing grayscale processing on the RGB color image to extract black and gray pixel points; performing binaryzation processing on the image after grayscale processing, and separating out the to-be-welded area by using binaryzation processing through the color difference between the to-be-welded area and the welding workbench; executing a region growing algorithm, selecting a starting pixel point from the image after binaryzation processing as a seed of a region, then expanding the pixel points around the seed until a preset similarity value or a boundary is reached; processing the expansion result, after obtaining the contour of the to-be-welded area, optimizing the contour of the to-be-welded area through morphological operation, eliminating noise points and holes, and saving the size and position information of the to-be-welded area on the two-dimensional RGB image;

[0053] The three-dimensional point cloud generation based on a two-dimensional RGB image and a depth image refers to respectively acquiring a two-dimensional RGB image and a depth image of the RGB-D camera, calibrating the two-dimensional RGB image and the depth image by using an existing method to unify them in the same coordinate system, reading the depth value in the depth image corresponding to the pixel value of the to-be-welded area contour by using the size and position information of the to-be-welded area on the two-dimensional RGB image obtained by the to-be-welded area identification based on a two-dimensional RGB image, and generating a three-dimensional point set with color information by using the position information and the depth value and relying on the internal parameter information of the RGB-D camera:

[0054]

[0055] Wherein, [X Y Z] is the position coordinates of the three-dimensional point, s d is a depth factor, d is a depth, [u v] is a pixel point in the two-dimensional RGB image, f x ,f yrespectively represent the length of the focal length of the pixel in the X-axis and Y-axis direction, (u0, v0) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system; the complete information of the three-dimensional point is [X Y Z R G B], and [R G B] is the color information of the red, green and blue three channels of the three-dimensional point;

[0056] The output of the to-be-welded region contour based on the three-dimensional point cloud refers to outputting a to-be-welded region contour three-dimensional point set according to the size and position information of the to-be-welded region on the two-dimensional RGB image and the three-dimensional point cloud:

[0057] P = (p1 p2 p3…p n-1 p n ),p i =[X i Y i Z i R i G i B i ];

[0058] The output to-be-welded region contour three-dimensional point set is in the RGB-D camera coordinate system.

[0059] S2, control the humanoid welding robot to move to the welding workbench along the set walking route, and connect the welding power source, the wire feeding system and the gas bottle through a cable.

[0060] Specifically, in step S2, the humanoid welding robot moves to the welding workbench along the set walking route, which means that the walking route of the humanoid welding robot is planned in a factory environment, and a yellow line is laid on the ground; the humanoid welding robot moves to the welding workbench by recognizing the yellow line through a binocular camera, and the binocular camera recognizing the yellow line includes the following steps: binarizing the binocular camera from a bird's eye view; converting the BGR color channel of the image to the HSV color channel, and setting upper and lower limit values for yellow; setting a region of interest (ROI) and detecting yellow line pixels by a sliding window method; fitting a curve and outputting the position of the yellow line.

[0061] Connecting the welding power source, the wire feeding system and the gas bottle to the humanoid welding robot through a cable means that after the humanoid welding robot moves to the welding workbench, the welding power source, the wire feeding system and the gas bottle are connected to the humanoid welding robot by a cable for subsequent process parameter control.

[0062] S3, based on the welding process parameter library established in the previous welding experiment, determine the process parameters according to the material of the to-be-welded workpiece; the process parameters include welding current, wire feeding speed, gas inlet speed, welding speed and welding angle.

[0063] The step S3, the welding process parameter library refers to that the corresponding relationship between the to-be-welded workpiece material and the welding process parameters is established through the previous process experiment, the welding current, the wire feeding speed, the gas feeding speed, the welding speed and the welding angle are respectively changed through the control variable method, the optimal process parameters are determined through the welding quality and performance, and thus the to-be-welded workpiece material-optimal process parameter mapping relationship is established.

[0064] S4, according to the to-be-welded area position, the contour three-dimensional point set P=(p1 p2 p3…p n-1 p n ) and the humanoid welding robot position, the motion trajectories of the two welding mechanical arms are respectively planned.

[0065] Specifically, in the step S4, the to-be-welded area position, the contour three-dimensional point set P refers to the to-be-welded area position, the contour three-dimensional point set P output in the step S1;

[0066] The position of the humanoid welding robot is obtained through the recognition of the welding torch by the RGB-D camera fixed above the welding workbench; the following steps are included: the left and right welding mechanical arms are controlled to move into the field of view of the RGB-D camera; the two-dimensional RGB image collected by the RGB-D camera is subjected to grayscale processing, and the black and gray pixel points are extracted; the region of interest ROI is set; the image after the grayscale processing is subjected to binaryzation processing, the welding torches of the left and right welding mechanical arms are separated through the color difference between the welding torches and the welding workbench; the region growing algorithm is executed, the pixel points around the welding torch are expanded until the preset similarity value or the boundary is reached; the welding torch contour is optimized through morphological operation, the noise points and the holes are eliminated, and the size and position information of the welding torches of the left and right welding mechanical arms on the two-dimensional RGB image are saved; the position information of the welding torch of the left welding mechanical arm is converted into the three-dimensional point T l =[X l Y l Z l ], and the position information of the welding torch of the right welding mechanical arm is converted into the three-dimensional point T r =[X r Y r Z r ];

[0067] The step of respectively planning the motion trajectories of the two welding mechanical arms refers to that the to-be-welded area contour three-dimensional point set P=(p1 p2 p3…p n-1 p n ) is decomposed into two segments of uniform and continuous three-dimensional point sets:

[0068] P l =(p1 p2 p3…p m-1 p m ), and P r =(p m+1 pm+2 p m+3 …p n-1 p n );

[0069] The three-dimensional point set P l The three-dimensional point set P is used for welding operations of the left welding robotic arm. r Used for welding operations of the right welding robot arm. The path for the left welding robot arm is set as follows: starting from T... l Let P be the starting position and the three-dimensional point set. l p1 is the termination point, and p1 is the optimal welding torch angle. This is achieved through the three-dimensional point set P. l =(p1 p2 p3…p m-1 p m The path of the right welding robot arm is set as follows: (using T...) r Let P be the starting position and the three-dimensional point set. r p m+1 p is the endpoint. m+1 The optimal welding torch angle is found at point P, through the three-dimensional point set P. r =(p m+1 p m+2 p m+3 …p n-1 p n The path of ). Wherein, the three-dimensional point set P l and P r The path is constrained by the optimal welding torch angle and welding speed. The RRT* algorithm is adopted, which is an improved method of Fast Random Exploration Tree (RRT). It plans the end-effector trajectories of the two welding robotic arms by sampling the three-dimensional space. Specifically, it uses cylinders to enclose the two welding robotic arms respectively, and uses cuboids to enclose the obstacles (workbench) in the space. The obstacle envelope is mapped to the obstacle avoidance space of the left welding robotic arm. The obstacle envelope and the left welding robotic arm envelope are mapped together to the obstacle avoidance space of the right welding robotic arm.

[0070] The random tree is generated with the starting position as the root node and the first 3D point of the 3D point set as the termination point. The generation method of the random tree is as follows: new nodes are generated in space using random sampling. If a new node does not collide with the obstacle avoidance space, the new node is added to the random tree as a node of the random tree. The random tree is expanded in space, and the corresponding parent node is recorded for each node. The cost from the starting point to each node is calculated until the termination point and the optimal welding torch angle are found. During the random sampling process, the nodes in the random tree are re-examined and reconnected to find the minimum cost path.

[0071] Afterwards, a random tree is generated with the three-dimensional point of the current three-dimensional point set as the starting point and the next three-dimensional point of the three-dimensional point set as the termination point; the execution is repeated until the last three-dimensional point of the three-dimensional point set is reached; the planned path is obtained by tracing back from the termination point to the starting position.

[0072] This way can ensure that the two-sided welding mechanical arms do not collide during welding.

[0073] S5, control the welding power supply to output ultrahigh frequency and fast response welding current, turn on the wire feeding system and gas bottle, control the two-sided welding mechanical arms to move along the planned trajectory, and complete the welding work.

[0074] In step S5, the control of the welding power supply to output ultrahigh frequency and fast response welding current refers to the use of ultrahigh frequency welding power supply to output welding current with inverter frequency of 200 kHz, rated current of 630 A, and response time of 300 μs. Compared with the output current of traditional industrial welding power supply, the ultrahigh frequency and fast response welding current is more suitable for controlling the welding quality, thereby ensuring the welding performance of the humanoid welding robot system.

[0075] Controlling the two-sided welding mechanical arms to move along the planned trajectory refers to controlling the two-sided welding mechanical arms to move along the planned trajectory in step S4 simultaneously after adjusting the process parameters through the cable connection, thereby completing the welding work.

[0076] S6, turn off the welding power supply, wire feeding system and gas bottle, and control the two-sided welding mechanical arms to return to the original point.

[0077] In step S6, turning off the welding power supply, wire feeding system and gas bottle refers to turning off the welding power supply, wire feeding system and gas bottle after the two-sided welding mechanical arms of the humanoid robot complete the welding work in step S5, thereby ending the supply of welding current, wire and gas.

[0078] Controlling the two-sided welding mechanical arms to return to the original point refers to controlling the left welding mechanical arm to move along a straight line in the three-dimensional space to the initial position T l , and the right welding mechanical arm to move along a straight line in the three-dimensional space to the initial position T m , after step S5 ends, the left welding mechanical arm is located at the endpoint p r of the three-dimensional point set P n , and the right welding mechanical arm is located at the endpoint p l of the three-dimensional point set P r .

[0079] S7, use the RGB-D camera fixed above the welding workbench to determine the welding quality.

[0080] The step S7 comprises the following sub-steps: using an RGB-D camera to shoot a post-welding RGB image and transmitting to a computing platform; according to the post-welding region contour three-dimensional point set P=(p1 p2 p3…pN) output in the step S1, a two-dimensional contour band ROI region with a width of 100mm is established on the RGB image; using a region growing algorithm, taking the post-welding region as a segmentation target, through similarity comparison with surrounding pixel points, similar pixel points in the RGB image are aggregated into a region, so as to realize segmentation of the post-welding region; saving size and position information of the post-welding region contour on the two-dimensional RGB image; converting the post-welding region contour into a three-dimensional point cloud, and making a difference with the contour three-dimensional point cloud set in the step S1, outputting a post-welding seam three-dimensional measurement result; judging whether there is a porosity, undercut and other welding quality problems for the post-welding seam three-dimensional measurement result. n-1 n ), a two-dimensional contour band ROI region with a width of 100mm is established on the RGB image; using a region growing algorithm, taking the post-welding region as a segmentation target, through similarity comparison with surrounding pixel points, similar pixel points in the RGB image are aggregated into a region, so as to realize segmentation of the post-welding region; saving size and position information of the post-welding region contour on the two-dimensional RGB image; converting the post-welding region contour into a three-dimensional point cloud, and making a difference with the contour three-dimensional point cloud set in the step S1, outputting a post-welding seam three-dimensional measurement result; judging whether there is a porosity, undercut and other welding quality problems for the post-welding seam three-dimensional measurement result.

[0081] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited by the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application should be equivalent replacement methods, and are all included in the protection scope of the present application.​

Claims

1. A method of intelligent welding for a humanoid welding robot, the method comprising: Be realized by a robot welding system; The robot welding system includes a humanoid welding robot and a welding workbench for fixing a workpiece to be welded; The humanoid welding robot carries a robot binocular camera and two side welding mechanical arms; The ends of the two side welding mechanical arms are respectively provided with a welding gun, and the two side welding mechanical arms respectively include a left welding mechanical arm and a right welding mechanical arm; The intelligent welding method of the humanoid welding robot includes the following steps: S1, record the image data of the welding workbench area using the RGB-D camera fixed above the welding workbench, run the three-dimensional measurement algorithm, and determine the position and contour three-dimensional point set of the to-be-welded area of the to-be-welded workpiece P =( p 1 p 2 p 3 … p n-1 p n ) S2, control the humanoid welding robot to move along the set walking route to the welding workbench; S3, based on the welding process parameter library, determine the process parameters according to the material of the workpiece to be welded; S4, according to the position of the welding area, the profile three-dimensional point set P and the position of the humanoid welding robot, respectively plan the motion trajectories of the welding mechanical arms on both sides; S5, control the two side welding mechanical arms to move along the planned trajectory and perform welding work; S6, after the welding work is completed, control the two side welding mechanical arms to return to the original position; In the step S1, recording the image data of the welding workbench area by using the RGB-D camera fixed above the welding workbench refers to recording the corrected RGB-D camera image data for three-dimensional measurement of the to-be-welded area, including two-dimensional RGB image and depth image; The three-dimensional measurement algorithm refers to identifying and high-precision three-dimensional measurement of the to-be-welded area by using the image data recorded by the RGB-D camera, including to-be-welded area identification based on two-dimensional RGB image, three-dimensional point cloud generation based on two-dimensional RGB image and depth image, and to-be-welded area contour output based on three-dimensional point cloud; The to-be-welded area identification based on two-dimensional RGB image refers to the following steps: grayscale processing of the RGB color image to extract black and gray pixel points; binaryzation processing of the grayscale processed image, separation of the to-be-welded area by using binaryzation processing through the color difference between the to-be-welded area and the welding workbench; performing region growing algorithm, selecting a starting pixel point from the binaryzation processed image as a seed of a region, then expanding the pixel points around the seed until a preset similarity value or boundary is reached; processing the expansion result, after obtaining the to-be-welded area contour, optimizing the to-be-welded area contour through morphological operation to eliminate noise and holes, and saving the size and position information of the to-be-welded area on the two-dimensional RGB image; The three-dimensional point cloud generation based on two-dimensional RGB image and depth image refers to obtaining the two-dimensional RGB image and depth image of the RGB-D camera respectively, calibrating the two-dimensional RGB image and depth image to be unified in the same coordinate system; using the size and position information of the to-be-welded area on the two-dimensional RGB image obtained by the to-be-welded area identification based on two-dimensional RGB image, reading the depth value of the to-be-welded area contour pixel value in the depth image; using the position information and depth value, generating a three-dimensional point set with color information by using the internal parameter information of the RGB-D camera: ； in,[ X Y Z [ represents the position coordinates of a 3D point] s d For depth factor, d For depth, [ u v [ ] represents a pixel in a two-dimensional RGB image. f x ,f y These represent the focal length of the pixel along the X and Y axes, respectively. u 0 , v 0 ) represents the coordinates of the origin of the image coordinate system in the pixel coordinate system; the complete information of a 3D point is [ X Y Z R G B ], [ R G B [This refers to the color information of the red, green, and blue channels of a 3D point;] The to-be-welded area contour output based on three-dimensional point cloud refers to outputting the to-be-welded area contour three-dimensional point set according to the size and position information of the to-be-welded area on the two-dimensional RGB image and the three-dimensional point cloud: P =( p 1 p 2 p 3 … p n-1 p n )， p i =[ X i Y i Z i R i G i B i ]； The output to-be-welded area contour three-dimensional point set is in the RGB-D camera coordinate system; In the step S4, the position of the area to be welded, the profile three-dimensional point set P is referred to the position of the area to be welded, the profile three-dimensional point set P output in the step S1. The human-shaped welding robot position is obtained by an RGB-D camera fixed above a welding workbench; comprising the following steps: controlling the left and right welding mechanical arms to move into the field of view of the RGB-D camera; carrying out grayscale processing on the two-dimensional RGB image collected by the RGB-D camera, and extracting black and gray pixel points; setting a region of interest (ROI); carrying out binaryzation processing on the grayscale-processed image, separating the welding torches of the left and right welding mechanical arms by using binaryzation processing due to the obvious color difference between the welding torches and the welding workbench; executing a region growing algorithm, expanding the pixel points around the welding torches until a preset similarity value or a boundary is reached; optimizing the welding torch contour by morphological operation, eliminating noise points and holes, and saving the size and position information of the welding torches of the left and right welding mechanical arms on the two-dimensional RGB image; converting the position information of the welding torch of the left welding mechanical arm into a three-dimensional point T l =[ X l Y l Z l ] and the position information of the welding torch of the right welding mechanical arm into a three-dimensional point T r =[ X r Y r Z r ] The separately planning the motion trajectories of the welding mechanical arms on two sides refers to decomposing the three-dimensional point set of the region to be welded contour P =( p 1 p 2 p 3 … p n-1 p n ) into two even and continuous three-dimensional point sets: P l =( p 1 p 2 p 3 … p m-1 p m )， P r =( p m+1 p m+2 p m+3 … p n-1 p n )； Set the path of the left welding robot arm as follows: T l Starting position, three-dimensional point set P l of p 1 As the endpoint, p 1 The optimal welding torch angle is found at a point defined by a three-dimensional point set. P l =( p 1 p 2 p 3 … p m-1 p m The path of the right welding robot arm is set as follows: T r Starting position, three-dimensional point set P r of p m+1 As the endpoint, p m+1 The optimal welding torch angle is found at a point defined by a three-dimensional point set. P r =( p m+1 p m+2 p m+3 … p n-1 p n The path of a 3D point set; P l and P r The path is constrained by the optimal welding torch angle and welding speed; where, the three-dimensional point set P l and P r The path planning method refers to: enclosing the two welding robotic arms with cylinders respectively, and enclosing the obstacles in the space with cuboids; mapping the obstacle envelopes to the obstacle avoidance space of the left welding robotic arm; and mapping the obstacle envelopes and the left welding robotic arm envelopes together to the obstacle avoidance space of the right welding robotic arm. The random tree is generated with a starting position as a root node of the random tree and a first three-dimensional point of the three-dimensional point set as a terminal point; the random tree is generated by using random sampling to generate a new node in the space; if the new node does not collide with the obstacle-avoiding space, the new node is added to the random tree as a node of the random tree; the random tree is expanded in the space, the corresponding parent node of each node is recorded, and the cost from the starting point to each node is calculated until the terminal point and the optimal welding gun angle are found; in the random sampling process, the nodes in the random tree are rechecked and reconnected to find the minimum cost path; Then, a random tree is generated with a three-dimensional point of the current three-dimensional point set as a starting point and a next three-dimensional point of the three-dimensional point set as a terminal point; the process is repeated until the last three-dimensional point of the three-dimensional point set is reached; the planned path is obtained by tracing back from the terminal point to the starting position; In the step S5, the control of the movement of the two welding mechanical arms along the planned trajectory refers to the control of the movement of the two welding mechanical arms along the planned trajectory in the step S4 at the same time, so as to complete the welding work; In the step S2, the movement of the humanoid welding robot along the set walking route to the welding workbench refers to the planning of the walking route of the humanoid welding robot in the factory environment and the layout of the yellow line on the ground; the humanoid welding robot moves to the welding workbench by recognizing the yellow line through the binocular camera, and the recognition of the yellow line by the binocular camera includes the following steps: binarization of the binocular camera in the bird's eye view; conversion of the BGR color channel of the image into the HSV color channel, and setting of upper and lower limit values for the yellow color; setting of a region of interest (ROI) and detection of yellow line pixels by a sliding window method; fitting of a curve and output of the position of the yellow line.

2. The anthropomorphic welding robot intelligent welding method of claim 1, wherein: In the step S3, the process parameters include welding current, wire feeding speed, gas feeding speed, welding speed and welding angle; the welding process parameter library refers to the establishment of the corresponding relationship between the to-be-welded workpiece material and the welding process parameters through preliminary process experiments, the change of the welding current, the wire feeding speed, the gas feeding speed, the welding speed and the welding angle by the control variable method, the determination of the optimal process parameters by the welding quality and performance, and the establishment of the mapping relationship between the to-be-welded workpiece material and the optimal process parameters.

3. The anthropomorphic welding robot intelligent welding method of claim 1, wherein: In the step S6, the control of the two welding mechanical arms to return to the original position means that after the end of the step S5, the left welding mechanical arm is located at the end point of the three-dimensional point set P l , p m the right welding mechanical arm is located at the end point of the three-dimensional point set P r , p n , the control of the two welding mechanical arms to respectively move along the straight line in the three-dimensional space to the initial position T l , T r .

4. The anthropomorphic welding robot intelligent welding method of claim 1, wherein: After the step S6, there is also a step S7: determining the welding quality by using an RGB-D camera fixed above the welding workbench.

5. The anthropomorphic welding robot intelligent welding method of claim 4, wherein: The step S7 comprises the following sub-steps: using an RGB-D camera to shoot a post-welding RGB image; according to the to-be-welded region contour three-dimensional point set output in the step S1 P =( p 1 p 2 p 3 … p n-1 p n ), a two-dimensional contour band ROI region is established on the RGB image; using a region growing algorithm, taking the post-welding region as a segmentation target, similar pixel points in the RGB image are aggregated into a region through similarity comparison with surrounding pixel points, so that the segmentation of the post-welding region is realized; the size and position information of the post-welding region contour on the two-dimensional RGB image is saved; the post-welding region contour is converted into a three-dimensional point cloud, and the three-dimensional point cloud is subtracted from the contour three-dimensional point cloud set in the step S1, so that a post-welding three-dimensional measurement result is output; whether there is a welding quality problem is judged according to the post-welding three-dimensional measurement result.

6. The anthropomorphic welding robot intelligent welding method of claim 1, wherein: The robot welding system also includes a support device; the support device includes a wire feeding system, a welding power source and a gas cylinder; the humanoid welding robot is connected with the support device. The robot welding system also includes a support device; the support device includes a wire feeding system, a welding power source and a gas cylinder; the humanoid welding robot is connected with the support device.

Images