An instruction-guided automated welding system and method

Abstract

This invention specifically relates to an automated welding system and method based on instruction guidance. The system includes a welding robot with a welding torch at one end; a welding indicator torch for pointing at a target welding point in a specific posture; and a vision subsystem for acquiring data from the welding indicator torch, determining the posture of the target welding point, and generating the motion trajectory of the welding robot based on the posture of the target welding point indicated by the welding indicator torch and the posture of the welding torch. The vision subsystem, with the assistance of the welding indicator torch, can acquire the target position and posture for welding, thereby enabling the welding robot to complete the welding under the guidance of the welding indicator torch. During welding tasks, the welding point is confirmed by manual instruction, avoiding the need for identifying workpiece features, thus making it easier to handle welding tasks involving complex workpieces.

Description

Technical Field

[0001] This invention relates to the field of automatic welding technology, and more specifically to an automatic welding system and method based on instruction guidance. Background Technology

[0002] Traditional manual welding suffers from problems such as poor weld quality consistency, high labor intensity for workers, and significant eye hazards. Furthermore, it is difficult to achieve good weld consistency during long-term welding operations.

[0003] Traditional automated welding systems use structured light, line laser and other vision methods to automatically identify welded parts. By automatically identifying weld features, the system performs path planning to achieve automated welding. However, on actual production lines, due to the strong reflectivity of steel components, structured light and other vision systems need to be detected at a good angle to obtain complete feature information. They are greatly affected by the secondary reflection of the workpiece itself and ambient light, and usually cannot perform automated welding accurately.

[0004] Traditional automated welding systems use a single camera, and without external camera detection, path planning during welding cannot perform obstacle avoidance in real time, making collisions easy. Therefore, automated welding of complex workpieces and large-scale applications is difficult to achieve.

[0005] Traditional welding path indication methods often involve manual teaching, which is complex, slow, and difficult for workers to master.

[0006] Therefore, optimizing the automatic welding method for complex and large workpieces, improving accuracy, and avoiding the environmental impact caused by surface reflection of the welded parts are problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention provides an automatic welding system and method based on instruction guidance, which can confirm the welding position and posture with the assistance of a welding indicator gun, avoiding the identification of workpiece features, and thus making it easier to handle welding tasks of complex workpieces.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] An instruction-guided automated welding system includes:

[0010] A welding robot, wherein one end of the welding robot has a welding torch;

[0011] A welding indicator gun is used to point at a target welding point in a specific positioning posture.

[0012] The vision subsystem is used to acquire images of the welding pointer gun, determine the pose of the target welding point, and generate the motion trajectory of the welding robot based on the pose of the target welding point indicated by the welding pointer gun and the pose of the welding gun.

[0013] Preferably, the welding robot includes: a robotic arm, the welding torch, and a loading platform;

[0014] One end of the robotic arm is fixed to the loading platform, and the other end is connected to the welding gun;

[0015] The loading platform is used to place the welding components.

[0016] Preferably, the vision subsystem is mounted on the loading platform or on the robotic arm.

[0017] Preferably, the vision subsystem includes a housing, a camera, a flash, and an area indicator light; the camera and the flash are disposed inside the housing; the area indicator light is disposed on the housing and is used to indicate the image acquisition area of ​​the camera by emitting visible light.

[0018] Preferably, the welding indicator gun includes an indicator component and a visual positioning component; the indicator component is fixedly connected to the visual positioning component;

[0019] The indicator component contacts the target welding point of the weldment;

[0020] When the indicating component comes into contact with the target welding point of the weldment, the vision subsystem acquires an image of the vision positioning component, identifies the marker points on the vision positioning component, and calculates the position and orientation of the indicating component using pre-calibrated vision positioning component calibration parameters.

[0021] Preferably, the visual positioning component includes a visual positioning plate, one end of which is connected to the indicating component; the surface of the visual positioning plate is used to set the marking points, which are made of a high-gloss material or device, and the visual positioning plate is connected to a handheld part.

[0022] Preferably, the marking points are attached with markings of a specific shape, feature points, or different colors;

[0023] The specific shape includes a ring, a checkerboard pattern, or stripes; the feature points include corner points, line intersections, or textures.

[0024] Preferably, the handheld part has a cavity structure, and a charging interface, a signal transmitting unit, an indicator switch, and a main power switch are installed inside the cavity of the handheld part.

[0025] An instruction-guided automated welding method includes the following steps:

[0026] The welding indicator gun is pointed at the target welding point in a specific posture;

[0027] Acquire one or more images of the welding indicator gun, identify the position and orientation of the welding indicator gun, and obtain one or more first poses;

[0028] The second pose is obtained based on the joint angles of the robotic arm;

[0029] Generate a motion trajectory from the second pose to the first pose; transform the motion trajectory generated from the first pose to the robot coordinate system through hand-eye calibration;

[0030] The welding robot is controlled to move to the target welding point according to the motion trajectory, and performs automatic welding based on the motion trajectory generated by connecting one or more first poses.

[0031] Preferably, identifying the position and orientation of the welding indicator gun includes:

[0032] Multiple marker points in the welding indicator gun are identified, and the pixel coordinates of the corresponding camera image of each marker point are confirmed; the position and orientation of the welding indicator gun are calculated using the PNP algorithm.

[0033] As can be seen from the above technical solution, compared with the prior art, the present invention discloses an automatic welding system and method based on instruction guidance. In addition to obtaining the current position and posture of the welding robot from its image, the vision subsystem can also acquire the target position and posture for welding with the assistance of the welding indicator gun, thus enabling the welding robot to complete welding under the guidance of the welding indicator gun. During welding tasks, the welding point is confirmed by manual instruction, avoiding the identification of workpiece features, thus making it easier to handle welding tasks of complex workpieces. The present invention is applicable to both "eye outside the hand" and "eye on the hand" installation methods, allowing for easy movement of the entire welding robot, thereby enabling the welding of large parts. In this invention, the marking points in the welding indicator gun are calibrated with the camera in the vision subsystem; the camera acquires images of the marking points to achieve guidance, thus making it applicable to various reflective material surfaces. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0035] Figure 1This invention provides a schematic diagram of an automatic welding system based on instruction guidance.

[0036] Figure 2 This is a schematic diagram of the welding robot in this invention.

[0037] Figure 3 This is a schematic diagram of the visual subsystem in this invention.

[0038] Figure 4 This is a top view of the welding indicator gun in this invention.

[0039] Figure 5 This is a front view of the welding indicator gun in this invention.

[0040] Among them, 1-robotic arm, 2-welding torch, 3-vision subsystem, 31-housing, 32-camera, 33-flash lamp; 4-carrying platform; 5-indicator component, 6-vision positioning component, 7-handheld part, 8-charging interface, 9-signal transmission unit, 10-indicator switch, 11-main power switch. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 An automated welding system based on instruction guidance includes: a welding robot with a welding torch at one end; a welding pointer torch for pointing at a target welding point in a specific pose; a vision subsystem for acquiring images of the welding pointer torch, determining the pose of the target welding point, and generating a motion trajectory of the welding robot based on the pose of the target welding point and the pose of the welding torch. It also includes a drive component for receiving control signals corresponding to the motion trajectory and driving the welding robot to perform automated welding.

[0044] Through prior coordinate system calibration, the transformation matrices between the camera coordinate system and the motion coordinate systems of the robotic arm and welding torch in vision subsystem 3, as well as the transformation matrix between the camera coordinate system and the motion coordinate system of the welding indicator torch, can be obtained. This achieves the unification of the motion coordinate systems of the welding indicator torch, robotic arm, and welding torch through the camera coordinate system. After vision subsystem 3 acquires the image of the welding indicator torch, it plans the motion trajectory from the initial pose to the target welding point pose in the same coordinate system, based on the coordinate transformation relationships.

[0045] In this embodiment, the vision subsystem, with the assistance of the welding pointer, can acquire the target position and orientation for welding, thereby enabling the welding robot to complete the welding under the guidance of the welding pointer. During welding tasks, the welding points are confirmed by manual instruction, avoiding the need for workpiece feature identification; therefore, it is easier to handle welding tasks involving complex workpieces.

[0046] In addition, in this embodiment, the vision subsystem can also acquire images of the welding indicator gun at different target welding points of the welded part multiple times to obtain the changes in the posture of the target welding point, obtain the target welding trajectory, and then generate a running trajectory that conforms to the target welding trajectory to guide the movement of the welding robot, thereby realizing automatic welding of different welding points of the welded part.

[0047] like Figure 2 To further implement the above technical solution, one aspect of this invention is to place the vision subsystem outside the welding robot. This ensures that the movement of the welding robot does not affect the vision subsystem, achieving a "vision outside the hand" arrangement. The welding robot includes: a robotic arm 1, a welding torch 2, and a loading platform 4; one end of the robotic arm 1 is fixed to the loading platform 4, and the other end is connected to the welding torch 2; in addition, the loading platform 4 is also used to place the workpiece; to achieve the "vision outside the hand" arrangement, the vision subsystem is mounted on the loading platform.

[0048] To improve the flexibility of welding operations, the loading platform 4 can be configured as a mobile platform; the robotic arm 1 is a six-axis robotic arm.

[0049] To further implement the above technical solution, the vision subsystem 3 includes a housing 31, a camera 32, and a flash 33; the camera 32 and the flash are disposed inside the housing 31; a region indicator target 34 is provided on the housing 31, which is used to emit visible light to indicate the image acquisition area of ​​the camera 32. Multiple region indicator targets 34 are provided on the housing 31, respectively arranged on the edge or corner of the housing 31 on the lens side of the camera 32.

[0050] To further implement the above technical solution, the welding indicator gun includes an indicator component 5 and a visual positioning component 6; the indicator component 5 is fixedly connected to the visual positioning component 6; the indicator component 5 indicates the welding position and posture by contacting the workpiece.

[0051] When the indicator component gives an instruction, the vision subsystem acquires an image of the vision positioning component, identifies the marker points on the vision positioning component, and calculates the position and orientation of the indicator component using pre-calibrated calibration parameters.

[0052] The visual positioning component 6 includes a visual positioning plate, one end of which is connected to the indicator component 5. The surface of the visual positioning plate is used to set marker points, which are made of a total reflective material and are attached to the surface of the visual positioning plate. The visual positioning plate is connected to a handheld part 7. The handheld part 7 has a cavity structure, and a charging interface 8, a signal transmitting unit 9, an indicator switch 10, and a main power switch 11 are installed inside the cavity. The signal transmitting unit 3 can use a chip such as ESP32. The welding indicator gun clicks the indicator switch 11 at the welding target and sends a command to the vision subsystem 3. After receiving the command, the vision subsystem 3 acquires image information from the visual positioning plate, determines the position and orientation of the target welding point based on the image information, sets the welding parameters, and finally plans the path according to the point indicated by the indicator gun.

[0053] The calculation of the target welding point position includes the following steps: constructing a welding indicator gun coordinate system, and calculating the pose between the welding indicator gun coordinate system and the camera as R, t. Simultaneously, obtaining the position (x, y, z) of the end of the indicator component 5 that contacts the weldment in the welding indicator gun coordinate system, and then calculating its position in the camera coordinate system at this moment as R*(x, y, z) + t; where R represents the rotation transformation matrix between the welding indicator gun coordinate system and the camera coordinate system, and t represents the translation transformation matrix between the welding indicator gun coordinate system and the camera coordinate system.

[0054] To determine the pose of the visual positioning component 6 when the indicating component 5 is providing guidance, the camera coordinate system and the visual positioning plate coordinate system need to be pre-calibrated. This calibration records the intrinsic parameters of the camera 32 in the vision subsystem 3 and the extrinsic parameters of the welding indicator gun at different angles. Based on the calibrated parameters, the pose is calculated using the PNP algorithm. The calibration process for the welding indicator gun and camera is as follows:

[0055] S11: Prepare a visual positioning board: First, you need to prepare a visual positioning board. A checkerboard visual positioning board is usually chosen because it has obvious feature points that are easy to identify. Ensure the visual positioning board is flat and that the feature points are clearly identifiable.

[0056] S12: Image Acquisition: Acquire multiple images of the visual positioning plate using a monocular camera. Throughout the process, the visual positioning plate needs to be kept in different positions and angles to capture images of the visual positioning plate from different perspectives.

[0057] S13: Corner Extraction: For each image, image processing methods are used to extract the corner information on the visual positioning board. These corners will be used for subsequent calibration calculations; these corners are referred to as marker points.

[0058] S14: Calculate intrinsic parameters: Using the extracted corner data, calculate the camera's intrinsic parameters, including focal length, principal point position, etc., through a mathematical model of camera calibration (such as a pinhole camera model).

[0059] S15: Calculate distortion parameters: Correct camera lens distortion, including radial distortion and tangential distortion.

[0060] S16: Calculate extrinsic parameters: Calculate the camera's extrinsic parameters, including camera pose (position and orientation), using intrinsic parameters and corner information on the image.

[0061] S17: Evaluate calibration results: Use the calibration results to correct image distortion and verify the accuracy of the calibration results.

[0062] Several methods can be used to distinguish marker points. Special shape markers can be added, such as rings, checkerboard patterns, stripes, etc., as these shapes are easier to detect and identify. Feature points in natural scenes can also be incorporated as markers, such as corner points, line intersections, and textures. Color marking can also be used, employing different colored markers, such as red, blue, and green, which can be identified using color segmentation algorithms.

[0063] In addition, in order to guide the position and orientation of the welding torch, it is necessary to perform hand-eye calibration of the welding torch and the vision subsystem in advance. The calibration steps are as follows:

[0064] S21: Fix the camera outside the robotic arm and ensure that the movement of the robotic arm will not collide with the camera.

[0065] S22: Collect data, move the end effector of the robotic arm to different positions (at least 4, more is better), and record the transformation matrix (\mathbf{A}_i) from the camera coordinate system to the robotic arm base coordinate system and the transformation matrix (\mathbf{B}_i) of the camera from its initial position at each position.

[0066] S23: Parameter Solving. X is the hand-eye calibration matrix to be solved, and the solution formula is as follows:

[0067] A point on the vision positioning plate has coordinates P1 in the vision positioning plate coordinate system and P3 in the robot arm end-effector coordinate system. The transformation relationship in pose 1 is as follows:

[0068] P1TcwXTeb=P3(1)

[0069] Transformation relationship in pose 2:

[0070] P1Tcw'XTeb'=P3(2)

[0071] (2) In the formula, Teb and Teb' can be obtained from the robot's pose output, and Tcw and Tcw' can be obtained from the calibrated camera extrinsic parameters. The result can be obtained by combining the two formulas.

[0072] TcwXTeb=Tcw'XTeb'(3)

[0073] Further conversion to:

[0074] TcwXTcw'-1=TebXTeb'-1(4)

[0075] Formula (4) can be understood as AX = XB, where A = TcwXTcw'-1 and B = TebXTeb'-1. A and B are known values. By changing the robot's pose multiple times, the hand-eye calibration equation AX = XB can be solved to obtain the value of the hand-eye conversion matrix X.

[0076] Application of calibration parameters: Once calibration is completed and verified, the obtained intrinsic and extrinsic parameters can be used in welding pen pose adjustment applications. By acquiring the attitude and position of the welding indicator gun, the position information of the welding indicator gun can be sent to the welding torch on the robotic arm, thereby guiding the welding torch on the robotic arm to complete the welding task.

[0077] Optimization and correction:

[0078] Using the collected data, the error function is minimized by continuously adjusting (\mathbf{X}) through iterative optimization algorithms (such as gradient descent, genetic algorithms, etc.).

[0079] After optimization, the obtained (\mathbf{X}) is the transformation matrix from the camera coordinate system to the robot arm base coordinate system.

[0080] The diversity and precision of data acquisition directly affect the accuracy of calibration results. The calibration process may require repeated iterations, especially given the limitations of mechanical system precision and camera resolution. This invention uses markers on a visual positioning plate that facilitate image recognition to improve the accuracy of camera observations.

[0081] To further implement the above technical solution, the vision subsystem is also used to perform calibration and store calibration parameters.

[0082] Example 2

[0083] Based on the same inventive concept, this invention discloses an automatic welding method based on instruction guidance, comprising the following steps:

[0084] S1: Point the welding indicator gun at the target welding point in a specific posture.

[0085] S2: Acquire an image of the welding indicator gun, identify the position and orientation of the welding indicator gun, and obtain one or more first poses. In S2, each first pose can confirm a trajectory point in the welding trajectory. Connect the trajectory points corresponding to multiple first poses to generate the welding trajectory.

[0086] S3: Obtain the second pose based on the angles of each axis of the robotic arm.

[0087] S4: Generate the motion trajectory from the second pose to the first pose. The welding trajectory, represented by multiple first poses, is transformed to the robot coordinate system through hand-eye calibration. Specifically, the PNP algorithm is used to transform the acquired pixel coordinates to the camera coordinate system using the camera's intrinsic parameters. Then, the robot-camera coordinate transformation matrix obtained during pre-calibration is used to transform it to the robot coordinate system. Alternatively, other industry-known calculation methods can also be used to achieve coordinate system transformation.

[0088] S5: Control the welding robot to move to the target welding point according to the motion trajectory, and perform automatic welding according to the motion trajectory generated by S2.

[0089] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0090] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An automatic welding system based on instruction guidance, characterized in that, The welding robot includes a robotic arm, a welding torch, and a loading platform; one end of the robotic arm is fixed to the loading platform, and the other end is connected to the welding torch; the loading platform is used to place the workpiece for welding. A welding indicator gun is used to point at a target welding point in a specific positioning posture. A vision subsystem is used to acquire images of the welding pointer gun, determine the pose of the target welding point, and generate the motion trajectory of the welding robot based on the pose of the target welding point indicated by the welding pointer gun and the pose of the welding gun; the vision subsystem is mounted on the platform. The vision subsystem is used to acquire the target position and posture of welding with the assistance of the welding indicator gun, so that the welding robot can complete the welding under the guidance of the welding indicator gun. When performing the welding task, the welding point is confirmed by manual instruction and the identification of workpiece features is avoided. The vision subsystem needs to perform hand-eye calibration before determining the pose. X is the hand-eye calibration matrix to be solved, and the solution formula is as follows: The coordinates of a point on the vision positioning plate are P1 in the vision positioning plate coordinate system and P3 in the robot arm end-effector coordinate system. The transformation relationship in pose 1 is as follows: P1TcwXTeb=P3 Transformation relationship in pose 2: P1Tcw'XTeb'=P3 In the formula, Teb and Teb' can be obtained from the robot's pose output, and Tcw and Tcw' can be obtained from the calibration of the camera's extrinsic parameters. The two formulas can be combined to obtain the final result. TcwXTeb=Tcw'XTeb' Further conversion to: TcwXTcw'-1=TebXTeb'-1 The value of the hand-eye transformation matrix X can be obtained by solving the problem.

2. The automatic welding system based on instruction guidance according to claim 1, characterized in that, The vision subsystem includes a housing, a camera, a flash, and an area indicator light; the camera and the flash are disposed inside the housing; the area indicator light is disposed on the housing and is used to indicate the image acquisition area of ​​the camera by emitting visible light.

3. The automatic welding system based on instruction guidance according to claim 1, characterized in that, The welding indicator gun includes an indicator component and a visual positioning component; the indicator component is fixedly connected to the visual positioning component. The indicator component contacts the target welding point of the weldment; When the indicating component comes into contact with the target welding point of the weldment, the vision subsystem acquires an image of the vision positioning component, identifies the marker points on the vision positioning component, and calculates the position and orientation of the indicating component using pre-calibrated vision positioning component calibration parameters.

4. The automatic welding system based on instruction guidance according to claim 3, characterized in that, The visual positioning component includes a visual positioning plate, one end of which is connected to the indicator component; the surface of the visual positioning plate is used to set the marking points, which are made of a high-gloss material or device, and the visual positioning plate is connected to a handheld part.

5. An automatic welding system based on instruction guidance according to claim 4, characterized in that, The markers have markings of specific shapes, features, or different colors attached to them; The specific shape includes a ring, a checkerboard pattern, or stripes; the feature points include corner points, line intersections, or textures.

6. The automatic welding system based on instruction guidance according to claim 5, characterized in that, The handheld part has a cavity structure, and a charging interface, a signal transmitting unit, an indicator switch, and a main power switch are installed inside the cavity of the handheld part.

7. An automatic welding method based on instruction guidance, characterized in that, The automatic welding system according to any one of claims 1-6 includes the following steps: The welding indicator gun is pointed at the target welding point in a specific posture; Acquire one or more images of the welding indicator gun, identify the position and orientation of the welding indicator gun, and obtain one or more first poses; The second pose is obtained based on the joint angles of the robotic arm; Generate a motion trajectory from the second pose to the first pose; transform the motion trajectory generated from the first pose to the robot coordinate system through hand-eye calibration; The welding robot is controlled to move to the target welding point according to the motion trajectory, and performs automatic welding based on the motion trajectory generated by connecting one or more first poses.

8. The automatic welding method based on instruction guidance according to claim 7, characterized in that, The identification of the position and orientation of the welding indicator gun includes: Multiple marker points in the welding indicator gun are identified, and the pixel coordinates of the camera image corresponding to each marker point are confirmed; the position and orientation of the welding indicator gun are calculated using the PNP algorithm.

Images