Demonstration-free robot welding pose planning method and system based on two-dimensional visual inspection

Abstract

The invention discloses a teaching-free robot welding pose planning method based on two-dimensional visual inspection. The teaching-free robot welding pose planning method comprises the following steps that a welding workpiece image is collected; marking two end points of a welding seam for the welding workpiece image, and training an end point detection model; performing image preprocessing and perspective correction on the welding workpiece image to obtain a standard image with a set size; the standard image outputs two endpoint target frames through a trained endpoint detection model, and the center point of each target frame serves as a welding seam endpoint pixel coordinate; converting the endpoint pixel coordinates into spatial points under a robot-based coordinate system; determining a welding seam direction vector according to the two space points, and generating an arc striking preparation point in front of a starting point; and a posture constraint that the advancing direction of the welding gun is aligned with the welding seam direction is constructed, the tool posture is solved, the arc striking preparation point posture is obtained, and the mechanical arm is guided to move to the arc striking point. According to the method, the welding pose and track under the robot base coordinate system can be automatically generated, and teaching-free welding pose planning is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot welding pose control, and particularly relates to a teaching-free robot welding pose planning method based on two-dimensional visual detection. BACKGROUND

[0002] The existing robot welding generally relies on manual teaching or offline programming, and is strongly dependent on manual experience, and is difficult to adapt to clamping errors, changes in groove gaps, thermal deformation and flying dust and other site fluctuations, often leading to inaccurate arc point positioning, difficult welding gun posture alignment, long debugging period and poor consistency. The existing technical route is usually based on teaching or CAD offline trajectory to generate a nominal path, and then uses vision, laser or arc sensing to detect deviations and compensate online. The three-dimensional profile of the weld is obtained by using structured light triangulation, and the weld positioning and tracking are realized by geometric feature extraction. And using two-dimensional camera image for filtering enhancement, edge extraction or template matching to identify key features such as weld edge and center line. Although these methods have their own application scenarios, they are still limited by the consistency requirements of teaching or models in actual engineering, and are sensitive to changes in light, occlusion, flying dust and other noise. At the same time, there is a lack of unified and complete conversion and planning link between the visual detection results and the six-dimensional pose and welding trajectory that the robot can directly execute, resulting in the need for more calibration, debugging and secondary development when landing. SUMMARY

[0003] In order to overcome the defects and deficiencies of the prior art, the present application provides a teaching-free robot welding pose planning method based on two-dimensional visual detection. The present application identifies key features of the weld based on two-dimensional visual detection, and automatically generates welding pose and trajectory in the robot base coordinate system through calibration mapping and attitude constraint solving, realizing teaching-free welding pose planning.

[0004] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0005] The present application provides a teaching-free robot welding pose planning method based on two-dimensional visual detection, comprising the following steps:

[0006] Collecting a welding workpiece image;

[0007] Labeling the two endpoints of the weld on the welding workpiece image, and training an endpoint detection model;

[0008] Image preprocessing and perspective correction are performed on the welding workpiece image to obtain a standard image of a specified size;

[0009] The two endpoint target boxes output by the trained endpoint detection model of the standard image are taken as the pixel coordinates of the weld endpoints;

[0010] Convert the endpoint pixel coordinates into spatial points in the robot base coordinate system;

[0011] Determine the weld direction vector according to two spatial points, and generate an arc striking preparation point before the starting point;

[0012] Construct the pose constraint of the welding gun travel direction aligning the weld direction, solve the tool pose, obtain the arc striking preparation point pose, and guide the movement of the mechanical arm to the arc striking point.

[0013] As a preferred technical solution, the two endpoints of the weld are labeled on the weld workpiece image, specifically including:

[0014] For each workpiece image, determine the two endpoints of the weld on the image coordinate system;

[0015] Draw a square frame with a set side length centered on each endpoint to obtain two endpoint frames, and take the center points of the two target frames as the two endpoint coordinates.

[0016] As a preferred technical solution, the weld workpiece image is preprocessed and perspective corrected, specifically by using a homography perspective transformation to transform the original image to a unified perspective and fixed size.

[0017] As a preferred technical solution, the endpoint detection model uses a single-stage target detection network, and the endpoint detection model outputs an endpoint frame set. When multiple candidate endpoint frames are detected, the top two items are selected according to the confidence score, or two endpoint frames are selected by combining spatial position constraints.

[0018] As a preferred technical solution, the coordinates of the two endpoints of the weld are directly output by a key point regression network.

[0019] As a preferred technical solution, the endpoint pixel coordinates are converted into spatial points in the robot base coordinate system, specifically including:

[0020] Normalize the endpoint pixel coordinates, and perform linear mapping based on the scale factor and bias to obtain 、 axis coordinate points, and take the axis coordinate points as the workpiece plane height or the calibrated height.

[0021] As a preferred technical solution, the weld direction vector is determined according to two spatial points, and an arc striking preparation point is generated before the starting point, specifically including:

[0022] Let the two spatial points of the weld be 、 , then the weld direction unit vector is:

[0023] ;

[0024] Retreat a distance of , and then along The shaft is raised in the opposite direction by a certain distance. Generate the arc-starting preparation point:

[0025] ;

[0026] in, Indicates the welding start point. This indicates the preparation point for arc initiation.

[0027] As a preferred technical solution, the posture constraint of aligning the welding torch travel direction with the weld direction is constructed, specifically including:

[0028] Attitude constraints ensure that the tool or sensor coordinate system satisfies the following condition: the x-axis is along the weld direction. The axes are opposite in direction to the base coordinate system and orthogonal to the x-axis. The y-axis is obtained by the cross product, forming a right-handed coordinate system.

[0029] As a preferred technical solution, guiding the robotic arm to move to the arc initiation point also includes: determining the arc initiation point based on line laser scanning, and performing weld seam tracking compensation based on line laser.

[0030] The present invention also provides a teachless robot welding pose planning system based on two-dimensional vision detection, which is used in the above-mentioned teachless robot welding pose planning method based on two-dimensional vision detection, including: an image acquisition module, a data annotation module, a model training module, a standard image generation module, an endpoint target box output module, a coordinate transformation module, a pose planning module, and an execution control module.

[0031] The image acquisition module is used to acquire images of the welded workpiece;

[0032] The data annotation module is used to annotate the two ends of the weld seam in the image of the welded workpiece.

[0033] The model training module is used to train the endpoint detection model;

[0034] The standard image generation module is used to perform image preprocessing and perspective correction on the image of the welded workpiece to obtain a standard image of a set size.

[0035] The endpoint target box output module is used to input the standard image into the trained endpoint detection model to obtain two endpoint target boxes, and the center point of each target box is used as the pixel coordinate of the weld endpoint.

[0036] The coordinate transformation module is used to convert the endpoint pixel coordinates into spatial points in the robot's base coordinate system;

[0037] The pose planning module is used to determine the weld direction vector based on two spatial points, generate an arc-starting preparation point before the starting point, construct a posture constraint that aligns the welding torch travel direction with the weld direction, solve the tool posture, and obtain the pose of the arc-starting preparation point.

[0038] The execution control module is used to guide the mechanical arm to move to the arc starting point.

[0039] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0040] (1) The existing two-dimensional vision is mainly dependent on filtering, edge detection and template matching to obtain edges or center lines, and then fitting end points, which is easy to be affected by light, splashing and threshold parameters, and the visual output is usually difficult to be directly converted into executable six-dimensional pose; although some demonstration-free systems provide a framework of positioning, planning and execution, the end point extraction method and coordinate / pose conversion link are often described generally, and more secondary calibration and parameter adjustment are still needed in engineering landing, the present application determines the end points on both sides of the weld on each workpiece image through the data labeling strategy of the end point fixed size labeling box, and draws a square labeling box with a fixed side length with the end point as the center, so that the center point of the target box output by reasoning can be directly used as the weld end point coordinate, thereby the end point positioning is concretized as reading the center point of the detection box, the post-processing such as key point regression and edge fitting is significantly reduced, the end point supervision and output form are more determined, and the stability and consistency of the end point positioning are improved, and the algorithm link complexity is reduced.

[0041] (2) The present application normalizes the image perspective through homography perspective transformation, establishes a mapping model of image coordinates to robot base coordinates, maps the end point space coordinates, determines the weld direction from the two end points and generates an arc starting preparation point, solves the pose constraint of aligning the weld direction with the welding gun advancing direction, outputs the welding pose and trajectory which can be directly executed, forms a complete automatic conversion link from two-dimensional detection results to robot executable six-dimensional pose and welding trajectory, and can cooperate with a line laser to realize accurate positioning before welding and tracking compensation during welding, improve the welding precision, reduce the cost of on-site secondary development and parameter adjustment, and improve the deployment efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 FIG. 1 is a flowchart of the two-dimensional vision detection-based demonstration-free robot welding pose planning method of the present application. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application, and are not used to limit the present application.

[0044] Example 1

[0045] As Figure 1As shown, the embodiment provides a two-dimensional visual detection based teaching-free robot welding pose planning method. The method uses a two-dimensional camera to collect welding workpiece images. To improve the accuracy and stability of the welding seam endpoint positioning, a labeling strategy for welding seam endpoint positioning is proposed to label the collected welding workpiece images. Then, the endpoint detection model YOLO is used to identify the key features of the welding seam. In combination with camera calibration and coordinate mapping, the key points of the welding seam in the image coordinates are converted into three-dimensional position points in the robot base coordinate system. Subsequently, through welding seam direction constraint and pose constraint solving, the welding starting point, welding direction and six-dimensional pose of the welding torch are automatically generated, and an executable welding motion trajectory is generated. Finally, it is issued to the robot controller for execution. Specifically, it includes the following steps:

[0046] In the offline preparation stage:

[0047] S1: Collecting welding area images and establishing a sample library;

[0048] S2: Labeling the two endpoints of the welding seam according to the set labeling strategy, specifically including:

[0049] (1) Data collection: The two-dimensional camera uses the placement mode of eyes outside the hand. After fixing the workpiece placement area and the camera support position, start collecting workpiece images. To cover the actual working conditions, the collected data should include samples with different light, different surface reflectivity and different placement angles;

[0050] (2) Endpoint determination: For each workpiece image, determine the two endpoints of the welding seam in the image coordinate system. The endpoints can be manually labeled, semi-automatically assisted or determined based on prior rules, but the final endpoint pixel coordinates 、 are used as the labeling benchmark;

[0051] (3) Fixed size endpoint box labeling: Draw a square box with a fixed side length centered on each endpoint, with the side length set to 30 pixels (which can also be adjusted to other fixed values according to the camera resolution and welding seam width). Get two endpoint boxes;

[0052] Endpoint box 1: center , width and height are both 30 pixels;

[0053] Endpoint box 2: center , width and height are both 30 pixels;

[0054] When the endpoint is close to the image boundary, the endpoint box can be boundary cropped or the center point remains unchanged, and the box range is adjusted to the inside of the image to ensure the effectiveness of the labeling box.

[0055] (4) Category definition and annotation format: both endpoints are annotated as weld endpoints, and the two target box center points obtained in the reasoning stage are the two endpoint coordinates. The annotation format can adopt the YOLO standard format, including category, normalized center coordinates, normalized width and height, that is:

[0056] ;

[0057] wherein, represents the category, represents the normalized coordinates of the target box center point, represents the pixel coordinates of the target box center point, and,

[0058] S3: training of the endpoint detection model to obtain an endpoint detection weight file;

[0059] The endpoint detection model of the embodiment is a single-stage target detection network YOLO.

[0060] S4: completion of camera calibration, perspective correction matrix, and image-to-base coordinate mapping parameter calibration;

[0061] In the embodiment, the two endpoint coordinates can also be directly output by a key point regression network (heat map regression, corner detection, etc.).

[0062] Online running stage:

[0063] S1: initialization and communication establishment: loading of model weight, perspective correction matrix, mapping parameter, establishment of connection with the camera, robot controller, and line laser sensor;

[0064] S2: two-dimensional camera acquisition of workpiece image;

[0065] S3: image preprocessing and perspective correction to obtain a fixed size standard image;

[0066] In the embodiment, the original image is transformed to a unified perspective and fixed size based on homography perspective transformation Homography, for example, the output is a standard image of The perspective matrix is obtained from a calibration point or a calibration board and is saved for direct online calling. This step ensures that the endpoint detection model works in a stable scale and perspective, improving detection consistency and generalization ability.

[0067] S4: endpoint detection: input of the standard image to the endpoint detection model YOLO, output of two endpoint target boxes, and reading of each target box center point as a weld endpoint pixel coordinate;

[0068] ​​In this embodiment, the endpoint detection model YOLO outputs a set of endpoint boxes. Due to the targeted data labeling method, the target box center point can be directly read in the inference stage As endpoint coordinates, when multiple candidate endpoint boxes are detected, the top two items can be sorted according to the confidence, or combined with spatial position constraints (such as an endpoint distance threshold, and the endpoints should be located at the two ends of the weld seam area) for screening to improve stability.

[0069] S5: Endpoint coordinate mapping: converting the endpoint pixel coordinates into spatial points in the robot base coordinate system (at least two points , , ) are obtained;

[0070] In this embodiment, the endpoint pixel coordinates are first normalized, and then linearly mapped using a scale factor and a bias to obtain , , and is taken as the workpiece plane height or the calibration height. Essentially, it is a linear mapping from 2D to planar 3D, which is suitable for welding scenarios of planar workpieces. Since the height in the axis direction is set in advance, a line laser sensor is introduced for scanning and positioning to find the accurate welding starting point, reducing errors. However, since the pose of the arc initiation preparation point has been previously calculated, there is no need for manual dragging of the robot arm to the vicinity of the welding endpoint for scanning. The robot arm can be automatically guided to the arc initiation preparation point, improving the precision and efficiency of automatic welding.

[0071] S6: Weld seam direction and trajectory key point generation: determining the weld seam direction vector according to the two spatial points, and generating an arc initiation preparation point before the starting point. The arc initiation preparation point is a weld seam starting point located by two-dimensional image recognition, which is retreated a certain distance along the weld seam direction, serving as the starting point for pre-scanning by the line laser sensor.

[0072] In this embodiment, let the two spatial points of the weld seam be , , then the weld seam direction unit vector is:

[0073] ;

[0074] To ensure the smoothness of arc initiation and entry into the weld seam, retreat a distance of (e.g., 100 mm) along the opposite direction of the weld seam before the welding starting point, and then raise a distance of along the opposite direction of the axis to generate a preparation point:

[0075] ;

[0076] It can further generate transition points such as approach point and retreat point to form a complete trajectory point sequence, which are: photo point, approach point, arc preparation point, welding start point, welding end point, and retreat point.

[0077] S7: Attitude planning: Construct attitude constraints to align the welding torch travel direction with the weld direction, solve the tool attitude in combination with welding process requirements, and obtain the arc-starting preparation point pose.

[0078] In this embodiment, the attitude constraint ensures that the tool or sensor coordinate system satisfies the following condition: the x-axis is along the weld direction. , The axis should be aligned as closely as possible with the direction of the base coordinate system. (usually taken) The x-axis is reversed and orthogonal to the x-axis; the y-axis is obtained by the cross product, forming a right-handed system.

[0079] S8: Trajectory Generation and Execution: Guides the robotic arm to the arc initiation point, uses line laser scanning to determine the precise arc initiation point, and uses line laser weld seam tracking to achieve precise welding;

[0080] S9: End and Reset: Stop tracking (if any), log the information, the robot returns to the safe position and exits automatic mode.

[0081] In this embodiment, visual coarse positioning is combined with pre-scanning before line laser welding and tracking compensation during welding. The visual system plans the arc preparation point and automatically guides the robot to the position. Based on this, the line laser obtains the precise arc starting point and performs online correction, realizing a two-stage closed loop of coarse and fine positioning. Compared with relying solely on teaching trajectories or a single sensing scheme, this can improve welding accuracy and robustness, while further reducing the dependence on teaching by skilled personnel.

[0082] This invention achieves the automatic generation and distribution of welding start point, welding direction and welding torch pose by stably acquiring key features of the weld and completing coordinate mapping and attitude constraint solution, thereby improving the automation level of welding pose planning without increasing hardware complexity.

[0083] Example 2

[0084] This embodiment provides a teachless robot welding pose planning system based on two-dimensional vision detection, which is used to implement the teachless robot welding pose planning method based on two-dimensional vision detection in Embodiment 1. The system includes: an image acquisition module, a data annotation module, a model training module, a standard image generation module, an endpoint target box output module, a coordinate transformation module, a pose planning module, and an execution control module.

[0085] In this embodiment, the image acquisition module is used to acquire images of the welded workpiece;

[0086] In the embodiment, the data labeling module is configured to label the two endpoints of the weld seam in the welding workpiece image, so that the center point of the target frame output by the endpoint detection model corresponds to the endpoint coordinate of the weld seam, thereby converting the weld seam endpoint positioning into reading of the center point of the target detection frame, reducing the complexity of post-processing and improving the consistency of endpoint positioning, and then performing data enhancement methods such as cropping, flipping, and increasing exposure rate on the labeled image to improve the diversity of the data and enhance the generalization ability of the model.

[0087] In the embodiment, the model training module is configured to train the endpoint detection model,

[0088] In the embodiment, the standard image generation module is configured to perform image preprocessing and perspective correction on the welding workpiece image to obtain a standard image with a set size.

[0089] Specifically, the image is corrected in perspective by using a homography perspective transformation, so that the welding area is expressed under a fixed perspective and a fixed scale, to improve the detection stability and the subsequent mapping accuracy, and to establish a mapping relationship between the image coordinate system and the robot base coordinate system, to convert the two-dimensional key points to three-dimensional positions in the base coordinate system.

[0090] In the embodiment, the endpoint target frame output module is configured to input the standard image into the trained endpoint detection model to obtain two endpoint target frames, and to take each center point of the target frame as a weld seam endpoint pixel coordinate.

[0091] In the embodiment, the coordinate conversion module is configured to convert the endpoint pixel coordinate into a spatial point in the robot base coordinate system.

[0092] In the embodiment, the pose planning module is configured to determine a weld seam direction vector according to the two spatial points, and to generate an arc striking preparation point in front of the starting point, to construct a pose constraint of aligning the welding gun travel direction with the weld seam direction, to solve the tool pose, and to obtain the pose of the arc striking preparation point.

[0093] In the embodiment, the execution control module is configured to guide the movement of the mechanical arm to the arc striking point, and to issue robot motion instructions and welding process instructions through an SDK or a controller API.

[0094] In the embodiment, a line laser sensor is further provided to perform accurate alignment and online tracking compensation in the execution stage, so as to realize stable welding without teaching, search and center the weld seam before welding, and track and compensate during welding, thereby improving the adaptability to assembly errors and thermal deformation.

[0095] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited by the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods and shall be included in the protection scope of the present application.

Claims

1. A method for teaching-free robot welding pose planning based on two-dimensional vision detection, characterized in that, Includes the following steps: Acquire images of the welded workpiece; Annotate the two ends of the weld seam in the image of the welded workpiece and train the endpoint detection model; Image preprocessing and perspective correction are performed on the image of the welded workpiece to obtain a standard image of a set size; The standard image is trained by the endpoint detection model, which outputs two endpoint bounding boxes. The center point of each bounding box is used as the pixel coordinate of the weld endpoint. Convert the endpoint pixel coordinates to spatial points in the robot's base coordinate system; The weld direction vector is determined based on two spatial points, and an arc-starting preparation point is generated before the starting point. Construct attitude constraints to align the welding torch's travel direction with the weld seam direction, solve for the tool's attitude, obtain the arc-starting preparation point pose, and guide the robotic arm to move to the arc-starting point.

2. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, Mark the two ends of the weld on the image of the welded workpiece, specifically including: For each workpiece image, determine the endpoints on both sides of the weld in the image coordinate system; Draw a square frame with a set side length centered on each endpoint to obtain two endpoint frames. Use the center point of the two target frames as the coordinates of the endpoints.

3. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, Image preprocessing and perspective correction are performed on the image of the welded workpiece. Specifically, homography perspective transformation is used to transform the original image to a unified viewpoint and fixed size.

4. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, The endpoint detection model uses a single-stage target detection network. The endpoint detection model outputs a set of endpoint boxes. When multiple candidate endpoint boxes are detected, the top two are selected according to their confidence scores, or two endpoint boxes are selected by combining spatial location constraints.

5. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, The coordinates of the two ends of the weld are directly output using a key point regression network.

6. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, Converting endpoint pixel coordinates to spatial points in the robot's base coordinate system specifically includes: The endpoint pixel coordinates are normalized, and a linear mapping based on the scaling factor and the offset is obtained. , axis coordinate points, and The axis coordinate point is taken as the workpiece plane height or the calibration height.

7. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, The weld direction vector is determined based on two spatial points, and an arc-starting preparation point is generated before the starting point, specifically including: Let the spatial points at both ends of the weld be... , Then the unit vector in the weld direction is: ； The distance the weld seam retracts in the opposite direction at the welding start point. , and then along The shaft is raised in the opposite direction by a certain distance. Generate the arc-starting preparation point: ； in, Indicates the welding start point. This indicates the preparation point for arc initiation.

8. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, Constructing attitude constraints to align the welding torch travel direction with the weld direction, specifically including: Attitude constraints ensure that the tool or sensor coordinate system satisfies the following condition: the x-axis is along the weld direction. The axes are opposite in direction to the base coordinate system and orthogonal to the x-axis. The y-axis is obtained by the cross product, forming a right-handed coordinate system.

9. The method for welding pose planning of a teachless robot based on two-dimensional vision detection according to claim 1, characterized in that, Guiding the robotic arm to move to the arc initiation point also includes: determining the arc initiation point based on line laser scanning, and performing weld seam tracking compensation based on line laser.

10. A teach-free robot welding pose planning system based on two-dimensional vision detection, characterized in that, The method for implementing the welding pose planning method for a teachless robot based on two-dimensional vision detection as described in any one of claims 1-9 includes: an image acquisition module, a data annotation module, a model training module, a standard image generation module, an endpoint target box output module, a coordinate transformation module, a pose planning module, and an execution control module. The image acquisition module is used to acquire images of the welded workpiece; The data annotation module is used to annotate the two ends of the weld seam in the image of the welded workpiece. The model training module is used to train the endpoint detection model; The standard image generation module is used to perform image preprocessing and perspective correction on the image of the welded workpiece to obtain a standard image of a set size. The endpoint target box output module is used to input the standard image into the trained endpoint detection model to obtain two endpoint target boxes, and the center point of each target box is used as the pixel coordinate of the weld endpoint. The coordinate transformation module is used to convert the endpoint pixel coordinates into spatial points in the robot's base coordinate system; The pose planning module is used to determine the weld direction vector based on two spatial points, generate an arc-starting preparation point before the starting point, construct a posture constraint that aligns the welding torch travel direction with the weld direction, solve the tool posture, and obtain the pose of the arc-starting preparation point. The execution control module is used to guide the robotic arm to move to the arc initiation point.

Images