Method for initial positioning of welding arc starting point based on machine vision and autonomous guiding of robot
By using machine vision methods based on RGB-D cameras, rapid and accurate positioning and autonomous guidance of the welding start point were achieved, solving the problem of existing welding robots relying on manual teaching and prior models, and improving the flexibility and stability of welding production.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTH CHINA UNIV OF TECH
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-29
AI Technical Summary
Existing welding robot starting point positioning schemes rely on manual teaching, which is inefficient and inconsistent. They are difficult to adapt to the clamping errors and thermal deformation of small batches of multi-variety and complex workpieces. Furthermore, they rely too heavily on prior models, which affects flexible manufacturing capabilities.
A machine vision method based on an RGB-D camera is adopted. By acquiring RGB images and depth maps of the workpiece surface, semantic segmentation and point cloud construction are performed to extract candidate areas of the weld. Geometric feature calculation and fitting are performed to generate the initial welding trajectory and arc starting point position, guiding the robot to move to the vicinity of the candidate arc starting point.
It enables rapid and accurate positioning of the arc initiation point without the need for complex scanning trajectories and strong prior models, improving system flexibility and adaptability to clamping deviations and workpiece position changes, reducing reliance on offline programming, and improving the stability and efficiency of welding production.
Smart Images
Figure CN122115557A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent robotic welding technology, specifically to a method for preliminary positioning of the welding arc initiation point and autonomous robot guidance based on machine vision. Background Technology
[0002] Currently, in automated welding production lines, welding robots still primarily rely on manual teaching: operators provide the welding torch's trajectory and process posture via a teach pendant or offline programming, and the robot repeatedly executes the predetermined trajectory during production. This method offers high stability for welding large batches of standardized parts with consistent structures. However, when workpieces are produced in small batches, have multiple varieties, complex structures, or are subject to disturbances such as clamping errors or thermal deformation, the taught trajectory struggles to adapt to the actual working conditions. The teaching process is highly dependent on human experience and is easily affected by factors such as workpiece clamping deviations, end effector calibration errors, and workpiece thermal deformation, leading to accumulated deviations between the arc initiation point and the weld trajectory. Because traditional teach-based welding lacks online sensing capabilities for workpiece status and deviations, it is difficult to achieve real-time compensation and dynamic adjustment of deviations, thus limiting the flexibility and intelligence of welding production.
[0003] With the development of machine vision technology, vision sensors are gradually being introduced into welding robot systems. These sensors utilize image and point cloud information to achieve workpiece recognition, weld seam recognition, and guidance control, thereby reducing reliance on manual teaching. However, some existing welding start-point (arc ignition point) positioning schemes rely on pre-imported workpiece CAD models or preset scanning trajectories, and use line laser sensors for line scanning positioning. This type of scheme typically has the following shortcomings: First, the sensing range of line scanning is relatively limited, often requiring scanning path planning, which affects positioning efficiency; second, its adaptability to working conditions such as workpiece placement changes, occlusion, spatter, and arc light interference is limited; third, the system's reliance on prior models or fixed processes reduces flexible manufacturing capabilities.
[0004] Existing technologies use a combination of 2D vision sensors and 3D cameras to identify and track weld seams. When 3D point cloud acquisition is limited, the system switches to a 2D module and uses line laser scanning for initial weld point location. This approach may still require additional line scanning procedures and multi-module collaboration in some scenarios. Therefore, there is still room for optimization in terms of further simplifying the rapid arc-starting point location process, reducing system integration complexity, and decreasing reliance on prior information such as models.
[0005] Therefore, there is still an urgent need for a solution that can achieve preliminary positioning of the arc initiation point based on an RGB-D camera without the need for pre-setting complex scanning trajectories or relying on strong prior models, and guide the robot to quickly reach the vicinity of the candidate arc initiation point, so as to provide reliable initial values for subsequent accurate positioning and weld seam tracking. Summary of the Invention
[0006] To overcome the problems of low efficiency and poor consistency in the existing welding robot arc starting point positioning process, which relies on manual teaching, this invention provides a machine vision-based method for preliminary positioning of the welding arc starting point and autonomous robot guidance. This invention uses machine vision based on an RGB-D camera to achieve preliminary positioning of the welding arc starting point and autonomous robot movement guidance. By identifying the workpiece and weld, its geometric information is obtained, an initial welding posture that meets the welding requirements is generated, and the robot is guided to move to the vicinity of the preliminary positioning position of the arc starting point, thereby providing reliable initial conditions and basic data for subsequent precise positioning of the arc starting point and weld tracking.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] This invention provides a method for preliminary positioning of the welding arc initiation point and autonomous robot guidance based on machine vision, comprising the following steps:
[0009] Acquire RGB images and depth maps of the workpiece surface to generate point clouds aligned with RGB pixels;
[0010] Semantic segmentation is performed on the RGB image of the workpiece surface to obtain multiple surface masks;
[0011] Based on the geometric relationship between the two surfaces of the workpiece, a candidate strip region for weld seam is constructed in a two-dimensional mask space;
[0012] The candidate strip region of the weld is mapped to a 3D point cloud to obtain the point cloud of the candidate weld region. The point cloud is then clipped and surface semantic labels are added to the point cloud to obtain a semantic point cloud.
[0013] Point cloud preprocessing and geometric feature calculation are performed to obtain surface patches. Edges are extracted based on the boundaries of the surface patches to obtain a set of edge candidate points.
[0014] Extract the weld point set from the edge candidate point set;
[0015] The initial welding trajectory point sequence and arc starting point position are generated based on geometric fitting and trajectory smoothing constraints;
[0016] Generate a welding torch pose for each welding trajectory point, convert the welding torch pose to the robot coordinate system, output the arc starting point and initial guide trajectory, and guide the robot to move to the arc starting point.
[0017] As a preferred technical solution, a candidate strip region for weld seam is constructed in a two-dimensional mask space based on the geometric relationship between the two surfaces of the workpiece, specifically represented as follows:
[0018] ;
[0019] ;
[0020] ;
[0021] in, Indicates the candidate strip area for weld seam. This represents the bandwidth retained along the interior of the surface. This represents the threshold for moving closer to another surface. Represents pixels The shortest distance from inside surface A to the boundary of surface A. Represents pixels The shortest distance from the interior of surface B to the boundary of surface B. Represents pixels The shortest distance to area A. Represents pixels The shortest distance to area B. This represents the set of pixels in area A. This represents the set of pixels in the B-side region.
[0022] As a preferred technical solution, the candidate weld zone is mapped onto a three-dimensional point cloud to obtain the candidate weld zone point cloud, which is represented as follows:
[0023] ;
[0024] in, Represents a depth map. This represents the point cloud of the candidate weld area.
[0025] As a preferred technical solution, point cloud preprocessing and geometric feature calculation are performed, specifically including:
[0026] Voxel downsampling is performed on the point cloud of the candidate weld area. The space is divided according to the voxel side length. Points inside the voxel are represented by the centroid. A KD-tree index is constructed to support k-nearest neighbor search.
[0027] For any point Take its k nearest neighbor Calculate the covariance matrix:
[0028] ;
[0029] Find eigenvalues The corresponding minimum eigenvector is used as the normal estimate, and a curvature index is defined:
[0030] ;
[0031] in, This indicates the curvature index.
[0032] As a preferred technical solution, a surface patch is obtained, and edges are extracted based on the boundaries of the surface patch to obtain a set of candidate edge points, specifically including:
[0033] Perform region growing on the point cloud, where the angle between the normal of a neighboring point and the current region is less than a threshold. And the curvature is less than the threshold The current surface patch is then incorporated to form a set of multiple surface patches.
[0034] The boundary points between surface patches and those exceeding the threshold The curvature points converge to form an edge candidate set.
[0035] As a preferred technical solution, the weld point set is extracted from the edge candidate point set, specifically including:
[0036] The constraint is set as follows: curvature exceeds a threshold. The weld seam point set is extracted from the edge candidate point set, either in the high quantile interval or in the semantic label neighborhood of the point that simultaneously contains both side A and side B.
[0037] As a preferred technical solution, the initial welding trajectory point sequence and arc initiation point position are generated based on geometric fitting and trajectory smoothing constraints, specifically including:
[0038] When the weld seam is approximately located in the same plane, random sampling consistency is used to fit the global plane.
[0039] When the weld is a spatial curve, a sliding window is used to perform local plane fitting on the trajectory segment to obtain a piecewise coordinate system and splice it to generate the trajectory.
[0040] Using the plane unit normal as Principal component analysis was performed on the weld joint, and the first principal direction was taken and projected onto the plane to obtain the result. Then by Construct a right-handed orthogonal coordinate system, with reference point Using the origin as the coordinate point, perform coordinate transformation on the point;
[0041] The weld points are sorted according to the principal direction parameters, and a continuous curve is obtained by cubic spline fitting. Then, path points are generated by uniform sampling according to the arc length and a set step size. The path points are mapped back to three dimensions through inverse transformation to obtain the three-dimensional initial welding trajectory point sequence. The arc starting point is taken as the trajectory first point or the optimal point that satisfies the process constraints.
[0042] As a preferred technical solution, the welding torch pose is generated for each welding trajectory point, specifically including:
[0043] The tangent is obtained by the difference between adjacent points. The mean value of the neighborhood normal of the weld point cloud is taken as the local normal. The constraints of the welding torch axis and the forward direction are determined. An orthogonal tool coordinate system is constructed to form a rotation matrix to output the pose matrix, or output the position and Euler angles.
[0044] As a preferred technical solution, the welding torch pose is transformed to the robot coordinate system, specifically as follows:
[0045] ;
[0046] in, , , Let represent the coordinate system transformation matrices of base B, camera C, and trajectory point i, respectively.
[0047] This invention also provides a machine vision-based preliminary positioning of welding arc starting point and autonomous robot guidance system, used to realize the above-mentioned machine vision-based preliminary positioning of welding arc starting point and autonomous robot guidance method, including: a data acquisition module, a semantic segmentation module, a weld seam candidate strip region construction module, a semantic point cloud construction module, an edge candidate point set construction module, a weld seam point set extraction module, a welding trajectory generation module, a welding torch pose generation module, and an execution module.
[0048] The data acquisition module is used to acquire RGB images and depth maps of the workpiece surface and generate point clouds aligned with RGB pixels.
[0049] The semantic segmentation module is used to perform semantic segmentation on the RGB image of the workpiece surface to obtain multiple surface masks;
[0050] The weld seam candidate strip region construction module is used to construct weld seam candidate strip regions in a two-dimensional mask space based on the geometric relationship between the two surfaces of the workpiece.
[0051] The semantic point cloud construction module is used to map the candidate strip region of the weld seam to a three-dimensional point cloud, obtain the point cloud of the candidate weld seam region, and trim it. Then, surface semantic labels are added to the point cloud to obtain the semantic point cloud.
[0052] The edge candidate point set construction module is used to perform point cloud preprocessing and geometric feature calculation, obtain surface patches, extract edges based on the boundaries of surface patches, and obtain the edge candidate point set;
[0053] The weld seam point set extraction module is used to extract the weld seam point set from the edge candidate point set;
[0054] The welding trajectory generation module is used to generate an initial welding trajectory point sequence and arc starting point position based on geometric fitting and trajectory smoothing constraints.
[0055] The welding torch pose generation module is used to generate the welding torch pose for each welding trajectory point.
[0056] The execution module is used to convert the welding torch pose to the robot coordinate system, output the arc starting point and initial guide trajectory, and guide the robot to move to the arc starting point.
[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0058] (1) In the initial positioning stage of the arc starting point, the present invention uses a single RGB-D sensor as the main information source, constructs a semantic point cloud mask by 2D semantic segmentation to 3D point cloud mapping, and completes the extraction of weld candidate areas, calculation of geometric features and generation of initial pose of arc starting point within a limited ROI. The process is more compact, and line laser and other methods can be used as optional enhancements for subsequent fine positioning.
[0059] (2) The present invention constructs identification, coarse localization, posture generation and motion guidance into a continuous link driven by the same source sensor data, which can directly output the initial value of the candidate pose of the arc starting point for robot guidance, reduce the dependence on offline programmable external planning modules, and reduce the data link integration cost.
[0060] (3) The present invention has a lower dependence on external prior information. In the initial positioning stage, it can generate candidate arc starting points without relying on strong prior information such as workpiece CAD model and weld type template, providing stable initial values for subsequent accurate positioning and weld tracking, and improving adaptability and consistency to working conditions such as clamping deviation and workpiece position change. Attached Figure Description
[0061] Figure 1 This is a flowchart illustrating the preliminary positioning of the welding arc initiation point and the autonomous robot guidance method based on machine vision according to the present invention. Detailed Implementation
[0062] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0063] Example 1
[0064] like Figure 1As shown, this embodiment provides a machine vision-based method for preliminary positioning of the arc initiation point in welding and autonomous robot guidance. This method uses color images and point cloud data acquired by an RGB-D camera as input. By identifying and segmenting the workpiece surface and weld area, semantic information of key workpiece surfaces is obtained and fused with the 3D point cloud to form a corresponding semantic point cloud region. Based on this, the target region point cloud is cropped, downsampled, and segmented to extract geometric feature points such as weld edges / boundaries. Further, combining geometric modeling and trajectory generation strategies, a continuous and smooth initial welding trajectory that meets welding requirements is constructed, obtaining candidate positions of the arc initiation point and corresponding initial welding poses (position and attitude). Through this method, the welding robot can be guided to move quickly to the vicinity of the candidate arc initiation point, providing reliable initial values and constraints for subsequent precise arc initiation point positioning and weld tracking. Preferably, it can be further combined with devices such as line laser sensors to achieve higher-precision online positioning and tracking. Compared with positioning methods that rely on workpiece models or preset scanning trajectories, this embodiment achieves preliminary arc initiation point positioning without relying on complex prior models, improving system flexibility and deployment efficiency. Specifically, it includes the following steps:
[0065] S1: Data Acquisition and Alignment: Acquire RGB images and depth maps, and generate aligned point clouds;
[0066] In this embodiment, the RGB-D camera acquires a frame of color image of the workpiece. and depth map Invalid pixels (depth of 0 or null) are removed or interpolated for repair, and the results are obtained using the camera intrinsic parameter matrix. The pixel back-projection onto the camera coordinate system point cloud is represented as:
[0067] ;
[0068] Obtaining point clouds And maintain a one-to-one correspondence between RGB and point cloud to support subsequent semantic fusion;
[0069] Of course, the RGB-D camera in this embodiment can also be replaced by any sensor or combination of solutions that can acquire depth information or three-dimensional point clouds (such as structured light depth camera, binocular stereo vision, 2D camera combined with depth estimation or multi-frame fusion reconstruction, etc.). The input form can be a depth map or a point cloud, both of which can achieve the same three-dimensional positioning effect of the weld area.
[0070] S2: Semantic segmentation of workpiece surface: Obtaining the surface mask set based on the lightweight segmentation model MobileSAM;
[0071] In this embodiment, an RGB image is input into the lightweight segmentation model MobileSAM, and several surface mask sets are obtained by combining the prompt. The prompt can be generated manually by clicking the seed, or automatically generated by key points in the workpiece area detected by YOLOv8.
[0072] To improve stability, the mask set is filtered and denoised, including area thresholding, connected component preservation, hole filling, and boundary smoothing, ultimately yielding the target surface mask (such as the A-side mask) for boundary analysis. B-side mask );
[0073] S3: Construction of weld candidate zones under boundary constraints: Construct weld candidate regions in 2D mask space and assign A-side / B-side labels. The A-side / B-side labels are used to indicate which surface a point / pixel belongs to on either side of the boundary, and are used to screen weld points for boundary consistency.
[0074] In this embodiment, to construct a candidate weld zone near the interface between two surfaces in 2D space, this embodiment generates a strip-shaped region based on a distance field. That is, the shortest Euclidean distance from the pixel to the target region / boundary is converted into a distance map to generate the strip-shaped region. Specifically, the pixel is regarded as a point. Calculate using Euclidean distance transformation:
[0075] : pixel The shortest distance from the interior of surface A to the boundary of surface A (representing the interior region near the boundary of surface A);
[0076] : pixel The shortest distance to area B (reflecting proximity to area B).
[0077] Based on this construction:
[0078] ;
[0079] ;
[0080] ;
[0081] Among them, parameters This represents the bandwidth retained along the interior of the surface. This represents the threshold for moving closer to another surface, which is used for subsequent boundary consistency judgments. and Assign labels to side A and side B respectively.
[0082] S4: Back projection fusion to generate semantic point cloud: Map the 2D weld seam candidate strip to the 3D point cloud and trim it, add surface semantic labels to the point cloud to obtain semantic point cloud;
[0083] In this embodiment, the set of masked pixels of the weld candidate area is back-projected into the point cloud to obtain the point cloud of the weld candidate area:
[0084] ;
[0085] At the same time, based on the pixel belonging or Add semantic tags to point clouds To form a semantic point cloud This provides prior knowledge of both sides of the weld surface for subsequent weld refining.
[0086] S5: Point cloud preprocessing and geometric feature calculation: downsampling, KD-tree indexing, normal and curvature estimation;
[0087] In this embodiment, the point cloud of the weld candidate region is... To improve efficiency, voxel downsampling is performed by dividing the space into grid cells and replacing voxel points with representative points to reduce the number of points, based on voxel edge length. The space is partitioned, with points within voxels represented by their centroids, and then a KD-tree index is constructed to support k-nearest neighbor search.
[0088] For any point Take its k nearest neighbor Calculate the covariance matrix:
[0089] ;
[0090] Finding eigenvalues from the covariance matrix The variance of these k points is represented by the degree of dispersion in various directions. The variance in the direction perpendicular to the plane is the smallest, so the minimum eigenvector corresponding to this eigenvalue is used as the normal estimate, that is, the unit vector estimate of the vertical direction of the local surface, and the curvature index is defined:
[0091] ;
[0092] in, The larger the value, the more likely the local area is to be an edge or a geometric abrupt change.
[0093] S6: Surface patch region growth, segmentation, and edge extraction: Obtain surface point sets and edge point sets;
[0094] In this embodiment, it is preferable to first obtain the surface patches and then extract the edges from the intersection of the surface patches, which better matches the physical structure of the weld being located at the intersection of two surfaces. Specifically:
[0095] Use low curvature points as seed set Perform region growing on the point cloud. When the angle between the normal of the neighboring points and the current region is less than a threshold... And the curvature is less than the threshold When merged into the current surface patch;
[0096] Forming multiple surface plate assemblies ;
[0097] The boundary points between surface patches and the high curvature points are grouped into an edge candidate set. ;
[0098] Among them, threshold , The parameters can be adaptively set based on point cloud statistics (e.g., taking the curvature quantile or the mean ± standard deviation range of the normal angle distribution) to avoid the inefficiency of fixed thresholds for different workpiece scales.
[0099] S7: Weld point set refinement: Extracting fine weld point sets by combining high curvature and consistency of the interface between two surfaces;
[0100] In this embodiment, from the edge candidate set Extracting fine weld point sets At least the following two conditions must be met:
[0101] Condition 1 (Geometric Abruptness): Curvature Greater than the threshold or in the high quantile range;
[0102] Condition 2 (Boundary Consistency): The semantic label neighborhood of a point simultaneously includes both side A and side B (or is small in distance from both surface patches) to exclude isolated noise edges that are not weld seams.
[0103] This results in a more stable set of weld points. ;
[0104] S8: Initial Welding Trajectory Generation: Planar modeling, coordinate transformation, spline fitting, and uniform sampling to obtain 3D trajectory points, specifically including:
[0105] (1) Planar / local planar modeling:
[0106] When the entire weld seam is approximately located in the same plane (in the case of fillet welds / lap welds), the global plane is fitted using Random Sampling Consensus (RANSAC). .
[0107] When the weld is a spatial curve, a sliding window is used to perform local plane fitting on the trajectory segment to obtain a segmented coordinate system and splice it to generate the trajectory.
[0108] (2) Coordinate system construction and point set flattening:
[0109] Using the plane unit normal as Principal component analysis (PCA) was performed on the weld joint, and the first principal direction was taken and projected onto the plane to obtain the desired result. Then by Construct a right-handed orthogonal coordinate system, with reference point (If the centroid of a point set is the origin, perform coordinate transformations on the points:)
[0110] ;
[0111] in, This represents a 3×3 rotation matrix used to rotate a point from the camera coordinate system to the planar coordinate system. The above formula first translates the point, then aligns and rotates it to ensure that the weld point satisfies the following conditions in the planar coordinate system. Only two-dimensional coordinates are needed Perform curve fitting.
[0112] (3) Spline fitting and arc length sampling:
[0113] The weld points are sorted according to the principal direction parameters (or the shortest path sequence), and a continuous curve is obtained by cubic spline fitting. Then, according to the arc length in step size... Uniform sampling generates path points Finally, it is mapped back to 3D through an inverse transformation:
[0114] ;
[0115] in, The transpose of the rotation matrix yields the sequence of initial three-dimensional welding trajectory points. The starting point of the arc can be taken as the first point of the trajectory or the optimal point that satisfies the process constraints (accessibility, obstacle avoidance).
[0116] S9: Welding torch pose generation and coordinate system transformation: Generates the welding torch pose and transforms it to the robot coordinate system, outputting the arc starting point and initial guide path.
[0117] For each trajectory point Generate the executable pose of the welding torch. First, obtain the tangential direction by differential analysis of adjacent points:
[0118] ;
[0119] in, The vector is normalized, and then the mean of the neighborhood normals of the weld point cloud near that point is taken as the local normal. Adjustable parameters are introduced based on the welding process: working angle Lean angle muzzle height Safety bias This determines the welding torch axis. (Pointing to the workpiece) and the direction of travel constraints, construct an orthogonal tool coordinate system. Forming a rotation matrix Output a 4×4 pose matrix:
[0120] ;
[0121] Alternatively, output position and Euler angles (Roll / Pitch / Yaw).
[0122] Finally, the hand-eye calibration was obtained. Transform the pose from the camera coordinate system to the robot base coordinate system:
[0123] ;
[0124] in, , , Let represent the coordinate system transformation matrices of base B, camera C, and trajectory point i, respectively;
[0125] Output the starting point and initial guide trajectory that the robot can directly execute.
[0126] This invention utilizes color images and depth point clouds acquired by a single RGB-D camera. It introduces a large visual model, first segmenting the workpiece surface / weld candidate region in the 2D image domain. The 2D segmentation results are then mapped to a 3D point cloud using camera intrinsic and extrinsic parameters, forming a semantic point cloud mask to limit the subsequent point cloud processing range. Within the semantic point cloud, the point cloud is cropped, downsampled, and segmented (e.g., adaptive region growing based on KD-tree neighborhood), extracting weld edge feature points at surface boundaries / intersections. Furthermore, geometric fitting (e.g., plane / straight line / curve fitting) and trajectory smoothing constraints generate candidate arc-starting points and initial welding poses that meet welding requirements. Based on these initial welding poses, the robot is guided to move to the vicinity of the candidate arc-starting points, providing stable initial values and constraints for accurate positioning and weld tracking.
[0127] Example 2
[0128] The present invention also provides a machine vision-based preliminary positioning of welding arc starting point and autonomous robot guidance system, which is used to implement the machine vision-based preliminary positioning of welding arc starting point and autonomous robot guidance method of embodiment 1 above, including: a data acquisition module, a semantic segmentation module, a weld seam candidate strip region construction module, a semantic point cloud construction module, an edge candidate point set construction module, a weld seam point set extraction module, a welding trajectory generation module, a welding torch pose generation module, and an execution module;
[0129] In this embodiment, the data acquisition module is used to acquire RGB images and depth maps of the workpiece surface and generate point clouds aligned with RGB pixels;
[0130] In this embodiment, the semantic segmentation module is used to perform semantic segmentation on the RGB image of the workpiece surface to obtain multiple surface masks;
[0131] In this embodiment, the weld seam candidate strip region construction module is used to construct weld seam candidate strip regions in a two-dimensional mask space based on the geometric relationship between the two surfaces of the workpiece.
[0132] In this embodiment, the semantic point cloud construction module is used to map the candidate strip region of the weld seam to a three-dimensional point cloud, obtain the point cloud of the candidate weld seam region, and trim it. Then, surface semantic labels are added to the point cloud to obtain the semantic point cloud.
[0133] In this embodiment, the edge candidate point set construction module is used to perform point cloud preprocessing and geometric feature calculation, obtain surface patches, extract edges based on the boundaries of the surface patches, and obtain the edge candidate point set;
[0134] In this embodiment, the weld point set extraction module is used to extract the weld point set from the edge candidate point set;
[0135] In this embodiment, the welding trajectory generation module is used to generate an initial welding trajectory point sequence and arc starting point position based on geometric fitting and trajectory smoothing constraints;
[0136] In this embodiment, the welding torch pose generation module is used to generate the welding torch pose for each welding trajectory point;
[0137] In this embodiment, the execution module is used to convert the welding torch pose to the robot coordinate system, output the arc starting point and the initial guide trajectory, and guide the robot to move to the arc starting point.
[0138] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision, characterized in that, Includes the following steps: Acquire RGB images and depth maps of the workpiece surface to generate point clouds aligned with RGB pixels; Semantic segmentation is performed on the RGB image of the workpiece surface to obtain multiple surface masks; Based on the geometric relationship between the two surfaces of the workpiece, a candidate strip region for weld seam is constructed in a two-dimensional mask space; The candidate strip region of the weld is mapped to a 3D point cloud to obtain the point cloud of the candidate weld region. The point cloud is then clipped and surface semantic labels are added to the point cloud to obtain a semantic point cloud. Point cloud preprocessing and geometric feature calculation are performed to obtain surface patches. Edges are extracted based on the boundaries of the surface patches to obtain a set of edge candidate points. Extract the weld point set from the edge candidate point set; The initial welding trajectory point sequence and arc starting point position are generated based on geometric fitting and trajectory smoothing constraints; Generate a welding torch pose for each welding trajectory point, convert the welding torch pose to the robot coordinate system, output the arc starting point and initial guide trajectory, and guide the robot to move to the arc starting point.
2. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, Based on the geometric relationship between the two surfaces of the workpiece, a candidate strip region for weld seam is constructed in a two-dimensional mask space, specifically represented as follows: ； ； ； in, Indicates the candidate strip area for weld seam. This represents the bandwidth retained along the interior of the surface. This represents the threshold for moving closer to another surface. Represents pixels The shortest distance from inside surface A to the boundary of surface A. Represents pixels The shortest distance from the interior of surface B to the boundary of surface B. Represents pixels The shortest distance to area A. Represents pixels The shortest distance to area B. This represents the set of pixels in area A. This represents the set of pixels in the B-side region.
3. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, Mapping the candidate weld strip region onto a 3D point cloud yields the weld candidate region point cloud, represented as: ； in, Represents a depth map. This represents the point cloud of the candidate weld area.
4. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, Point cloud preprocessing and geometric feature calculation are performed, specifically including: Voxel downsampling is performed on the point cloud of the candidate weld area. The space is divided according to the voxel side length. Points inside the voxel are represented by the centroid. A KD-tree index is constructed to support k-nearest neighbor search. For any point Take its k nearest neighbor Calculate the covariance matrix: ； Find eigenvalues The corresponding minimum eigenvector is used as the normal estimate, and a curvature index is defined: ； in, This indicates the curvature index.
5. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, Obtain surface patches, extract edges based on the boundaries of the surface patches, and obtain a set of candidate edge points, specifically including: Perform region growing on the point cloud, where the angle between the normal of a neighboring point and the current region is less than a threshold. And the curvature is less than the threshold The current surface patch is then incorporated to form a set of multiple surface patches. The boundary points between surface patches and those exceeding the threshold The curvature points converge to form an edge candidate set.
6. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, Extracting the weld point set from the edge candidate point set specifically includes: The constraint is set as follows: curvature exceeds a threshold. The weld seam point set is extracted from the edge candidate point set, either in the high quantile interval or in the semantic label neighborhood of the point that simultaneously contains both side A and side B.
7. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, The initial welding trajectory point sequence and arc initiation point position are generated based on geometric fitting and trajectory smoothing constraints, specifically including: When the weld seam is approximately located in the same plane, random sampling consistency is used to fit the global plane. When the weld is a spatial curve, a sliding window is used to perform local plane fitting on the trajectory segment to obtain a piecewise coordinate system and splice it to generate the trajectory. Using the plane unit normal as Principal component analysis was performed on the weld joint, and the first principal direction was taken and projected onto the plane to obtain the result. Then by Construct a right-handed orthogonal coordinate system, with reference point Using the origin as the coordinate point, perform coordinate transformation on the point; The weld points are sorted according to the principal direction parameters, and a continuous curve is obtained by cubic spline fitting. Then, path points are generated by uniform sampling according to the arc length and a set step size. The path points are mapped back to three dimensions through inverse transformation to obtain the three-dimensional initial welding trajectory point sequence. The arc starting point is taken as the trajectory first point or the optimal point that satisfies the process constraints.
8. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, For each welding trajectory point, the welding torch pose is generated, specifically including: The tangent is obtained by the difference between adjacent points. The mean value of the neighborhood normal of the weld point cloud is taken as the local normal. The constraints of the welding torch axis and the forward direction are determined. An orthogonal tool coordinate system is constructed to form a rotation matrix to output the pose matrix, or output the position and Euler angles.
9. The method for preliminary positioning of welding arc initiation point and autonomous robot guidance based on machine vision according to claim 1, characterized in that, The welding torch pose is transformed into the robot coordinate system, specifically represented as follows: ； in, , , Let represent the coordinate system transformation matrices of base B, camera C, and trajectory point i, respectively.
10. A machine vision-based preliminary positioning system for welding arc initiation point and autonomous robot guidance system, characterized in that, The method for preliminary localization of welding arc initiation point and autonomous robot guidance based on machine vision as described in any one of claims 1-9 includes: a data acquisition module, a semantic segmentation module, a weld seam candidate strip region construction module, a semantic point cloud construction module, an edge candidate point set construction module, a weld seam point set extraction module, a welding trajectory generation module, a welding torch pose generation module, and an execution module. The data acquisition module is used to acquire RGB images and depth maps of the workpiece surface and generate point clouds aligned with RGB pixels. The semantic segmentation module is used to perform semantic segmentation on the RGB image of the workpiece surface to obtain multiple surface masks; The weld seam candidate strip region construction module is used to construct weld seam candidate strip regions in a two-dimensional mask space based on the geometric relationship between the two surfaces of the workpiece. The semantic point cloud construction module is used to map the candidate strip region of the weld seam to a three-dimensional point cloud, obtain the point cloud of the candidate weld seam region, and trim it. Then, surface semantic labels are added to the point cloud to obtain the semantic point cloud. The edge candidate point set construction module is used to perform point cloud preprocessing and geometric feature calculation, obtain surface patches, extract edges based on the boundaries of surface patches, and obtain the edge candidate point set; The weld seam point set extraction module is used to extract the weld seam point set from the edge candidate point set; The welding trajectory generation module is used to generate an initial welding trajectory point sequence and arc starting point position based on geometric fitting and trajectory smoothing constraints. The welding torch pose generation module is used to generate the welding torch pose for each welding trajectory point. The execution module is used to convert the welding torch pose to the robot coordinate system, output the arc starting point and initial guide trajectory, and guide the robot to move to the arc starting point.