An automatic tracking system and method for a large-scale spatial weld seam moving robotic arm

Abstract

This invention relates to an automatic tracking system and method for a large-scale spatial weld seam using a mobile robotic arm. The system includes a mobile robotic arm, a vision sensor, a welding torch, and a computer. The vision sensor acquires point cloud data of the surface of the large-scale weldment to be tracked from multiple perspectives and transmits it to the computer. The computer uses algorithms to register and stitch the point cloud data, identifies and extracts weld seam feature points, and plans the end effector trajectory of the six-degree-of-freedom robotic arm and the movement path of the three-degree-of-freedom mobile chassis based on the spatial trajectory of the weld seam feature points, converting them into motion commands. The computer sends the motion commands to the mobile robotic arm. The six-degree-of-freedom robotic arm drives the welding torch to first accurately track the spatial trajectory of the weld seam feature points along the end effector trajectory, and then the three-degree-of-freedom mobile chassis tracks the weld seam along the movement path. Through the alternating trajectory tracking of the six-degree-of-freedom robotic arm and the three-degree-of-freedom mobile chassis, the mobile tracking of the large-scale spatial weld seam is achieved.

Description

Technical Field

[0001] This invention relates to the field of mobile robotic arm welding technology, and in particular to an automatic tracking system and method for a large-scale spatial weld mobile robotic arm. Background Technology

[0002] As modern industry expands, the demand for welding large and complex components such as ship hulls, aerospace equipment, bridges, and spherical tanks (storage tanks) is increasing, and the welding of these large components often needs to be carried out on the construction site.

[0003] Traditional welding robotic arms have their bases fixed in specific locations, limiting their reach and flexibility, making them suitable only for mass production assembly line scenarios. Mobile welding carriages, on the other hand, need to move on pre-laid tracks or gantry frames, with restrictions on their direction and distance of movement, making it difficult to weld various complex weld seams in different spatial positions, and deployment costs are high. Mobile welding robotic arms, composed of a mobile chassis and a multi-degree-of-freedom robotic arm, combine the advantages of both. They can autonomously move and change working positions according to different welding scenarios, and the welding torch posture can be flexibly adjusted during welding movements, making them ideal for welding large and complex components. Therefore, utilizing mobile welding robotic arms to achieve intelligent and automated welding has become a research hotspot and development trend in the welding field.

[0004] Current technological limitations include: irregular curvature variations in large and complex components, long weld lengths, and variable welding postures; insufficient positioning accuracy of the mobile chassis; and unavoidable vibrations and slippage during movement, severely impacting the weld extraction and tracking accuracy of the mobile welding robot arm. Patent CN108941848A discloses a planar autonomous mobile welding robot weld initial detection and positioning system, using a monocular vision sensor to acquire weld images and enabling welding of right-angle welds on grid-shaped components at the bottom of a ship's cabin with irregular drainage holes. Patent CN119589235A discloses a permanent magnet adsorption-type automatic obstacle-crossing and wall-climbing welding robot, which achieves weld tracking by coordinating the movement of the mobile carriage and the cross-slider mechanism; however, this is only suitable for high-altitude straight-line welding and cannot achieve autonomous adjustment of the welding torch posture during welding. Patent CN114769962A discloses a mobile robot based on vision sensing... The weld seam recognition and tracking system of the welding robot uses a camera to capture multiple frames of optical images of the weld seam, and stitches them together to form digital image data of the weld seam based on image processing algorithms. The target trajectory points are extracted to obtain the weld seam trajectory data. However, it is sensitive to ambient light and has limited accuracy in weld seam extraction. Patent CN116423114A discloses a mobile welding robot arm collaborative tracking method, which constructs a speed, angular velocity and TCP displacement model through a speed and heading-chassis coordinate system model and a vehicle motion state estimation model, and deploys a mobile welding manipulator coordinated control architecture. However, it ignores the slippage that the vehicle body will experience during turning and lacks consideration of the constraints of the actual scenario. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an automatic tracking system and method for large-scale spatial weld seams using a mobile robotic arm. This system uses a three-degree-of-freedom mobile chassis and a six-degree-of-freedom robotic arm to expand the welding range; it acquires point cloud data of the weldment using a binocular structured light camera, resulting in high image accuracy and minimal interference from ambient light; and it employs a multi-station segmented weld seam tracking strategy using the mobile robotic arm to avoid tracking accuracy issues caused by factors such as chassis slippage and vibration during the weld seam tracking process. It is applicable to various types of unknown large-scale butt welded components, solving the technical challenge of continuous tracking of large-scale spatial weld seams.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an automatic tracking system for a large-scale spatial weld seam mobile robotic arm, comprising a mobile robotic arm, a vision sensor, a welding torch, a computer, and a large-scale weldment to be tracked; the mobile robotic arm includes a three-degree-of-freedom mobile chassis and a six-degree-of-freedom robotic arm; the vision sensor is a binocular structured light camera; the vision sensor and the welding torch are mounted at the end of the six-degree-of-freedom robotic arm; the vision sensor captures images of the surface of the large-scale weldment to be tracked from different perspectives, generating at least three frames of 3D point cloud data, and transmits the point cloud data to the computer; the computer registers and stitches the point cloud data using an algorithm, identifies and... Weld feature points are extracted from the 3D point cloud to obtain their spatial trajectories. Based on these spatial trajectories, the end effector trajectory of the six-degree-of-freedom robotic arm and the movement path of the three-degree-of-freedom mobile chassis are planned and converted into motion commands. The computer sends these motion commands to the robotic arm, which first drives the welding torch to accurately track the spatial trajectory of the weld feature points along the end effector trajectory. Then, the three-degree-of-freedom mobile chassis tracks the weld feature points along the movement path. Through the alternating trajectory tracking of the six-degree-of-freedom robotic arm and the three-degree-of-freedom mobile chassis, large-scale spatial weld tracking is achieved.

[0007] In a preferred embodiment, the large-scale weldment to be tracked is a planar weldment or an irregular curved surface weldment with symmetrical V-shaped, X-shaped, or U-shaped bevels, and the weld is a spatial straight line, oblique line, or irregular curve.

[0008] This invention also provides an automatic tracking method for a large-scale spatial weld seam moving robotic arm. The method employs a multi-station segmented weld seam tracking strategy and is implemented based on the aforementioned large-scale spatial weld seam moving robotic arm automatic tracking system. The method includes the following steps:

[0009] Step S1: Control the mobile robotic arm to move to the initial position;

[0010] Step S2: Keep the three-degree-of-freedom moving chassis stationary, and control the six-degree-of-freedom robotic arm equipped with a vision sensor to take pictures of the large-scale weldment to be tracked in multiple preset poses, generating at least three frames of original point cloud images.

[0011] Step S3: Perform point cloud preprocessing on each frame of the original point cloud image;

[0012] Step S4: At least three preprocessed point cloud images are fused using point cloud registration and stitching techniques to obtain a local three-dimensional point cloud model of the weldment at the current station location.

[0013] Step S5: In the local three-dimensional point cloud model of the weldment, segment out the point cloud set of the bevel region;

[0014] Step S6: Extract the edge points of the bevel area point cloud and obtain the weld feature point set based on this;

[0015] Step S7: Fit the weld feature point set to generate a continuous weld trajectory, interpolate the continuous weld trajectory to generate a welding path point sequence, plan the pose matrix of the welding torch at each welding path point, convert the pose matrix sequence into motion commands and drive the six-degree-of-freedom robotic arm to drive the welding torch to complete the accurate tracking of the continuous weld trajectory.

[0016] Step S8: Based on the continuous weld seam trajectory and combined with the operability constraints of the six-degree-of-freedom robotic arm, plan the movement path of the three-degree-of-freedom mobile chassis, and drive the mobile robotic arm to move from the current station to the next station approximately along the movement path.

[0017] Step S9: Repeat steps S2-S8 above to achieve continuous tracking of large-scale spatial welds through multi-station switching operations. The control process is terminated when the weld endpoint is detected as empty, i.e., the weld feature point set is empty.

[0018] In a preferred embodiment, step S3 includes: processing the acquired single-frame original point cloud image, establishing the topological relationship between discrete point cloud data through a KD-tree spatial index structure; reducing the point cloud density using an improved voxel filtering method, calculating the point within each voxel grid that has the closest Euclidean distance to the centroid point to replace the centroid point; and then using a statistical filtering algorithm to remove outlier noise points from the point cloud.

[0019] In a preferred embodiment, the specific implementation process of registering and stitching at least three frames of point cloud images in step S4 is as follows:

[0020] Repeat the preset point cloud image registration and stitching steps until all frame point cloud images are stitched together to obtain the local three-dimensional point cloud model of the weldment at the current station position.

[0021] The preset point cloud image registration and stitching steps include:

[0022] Step S4-1: Equip the six-DOF robotic arm with a vision sensor to capture two large-scale point cloud images of the weldment to be tracked in two adjacent poses, and then use a hand-eye calibration matrix to generate the images. The pose matrix of the six-DOF robotic arm end effector corresponding to the acquisition of a single frame point cloud image. The point cloud coordinates are unified to the six-DOF robot arm base coordinate system, and represented as the first frame of point cloud. Set and second frame point cloud set To achieve coarse stitching of point clouds, the formula is as follows: In the formula, This represents point cloud data in the base coordinate system of a six-DOF robotic arm. This represents preprocessed point cloud data in the visual sensor coordinate system; where the first frame point cloud set... Second frame point cloud There are partially overlapping areas;

[0023] Step S4-2: Select the first frame point cloud set with the largest number of point clouds. As the target point cloud, the second frame point cloud set Using the source point cloud as an example, the first frame point cloud set is extracted based on the octree spatial index structure and Euclidean distance constraints. Second frame point cloud First overlapping region point cluster Second overlapping region point cloud ;

[0024] Step S4-3: Set the points in the first overlapping region. Each point in Perform a radius search to obtain its neighborhood point set, and solve for the point. eigenvalues ​​of the weighted covariance matrix formed by its neighborhood point set ,in The weighted covariance matrix is: In the formula, T represents the transpose of the matrix. Indicates the number of neighboring points. Representative point Its first Neighboring points The Euclidean distance between them, if the eigenvalues ​​satisfy the following condition: Then point Define the points as ISS feature points. Add to the first ISS feature point set In the middle; the second overlapping region point cloud was selected using the same method. The ISS feature points in the set are added to the second ISS feature point set. middle;

[0025] Step S4-4: For the first ISS feature point set Second ISS feature point set The nearest point iteration algorithm (ICP) between two point clouds is used for registration to obtain the rigid body transformation matrix of the two point clouds.

[0026] Step S4-5: Apply the rigid body transformation matrix to the second frame point cloud. Perform a linear transformation to obtain the point cloud in the first frame. Point clouds in a coordinate system, merging two point clouds into one. To achieve precise point cloud stitching;

[0027] After stitching together all frame point cloud images to obtain the local three-dimensional point cloud model of the weldment at the current station location, the improved voxel filtering method is used to reduce the redundant point cloud density in the overlapping area.

[0028] In a preferred embodiment, step S5 is specifically implemented as follows:

[0029] Step S5-1: For the local 3D point cloud model of the weldment, the weighted principal component analysis method based on Euclidean distance is used to calculate the normal vector of the weldment point cloud. Based on KD-tree, the normal vector of each point is calculated. Perform a k-nearest neighbor search to obtain its neighborhood point set, and solve for the point. The eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by the k-neighborhood point set is the point's eigenvector. unit normal vector The formula for the covariance matrix is: In the formula, Indicates the number of neighboring points. Representative point Its first Neighboring points The Euclidean distance between them Represents a number that approaches 0. Point The geometric centroid of its neighborhood set of points; the spatial position of the visual sensor in the six-DOF robot arm's base coordinate system. As a reference viewpoint, the point cloud normal vectors are redirected to ensure they all point in the camera direction. The method is as follows: ;

[0030] Step S5-2: Construct a surface change feature descriptor SVFD based on point cloud normal vector information, and perform KD-tree analysis on each point. Perform a radius search to obtain its neighborhood point set, and traverse it. The neighborhood point set, calculate the point With each neighboring point The angle between the normal vectors and its arithmetic mean The calculation formula is: , Using variance representation points The local SVFD is calculated using the following formula: ; basis With all its neighboring points Local SVFD, starting point The global SVFD, or surface variation feature descriptor SVFD, is calculated using the following formula: ,in, Representing neighborhood points Local SVFD, Point The arithmetic mean of the local SVFD of all neighboring points.

[0031] Step S5-3: Set an appropriate filtering threshold to filter out points whose surface change feature descriptor (SVFD) is greater than the filtering threshold and construct a new point cloud set. Use the Euclidean clustering algorithm to cluster and segment the point cloud set, sort the clustering results, and select the category with the most point clouds as the point cloud set for the bevel area.

[0032] In a preferred embodiment, step S6 is specifically implemented as follows:

[0033] Step S6-1: Traverse the point cloud set of the slope area, and process each point based on a KD-tree. Perform a radius search to obtain its neighborhood point set, and calculate the point. The point is obtained by the angle between the vector formed by the projections of the point onto the fitting tangent plane and all its neighboring points. Given the set of included angles, take the maximum value from the set of included angles. ,like greater than the set angle threshold Then the decision point As edge points, add them to the edge point set of the bevel region;

[0034] Step S6-2: Traverse the set of edge points in the bevel region and use principal component analysis to calculate the value of each point. Local tangent vector For point Find points that meet the conditions in the set of edge points of the bevel region. And add it to the corresponding candidate point set, and then select the point from the candidate point set. The point closest to the Euclidean distance is used as the corresponding point on both sides of the edge. The filtering criteria are: In the formula, This is the bevel width value. Angle threshold;

[0035] Step S6-3, Calculate points Its corresponding point geometric center with the mean of the normal vector The calculation formula is: , In the formula For point The normal vector, For corresponding points The normal vector, based on the known bevel depth ,according to Calculate the characteristic points at the bottom of the slope, which are the weld characteristic points, and add them to the weld characteristic point set.

[0036] In a preferred embodiment, in step S7, the non-uniform rational B-spline algorithm (NURBS) is used to fit the weld feature points to obtain a continuous weld trajectory, and the continuous weld trajectory is interpolated based on the principle of equal arc length to obtain a welding path point sequence.

[0037] In a preferred embodiment, in step S8, the operability of the six-degree-of-freedom robotic arm is analyzed, and a TCP operability distribution space is plotted based on the Monte Carlo method. A dexterous working radius is set within the operability distribution space as the tracking radius between the three-degree-of-freedom mobile chassis and the large-scale weldment to be tracked. Based on the tracking radius, the weld trajectory is offset at equal intervals to obtain the movement path of the three-degree-of-freedom mobile chassis.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. This invention proposes an automatic tracking system for a large-scale spatial weld seam moving robotic arm, which has certain versatility and is applicable to various types of unknown large-scale butt welded parts, including but not limited to planar welded parts or irregular curved surface welded parts with symmetrical V-shaped, X-shaped, and U-shaped bevels. The weld seam is a spatial straight line, oblique line, or irregular curve, which expands the application scenarios of the moving welding robotic arm and improves the level of intelligence and precision in the welding of large and complex components.

[0040] 2. The multi-station segmented weld seam tracking strategy for mobile robotic arms provided by this invention ensures that the three-degree-of-freedom mobile chassis remains stationary while the six-degree-of-freedom robotic arm performs weld seam trajectory tracking tasks. This effectively avoids the problem of low tracking accuracy caused by factors such as slippage and shaking of the mobile chassis during the weld seam tracking process.

[0041] 3. The point cloud image registration and stitching technology provided by this invention extracts the ISS feature point set of the point cloud in the overlapping area of ​​two point cloud images, and uses the nearest point iterative (ICP) registration algorithm based on the ISS feature point set, which can effectively improve the efficiency and accuracy of high-density point cloud registration; the mobile robotic arm can acquire at least three frames of images through pose transformation in a single station to obtain a large field of view three-dimensional imaging of the weldment, which effectively solves the problem of limited field of view of the binocular structured light camera when facing large-scale weldment point cloud acquisition, and improves weld seam tracking efficiency.

[0042] 4. The weld feature point identification and extraction method provided by the present invention divides the weld bevel region point cloud by using surface variation feature descriptor SVFD and Euclidean clustering algorithm, extracts the edge points of the bevel point cloud based on the method of determining the angle between projection vectors, and then obtains the weld feature points based on the position of the corresponding points on both sides and their normal vector information. This method can greatly improve the accuracy and efficiency of weld feature point extraction for butt welds. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the overall system of the present invention;

[0044] Figure 2 This is a schematic diagram of the multi-station segmented weld seam tracking strategy of the present invention;

[0045] Figure 3 This is a flowchart illustrating the overall method implementation of the present invention;

[0046] Figure 4 This is a flowchart illustrating the point cloud registration and stitching technology of the present invention.

[0047] Figure 5 This is a point cloud stitching effect diagram of the present invention;

[0048] Figure 6 This is a schematic diagram of weld feature point extraction according to the present invention;

[0049] Figure label:

[0050] 2. Vision sensor; 3. Welding torch; 4. Computer; 5. Large-scale workpiece to be tracked; 11. Three-degree-of-freedom mobile chassis; 12. Six-degree-of-freedom robotic arm. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0052] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0053] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0054] Example 1

[0055] like Figure 1-6 As shown, this invention provides an automatic tracking system for a large-scale spatial weld seam mobile robotic arm, applicable to welding scenarios of large and complex components such as ship hulls, aerospace equipment, bridge structures, and spherical tanks (storage tanks). The system includes a mobile robotic arm, a vision sensor 2, a welding torch 3, a computer 4, and a large-scale workpiece 5 to be tracked. The mobile robotic arm includes a three-degree-of-freedom mobile chassis 11 and a six-degree-of-freedom robotic arm 12; the vision sensor 2 is a binocular structured light camera; the vision sensor 2 and the welding torch 3 are mounted at the end of the six-degree-of-freedom robotic arm 12; the vision sensor 2 captures images of the surface of the large-scale workpiece 5 from different perspectives, generating at least three frames of 3D point cloud data, and transmits the point cloud data to the computer 4; the computer 4 uses algorithms to register and stitch the point cloud data, identifies and extracts weld seam feature points from the 3D point cloud, obtains the spatial trajectory of the weld seam feature points, and, based on the spatial... The trajectory plans the end effector trajectory of the six-degree-of-freedom robotic arm 12 and the movement path of the three-degree-of-freedom mobile chassis 11, and converts them into motion commands. The computer 4 sends the motion commands to the mobile robotic arm. The six-degree-of-freedom robotic arm 12 drives the welding torch 3 to first complete the precise tracking of the spatial trajectory of the weld feature points along the end effector trajectory. Then, the three-degree-of-freedom mobile chassis 11 tracks and moves along the movement path. Through the alternating trajectory tracking of the six-degree-of-freedom robotic arm 12 and the three-degree-of-freedom mobile chassis 11, the movement tracking of the large-scale spatial weld is realized.

[0056] In this embodiment, the six-degree-of-freedom robotic arm 12 is responsible for accurately tracking the weld trajectory and changing the position of the welding torch 3 according to the calculated welding path points; the three-degree-of-freedom mobile chassis 11 is responsible for roughly tracking the surface of the large-scale weldment 5. Based on the idea of ​​equidistant offset of the weld trajectory, it ensures that the heading of the three-degree-of-freedom mobile chassis 11 is always basically consistent with the tangential direction of the weld trajectory, thus avoiding collision with the large-scale weldment 5.

[0057] In this embodiment, the three-degree-of-freedom mobile chassis 11 is omnidirectional Mecanum wheel drive, and the vehicle body fully integrates an onboard computer, front and rear lidar, encoder and IMU sensor, which can provide accurate positioning in dynamic environments; the six-degree-of-freedom robotic arm 12 adopts a body integrated controller design (no control cabinet), which makes the system space deployment more convenient.

[0058] In this embodiment, the large-scale weldment 5 to be tracked is a planar weldment or an irregular curved surface weldment with symmetrical V-shaped, X-shaped, or U-shaped bevels, and the weld is a spatial straight line, oblique line, or irregular curve.

[0059] Example 2

[0060] In addition, such as Figure 2, Figure 3 As shown, this invention also proposes an automatic tracking method for a large-scale spatial weld seam moving robotic arm. The method employs a multi-station segmented weld seam tracking strategy, implemented based on a large-scale spatial weld seam moving robotic arm automatic tracking system, and includes the following steps:

[0061] Step S1: Control the mobile robotic arm to move to the initial position;

[0062] Step S2: Keep the three-degree-of-freedom mobile chassis 11 stationary, and control the six-degree-of-freedom robotic arm 12 equipped with the vision sensor 2 to take pictures of the large-scale weldment 5 to be tracked in multiple preset poses, generating at least three frames of original point cloud images. The multiple preset poses can be planned and taught manually by combining factors such as the type of the large-scale weldment 5 to be tracked, the weld position, and the field of view of the vision sensor 2 in a single frame.

[0063] Step S3: Perform point cloud preprocessing on each frame of the original point cloud image;

[0064] Step S4: At least three preprocessed point cloud images are fused using point cloud registration and stitching techniques to obtain a local three-dimensional point cloud model of the weldment at the current station location.

[0065] Step S5: In the local three-dimensional point cloud model of the weldment, segment out the point cloud set of the bevel region;

[0066] Step S6: Extract the edge points of the bevel area point cloud and obtain the weld feature point set based on this;

[0067] Step S7: Fit the weld feature point set to generate a continuous weld trajectory, interpolate the continuous weld trajectory to generate a welding path point sequence, plan the pose matrix of the welding torch 3 at each welding path point, convert the pose matrix sequence into motion commands and drive the six-degree-of-freedom robotic arm 12 to drive the welding torch 3 to complete the accurate tracking of the continuous weld trajectory.

[0068] Step S8: Based on the continuous weld seam trajectory and combined with the operability constraints of the six-degree-of-freedom robotic arm 12, plan the movement path of the three-degree-of-freedom mobile chassis 11, and drive the mobile robotic arm to move from the current station to the next station approximately along the movement path.

[0069] Step S9: Repeat steps S2-S8 above to achieve continuous tracking of large-scale spatial welds through multi-station switching operations. The control process is terminated when the weld endpoint is detected as empty, i.e., the weld feature point set is empty.

[0070] In this embodiment, step S3 includes: processing the acquired single-frame original point cloud image, establishing the topological relationship between discrete point cloud data through the KD-tree spatial index structure, which can reduce the search time for neighboring points and improve the efficiency of point cloud processing; the original voxel filtering method calculates the centroid point in each voxel grid to replace all point clouds in that voxel grid to achieve point cloud downsampling, this embodiment uses an improved voxel filtering method to reduce point cloud density, calculates the point in each voxel grid with the closest Euclidean distance to the centroid point to replace the centroid point, ensuring that the downsampled point cloud is completely composed of original data points, thereby preserving the geometric feature information of the original point cloud to the maximum extent; and using a statistical filtering algorithm to remove outlier noise points in the point cloud to avoid interfering with subsequent point cloud registration, edge extraction and other methods.

[0071] In this embodiment, as Figure 4 As shown, the specific implementation process of registering and stitching at least three frames of point cloud images in step S4 is as follows:

[0072] Repeat the preset point cloud image registration and stitching steps until all frame point cloud images are stitched together to obtain the local three-dimensional point cloud model of the weldment at the current station position.

[0073] The preset point cloud image registration and stitching steps include:

[0074] Step S4-1: Equip the six-DOF robotic arm with a vision sensor to capture two large-scale point cloud images of the weldment to be tracked in two adjacent poses, and then use a hand-eye calibration matrix to generate the images. The pose matrix of the six-DOF robotic arm end effector corresponding to the acquisition of a single frame point cloud image. The point cloud coordinates are unified to the six-DOF robot arm base coordinate system and represented as frame point cloud sets. And Frame Point Cloud Collection To achieve coarse stitching of point clouds, the formula is as follows: In the formula, This represents point cloud data in the base coordinate system of a six-DOF robotic arm. This represents preprocessed point cloud data in the coordinate system of the visual sensor 2, where the frame point cloud set... And Frame Point Cloud Collection There are some overlapping areas. At this time, due to slight errors in the calibration of visual sensors, hand-eye calibration, or coordinate system transformation of robotic arms, the two point clouds may not be able to be accurately and completely stitched together.

[0075] Step S4-2: Select the frame point cloud set with the largest number of point clouds. As the target point cloud, the frame point cloud set As the source point cloud, the frame point cloud set is extracted based on the octree spatial index structure and Euclidean distance constraints. and Frame Cloud Overlapping area point clusters and overlapping area point clusters Specifically, it is done by traversing the frame cloud. , gather frame points As a search space, a radius search is performed based on the KD-tree, and points that meet the distance conditions are marked as overlapping region point clouds;

[0076] Step S4-3: Gather points in the overlapping region Each point in Perform a radius search to obtain its neighborhood point set, and solve for the point. eigenvalues ​​of the weighted covariance matrix formed by its neighborhood point set ,in The weighted covariance matrix is: In the formula, Indicates the number of neighboring points. Representative point Its first Neighboring points The Euclidean distance between them, if the eigenvalues ​​satisfy the following condition: Then point Define the points as ISS feature points. Add to ISS feature point set In the middle; the same method was used to filter out the overlapping region point cloud. The ISS feature points are added to the ISS feature point set. middle;

[0077] Step S4-4: Process the ISS feature point set and The nearest point iteration (ICP) algorithm between two point clouds is used for registration to obtain the rigid body transformation matrix of the two point clouds.

[0078] Step S4-5: Apply the rigid body transformation matrix to the frame point cloud. Perform a linear transformation to obtain the frame point cloud. Point clouds in a coordinate system, merging two point clouds into one. To achieve precise point cloud stitching;

[0079] After stitching together all frame point cloud images to obtain the local three-dimensional point cloud model of the weldment at the current station location, the improved voxel filtering method is used to reduce the redundant point cloud density in the overlapping area. Otherwise, the repeated point clouds will affect the weights of the subsequent weighted principal component analysis method and the accuracy of the point cloud normal vector calculation. Figure 5 This is the effect of stitching together two point cloud frames.

[0080] In this embodiment, the specific implementation steps of step S5 are as follows:

[0081] Step S5-1: For the local 3D point cloud model of the weldment, the weighted principal component analysis method based on Euclidean distance is used to calculate the normal vector of the weldment point cloud. Based on KD-tree, the normal vector of each point is calculated. Perform a k-nearest neighbor search to obtain its neighborhood point set, and solve for the point. The eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by the k-neighborhood point set is the point's eigenvector. unit normal vector The formula for the covariance matrix is: In the formula, Indicates the number of neighboring points. Representative point Its first Neighboring points The Euclidean distance between them Represents a number that approaches 0. Point The geometric centroid of its neighborhood set of points; the spatial position of the visual sensor in the six-DOF robot arm's base coordinate system. As a reference viewpoint, the point cloud normal vectors are redirected to ensure they all point in the camera direction. The method is as follows: ;

[0082] Step S5-2: By analyzing the changes in the normal vector direction within the neighborhood of the point cloud, it can be seen that in the flat base material surface area, the normal vector direction of the point cloud tends to be consistent and the included angle value is relatively small. However, in the bevel area with geometrical abrupt changes, the normal vector direction of the point cloud shows obvious abrupt changes and the included angle value is relatively large. Therefore, a surface change feature descriptor SVFD is constructed based on the point cloud normal vector information to describe the unevenness and convexity of the surface of the large-scale weldment 5 to be tracked. Based on KD-tree, each point... Perform a radius search to obtain its neighborhood point set, and traverse it. The neighborhood point set, calculate the point With each neighboring point The angle between the normal vectors and its arithmetic mean The calculation formula is: , Using variance representation points The local SVFD is calculated using the following formula: ; basis With all its neighboring points Local SVFD, starting point The global SVFD, or surface variation feature descriptor SVFD, is calculated using the following formula: ,in, Representing neighborhood points Local SVFD, Point The arithmetic mean of the local SVFD of all neighboring points;

[0083] Step S5-3: Set an appropriate filtering threshold to filter out points whose surface change feature descriptor (SVFD) values ​​are greater than the threshold and construct a new point cloud set. Due to factors such as material properties, processing technology, and operating environment, there may still be uneven areas on the surface of the large-scale weldment 5 to be tracked. When applying SVFD to extract the welding bevel area, these uneven areas may be incorrectly extracted. The incorrectly extracted uneven areas are usually spatially discrete and have a small number of point clouds. Use the Euclidean clustering algorithm to cluster and segment the point cloud set, and sort the clustering results. Select the category with the most point clouds as the bevel area point cloud set. At this time, the bevel area point cloud set can be extracted completely and effectively.

[0084] In this embodiment, the specific implementation steps of step S6 are as follows:

[0085] Step S6-1: Traverse the point cloud set of the slope area, and process each point based on a KD-tree. Perform a radius search to obtain its neighborhood point set, and calculate the point. The point is obtained by the angle between the vector formed by the projections of the point onto the fitting tangent plane and all its neighboring points. Given the set of included angles, take the maximum value from the set of included angles. ,like greater than the set angle threshold Then the decision point As edge points, add them to the edge point set of the bevel region;

[0086] Step S6-2: Traverse the set of edge points in the bevel region and use principal component analysis to calculate the value of each point. Local tangent vector For point By estimating the tangent vector of the point cloud and applying orthogonal constraints, points satisfying the conditions are found in the set of edge points in the bevel region. And add it to the corresponding candidate point set, and then select the point from the candidate point set. The point closest to the Euclidean distance is used as the corresponding point on both sides of the edge. The filtering criteria are: In the formula, This is the bevel width value. Angle threshold;

[0087] Step S6-3, Calculate points Its corresponding point geometric center with the mean of the normal vector The calculation formula is: , In the formula For point The normal vector, For corresponding points The normal vector, based on the known bevel depth ,according to Calculate the characteristic points at the bottom of the slope, which are the weld characteristic points, and add them to the weld characteristic point set.

[0088] In this embodiment, in step S7, the non-uniform rational B-spline (NURBS) algorithm is used to fit the weld feature points to obtain the continuous weld trajectory. Based on the principle of equal arc length, the continuous weld trajectory is interpolated to obtain the welding path point sequence, ensuring that the welding path point sequence sent to the six-degree-of-freedom robotic arm 12 is smooth and orderly, thereby minimizing the influence of outliers and invalid points.

[0089] In this embodiment, in step S8, a kinematic model and a kinematic Jacobian matrix of the six-degree-of-freedom robotic arm 12 equipped with the welding torch 3 are established. Using the motion Jacobian matrix Using the determinant as a metric, the operability of the six-DOF robotic arm 12 is analyzed: ,in The determinant of the Jacobian matrix is ​​represented; and the TCP operability distribution space is plotted based on the Monte Carlo method. A smart working radius is set in the operability distribution space as the tracking radius between the three-degree-of-freedom mobile chassis 11 and the large-scale weldment 5 to be tracked. The weld trajectory is offset by an equal distance based on the tracking radius to obtain the movement path of the three-degree-of-freedom mobile chassis 11.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; any changes made according to the technical solutions of the present invention that do not exceed the scope of the technical solutions of the present invention shall fall within the protection scope of the present invention.

Claims

1. A large-scale space weld moving robot automatic tracking method, characterized in that, The method employs a multi-station segmented weld seam tracking strategy, including the following steps: Step S1: Control the mobile robotic arm to move to the initial position; Step S2: Keep the three-degree-of-freedom moving chassis stationary, and control the six-degree-of-freedom robotic arm equipped with a vision sensor to take pictures of the large-scale weldment to be tracked in multiple preset poses, generating at least three frames of original point cloud images. Step S3: Perform point cloud preprocessing on each frame of the original point cloud image; Step S4: At least three preprocessed point cloud images are fused using point cloud registration and stitching techniques to obtain a local three-dimensional point cloud model of the weldment at the current station location. Step S5: In the local three-dimensional point cloud model of the weldment, segment out the point cloud set of the bevel region; Step S6: Extract the edge points of the bevel area point cloud and obtain the weld feature point set based on this; Step S7: Fit the weld feature point set to generate a continuous weld trajectory, interpolate the continuous weld trajectory to generate a welding path point sequence, plan the pose matrix of the welding torch at each welding path point, convert the pose matrix sequence into motion commands and drive the six-degree-of-freedom robotic arm to drive the welding torch to complete the accurate tracking of the continuous weld trajectory. Step S8: Based on the continuous weld seam trajectory and combined with the operability constraints of the six-degree-of-freedom robotic arm, plan the movement path of the three-degree-of-freedom mobile chassis, and drive the mobile robotic arm to move from the current station to the next station approximately along the movement path. Step S9: Repeat steps S2-S8 above to achieve continuous tracking of large-scale spatial welds through multi-station switching operations. The control process is terminated when the weld endpoint is detected as empty, i.e., the weld feature point set is empty. The specific implementation steps of step S5 are as follows: Step S5-1: For the local 3D point cloud model of the weldment, the weighted principal component analysis method based on Euclidean distance is used to calculate the normal vector of the weldment point cloud. Based on KD-tree, the normal vector of each point is calculated. Perform a k-nearest neighbor search to obtain its neighborhood point set, and solve for the point. The eigenvector corresponding to the smallest eigenvalue of the covariance matrix formed by the k-neighborhood point set is the point's eigenvector. unit normal vector The formula for the covariance matrix is: In the formula, Indicates the number of neighboring points. Representative point Its first Neighboring points The Euclidean distance between them Represents a number that approaches 0. Point The geometric centroid of its neighborhood set of points; Spatial position of the vision sensor in the six-degree-of-freedom robot arm's base coordinate system As a reference viewpoint, the point cloud normal vectors are redirected to ensure they all point in the camera direction. The method is as follows: ; Step S5-2: Construct a surface change feature descriptor SVFD based on point cloud normal vector information, and perform KD-tree analysis on each point. Perform a radius search to obtain its neighborhood point set, and traverse it. The neighborhood point set, calculate the point With each neighboring point The angle between the normal vectors and its arithmetic mean The calculation formula is: , Using variance representation points The local SVFD is calculated using the following formula: ; basis With all its neighboring points Local SVFD, starting point The global SVFD, or surface variation feature descriptor SVFD, is calculated using the following formula: ,in, Representing neighborhood points Local SVFD, Point The arithmetic mean of the local SVFD of all neighboring points; Step S5-3: Set an appropriate filtering threshold to filter out points whose surface change feature descriptor (SVFD) is greater than the filtering threshold and construct a new point cloud set. Use the Euclidean clustering algorithm to cluster and segment the point cloud set, sort the clustering results, and select the category with the most point clouds as the point cloud set for the bevel area.

2. The automatic tracking method for a large-scale spatial weld seam moving robotic arm according to claim 1, characterized in that, Step S3 includes: processing the acquired single-frame original point cloud image, establishing the topological relationship between discrete point cloud data through the KD-tree spatial index structure; reducing the point cloud density using an improved voxel filtering method, calculating the point in each voxel grid that has the closest Euclidean distance to the centroid to replace the centroid; and then using a statistical filtering algorithm to remove outlier noise points in the point cloud.

3. The automatic tracking method for a large-scale spatial weld seam moving robotic arm according to claim 1, characterized in that, In step S4, the specific implementation process of registering and stitching at least three frames of point cloud images is as follows: Repeat the preset point cloud image registration and stitching steps until all frame point cloud images are stitched together to obtain the local three-dimensional point cloud model of the weldment at the current station position. The preset point cloud image registration and stitching steps include: Step S4-1: The six-DOF robotic arm equipped with a vision sensor captures two large-scale point cloud images of the weldment to be tracked in two adjacent poses, and then uses a hand-eye calibration matrix to generate the images. The pose matrix of the six-DOF robotic arm end effector corresponding to the acquisition of a single frame point cloud image. The point cloud coordinates are unified to the six-DOF robot arm base coordinate system, and are represented as the first frame point cloud set. Second frame point cloud To achieve coarse stitching of point clouds, the formula is as follows: In the formula, This represents point cloud data in the base coordinate system of a six-DOF robotic arm. This represents preprocessed point cloud data in the visual sensor coordinate system; where the first frame point cloud set... Second frame point cloud There are partially overlapping areas; Step S4-2: Select the first frame point cloud set with the largest number of point clouds. As the target point cloud, the second frame point cloud set Using the source point cloud as an example, the first frame point cloud set is extracted based on the octree spatial index structure and Euclidean distance constraints. Second frame point cloud First overlapping region point cluster Second overlapping region point cloud ; Step S4-3: Set the points in the first overlapping region. Each point in Perform a radius search to obtain its neighborhood point set, and solve for the point. eigenvalues ​​of the weighted covariance matrix formed by its neighborhood point set ,in The weighted covariance matrix is: In the formula, T represents the transpose of the matrix. Indicates the number of neighboring points. Representative point Its first Neighboring points The Euclidean distance between them, if the eigenvalues ​​satisfy the following condition: Then point Define the points as ISS feature points. Add to the first ISS feature point set In the middle; the second overlapping region point cloud was selected using the same method. The ISS feature points in the set are added to the second ISS feature point set. middle; Step S4-4: For the first ISS feature point set Second ISS feature point set The nearest point iteration algorithm (ICP) between two point clouds is used for registration to obtain the rigid body transformation matrix of the two point clouds. Step S4-5: Apply the rigid body transformation matrix to the second frame point cloud. Perform a linear transformation to obtain the point cloud in the first frame. Point clouds in a coordinate system, merging two point clouds into one. To achieve precise point cloud stitching; After stitching together all frame point cloud images to obtain the local 3D point cloud model of the weldment at the current station location, an improved voxel filtering method is used to reduce the redundant point cloud density in the overlapping area.

4. The automatic tracking method for a large-scale spatial weld seam moving robotic arm according to claim 1, characterized in that, The specific implementation steps of step S6 are as follows: Step S6-1: Traverse the point cloud set of the slope area, and process each point based on a KD-tree. Perform a radius search to obtain its neighborhood point set, and calculate the point. The point is obtained by the angle between the vector formed by the projections of the point onto the fitting tangent plane and all its neighboring points. Given the set of included angles, take the maximum value from the set of included angles. ,like greater than the set angle threshold Then the decision point As edge points, add them to the edge point set of the bevel region; Step S6-2: Traverse the set of edge points in the bevel region and use principal component analysis to calculate the value of each point. Local tangent vector For point Find points that meet the conditions in the set of edge points of the bevel region. And add it to the corresponding candidate point set, and then select the point from the candidate point set. The point closest to the Euclidean distance is used as the corresponding point on both sides of the edge. The filtering criteria are: In the formula, This is the bevel width value. Angle threshold; Step S6-3, Calculate points Its corresponding point geometric center with the mean of the normal vector The calculation formula is: , In the formula For point The normal vector, For corresponding points The normal vector, based on the known bevel depth ,according to Calculate the characteristic points at the bottom of the slope, which are the weld characteristic points, and add them to the weld characteristic point set.

5. The automatic tracking method for a large-scale spatial weld seam moving robotic arm according to claim 1, characterized in that, In step S7, the non-uniform rational B-spline algorithm (NURBS) is used to fit the weld feature points to obtain the continuous weld trajectory, and the continuous weld trajectory is interpolated based on the principle of equal arc length to obtain the welding path point sequence.

6. The automatic tracking method for a large-scale spatial weld seam moving robotic arm according to claim 1, characterized in that, In step S8, the operability of the six-degree-of-freedom robotic arm is analyzed, and the operability distribution space of TCP is plotted based on the Monte Carlo method. A dexterous working radius is set in the operability distribution space as the tracking radius between the three-degree-of-freedom mobile chassis and the large-scale weldment to be tracked. Based on the tracking radius, the weld trajectory is offset at equal intervals to obtain the movement path of the three-degree-of-freedom mobile chassis.

Images