Automatic tracking system and method for large-scale space welding seam mobile mechanical arm

Abstract

The invention relates to an automatic tracking system and method for a large-scale space welding seam mobile mechanical arm. The system comprises the mobile mechanical arm, a visual sensor, a welding gun and a computer. The visual sensor collects large-scale to-be-tracked weldment surface point cloud data in a multi-view mode and transmits the data to the computer. The computer registers and splices the point cloud data through an algorithm, identifies and extracts weld joint feature points, plans a tail end moving track of the six-degree-of-freedom mechanical arm and a moving path of the three-degree-of-freedom moving chassis according to space tracks of the weld joint feature points, and converts the moving tracks and the moving path into motion instructions; the computer sends a motion instruction to the movable mechanical arm, the six-degree-of-freedom mechanical arm drives the welding gun to finish precise tracking of a space trajectory of a welding seam feature point along a tail end moving trajectory, the three-degree-of-freedom movable chassis tracks and moves along a moving path, and the welding seam feature point is obtained through alternate trajectory tracking of the six-degree-of-freedom mechanical arm and the three-degree-of-freedom movable chassis. And moving tracking of the large-scale space welding seam is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of mobile robotic arm welding, in particular to an automatic tracking system and method for a mobile robotic arm for large-scale space welds. Background Art

[0002] With the gradual expansion of the scale of modern industry, the welding requirements for large and complex components such as ship cabins, aerospace equipment, bridge buildings, spherical tanks (storage tanks), etc. are increasing day by day, and the welding work of such large components often needs to be carried out at the construction site.

[0003] For traditional welding robotic arms, their bases are fixedly installed at specific positions, with limited arm reach space and insufficient flexibility, and are only suitable for the assembly line work scenarios of mass production; mobile welding carts need to move on pre-laid tracks or gantry frames by workers, and their moving directions and distances are restricted, making it difficult to perform welding operations on various complex forms of welds at different spatial positions, and the deployment cost is expensive. The mobile welding robotic arm composed of a mobile chassis and a multi-degree-of-freedom robotic arm combines the advantages of both, can move autonomously and change the working position according to different welding scenarios, and the posture of the welding torch can be adjusted flexibly during the welding movement, which is very suitable for the welding operation of large and complex components. Therefore, using a mobile welding robotic arm to achieve intelligent and automatic welding has become a research hotspot and development trend in the welding field.

[0004] The problems and deficiencies existing in the current technology are as follows: The curvature of large and complex components changes irregularly, the weld length is long and the welding posture is variable, the positioning accuracy of the mobile chassis is insufficient, and there are inevitable jitters, skidding, etc. during the movement, which seriously affect the weld extraction and tracking accuracy of the mobile welding manipulator. Patent CN108941848A discloses a weld initial detection and positioning system for a planar autonomous mobile welding robot, which uses a monocular vision sensor to collect weld images and realizes the welding operation of the right-angle welds of the lattice components at the bottom of the cabin with irregular flow holes; Patent CN119589235A discloses a permanent magnet adsorption type automatic obstacle-crossing wall-climbing welding robot, which realizes weld tracking by coordinating the movement of the mobile trolley and the cross-slider mechanism, and is only applicable to high-altitude straight-line welding and cannot realize the autonomous adjustment of the welding torch posture during the welding process; Patent CN114769962A discloses a weld recognition and tracking system for a mobile welding robot based on visual sensing, which uses a camera to capture multiple frames of weld optical images, stitches them into weld digital image data based on image processing algorithms, extracts target trajectory points to obtain weld trajectory data, is sensitive to ambient light, and has limited weld extraction accuracy; Patent CN116423114A discloses a method for collaborative tracking of the arm of a mobile welding robot, which constructs a speed and angular velocity - TCP displacement model through a speed and heading - chassis coordinate system model and a vehicle body motion state estimation model, and deploys a coordinated control architecture for the mobile welding manipulator, but ignores the slippage that will occur during the turning process of the vehicle body and lacks consideration of the actual scene constraints. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a large-scale space weld mobile manipulator automatic tracking system and method. This system combines a three-degree-of-freedom mobile chassis and a six-degree-of-freedom manipulator to form a mobile manipulator, expanding the welding range; collects the point cloud data of the welded part through a binocular structured light camera, with high image accuracy and weak interference from ambient light; adopts a multi-station segmented weld tracking strategy for the mobile manipulator to avoid the problem of low tracking accuracy caused by factors such as slippage and jitter of the mobile chassis during the weld tracking process. It is applicable to multiple types of unknown large-scale butt welded parts and solves the technical problem of continuous tracking of large-scale space welds.

[0006] To achieve the above object, the present invention adopts the following technical solutions: A large-scale spatial weld moving robotic arm automatic tracking system, comprising a moving robotic arm, a vision sensor, a welding torch, a computer, and a large-scale weldment to be tracked; the moving robotic arm includes a three-degree-of-freedom moving chassis and a six-degree-of-freedom robotic arm; the vision sensor is a binocular structured light camera; the vision sensor and the welding torch are installed at the end of the six-degree-of-freedom robotic arm; the vision sensor captures the surface of the large-scale weldment to be tracked from different perspectives to generate at least three frames of 3D point cloud data, and transmits the point cloud data to the computer; the computer registers and stitches the point cloud data through algorithms, identifies and extracts the weld feature points in the 3D point cloud, obtains the spatial trajectory of the weld feature points, respectively plans the end movement trajectory of the six-degree-of-freedom robotic arm and the movement path of the three-degree-of-freedom moving chassis according to the spatial trajectory, and converts them into motion instructions; the computer sends the motion instructions to the moving robotic arm, the six-degree-of-freedom robotic arm drives the welding torch to first accurately track the spatial trajectory of the weld feature points along the end movement trajectory, and then the three-degree-of-freedom moving chassis moves along the movement path for tracking. Through the alternating trajectory tracking of the six-degree-of-freedom robotic arm and the three-degree-of-freedom moving chassis, the moving tracking of the large-scale spatial weld is realized.

[0007] In a preferred embodiment, the large-scale weldment to be tracked is a planar weldment or an irregular curved surface weldment with symmetric V-shaped, X-shaped, or U-shaped grooves, and the weld is a spatial straight line, inclined line, or irregular curve.

[0008] The present invention also provides a large-scale spatial weld moving robotic arm automatic tracking method. The method adopts a multi-station segmented weld tracking strategy and is implemented based on the large-scale spatial weld moving robotic arm automatic tracking system according to claim 1 or 2, and includes the following steps:

[0009] Step S1, control the moving robotic arm to move to the initial station;

[0010] Step S2, keep the three-degree-of-freedom moving chassis stationary, and control the six-degree-of-freedom robotic arm to carry the vision sensor to capture the large-scale weldment to be tracked at a preset number of poses to generate at least three frames of original point cloud images;

[0011] Step S3, perform point cloud preprocessing on each frame of the original point cloud image;

[0012] Step S4, fuse at least three frames of preprocessed point cloud images by using point cloud registration and stitching technology to obtain the corresponding local three-dimensional point cloud model of the weldment at the current station;

[0013] Step S5, in the local three-dimensional point cloud model of the weldment, segment out the point cloud set of the groove area;

[0014] Step S6: Extract the edge points of the groove area point cloud, and based on this, obtain the weld feature point set;

[0015] Step S7: Fit the weld feature point set to generate a continuous weld track, interpolate points on the continuous weld track to generate a welding path point sequence, plan the pose matrix of the welding torch at each welding path point, convert the pose matrix sequence into a motion instruction, and drive the six-degree-of-freedom robotic arm to drive the welding torch to accurately track the continuous weld track;

[0016] Step S8: Based on the continuous weld track, combined with the manipulability constraint of the six-degree-of-freedom robotic arm, plan the moving path of the three-degree-of-freedom mobile chassis, and drive the mobile robotic arm to move from this station to the next station roughly along the moving path;

[0017] Step S9: Repeat the above steps S2 - S8, and achieve continuous tracking of large-scale space welds through multi-station switching operations. When the weld end point is detected, that is, when the weld feature point set is empty, terminate the control process.

[0018] In a preferred embodiment, the step S3 includes: processing the acquired single-frame original point cloud image, establishing the topological relationship between discrete point cloud data through the KD-tree spatial index structure; using an improved voxel filtering method to reduce the point cloud density, calculating the point with the closest Euclidean distance to the centroid point within each voxel grid to replace the centroid point; and then using a statistical filtering algorithm to remove the outlier noise points of the point cloud.

[0019] In a preferred embodiment, in the step S4, the specific implementation process of registering and stitching at least three frames of point cloud images is as follows:

[0020] Repeat the preset point cloud image registration and stitching steps until the stitching of all frames of point cloud images is completed, and obtain the corresponding local three-dimensional point cloud model of the welded part at the current station;

[0021] The preset point cloud image registration and stitching steps include:

[0022] Step S4-1: The six-degree-of-freedom robotic arm equipped with a vision sensor captures two frames of point cloud images generated by the large-scale welded part to be tracked at adjacent two poses. Through the hand-eye calibration matrix E T C and the pose matrix B T E of the end of the six-degree-of-freedom robotic arm corresponding to the acquisition of a single-frame point cloud image, unify the point cloud coordinates to the base coordinate system of the six-degree-of-freedom robotic arm, and represent them as the first-frame point cloud P i set and the second-frame point cloud set P j , to achieve rough stitching of the point cloud. The implementation formula is: B P = B T EE T C C P, where B P represents the point cloud data in the base coordinate system of a six-degree-of-freedom robotic arm, C and P represents the preprocessed point cloud data in the coordinate system of the vision sensor; among them, the first point cloud set P i and the second point cloud set P j have a partial overlapping area;

[0023] Step S4-2: Select the first point cloud set P with the largest number of points i as the target point cloud, and the second point cloud set P j as the source point cloud. Based on the octree spatial index structure, use the Euclidean distance constraint to extract the first overlapping area point cloud set Q i of the first point cloud set P j and the second overlapping area point cloud set Q i of the second point cloud set P j ;

[0024] Step S4-3: For each point p i in the first overlapping area point cloud set Q i perform a radius search to obtain its neighborhood point set, and solve the eigenvalues of the weighted covariance matrix formed by point p i and its neighborhood point set where the weighted covariance matrix is: In the formula, T represents transposing the matrix, k represents the number of neighborhood points, and dist ij represents the Euclidean distance between point p i and its j-th neighborhood point p ij . If the eigenvalues satisfy the condition: then point p i is defined as an ISS feature point, and point p i is put into the first ISS feature point set S i ; Screen out the ISS feature points in the second overlapping area point cloud set Q j in the same way and put them into the second ISS feature point set S j ;

[0025] Step S4-4: For the first ISS feature point set S i and the second ISS feature point set S j , use the point-to-plane closest point iterative algorithm ICP for registration to obtain the rigid body transformation matrix of the two point clouds;

[0026] Step S4-5: Apply the rigid body transformation matrix to the second point cloud set P j for linear transformation to obtain in the first point cloud set Pi Point cloud in the coordinate system, combining two point clouds into O i , realizing precise stitching of the point cloud;

[0027] After completing the stitching of all frame point cloud images and obtaining the corresponding local three-dimensional point cloud model of the welded part at the current station, use the improved voxel filtering method to reduce the redundant point cloud density in the overlapping area.

[0028] In a preferred embodiment, the specific implementation steps of step S5 are as follows:

[0029] Step S5-1: For the local three-dimensional point cloud model of the welded part, use the weighted principal component analysis method based on the Euclidean distance to calculate the normal vector of the welded part point cloud. Based on the KD-tree, perform a k-nearest neighbor search for each point p m to obtain its neighborhood point set, and solve the eigenvector corresponding to the minimum eigenvalue of the covariance matrix formed by the k-neighborhood point set of point p m , and this eigenvector is the unit normal vector of point p m The covariance matrix formula is: In the formula, k represents the number of neighborhood points, dist represents the Euclidean distance between point p mn and its nth neighborhood point p m , ε represents a number approaching 0, and p represents the geometric centroid of point p mn and its neighborhood point set; taking the spatial position p m of the vision sensor in the base coordinate system of the six-degree-of-freedom robotic arm as the reference viewing point, redirect the point cloud normal vector to ensure that the point cloud normal vectors are uniformly directed towards the camera direction. The method is as follows: view m

[0030] Step S5-2: Construct the surface variation feature descriptor SVFD based on the point cloud normal vector information. Based on the KD-tree, perform a radius search for each point p m to obtain its neighborhood point set, traverse the neighborhood point set of p m , calculate the normal vector included angle value θ m between point p mn and each neighborhood point p mn and its arithmetic mean The calculation formula is: Use variance to characterize the local SVFD of point p m The calculation formula is: Based on the local SVFD of point p m and all its neighborhood points p mn , construct the global SVFD of point p m , that is, the surface variation feature descriptor SVFD. The calculation formula is: Among them, σmn Denote the neighborhood point p mn of the local SVFD, denote the point p m the arithmetic mean of the local SVFDs of all neighborhood points.

[0031] Step S5-3: Set an appropriate screening threshold, screen out the points whose surface variation feature descriptor SVFD is greater than the screening threshold to construct a new point cloud set, use the Euclidean clustering algorithm to perform clustering segmentation on this point cloud set, and sort the clustering results, and select the category with the largest number of point clouds as the point cloud set of the groove area.

[0032] In a preferred embodiment, the specific implementation steps of the said step S6 are as follows:

[0033] Step S6-1: Traverse the point cloud set of the groove area, perform radius search on each point p u based on the KD-tree to obtain its neighborhood point set, calculate the magnitude of the included angle of the vectors formed by the projection of the point p u and all its neighborhood points on the fitting tangent plane to obtain the included angle set of the point p u , take the maximum value α max in the included angle set, if α max is greater than the set angle threshold β, then determine that the point p u is an edge point and put it into the edge point set of the groove area;

[0034] Step S6-2: Traverse the edge point set of the groove area, use the principal component analysis method to calculate the local tangent vector of each point p u , for the point p , find the point p u satisfying the conditions in the edge point set of the groove area, and put it into the corresponding point candidate set, and then select the point with the closest Euclidean distance to the point p v from the candidate set as the corresponding point p u ' on both sides, and the screening conditions are: u where w is the groove width value and Δθ is the angle threshold; In the formula, w is the groove width value, and Δθ is the angle threshold;

[0035] Step S6-3: Calculate the geometric center w u of the point p u and its corresponding point p u ', and the mean value of the normal vectors The calculation formula is: In the formula is the normal vector of the point p u , is the normal vector of the corresponding point p u ', according to the known groove depth d, according to Calculate the feature points at the bottom of the groove, i.e., the weld feature points, and put them into the weld feature point set.

[0036] In a preferred embodiment, in step S7, the non-uniform rational B-spline algorithm NURBS is used to fit the weld feature points to obtain a continuous weld track, and the continuous weld track is interpolated based on the equal arc length principle to obtain a sequence of welding path points.

[0037] In a preferred embodiment, in step S8, the manipulability of the six-degree-of-freedom robotic arm is analyzed, and the TCP manipulability distribution space is drawn based on the Monte Carlo method. A dexterous working radius is set within the manipulability distribution space as the tracking radius of the three-degree-of-freedom mobile chassis and the large-scale weldment to be tracked. Based on the tracking radius, an equidistant offset is made to the weld track to obtain the moving path of the three-degree-of-freedom mobile chassis.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention proposes a large-scale space weld moving robotic arm automatic tracking system, which has a certain degree of generality and is applicable to multiple types of unknown large-scale butt weldments, including but not limited to planar weldments or irregular curved surface weldments with symmetric V-shaped, X-shaped, and U-shaped grooves, and the welds are space straight lines, inclined lines, or irregular curves, expanding the application scenarios of the moving welding robotic arm and improving the welding intelligence and refinement level of large complex components.

[0040] 2. The multi-station segmented weld tracking strategy provided by the present invention for the moving robotic arm can keep the three-degree-of-freedom mobile chassis stationary when the six-degree-of-freedom robotic arm performs the weld track tracking task, effectively avoiding the problem of low tracking accuracy caused by factors such as slipping and jitter of the mobile chassis during the weld tracking process.

[0041] 3. The point cloud image registration and stitching technology provided by the present invention can effectively improve the efficiency and accuracy of high-density point cloud set registration by extracting the ISS feature point set of the overlapping area of two frames of point cloud images and using the iterative closest point (ICP) registration algorithm based on the ISS feature point set; the moving robotic arm obtains at least three frames of images through pose transformation within a single station to obtain a three-dimensional imaging of the large-scale weldment with a large field of view, effectively solving the problem of limited field of view of the binocular structured light camera when collecting point clouds of large-scale weldments and improving the weld tracking efficiency.

[0042] 4. The weld feature point recognition and extraction method provided by the present invention can greatly improve the extraction accuracy and efficiency of weld feature points of butt weldments by using the surface variation feature descriptor SVFD and the Euclidean clustering algorithm to segment the point cloud set of the weldment groove area, extracting the edge points of the groove point cloud based on the method of judging the included angle of projection vectors, and then obtaining the weld feature points based on the position of the corresponding points on both sides of the edge and their normal vector information. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the overall system of the present invention;

[0044] Figure 2 is a schematic diagram of the multi-station segmented weld tracking strategy of the present invention;

[0045] Figure 3 is a flowchart of the overall method implementation of the present invention;

[0046] Figure 4 is a flowchart of the point cloud registration and stitching technology implementation of the present invention;

[0047] Figure 5 is an effect diagram of the point cloud stitching of the present invention;

[0048] Figure 6 is a schematic diagram of the extraction of weld feature points of the present invention;

[0049] Reference numerals:

[0050] 1. Mobile robotic arm; 2. Vision sensor; 3. Welding torch; 4. Computer; 5. Large-scale weldment to be tracked; 11. Three-degree-of-freedom mobile chassis; 12. Six-degree-of-freedom robotic arm. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The present invention will be further described below in conjunction with the drawings and embodiments.

[0052] It should be noted that the following detailed description is illustrative and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which the present application belongs.

[0053] It should be noted that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0054] Embodiment 1

[0055] As Figure 1-6As shown in the figure, an automatic tracking system for a large-scale space weld moving robotic arm provided by the present invention is applied to welding scenarios of large and complex components such as ship cabins, aerospace equipment, bridge buildings, spherical tanks (storage tanks), etc. The automatic tracking system for the large-scale space weld moving robotic arm includes a moving robotic arm 1, a vision sensor 2, a welding torch 3, a computer 4, and a large-scale weldment to be tracked 5. The moving robotic arm 1 includes a three-degree-of-freedom moving chassis 11 and a six-degree-of-freedom robotic arm 12; the vision sensor 2 is a binocular structured light camera; the vision sensor 2 and the welding torch 3 are installed at the end of the six-degree-of-freedom robotic arm 12; the vision sensor 2 generates at least three frames of 3D point cloud data by photographing the surface of the large-scale weldment to be tracked 5 from different perspectives, and transmits the point cloud data to the computer 4; the computer 4 registers and stitches the point cloud data through an algorithm, identifies and extracts the weld feature points in the 3D point cloud, obtains the spatial trajectory of the weld feature points, respectively plans the end movement trajectory of the six-degree-of-freedom robotic arm 12 and the movement path of the three-degree-of-freedom moving chassis 11 according to the spatial trajectory, and converts them into motion instructions; the computer 4 sends the motion instructions to the moving robotic arm 1, the six-degree-of-freedom robotic arm 12 drives the welding torch 3 to first accurately track the spatial trajectory of the weld feature points along the end movement trajectory, and then the three-degree-of-freedom moving chassis 11 tracks and moves along the movement path. Through the alternating trajectory tracking of the six-degree-of-freedom robotic arm 12 and the three-degree-of-freedom moving chassis 11, the moving tracking of the large-scale space weld is realized.

[0056] In this embodiment, the six-degree-of-freedom robotic arm 12 is responsible for accurately tracking the weld trajectory and changing the pose of the welding torch 3 according to the calculated welding path points; the three-degree-of-freedom moving chassis 11 is responsible for roughly tracking the surface of the large-scale weldment to be tracked 5. Based on the idea of equidistant offset of the weld trajectory, it is ensured that the heading of the three-degree-of-freedom moving chassis 11 is always basically consistent with the tangent direction of the weld trajectory to avoid collision with the large-scale weldment to be tracked 5.

[0057] In this embodiment, the three-degree-of-freedom moving chassis 11 is driven by omnidirectional Mecanum wheels, and the vehicle body is fully integrated with an on-board computer, front and rear lidar, encoders, and IMU sensors, which can provide precise positioning in a dynamic environment; the six-degree-of-freedom robotic arm 12 adopts an integrated controller design for the body (without a control cabinet), which is more convenient for system space deployment.

[0058] In this embodiment, the large-scale weldment to be tracked 5 is a planar weldment or an irregular curved surface weldment with symmetric V-shaped, X-shaped, or U-shaped grooves, and the weld is a space straight line, oblique line, or irregular curve.

[0059] Embodiment 2

[0060] In addition, as Figure 2 、Figure 3 As shown in the figure, the present invention also proposes an automatic tracking method for a large-scale space weld moving manipulator, which is characterized in that the method adopts a multi-station segmented weld tracking strategy and is implemented based on an automatic tracking system for a large-scale space weld moving manipulator, including the following steps:

[0061] Step S1: Control the moving manipulator 1 to move to the initial station;

[0062] Step S2: Keep the three-degree-of-freedom moving chassis 11 stationary, and control the six-degree-of-freedom manipulator 12 carrying the vision sensor 2 to capture the large-scale weldment 5 to be tracked at a plurality of preset poses, generating at least three frames of original point cloud images. The plurality of preset poses can be planned and taught manually by combining factors such as the type of the large-scale weldment 5 to be tracked, the weld position, and the single-frame field of view range of the vision sensor 2;

[0063] Step S3: Perform point cloud preprocessing on each frame of the original point cloud image;

[0064] Step S4: Use point cloud registration and stitching technology to fuse at least three frames of point cloud images after preprocessing to obtain the corresponding local three-dimensional point cloud model of the weldment at the current station;

[0065] Step S5: In the local three-dimensional point cloud model of the weldment, segment out the point cloud set of the groove area;

[0066] Step S6: Extract the edge points of the point cloud set of the groove area, and obtain the weld feature point set based on this;

[0067] Step S7: Fit the weld feature point set to generate a continuous weld track, interpolate points on the continuous weld track to generate a welding path point sequence, plan the pose matrix of the welding torch 3 at each welding path point, convert the pose matrix sequence into a motion instruction, and drive the six-degree-of-freedom manipulator 12 to drive the welding torch 3 to complete the precise tracking of the continuous weld track;

[0068] Step S8: Based on the continuous weld track, combined with the manipulability constraint of the six-degree-of-freedom manipulator 12, plan the moving path of the three-degree-of-freedom moving chassis 11, and drive the moving manipulator 1 to move from this station to the next station roughly along the moving path;

[0069] Step S9: Repeat the above steps S2 - S8 to achieve continuous tracking of the large-scale space weld through multi-station switching operation, and terminate the control process when the weld end point is detected, that is, when the weld feature point set is empty.

[0070] In this embodiment, the step S3 includes: processing the acquired single-frame original point cloud image, establishing the topological relationship between discrete point cloud data through the KD-tree spatial index structure, which can reduce the neighborhood point search time and improve the point cloud processing efficiency; the original voxel filtering method realizes point cloud downsampling by calculating the centroid point within each voxel grid to replace all the point clouds within that voxel grid. In this embodiment, an improved voxel filtering method is used to reduce the point cloud density, calculating the point with the closest Euclidean distance to the centroid point within each voxel grid to replace the centroid point, ensuring that the downsampled point cloud is completely composed of the original data points, thereby maximizing the retention of the geometric feature information of the original point cloud; the statistical filtering algorithm is used to eliminate the outlier noise points of the point cloud, avoiding interference with subsequent methods such as point cloud registration and edge extraction.

[0071] In this embodiment, as Figure 4 shown, in the step S4, the specific implementation process of registering and splicing at least three frames of point cloud images is as follows:

[0072] Repeat the preset point cloud image registration and splicing steps until the splicing of all frames of point cloud images is completed, and obtain the corresponding local three-dimensional point cloud model of the welded part at the current station;

[0073] The preset point cloud image registration and splicing steps include:

[0074] Step S4-1: Mount the six-degree-of-freedom robotic arm with a vision sensor to capture two frames of point cloud images of the large-scale welded part to be tracked at adjacent poses. Through the hand-eye calibration matrix E T C and the six-degree-of-freedom robotic arm end pose matrix B T E corresponding to the acquisition of a single-frame point cloud image, unify the point cloud coordinates to the six-degree-of-freedom robotic arm base coordinate system, and represent them as the frame point cloud set P i and the frame point cloud set P j respectively, to achieve rough splicing of the point cloud. The implementation formula is: B P = B T E E T C C P. In the formula, B P represents the point cloud data in the six-degree-of-freedom robotic arm base coordinate system, C P represents the preprocessed point cloud data in the vision sensor 2 coordinate system. Among them, there is a partial overlap area between the frame point cloud set P i and the frame point cloud set P j . At this time, due to possible small errors in the vision sensor calibration, hand-eye calibration, or robotic arm coordinate system conversion process, the two frames of point clouds may not be accurately and completely spliced together;

[0075] Step S4-2: Select the frame point cloud set P with the largest number of point clouds i as the target point cloud, and the frame point cloud set P j as the source point cloud. Based on the octree spatial index structure, use the Euclidean distance constraint to extract the frame point cloud set P i and the frame point cloud set P j to obtain the overlapping region point cloud set Q i and the overlapping region point cloud set Q j , specifically by traversing the frame point cloud set P i , using the frame point cloud set P j as the search space, performing a radius search based on the KD-tree, and marking the points that meet the distance condition as the overlapping region point cloud;

[0076] Step S4-3: For each point p i in the overlapping region point cloud set Q i , perform a radius search to obtain its neighborhood point set, and solve the eigenvalues of the weighted covariance matrix formed by the point p i and its neighborhood point set where λ i 1 > λ i 2 > λ i 3 , and the weighted covariance matrix is: In the formula, k represents the number of neighborhood points, and dist ij represents the Euclidean distance between the point p i and its jth neighborhood point p ij . If the eigenvalues meet the condition: then the point p i is defined as an ISS feature point, and the point p i is put into the ISS feature point set S i ; Use the same method to screen out the ISS feature points in the overlapping region point cloud set Q j , and put them into the ISS feature point set S j ;

[0077] Step S4-4: For the ISS feature point sets S i and S j , use the point-to-plane closest point iterative (ICP) algorithm for registration to obtain the rigid body transformation matrix of the two point clouds;

[0078] Step S4-5: Apply the rigid body transformation matrix to the frame point cloud set P j for linear transformation to obtain the point cloud in the coordinate system of the frame point cloud set P i , and merge the two point clouds into O i to achieve precise point cloud stitching;

[0079] After completing the stitching of all frame point cloud images and obtaining the local three-dimensional point cloud model of the welded part corresponding to the current standing position, use the improved voxel filtering method to reduce the redundant point cloud density in the overlapping area. Otherwise, the repeated point clouds will affect the weights of the subsequent weighted principal component analysis method and the accuracy of the point cloud normal vector calculation. Figure 5 is the stitching effect of two frames of point clouds.

[0080] In this embodiment, the specific implementation steps of step S5 are as follows:

[0081] Step S5-1: For the local three-dimensional point cloud model of the welded part, use the weighted principal component analysis method based on Euclidean distance to calculate the normal vector of the welded part point cloud. Based on the KD-tree, perform a k-nearest neighbor search for each point p m to obtain its neighborhood point set, and solve the eigenvector corresponding to the minimum eigenvalue of the covariance matrix formed by the k-neighborhood point set of point p m . This eigenvector is the unit normal vector of point p m The covariance matrix formula is: In the formula, k represents the number of neighborhood points, dist mn represents the Euclidean distance between point p m and its nth neighborhood point p mn , ε represents a number approaching 0, and p represents the geometric centroid of point p m and its neighborhood point set; taking the spatial position p view of the vision sensor in the base coordinate system of the six-degree-of-freedom robotic arm as the reference viewing point, redirect the point cloud normal vector to ensure that the point cloud normal vector uniformly points to the camera direction. The method is as follows:

[0082] Step S5-2: By analyzing the change of the normal vector direction in the neighborhood of the point cloud, it can be obtained that in the flat base material surface area, the normal vector directions of the point cloud tend to be consistent and the included angle values are relatively small, while in the groove area with geometric mutations, the normal vector directions of the point cloud show obvious mutations and the included angle values are relatively large. Therefore, construct a surface change feature descriptor SVFD based on the point cloud normal vector information to describe the concave and convex changes on the surface of the large-scale welded part 5 to be tracked; based on the KD-tree, perform a radius search for each point p m to obtain its neighborhood point set, traverse the neighborhood point set of p m , calculate the included angle value θ m between point p mn and each neighborhood point p mn and its arithmetic mean . The calculation formula is: Use variance to characterize the local SVFD of point p m . The calculation formula is: According to point p mWith respect to the local SVFD of all its neighborhood points p mn a global SVFD of the starting point p m i.e., the surface variation feature descriptor SVFD, is constructed, and the calculation formula is: where σ mn represents the local SVFD of the neighborhood point p mn and represents the arithmetic mean of the local SVFDs of all neighborhood points of the point p m ;

[0083] Step S5-3: Set an appropriate screening threshold, and screen out the points with the surface variation feature descriptor SVFD greater than the screening threshold to construct a new point cloud set; due to factors such as material properties, processing technology, and operating environment, there may still be uneven areas on the surface of the large-scale weldment 5 to be tracked. When applying SVFD to extract the welding groove area, these uneven areas may be mis-extracted; and the mis-extracted uneven areas usually show spatial discrete distribution and a small number of point clouds. Use the Euclidean clustering algorithm to perform clustering segmentation on this point cloud set, and sort the clustering results, and select the category with the largest number of point clouds as the point cloud set of the groove area. At this time, the point cloud set of the groove area can be completely and effectively extracted.

[0084] In this embodiment, the specific implementation steps of the step S6 are as follows:

[0085] Step S6-1: Traverse the point cloud set of the groove area, perform radius search on each point p u based on the KD-tree to obtain its neighborhood point set, and calculate the size of the vector angle formed by the projection of the point p u and all its neighborhood points on the fitting tangent plane to obtain the angle set of the point p u . Take the maximum value α max in the angle set. If α max is greater than the set angle threshold β, then determine that the point p u is an edge point and put it into the edge point set of the groove area;

[0086] Step S6-2: Traverse the edge point set of the groove area, and use the principal component analysis method to calculate the local tangent vector u of each point p For the point p u , find the point p v that meets the conditions in the edge point set of the groove area through point cloud tangent vector estimation and orthogonal constraint conditions, and put it into the corresponding point candidate set. Then, select the point with the closest Euclidean distance to the point p u as the corresponding point p u ' on both sides. The screening conditions are: In the formula, w is the groove width value, and Δθ is the angle threshold;

[0087] Step S6-3, calculate point p u and its corresponding point p u ′s geometric center w u ′ and the mean value of the normal vector The calculation formula is as follows: In the formula is the normal vector of point p u , is the normal vector of the corresponding point p u ′. According to the known groove depth d, calculate the bottom feature points of the groove, that is, the weld feature points, according to and put them into the weld feature point set.

[0088] In this embodiment, in the step S7, the non-uniform rational B-spline (NURBS) algorithm is used to fit the weld feature points to obtain a continuous weld track, and the continuous weld track is interpolated based on the equal arc length principle to obtain a welding path point sequence, ensuring that the welding path point sequence sent to the six-degree-of-freedom robotic arm 12 is smooth and orderly, thereby minimizing the influence of outliers and invalid points.

[0089] In this embodiment, in the step S8, by establishing the kinematic model and the motion Jacobian matrix J(q) of the six-degree-of-freedom robotic arm 12 carrying the welding torch 3, the determinant of the motion Jacobian matrix J(q) is used as a metric index to analyze the manipulability of the six-degree-of-freedom robotic arm 12: where det represents the determinant of the Jacobian matrix; and based on the Monte Carlo method, the TCP manipulability distribution space is drawn, and a dexterous working radius is set in the manipulability distribution space as the tracking radius of the three-degree-of-freedom mobile chassis 11 and the large-scale weldment 5 to be tracked. Based on the tracking radius, an equidistant offset is made to the weld track to obtain the moving path of the three-degree-of-freedom mobile chassis 11.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; all changes made according to the technical solutions of the present invention, when the functions and effects produced do not exceed the scope of the technical solutions of the present invention, all belong to the protection scope of the present invention.

Claims

1. A large-scale space weld mobile robot automatic tracking system, characterized in that: The invention comprises a mobile robot arm, a visual sensor, a welding gun and a computer; the mobile robot arm comprises a three-degree-of-freedom mobile chassis and a six-degree-of-freedom robot arm; the visual sensor is a binocular structured light camera; the visual sensor and the welding gun are installed at the end of the six-degree-of-freedom robot arm; the visual sensor shoots the surface of a large-scale weldment to be tracked at different viewing angles to generate at least three frames of 3D point cloud data, and transmits the point cloud data to the computer; the computer aligns and splices the point cloud data through an algorithm, identifies and extracts weld feature points in the 3D point cloud, obtains the spatial trajectory of the weld feature points, and plans the end movement trajectory of the six-degree-of-freedom robot arm and the movement path of the three-degree-of-freedom mobile chassis respectively according to the spatial trajectory, and converts them into motion instructions; the computer sends the motion instruction to the mobile robot arm, the six-degree-of-freedom robot arm drives the welding gun to firstly complete the accurate tracking of the spatial trajectory of the weld feature points along the end movement trajectory, and the three-degree-of-freedom mobile chassis then tracks and moves along the movement path, and the mobile tracking of the large-scale spatial weld is realized through the alternating trajectory tracking of the six-degree-of-freedom robot arm and the three-degree-of-freedom mobile chassis.

2. The large-scale space weld mobile robot automatic tracking system according to claim 1 is characterized in that: The large-scale weldment to be tracked is a plane weldment or an irregular curved surface weldment with symmetrical V-shaped, X-shaped, or U-shaped grooves, and the weld is a spatial straight line, oblique line, or irregular curve.

3. A large-scale space weld mobile robot automatic tracking method, characterized in that: The method adopts a multi-station segmented weld tracking strategy and is implemented based on the large-scale space weld mobile robot automatic tracking system according to claim 1 or 2, and includes the following steps: Step S1, controlling the mobile robot arm to move to an initial position; Step S2, keeping the three-degree-of-freedom mobile chassis stationary, controlling the six-degree-of-freedom robotic arm equipped with a visual sensor to shoot a large-scale weldment to be tracked in multiple preset positions, and generating at least three frames of original point cloud images; Step S3, performing point cloud preprocessing on each frame of original point cloud image; Step S4, fusing at least three frames of pre-processed point cloud images using point cloud registration and stitching technology to obtain a local three-dimensional point cloud model of the weldment corresponding to the current position; Step S5, segmenting the groove area point cloud set in the local three-dimensional point cloud model of the weldment; Step S6, extracting edge points of the point cloud set in the groove area, and obtaining a weld feature point set based on the edge points; Step S7, fitting the weld feature point set to generate a continuous weld trajectory, interpolating points on the continuous weld trajectory to generate a welding path point sequence, planning the posture matrix of the welding gun at each welding path point, converting the posture matrix sequence into motion instructions and driving the six-degree-of-freedom robot arm to drive the welding gun to complete the precise tracking of the continuous weld trajectory; Step S8: Based on the continuous weld trajectory and in combination with the maneuverability constraints of the six-degree-of-freedom robot arm, a moving path of the three-degree-of-freedom mobile chassis is planned, and the mobile robot arm is driven to move from the current station to the next station approximately along the moving path; Step S9, repeat the above steps S2-S8, realize continuous tracking of large-scale spatial welds through multi-station switching operations, and terminate the control process when the weld end point is detected, that is, the weld feature point set is empty.

4. The method for automatically tracking a large-scale space weld mobile robot according to claim 3, characterized in that: The step S3 includes: processing the acquired single-frame original point cloud image, establishing the topological relationship between discrete point cloud data through the KD-tree spatial index structure; using the improved voxel filtering method to reduce the point cloud density, calculating the point in each voxel grid with the closest Euclidean distance to the center of gravity to replace the center of gravity; and then using the statistical filtering algorithm to eliminate point cloud outlier noise points.

5. The method for automatically tracking a large-scale space weld mobile robot according to claim 3, characterized in that: In step S4, the specific implementation process of registering and stitching at least three frames of point cloud images is as follows: Repeat the preset point cloud image registration and stitching steps until all frame point cloud images are stitched together to obtain the corresponding local 3D point cloud model of the weldment at the current station; The preset point cloud image registration and stitching steps include: Step S4-1: Use a six-degree-of-freedom robotic arm equipped with a visual sensor to capture two frames of point cloud images of a large-scale weldment to be tracked in two adjacent positions, and use the hand-eye calibration matrix E T C The six-DOF manipulator end pose matrix corresponding to the single-frame point cloud image acquisition B T E , unify the point cloud coordinates to the six-DOF robot base coordinate system, and represent them as the first frame point cloud set P i and the second frame point cloud set P j , to achieve the rough stitching of point cloud, the implementation formula is: B P= B T E E T C C P, where B P represents the point cloud data in the six-degree-of-freedom robot base coordinate system. C P represents the preprocessed point cloud data in the visual sensor coordinate system; among them, the first frame point cloud set P i and the second frame point cloud set P j There are some overlapping areas; Step S4-2: Select the first frame point cloud set P with the largest number of point clouds i As the target point cloud, the second frame point cloud set P j As the source point cloud, based on the octree spatial index structure, the first frame point cloud set P is extracted using the Euclidean distance constraint. i and the second frame point cloud set P j The first overlapping area point cloud set Q i and the second overlapping area point cloud set Q j ; Step S4-3: the first overlapping area point cloud set Q i Every point p in i Perform radius search to obtain its neighborhood point set and solve point p i The eigenvalue of the weighted covariance matrix formed by its neighborhood point set in The weighted covariance matrix is: In the formula, T represents the transposition of the matrix, k represents the number of neighborhood points, and dist ij Representative point p i and its jth neighboring point p ij The Euclidean distance between them, if the eigenvalues ​​meet the conditions: Then point p i Defined as the ISS feature point, point p i Put the first ISS feature point set S i In the same way, the second overlapping area point cloud set Q is selected j and put them into the second ISS feature point set S j middle; Step S4-4: For the first ISS feature point set S i and the second ISS feature point set S j , use the closest point iterative algorithm ICP to align and obtain the rigid body transformation matrix of the two point clouds; Step S4-5: Apply the rigid body transformation matrix to the second frame point cloud set P j Perform linear transformation to obtain the point cloud set P in the first frame i Point cloud in the coordinate system, merge the two point clouds into O i , to achieve precise point cloud stitching; After completing the stitching of all frame point cloud images and obtaining the local three-dimensional point cloud model of the weldment corresponding to the current position, the improved voxel filtering method is used to reduce the redundant point cloud density in the overlapping area.

6. The method for automatically tracking a large-scale space weld mobile robot according to claim 3, characterized in that: The specific implementation steps of step S5 are: Step S5-1: For the local 3D point cloud model of the weldment, a weighted principal component analysis method based on Euclidean distance is used to calculate the normal vector of the weldment point cloud. m Perform a k-nearest neighbor search to obtain its neighborhood point set and solve point p m The eigenvector corresponding to the minimum eigenvalue of the covariance matrix composed of the k-neighborhood point set is the eigenvector of point p m The unit normal vector The covariance matrix formula is: In the formula, k represents the number of neighborhood points, dist mn Representative point p m and its nth neighbor point p mn The Euclidean distance between them, ε represents a number close to 0, and p represents point p m The geometric centroid of its neighborhood point set; The spatial position p of the visual sensor in the six-degree-of-freedom robot base coordinate system view As the reference viewpoint, redirect the point cloud normal vector to ensure that the point cloud normal vector points to the camera direction uniformly. The method is: Step S5-2: construct a surface variation feature descriptor SVFD based on the point cloud normal vector information, and perform KD-tree analysis on each point p m Perform radius search to obtain its neighborhood point set and traverse p m Neighborhood point set, calculate point p m With each neighborhood point p mn The normal vector angle θ mn and its arithmetic mean The calculation formula is: Use variance to characterize point p m The local SVFD of is calculated as: According to point p m With all its neighboring points p mn The local SVFD of the starting point p is constructed m The global SVFD is the surface variation feature descriptor SVFD, and the calculation formula is: Among them, σ mn Represents the neighborhood point p mn The local SVFD of Represents point p m The arithmetic mean of the local SVFD of all neighborhood points. Step S5-3, set a suitable screening threshold, screen out points whose surface variation feature descriptor SVFD is greater than the screening threshold to construct a new point cloud set, use the Euclidean clustering algorithm to cluster and segment the point cloud set, sort the clustering results, and select the category with the largest number of point clouds as the groove area point cloud set.

7. The method for automatically tracking a large-scale space weld mobile robot according to claim 3, characterized in that: The specific implementation steps of step S6 are: Step S6-1, traverse the point cloud set in the groove area, and for each point p u Perform radius search to obtain its neighborhood point set and calculate point p u The point p is obtained by the size of the vector angle formed by the projection of all its neighboring points on the fitting tangent plane. u The angle set, take the maximum value α in the angle set max , if α max If the angle is greater than the set angle threshold β, then the point p is determined u is an edge point, which is put into the edge point set of the groove area; Step S6-2: traverse the set of edge points in the groove area and use the principal component analysis method to calculate the p of each point u The local tangent vector of For point p u , find the point p that meets the conditions in the set of edge points in the groove area v , and put it into the corresponding point candidate point set, and then select the corresponding point from the candidate point set. u The point with the closest Euclidean distance is taken as the corresponding point p on both sides of the edge u ′, the screening conditions are: In the formula, w is the groove width value, Δθ is the angle threshold; Step S6-3, calculate point p u Its corresponding point p u The geometric center w of ′ u ′ and the normal vector mean The calculation formula is: In the formula For point p u The normal vector of For the corresponding point p u ′, according to the known groove depth d, according to The slope bottom feature point, i.e., the weld feature point, is calculated and put into the weld feature point set.

8. The method for automatically tracking a large-scale space weld mobile robot according to claim 3, characterized in that: In the step S7, the non-uniform rational B-spline algorithm (NURBS) is used to fit the weld feature points to obtain a continuous weld trajectory, and the continuous weld trajectory is interpolated based on the equal arc length principle to obtain a welding path point sequence.

9. The method for automatically tracking a large-scale space weld mobile robot according to claim 3, characterized in that: In the step S8, the operability of the six-degree-of-freedom robot is analyzed, and the TCP operability distribution space is drawn based on the Monte Carlo method. A dexterous working radius is set in the operability distribution space as the tracking radius of the three-degree-of-freedom mobile chassis and the large-scale weldment to be tracked. Based on the tracking radius, the weld trajectory is equidistantly offset to obtain the moving path of the three-degree-of-freedom mobile chassis.

Images