A method and system for rapid generation of robot trajectories based on adaptive key point extraction

Abstract

This invention belongs to the technical field related to robot trajectory generation and discloses a method and system for rapid robot trajectory generation based on adaptive keypoint extraction. The method comprises the following steps: using a point cloud of a sample area to generate multiple sample trajectories for processing the sample area; merging these multiple sample trajectories to form a reference trajectory, each of which includes N trajectory processing points; extracting M processing points from the initial processing trajectory of the area to be processed as key points; and performing trajectory planning on these M key points and the N trajectory processing points in the reference trajectory to obtain a generalized trajectory passing through the M key points. This generalized trajectory serves as the final processing trajectory for the area to be processed. This invention solves the problem of rapid robot trajectory generation in complex scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field related to robot trajectory generation, and more specifically, relates to a method and system for quickly generating robot trajectories based on adaptive key point extraction. Background Art

[0002] Complex cabins are important components of spacecraft, spacecraft and missiles. Space cabins have the characteristics of many types, complex internal cavity features, uneven blank allowances, and large sizes. During measurement and processing, there are problems such as the difficulty of measuring point cloud data and the low efficiency of customized programming of processing trajectories.

[0003] Compared to traditional machining centers, industrial robots offer greater flexibility and lower cost, making them widely used in automated production lines and flexible manufacturing systems. The accuracy of robot machining paths determines part quality. There are two main methods for generating robot machining paths: those based on part CAD models and those based on point cloud data. CAD-based methods can be categorized by data format, using discrete triangular facets or parametric surface models. However, in actual machining, complex surfaces without CAD models often require scanning point cloud data to capture the workpiece surface topography.

[0004] There are currently two methods for generating robot machining trajectories based on point clouds. One involves reverse modeling point cloud data to obtain the corresponding parametric surface, which is then used to generate the robot machining trajectory. However, problems such as holes in the point cloud can affect the accuracy of the surface reconstruction, resulting in low precision in the generated machining trajectory. Another method directly generates machining or movement trajectories based on point cloud data. These methods include slicing, meshing, and feature point fitting. The slicing method is the most widely used. Its principle is to obtain the projection point and normal of the end effector based on point cloud slices. The robot's movement path in space is generated by offsetting the normals of key sampling points. These methods require adjustment for different scenarios, making it difficult to find a universal path planning strategy.

[0005] Therefore, there is an urgent need for a method to quickly generate robot processing trajectories in complex scenarios to improve the processing trajectory planning speed and processing efficiency of parts. Summary of the Invention

[0006] In response to the above defects or improvement needs of the prior art, the present invention provides a method and system for rapid generation of robot trajectories based on adaptive key point extraction to solve the problem of rapid generation of robot processing trajectories in complex scenes.

[0007] To achieve the above object, according to one aspect of the present invention, a method for rapidly generating robot trajectories based on adaptive key point extraction is provided, the method comprising the following steps:

[0008] Using the point cloud of the sample area, a plurality of sample trajectories for processing the sample area are generated, and the plurality of sample trajectories are merged to form a reference trajectory, wherein each reference trajectory includes N trajectory processing points;

[0009] Extract M processing points as key points from the initial processing trajectory of the area to be processed;

[0010] The M key points and the N trajectory processing points in the reference trajectory are subjected to trajectory planning to obtain a generalized trajectory passing through the M key points. The generalized trajectory is the final processing trajectory of the area to be processed.

[0011] Further preferably, the extracting of M processing points as key points is performed according to the following steps:

[0012] (a) connecting the starting point and the ending point of the trajectory to form a straight line, calculating the distance from each point on the trajectory to the straight line, discarding points whose distance is less than a preset threshold, and retaining points whose distance is greater than the preset threshold, the starting point, and the ending point;

[0013] (b) Split the trajectory into two trajectories using the point with the largest distance, and return to step (a) for each trajectory until there is no point on the trajectory with a distance greater than a preset threshold, and the retained point is regarded as the key point.

[0014] Further preferably, the point cloud of the sample area is used to generate a plurality of sample trajectories for processing the sample area according to the following steps:

[0015] Preprocess the point cloud of the sample area to remove noise and obtain the point cloud set Q after removing noise;

[0016] Calculate the centroid of all points in the point cloud set Q and the normal vector of each point;

[0017] The sample trajectory is generated by using the center of mass and the normal vector of each point using the least square method.

[0018] Further preferably, after the sample trajectory is obtained, the obtained sample trajectory is rotated around the X, Y or Z axis to generate a new sample trajectory, until a required number of sample trajectories are obtained.

[0019] Further preferably, the merging of the multiple sample trajectories to form a reference trajectory is performed according to the following steps:

[0020] The trajectory points corresponding to each moment on all sample trajectories are used as sample points to form a sample trajectory dataset;

[0021] Use Gaussian mixture model to estimate the joint distribution probability of all sample points;

[0022] The conditional probability distribution of each sample point is obtained by calculating the joint distribution probability of the sample points. The conditional probability distribution of each sample point constitutes a reference trajectory point data set, that is, a reference trajectory is formed.

[0023] Further preferably, the calculation of the conditional probability distribution adopts a mixed Gaussian regression method.

[0024] Further preferably, the generalized trajectory is obtained by generalizing the trajectory using the key points as extrapolation features of the imitation learning KMP algorithm.

[0025] According to another aspect of the present invention, a system for rapidly generating robot trajectories based on adaptive key point extraction is provided, comprising a processor configured to execute the method for rapidly generating robot trajectories based on adaptive key point extraction as described above.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for quickly generating a robot trajectory based on adaptive key point extraction as described above is implemented.

[0027] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0028] 1. In this invention, a sample area is used to obtain N trajectory processing points on a reference trajectory. Then, combined with M key points on the initial processing trajectory of the area to be processed, the processing trajectory of the area to be processed is obtained by generalization using M+N points. This method does not require adjustment according to different scenarios and solves the problem of rapid generation of robot processing trajectories in complex scenarios.

[0029] 2. In existing modeling and slicing methods for generating trajectories from point clouds, the loss of local trajectory points caused by point cloud holes will reduce the global trajectory accuracy. The present invention generates multiple sample trajectories from the sample area, and then merges the sample trajectories to obtain a reference trajectory to obtain N trajectory processing points. This trajectory generation method itself does not rely on complete point cloud data. It generalizes new trajectories from limited samples through imitation learning, avoiding the impact of incomplete point cloud data on the processing trajectory accuracy.

[0030] 3. Existing methods typically rely on complete point cloud data for inverse modeling or direct trajectory generation, which limits their adaptability to diverse and complex scenarios. In contrast, this invention uses a rapid trajectory generation method based on a generalized imitation learning KMP algorithm. Through imitation learning, it can extract key features from a limited number of sample trajectories and generalize new trajectories suitable for different machining scenarios, greatly expanding the application scope of robot machining trajectory generation.

[0031] 4. Existing point cloud-based trajectory generation methods typically involve complex inverse modeling and point cloud processing steps, which are computationally intensive and time-consuming, resulting in slow trajectory generation. However, a rapid trajectory generation method based on adaptive keypoint extraction significantly reduces trajectory generation time to approximately 2 seconds through efficient imitation learning and optimized data processing strategies. This enables the robot system to respond more quickly to environmental changes and task requirements, significantly enhancing the robot's adaptability and real-time processing capabilities in dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flow chart of a method for rapidly generating robot trajectories based on adaptive key point extraction constructed according to a preferred embodiment of the present invention;

[0033] Figure 2 It is the information transmission process between various tracks and methods constructed according to the preferred embodiment of the present invention;

[0034] Figure 3 is a projection diagram of a robot reference trajectory generated by GMM on the xy plane constructed according to a preferred embodiment of the present invention;

[0035] Figure 4 The key points of the target trajectory generated under different thinning thresholds constructed according to the preferred embodiment of the present invention are shown in Figure 1. (a) is a schematic diagram of the key points when the thinning threshold is 0.3, with a DTW distance of 122.4497 mm, and (b) is a schematic diagram of the key points when the thinning threshold is 1, with a DTW distance of 293.7336 mm.

[0036] Figure 5 The changes in DTW distance and calculation time under different thinning thresholds constructed according to the preferred embodiment of the present invention;

[0037] Figure 6 is a comparison diagram of the generalized trajectory constructed according to the preferred embodiment of the present invention, the reference trajectory and the target trajectory;

[0038] Figure 7 : This is a comparison diagram of various position errors between the generalized trajectory constructed according to the preferred embodiment of the present invention and the reference trajectory, wherein (a) is the comparison in the x-direction, (b) is the comparison in the y-direction, and (c) is the comparison in the z-direction;

[0039] Figure 8 It is a complete machining surface trajectory formed by splicing the generalized boundary trajectory, the sub-outer trajectory and the inner ring trajectory, etc., constructed according to the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0041] like Figure 1 and 2 As shown in FIG, a method for rapidly generating robot trajectories based on adaptive key point extraction includes the following steps:

[0042] The point cloud of S1 collected features is preprocessed by denoising and streamlining, and H sample trajectories are generated based on the moving least squares method. Each trajectory contains the position, velocity and acceleration of N points.

[0043] The specific sub-steps are as follows:

[0044] S1.1 uses a surface structured light camera to collect point cloud data, uses a straight-through filter to obtain the region of interest, and then uses Gaussian filtering to remove noise during point cloud measurement to obtain a point cloud set Q. Each point in the point set is q j (x j ,y j ,z j ).

[0045] S1.2 Use PCA principal component analysis to complete the normal feature estimation of the point cloud, calculate the three-dimensional mean of each point in the point set Q to obtain the point cloud centroid O i (x i ,y i ,z i ), the problem of obtaining the surface normal is converted into the problem of obtaining the eigenvalues ​​and eigenvectors of the covariance matrix C. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix C is the normal vector of the plane P.

[0046]

[0047] On this basis, normal vector orientation is completed based on the minimum spanning tree (MST).

[0048] S1.3. Generate sample trajectories based on the moving least squares method. The key to this sub-step is the surface approximation function:

[0049]

[0050] Among them, p j (x,y) is the basis function, m is the number of basis functions, a j (x,y) is the coefficient of the basis function, a j(x,y) can be obtained by weighted least squares fitting of the local fit.

[0051] S1.4 To obtain H tracks, a single track is rotated by a small angle Δθ around the XYZ axis, and the number of sample tracks for each feature is increased to 4.

[0052] S2, using Gaussian mixture model (GMM) to extract conditional mean and conditional covariance information from the sample trajectory to obtain the reference trajectory, such as Figure 3 As shown, the points that constitute the trajectory are the predicted means, the red ellipsoid is the corresponding Gaussian distribution, and the trajectory contains 86 points.

[0053] The specific sub-steps are as follows:

[0054] S2.1. The sample trajectory dataset is represented as where s n,h is the input, is the time, ξ n,h is the output, i.e. the position of the trajectory point, H and N represent the number of sample trajectories and the number of trajectory points respectively. (Here n=1,…,N,h=1,…,H) The Gaussian mixture model GMM trajectory is used to estimate the joint distribution probability P(s,ξ) of the sample points, i.e.

[0055]

[0056] where π c 、μ c and Σ c They represent the prior probability, mean and covariance of the cth Gaussian component respectively, and C represents the number of Gaussian components.

[0057] S2.2. Obtain reference trajectory based on Gaussian mixture regression (GMR) Among them and s n Each relevant point Described as having a mean and covariance The conditional probability distribution of , that is:

[0058]

[0059] Therefore, the reference trajectory dataset (reference dataset) can be expressed as

[0060]

[0061] S3. Extract the key intermediate points of the target trajectory and use these key points as extrapolation features of the imitation learning KMP method to generalize the trajectory.

[0062] The specific sub-steps are as follows:

[0063] S3.1. Connect the starting and ending points, that is, draw a virtual straight line between the starting point and the ending point of the trajectory.

[0064] S3.2. Calculate the distance from all points on the trajectory to the line, and find the maximum distance d. max .

[0065] S3.3. Compare the distance of each point to the line with the preset threshold D. Discard the points with a distance less than D and retain the points with a distance greater than D. Among the points retained, the point with the largest distance d is selected. max The point will split the trajectory into two trajectories, and return to step S3.1 for each trajectory until the distance di is less than D

[0066] Select and compare, set the threshold D, and set d max Compare with D. If d max If d is less than D, all intermediate points between the starting point and the end point on the trajectory are discarded; if d max If it is greater than or equal to D, then keep the starting point, end point and corresponding d max point, and the corresponding d max The points are used as boundaries to segment the trajectory. Here, the thresholds are set to 0.3 and 0.1 respectively. Figure 4 It can be seen that lowering the threshold and increasing the key points can better restore the geometric characteristics of the trajectory.

[0067] S3.4. Repeat the above steps for the segmented trajectory until all points on the trajectory between the retained points meet the conditions for being discarded, that is, d max are all less than D. At this time, depending on the setting of the threshold D, 6 to 16 key points can be calculated. However, since the calculation time of the trajectory thinning algorithm increases with the decrease of D, it is necessary to compare the time and curve similarity to comprehensively select the best thinning threshold. Figure 5 As shown in the figure, when the thinning threshold is less than 0.4, the calculation time will increase significantly. When the thinning threshold is greater than 0.5, the DTW distance increases from 145.0549 to 220.0648, which is a significant increase. Therefore, considering all factors, a threshold of 0.4 or 0.5 can be selected, and the two have the same effect. Here, D = 0.5 is selected.

[0068] S4. Update the reference trajectory using the 12 trajectory key points calculated in step S3. The new reference trajectory contains 98 points. Set the hyperparameters λ and λ in the kernel motion primitive KMP. c , according to the kernel function k(·,·), KMP is used to predict the trajectories passing through the key intermediate points to obtain the generalized trajectory. Here, the various hyperparameters of KMP are set as follows: the number of trajectories used n = 4, the time interval dt = 0.01s.

[0069] The specific sub-steps are as follows:

[0070] S4.1. Update the sample set and use the M trajectory key points obtained in S3 (12 points in the example) to expand the reference dataset Get the expanded reference dataset

[0071]

[0072] S4.2, construct the parameter trajectory as:

[0073] ξ(s)=Θ(s) T w

[0074] in, is the B-dimensional basis function, and the weights follow the normal distribution Therefore, the parameter trajectory can also be written as:

[0075]

[0076] S4.3. To ensure the minimum information loss during the imitation learning process, consider minimizing the target form.

[0077] After optimizing the distribution using KL divergence, we can get the optimization goal of minimizing the following two formulas:

[0078]

[0079] Among them, λ, λ c is the penalty term, here λ=5, λ c =10, which can effectively prevent the overfitting problem in the solution.

[0080] S4.4. Solving for μ w The optimal solution According to the dual transformation of KRR (ridge regression), we can get for:

[0081]

[0082] Where Φ, Σ and μ are:

[0083] Φ=[Θ(s1)Θ(s2)…Θ(s N )]

[0084]

[0085] According to the definition of parameter trajectory in S4.2, for a new input point s * , the expected value of the corresponding output is:

[0086]

[0087] In order to facilitate complex calculations when the data is high-dimensional, the above formula is kernelized and defined as follows:

[0088]

[0089] k * =[k(s * ,s1)k(s * ,s2)…k(s * ,s N )]

[0090] Where I is the identity matrix. Then Can be rewritten as:

[0091]

[0092] This enables prediction of output expectations.

[0093] S4.5, J(∑ w ) expression is set to 0, we can get:

[0094]

[0095] The optimal solution for:

[0096]

[0097] According to the Woodbury equation and the definition of the kernel matrix above, ξ(s * ) has a covariance of

[0098]

[0099] This enables the prediction of the mean and covariance of the KMP trajectory

[0100] The comparison between the generalized trajectory and the reference trajectory and the target trajectory is as follows: Figure 6 As shown, the position error comparison of the generalized trajectory x, y, and z directions is Figure 7 As shown in Figure 2, the marked points are the key point set P in the target trajectory. It can be seen that the RSME error of the predicted generalized trajectory is less than 0.35 mm for the three items XYZ compared with the target trajectory, which shows high accuracy.

[0101] Generate multiple unit trajectories for splicing, connect adjacent trajectories, and perform smoothing based on B-spline. The processing trajectory of the complete feature surface after splicing is as follows Figure 8 shown.

[0102] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for rapid generation of robot trajectories based on adaptive key point extraction, characterized in that: The method comprises the following steps: Using the point cloud of the sample area, a plurality of sample trajectories for processing the sample area are generated, and the plurality of sample trajectories are merged to form a reference trajectory, wherein each reference trajectory includes N trajectory processing points; Extract M processing points as key points from the initial processing trajectory of the area to be processed; Perform trajectory planning on the M key points and the N trajectory processing points in the reference trajectory to obtain a generalized trajectory passing through the M key points. The generalized trajectory is the final processing trajectory of the area to be processed; The extraction of M processing points as key points is performed according to the following steps: (a) connecting the starting point and the ending point of the trajectory to form a straight line, calculating the distance from each point on the trajectory to the straight line, discarding points with distances less than a preset threshold, and retaining points with distances greater than the preset threshold, the starting point, and the ending point; (b) Split the trajectory into two trajectories using the point with the largest distance, and return to step (a) for each trajectory until there are no points on the trajectory with a distance greater than a preset threshold, and the retained points are regarded as key points.

2. A method for rapid robot trajectory generation based on adaptive key point extraction as claimed in claim 1, characterized in that: The following steps are used to generate multiple sample trajectories for processing the sample area using the point cloud of the sample area: Preprocess the point cloud of the sample area to remove noise and obtain the point cloud set Q after removing noise; Calculate the centroid of all points in the point cloud set Q and the normal vector of each point; The sample trajectory is generated by using the center of mass and the normal vector of each point using the least square method.

3. A method for rapidly generating robot trajectories based on adaptive key point extraction according to claim 1 or 2, characterized in that: After the sample trajectory is obtained, the obtained sample trajectory is rotated around the X, Y or Z axis to generate a new sample trajectory until a required number of sample trajectories are obtained.

4. A method for rapid robot trajectory generation based on adaptive key point extraction as claimed in claim 2, characterized in that: The steps of merging the multiple sample trajectories to form a reference trajectory are as follows: The trajectory points corresponding to each moment on all sample trajectories are used as sample points to form a sample trajectory dataset; Use Gaussian mixture model to estimate the joint distribution probability of all sample points; The conditional probability distribution of each sample point is obtained by calculating the joint distribution probability of the sample points. The conditional probability distribution of each sample point constitutes a reference trajectory point data set, that is, a reference trajectory is formed.

5. A method for rapid robot trajectory generation based on adaptive key point extraction as claimed in claim 4, characterized in that: The calculation conditional probability distribution adopts Gaussian mixture regression method.

6. A method for rapid robot trajectory generation based on adaptive key point extraction as claimed in claim 1, characterized in that: The generalized trajectory is obtained by generalizing the trajectory using the key points as extrapolation features of the imitation learning KMP algorithm.

7. A robot trajectory rapid generation system based on adaptive key point extraction, characterized in that: The method comprises a processor configured to execute the method for rapidly generating robot trajectories based on adaptive key point extraction according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for rapidly generating a robot trajectory based on adaptive key point extraction according to any one of claims 1 to 6 is implemented.

Images