Robot trajectory rapid generation method and system based on adaptive key point extraction

Abstract

The invention belongs to the related technical field of robot trajectory generation, and discloses a robot trajectory rapid generation method and system based on adaptive key point extraction. The method comprises the following steps that point clouds of a sample area are used for generating and processing a plurality of sample tracks of the sample area, the plurality of sample tracks are combined to form a reference track, and each reference track comprises N track processing points; m machining points are extracted from the initial machining track of the to-be-machined area to serve as key points; trajectory planning is conducted on the M key points and N trajectory machining points in the reference trajectory, a generalization trajectory passing through the M key points is obtained, and the generalization trajectory is the final machining trajectory of the to-be-machined area. According to the invention, the problem of rapid generation of the robot processing track in a complex scene is solved.

Description

Technical Field

[0001] The present invention belongs to the technical field related to robot trajectory generation, and more specifically, relates to a method and system for quickly generating robot trajectories based on adaptive key point extraction. Background Art

[0002] Complex cabins are important components of spacecraft, spacecraft and missiles. Space cabins have the characteristics of many types, complex internal cavity features, uneven blank allowances, large sizes, etc. During measurement and processing, there are problems such as difficulty in measuring point cloud data and low efficiency in customized programming of processing trajectories.

[0003] Compared with traditional processing machine tools, industrial robots are highly flexible and inexpensive, and are widely used in automated production lines and flexible manufacturing systems. The accuracy of robot processing trajectory generation determines the quality of part processing. There are two main methods for robot processing trajectory generation: based on part CAD models and based on point cloud data. The CAD model-based method can be divided into discrete triangular patches and parametric surface models according to the data form. However, in actual processing, for complex surfaces without CAD models, it is usually necessary to scan point cloud data to reflect the surface morphology of the workpiece.

[0004] There are currently two methods for generating robot processing trajectories based on point clouds. One is to obtain the corresponding parametric surface by reverse modeling of point cloud data, and then generate the robot processing trajectory. Problems such as point cloud holes will affect the accuracy of surface reconstruction, resulting in low accuracy of the generated processing trajectory. Another method directly generates processing or movement trajectories based on point cloud data, including slicing method, grid method and feature point fitting method. Among them, the slicing method is widely used. Its principle is to obtain the projection point and normal of the end effector based on point cloud slicing, and generate the robot movement path in space by offsetting the normal of the key sampling points. These methods need to be adjusted according to different scenarios, and it is difficult to find a general path planning strategy.

[0005] Therefore, there is an urgent need for a method to quickly generate robot processing trajectories in complex scenarios in order to improve the processing trajectory planning speed and processing efficiency of parts. Summary of the invention

[0006] In view of the above defects or improvement needs of the prior art, the present invention provides a method and system for quickly generating robot trajectories based on adaptive key point extraction, which solves the problem of quickly generating robot processing trajectories in complex scenes.

[0007] To achieve the above object, according to one aspect of the present invention, a method for quickly generating robot trajectories based on adaptive key point extraction is provided, the method comprising the following steps:

[0008] Using the point cloud of the sample area, a plurality of sample trajectories for processing the sample area are generated, and the plurality of sample trajectories are merged to form a reference trajectory, wherein the reference trajectory includes N trajectory processing points;

[0009] Extract M processing points as key points from the initial processing trajectory of the area to be processed;

[0010] The M key points and the N trajectory processing points in the reference trajectory are subjected to trajectory planning to obtain a generalized trajectory passing through the M key points, and the generalized trajectory is the final processing trajectory of the area to be processed.

[0011] Further preferably, the extracting of M processing points as key points is performed according to the following steps:

[0012] (a) connecting the starting point and the ending point of the trajectory to form a straight line, calculating the distance from each point on the trajectory to the straight line, discarding points whose distance is less than a preset threshold, and retaining points whose distance is greater than the preset threshold, the starting point and the ending point;

[0013] (b) Using the point with the largest distance, the trajectory is split into two trajectories. For each trajectory, the process returns to step (a) until there is no point on the trajectory whose distance is greater than a preset threshold. The retained points are regarded as key points.

[0014] Further preferably, the point cloud of the sample area is used to generate a plurality of sample trajectories for processing the sample area according to the following steps:

[0015] Preprocess the point cloud of the sample area to remove noise, and obtain the point cloud set Q after removing noise;

[0016] Calculate the centroid of all points in the point cloud set Q and the normal vector of each point;

[0017] The sample trajectory is generated by using the center of mass and the normal vector of each point using the least square method.

[0018] Further preferably, after the sample trajectory is obtained, the obtained sample trajectory is rotated around the X, Y or Z axis to generate a new sample trajectory until a required number of sample trajectories are obtained.

[0019] Further preferably, the merging of the multiple sample trajectories to form a reference trajectory is performed according to the following steps:

[0020] The trajectory points corresponding to each moment on all sample trajectories are taken as sample points to form a sample trajectory data set;

[0021] Use Gaussian mixture model to estimate the joint distribution probability of all sample points;

[0022] The conditional probability distribution of each sample point is obtained by using the joint distribution probability of the sample points, and the conditional probability distribution of each sample point constitutes a reference trajectory point data set, that is, forms a reference trajectory.

[0023] Further preferably, the calculating conditional probability distribution adopts a mixed Gaussian regression method.

[0024] Further preferably, the generalized trajectory is obtained by generalizing the trajectory using the key points as extrapolated features of the imitation learning KMP algorithm.

[0025] According to another aspect of the present invention, a system for rapidly generating robot trajectories based on adaptive key point extraction is provided, comprising a processor, wherein the processor is used to execute the method for rapidly generating robot trajectories based on adaptive key point extraction as described above.

[0026] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for quickly generating robot trajectories based on adaptive key point extraction as described above is implemented.

[0027] In general, the above technical solutions conceived by the present invention have the following beneficial effects compared with the prior art:

[0028] 1. In the present invention, a sample area is used to obtain N trajectory processing points on a reference trajectory, and then the processing trajectory of the area to be processed is obtained by generalizing M+N points in combination with the M key points on the initial processing trajectory of the area to be processed. This method does not need to be adjusted according to different scenes, and solves the problem of rapid generation of robot processing trajectories in complex scenes.

[0029] 2. In the existing modeling and slicing methods of generating trajectories from point clouds, the missing of local trajectory points caused by point cloud holes will reduce the global trajectory accuracy. The present invention generates multiple sample trajectories from the sample area, and then merges the sample trajectories to obtain a reference trajectory to obtain N trajectory processing points. The trajectory generation method itself does not rely on complete point cloud data. It generalizes new trajectories from limited samples through imitation learning, thereby avoiding the influence of the incompleteness of point cloud data on the processing trajectory accuracy.

[0030] 3. Existing methods usually rely on complete point cloud data for reverse modeling or direct trajectory generation, which limits their adaptability in the face of diverse and complex scenarios. On the contrary, the present invention adopts a fast trajectory generation method based on the generalization of the imitation learning KMP algorithm. Through imitation learning, key features can be extracted from limited sample trajectories, and new trajectories suitable for different processing scenarios can be generalized, which greatly expands the application scope of robot processing trajectory generation.

[0031] 4. Existing point cloud-based trajectory generation methods usually involve complex inverse modeling and point cloud processing steps, which are computationally intensive and time-consuming, resulting in slow trajectory generation. The fast trajectory generation method based on adaptive key point extraction greatly shortens the trajectory generation time through efficient imitation learning and optimized data processing strategies, and its generation time is only about 2 seconds. It enables the robot system to respond to environmental changes and task requirements more quickly, significantly enhancing the robot's adaptability and real-time processing capabilities in dynamic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a flow chart of a method for quickly generating robot trajectories based on adaptive key point extraction constructed according to a preferred embodiment of the present invention;

[0033] Figure 2 It is the information transmission process between various tracks and methods constructed according to the preferred embodiment of the present invention;

[0034] Figure 3 is a projection diagram of a robot reference trajectory generated by GMM on the xy plane constructed according to a preferred embodiment of the present invention;

[0035] Figure 4 : are the key points of the target trajectory generated under different thinning thresholds constructed according to the preferred embodiment of the present invention, wherein (a) is a schematic diagram of the key points when the thinning threshold is 0.3, and the DTW distance is 122.4497 mm, and (b) is a schematic diagram of the key points when the thinning threshold is 1, and the DTW distance is 293.7336 mm;

[0036] Figure 5 The changes of DTW distance and calculation time under different thinning thresholds constructed according to the preferred embodiment of the present invention;

[0037] Figure 6 is a comparison diagram of the generalized trajectory constructed according to the preferred embodiment of the present invention, the reference trajectory and the target trajectory;

[0038] Figure 7 1 is a comparison diagram of various position errors between the generalized trajectory and the reference trajectory constructed according to the preferred embodiment of the present invention, wherein (a) is the comparison in the x direction, (b) is the comparison in the y direction, and (c) is the comparison in the z direction;

[0039] Figure 8 It is a complete machining surface trajectory formed by splicing unit trajectories such as the generalized boundary trajectory, the sub-outer layer trajectory and the inner ring trajectory constructed according to the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0041] like Figure 1 and 2 As shown, a method for quickly generating robot trajectories based on adaptive key point extraction includes the following steps:

[0042] The point cloud of S1 collection features is preprocessed by denoising and streamlining, and H sample trajectories are generated based on the moving least squares method. Each trajectory contains the position, velocity and acceleration of N points.

[0043] The specific sub-steps are as follows:

[0044] S1.1 uses a surface structured light camera to collect point cloud data, uses a straight-through filter to obtain the area of ​​interest, and then uses a Gaussian filter to remove the noise during point cloud measurement to obtain a point cloud set Q. Each point in the point set is q j (x j ,y j ,z j ).

[0045] S1.2 Use PCA principal component analysis to estimate the normal features of the point cloud, and calculate the three-way mean of each point in the point set Q to obtain the point cloud centroid O i (x i ,y i ,z i ), the problem of obtaining the surface normal is converted into the problem of obtaining the eigenvalue and eigenvector of the covariance matrix C. The eigenvector corresponding to the minimum eigenvalue of the covariance matrix C is the normal vector of the plane P.

[0046]

[0047] On this basis, the normal vector orientation is completed based on the minimum spanning tree (MST).

[0048] S1.3. Generate sample trajectories based on the moving least squares method. The key to this sub-step is the surface approximation function:

[0049]

[0050] Among them, p j (x, y) is the basis function, m is the number of basis functions, a j (x,y) is the coefficient of the basis function, a j(x,y) can be obtained by weighted least squares fitting of the local fit.

[0051] S1.4 In order to obtain H tracks, a single track is rotated by a small angle Δθ around the XYZ axis, and the number of sample tracks for each feature is increased to 4.

[0052] S2, using Gaussian mixture model (GMM) to extract conditional mean and conditional covariance information from the sample trajectory to obtain the reference trajectory, such as Figure 3 As shown, the points that make up the trajectory are the predicted means, the red ellipsoid is the corresponding Gaussian distribution, and the trajectory contains 86 points.

[0053] The specific sub-steps are as follows:

[0054] S2.1. The sample trajectory dataset is represented as where s n,h is the input, is the time, ξ n,h is the output, i.e. the position of the trajectory point, H and N represent the number of sample trajectories and the number of trajectory points respectively. (Here n=1,…,N, h=1,…,H) The Gaussian mixture model GMM trajectory is used to estimate the joint distribution probability P(s,ξ) of the sample points, i.e.

[0055]

[0056] where π c , μ c and Σ c They represent the prior probability, mean and covariance of the cth Gaussian component respectively, and C represents the number of Gaussian components.

[0057] S2.2. Obtaining reference trajectory based on Gaussian mixture regression (GMR) Among them and s n Each relevant point Described as having a mean and covariance The conditional probability distribution of is:

[0058]

[0059] Therefore, the reference trajectory dataset (reference dataset) can be expressed as

[0060]

[0061] S3. Extract the key intermediate points of the target trajectory and use them as the extrapolation features of the imitation learning KMP method to generalize the trajectory.

[0062] The specific sub-steps are as follows:

[0063] S3.1. Connect the starting and ending points, that is, virtually connect a straight line between the starting point and the ending point of the trajectory.

[0064] S3.2. Calculate the distance, find the distance from all points on the trajectory to this line, and find the maximum distance value d max .

[0065] S3.3. Compare the distance of each point to the line with the preset threshold D. Discard the points whose distance is less than D, and retain the points whose distance is greater than D. max The point of the trajectory is split into two trajectories, and for each trajectory, return to step S3.1 until the distance di is less than D

[0066] Select and compare, set the threshold D, and set d max Compare with D. If d max If d is less than D, all the intermediate points between the starting point and the end point on the trajectory are discarded; max If it is greater than or equal to D, then keep the starting point, the end point and the corresponding d max point, and the corresponding d max The points are used as boundaries to segment the trajectory. Here, the thresholds are set to 0.3 and 0.1 respectively. Figure 4 It can be seen that lowering the threshold and increasing the key points can better restore the geometric features of the trajectory.

[0067] S3.4, repeat the above steps for the segmented trajectory until all points on the trajectory between the retained points meet the conditions for being discarded, that is, d max are all less than D. At this time, according to the different settings of the threshold D, 6 to 16 key points can be calculated. However, since the calculation time of the trajectory thinning algorithm increases with the decrease of D, it is necessary to compare the time and curve similarity and comprehensively select the best thinning threshold. Figure 5 As shown in the figure, when the thinning threshold is less than 0.4, the calculation time will increase significantly, and when the thinning threshold is greater than 0.5, the DTW distance increases from 145.0549 to 220.0648, which is significantly increased. Therefore, considering comprehensively, the threshold of 0.4 or 0.5 can be selected, and the two have the same effect. Here, D=0.5 is selected.

[0068] S4. Update the reference trajectory using the 12 trajectory key points calculated in step S3. The new reference trajectory contains 98 points. Set the hyperparameters λ and λ in the kernel motion primitive KMP. c , according to the kernel function k(·,·), KMP is used to predict the trajectory passing through the key intermediate points to obtain the generalized trajectory. Here, the various hyperparameters of KMP are set as follows: the number of trajectories used n = 4, the time interval dt = 0.01s.

[0069] The specific sub-steps are as follows:

[0070] S4.1. Update the sample set and use the M trajectory key points (12 points in the example) obtained in S3 to expand the reference data set Get the extended reference dataset

[0071]

[0072] S4.2, construct the parameter trajectory as:

[0073] ξ(s)=Θ(s) T w

[0074] in, is the B-dimensional basis function, and the weights follow a normal distribution Therefore, the parameter trajectory can also be written as:

[0075]

[0076] S4.3. To ensure the minimum information loss in the imitation learning process, consider minimizing the target form.

[0077] After optimizing the distribution using KL divergence, we can get the optimization goal of minimizing the following two formulas:

[0078]

[0079] Among them, λ, λ c is the penalty term, here λ=5,λ c =10, which can effectively prevent the overfitting problem in the solution.

[0080] S4.4. Solving μ w The optimal solution According to the dual transformation of KRR (ridge regression), we can get for:

[0081]

[0082] Where Φ, Σ and μ are:

[0083] Φ=[Θ(s1)Θ(s2)…Θ(s N )]

[0084]

[0085] According to the definition of parameter trajectory in S4.2, for a new input point s * , the expected value of the corresponding output is:

[0086]

[0087] In order to facilitate complex calculations when the data is high-dimensional, the above formula is kernelized and defined as follows:

[0088]

[0089] k * =[k(s * ,s1)k(s * ,s2)…k(s * ,s N )]

[0090] Where I is the identity matrix. Then Can be rewritten as:

[0091]

[0092] This enables prediction of output expectations.

[0093] S4.5, J(∑ w ) is set to 0, we get:

[0094]

[0095] The optimal solution for:

[0096]

[0097] According to the Woodbury equation and the definition of the kernel matrix above, ξ(s * ) has a covariance of

[0098]

[0099] This enables the prediction of the mean and covariance of the KMP trajectory

[0100] The comparison between the generalized trajectory and the reference trajectory and the target trajectory is as follows: Figure 6 As shown, the position error comparison of the generalized trajectory x, y, and z is Figure 7 As shown in the figure, the marked points are the key point set P in the target trajectory. It can be seen that compared with the target trajectory, the RSME errors of the three items of XYZ of the predicted generalized trajectory are all less than 0.35mm, and the accuracy is relatively high.

[0101] Generate multiple unit trajectories for splicing, connect adjacent trajectories, and perform smoothing based on B-spline. The processing trajectory of the complete feature surface after splicing is as follows: Figure 8 shown.

[0102] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for rapid generation of robot trajectories based on adaptive key point extraction, characterized in that: The method comprises the following steps: Using the point cloud of the sample area, a plurality of sample trajectories for processing the sample area are generated, and the plurality of sample trajectories are merged to form a reference trajectory, wherein the reference trajectory includes N trajectory processing points; Extract M processing points as key points from the initial processing trajectory of the area to be processed; The M key points and the N trajectory processing points in the reference trajectory are subjected to trajectory planning to obtain a generalized trajectory passing through the M key points, and the generalized trajectory is the final processing trajectory of the area to be processed.

2. A method for rapid generation of robot trajectories based on adaptive key point extraction as claimed in claim 1, characterized in that: The extraction of M processing points as key points is performed according to the following steps: (a) connecting the starting point and the ending point of the trajectory to form a straight line, calculating the distance from each point on the trajectory to the straight line, discarding points whose distance is less than a preset threshold, and retaining points whose distance is greater than the preset threshold, the starting point and the ending point; (b) Using the point with the largest distance, the trajectory is split into two trajectories. For each trajectory, the process returns to step (a) until there is no point on the trajectory whose distance is greater than a preset threshold. The retained points are regarded as key points.

3. A method for rapid generation of robot trajectories based on adaptive key point extraction as claimed in claim 1 or 2, characterized in that: Using the point cloud of the sample area to generate multiple sample trajectories for processing the sample area is done in the following steps: Preprocess the point cloud of the sample area to remove noise, and obtain the point cloud set Q after removing noise; Calculate the centroid of all points in the point cloud set Q and the normal vector of each point; The sample trajectory is generated by using the center of mass and the normal vector of each point using the least square method.

4. A method for rapidly generating robot trajectories based on adaptive key point extraction as claimed in claim 1 or 2, characterized in that: After the sample trajectory is obtained, the obtained sample trajectory is rotated around the X, Y or Z axis to generate a new sample trajectory until a required number of sample trajectories are obtained.

5. A method for rapid generation of robot trajectories based on adaptive key point extraction as claimed in claim 3, characterized in that: The merging of the multiple sample trajectories to form a reference trajectory is performed according to the following steps: The trajectory points corresponding to each moment on all sample trajectories are taken as sample points to form a sample trajectory data set; Use Gaussian mixture model to estimate the joint distribution probability of all sample points; The conditional probability distribution of each sample point is obtained by using the joint distribution probability of the sample points, and the conditional probability distribution of each sample point constitutes a reference trajectory point data set, that is, forms a reference trajectory.

6. A method for rapid generation of robot trajectories based on adaptive key point extraction as claimed in claim 5, characterized in that: The calculation conditional probability distribution adopts Gaussian mixture regression method.

7. A method for rapid generation of robot trajectories based on adaptive key point extraction as claimed in claim 1, characterized in that: The generalized trajectory is obtained by generalizing the trajectory using the key points as extrapolated features of the imitation learning KMP algorithm.

8. A robot trajectory rapid generation system based on adaptive key point extraction, characterized in that: It includes a processor, which is used to execute the robot trajectory rapid generation method based on adaptive key point extraction as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for rapidly generating robot trajectories based on adaptive key point extraction as described in any one of claims 1 to 7 is implemented.

Images