A robot arm polishing trajectory generation method and related device

Abstract

The application discloses a mechanical arm polishing track generation method and related devices, and relates to the technical field of machine vision, the method comprises the following steps: based on the polishing demonstration video, the motion track and the posture information of the polishing demonstration object are analyzed by using a posture estimation model; based on the motion track and the posture information, a polishing contact point track is generated, and filtering processing is performed on the polishing contact point track; a density-based clustering algorithm is used to extract the start and end points of the polishing contact point track, and the polishing contact point track is down-sampled based on the start and end points; based on a hand-eye calibration matrix, the point cloud data of a workpiece to be polished and the down-sampled polishing contact point track are used to generate a polishing track in a mechanical arm base coordinate system, and the polishing track is coarsely optimized by using an optimization function; the coarsely optimized polishing track is finely optimized, and a target polishing track of the mechanical arm is obtained. The application guarantees high precision and high efficiency of the generated robot polishing track, and reduces the professional requirements for generating the polishing track.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method and apparatus for generating grinding trajectories for a robotic arm. Background Technology

[0002] Robots, due to their high flexibility, strong collaborative capabilities, and small space requirements, are widely used in intelligent manufacturing. They can replace humans in performing tedious, repetitive, high-intensity, or potentially dangerous tasks. Currently, industrial robots are often used for workpiece grinding. The main operating methods for industrial robots include online programming with a teach pendant and offline simulation. Teaching programming involves manually operating the teach pendant to drive the robotic arm and generate task paths, offering intuitive operation and strong adaptability. However, the motion trajectory generation process often takes a long time, and operating the teach pendant requires a high level of expertise, presenting a learning curve. Offline simulation can generate motion trajectories without using a real robot, avoiding downtime, but this process still requires specialized personnel, and the robotic arm operation also demands a high level of expertise, hindering widespread adoption. Therefore, how to ensure high accuracy and efficiency in generating robot grinding trajectories while reducing the expertise required for trajectory generation has become a key research issue for various companies. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and related device for generating a robotic arm grinding trajectory, which ensures high precision and high efficiency in generating the robot grinding trajectory while reducing the professional requirements for generating the grinding trajectory.

[0004] To address the aforementioned technical problems, this invention provides a method for generating a grinding trajectory for a robotic arm, the method comprising:

[0005] Based on the polishing demonstration video, the motion trajectory and attitude information of the polishing demonstration object were analyzed using an attitude estimation model;

[0006] The grinding contact point trajectory is generated based on the motion trajectory and attitude information, and the grinding contact point trajectory is filtered to obtain the filtered grinding contact point trajectory.

[0007] A density-based clustering algorithm is used to extract the start and end points of the filtered grinding contact point trajectory, and the filtered grinding contact point trajectory is downsampled based on the start and end points to obtain the downsampled grinding contact point trajectory.

[0008] Based on the hand-eye calibration matrix, the grinding trajectory in the coordinate system of the robot arm base is generated using the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling. The grinding trajectory is then coarsely optimized using an optimization function to obtain the coarsely optimized grinding trajectory.

[0009] The coarse-optimized grinding trajectory is then fine-optimized to obtain the target grinding trajectory for the robotic arm.

[0010] Optionally, the step of analyzing the motion trajectory and attitude information of the polishing demonstration object using a posture estimation model based on the polishing demonstration video includes:

[0011] The polishing demonstration image sequence is extracted from the polishing demonstration video, and then input into the attitude estimation model to analyze the motion trajectory and attitude information of the polishing demonstration object. The attitude estimation model adopts the Foundationpose attitude estimation model.

[0012] Optionally, the step of generating the grinding contact point trajectory based on the motion trajectory and attitude information, and filtering the grinding contact point trajectory to obtain the filtered grinding contact point trajectory, includes:

[0013] Contact point analysis is performed based on motion trajectory and posture information combined with the shape characteristics of the object being polished to obtain contact point location information and contact point posture information.

[0014] The grinding contact point trajectory is generated based on the contact point location information and contact point posture information;

[0015] The grinding contact point trajectory is processed by median filtering to obtain the grinding contact point trajectory after median filtering.

[0016] The grinding contact point trajectory after median filtering is subjected to mean filtering to obtain the filtered grinding contact point trajectory.

[0017] Optionally, the expression for the median filtering process is:

[0018] y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1}), 0≤i≤N1-k,

[0019] Among them, y i Here, denoted as median-filtered grinding contact point trajectory data, x represents the original grinding contact point trajectory data, median is the median filtering function, N1 is the length of the grinding contact point trajectory sequence used for filtering, and k is the size of the median filtering window.

[0020] The expression for the mean filtering process is:

[0021]

[0022] Among them, z iy represents the grinding contact point trajectory data after mean filtering, m is the mean filtering window size, y represents the grinding contact point trajectory data after median filtering, and N2 is the length of the grinding contact point trajectory sequence used for filtering.

[0023] Optionally, the step of extracting the start and end points of the filtered grinding contact point trajectory using a density-based clustering algorithm, and then downsampling the filtered grinding contact point trajectory based on the start and end points to obtain the downsampled grinding contact point trajectory, includes:

[0024] Set the first neighborhood radius and minimum number of samples for the density-based clustering algorithm, and cluster the trajectory points in the filtered grinding contact point trajectory based on the first neighborhood radius and minimum number of samples to obtain the clustering results. Then, determine the start and end points of the filtered grinding contact point trajectory based on the clustering results.

[0025] Several key-value pairs are generated based on the clustering results. Then, the grinding contact point trajectory after filtering by combining several key-value pairs with the start and end point pairs is downsampled to obtain the downsampled grinding contact point trajectory.

[0026] Optionally, the step of performing coarse optimization on the grinding trajectory using an optimization function to obtain a coarsely optimized grinding trajectory includes:

[0027] A minimization function is set as the optimization function. Based on the BFGS optimization method, the optimization function is used to iteratively optimize each trajectory point in the grinding trajectory to obtain a coarse-optimized grinding trajectory. The expression of the optimization function is:

[0028]

[0029] Where f is the optimization function, n is the number of trajectory points, min is the minimization function, and p i Let c be a trajectory point, and c be a point in the point cloud data;

[0030] The expression for the iterative optimization process is:

[0031]

[0032] Where, x k+1 x is the translation amount of the trajectory point after iterative optimization. k Let α be the trajectory point to be iteratively optimized. k Let H be the step size, k be the number of iterations, and H be the number of iterations. k It is the inverse of the approximate Hessian matrix. For the objective function in x k The gradient at that point.

[0033] Optionally, the step of fine-optimizing the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm includes:

[0034] All trajectory points in the coarse optimization and polishing trajectory are taken as points to be optimized. A second neighborhood radius is set, and the direction from which all points in the point cloud data within the second neighborhood radius point to each point to be optimized is taken as the reference direction.

[0035] Calculate the vectors for each reference direction, and use the sum of the vectors for all reference directions as the optimization direction;

[0036] Based on the optimization direction and preset distance threshold, each point to be optimized is optimized to form the target grinding trajectory of the robotic arm.

[0037] In addition, the present invention also provides a robotic arm grinding trajectory generation device, the device comprising:

[0038] Trajectory and attitude analysis module: used to analyze the motion trajectory and attitude information of the object being polished based on the polishing demonstration video using an attitude estimation model;

[0039] Trajectory filtering module: used to generate grinding contact point trajectory based on motion trajectory and attitude information, and to filter the grinding contact point trajectory to obtain the filtered grinding contact point trajectory;

[0040] Trajectory downsampling module: Used to extract the start and end points of the filtered grinding contact point trajectory using a density-based clustering algorithm, and to downsample the filtered grinding contact point trajectory based on the start and end points to obtain the downsampled grinding contact point trajectory.

[0041] The trajectory coarse optimization module is used to generate a grinding trajectory in the coordinate system of the robot arm base based on the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling, using the hand-eye calibration matrix. The grinding trajectory is then coarsely optimized using an optimization function to obtain the coarsely optimized grinding trajectory.

[0042] The trajectory fine optimization module is used to fine optimize the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm.

[0043] In addition, the present invention also provides an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the above-described robotic arm grinding trajectory generation method.

[0044] In addition, the present invention also provides a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the above-described robotic arm grinding trajectory generation method.

[0045] In this embodiment of the invention, a posture estimation model is used to analyze the motion trajectory and posture information of the object being polished based on a polishing demonstration video to generate a polishing contact point trajectory. Filtering of the polishing contact point trajectory reduces interference caused by jitter or stuttering during manual demonstrations. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory for downsampling. Based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory can be used to quickly generate the polishing trajectory in the coordinate system of the robotic arm base. A coarse optimization function is combined with a fine optimization process to further refine the polishing trajectory, making the obtained target polishing trajectory of the robotic arm more reliable. When generating the polishing trajectory, a point on the demonstration object is mapped to the actual polishing tool used. This does not limit the object used for demonstration and is independent of the surface and edge features of the object's point cloud. That is, the generated trajectory is not limited to the object's edge and can generate complex trajectories applicable to different polishing scenarios and complex workpieces, improving the versatility of robotic arm polishing trajectory generation. Generating grinding trajectories from grinding demonstration videos is more efficient than teaching grinding trajectories via a teach pendant. This no-programming trajectory generation method eliminates the need for a teach pendant, reducing the difficulty of using the robotic arm and lowering the required level of expertise during operation. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the robotic arm grinding trajectory generation method in an embodiment of the present invention.

[0048] Figure 2 This is a flowchart illustrating the robotic arm grinding trajectory generation method in another embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of the structural composition of the robotic arm grinding trajectory generation device in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention;

[0051] Figure 5 This is a rendering of the grinding trajectory in the coordinate system of the robotic arm base in an embodiment of the present invention;

[0052] Figure 6 This is a diagram showing the effect of coarse optimization of the grinding trajectory in an embodiment of the present invention;

[0053] Figure 7 This is a diagram showing the effect of the finely optimized grinding trajectory in an embodiment of the present invention. Detailed Implementation

[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Example 1

[0056] Please see Figure 1 , Figure 1 This is a flowchart illustrating the robotic arm grinding trajectory generation method in an embodiment of the present invention, the method comprising:

[0057] S11: Based on the polishing demonstration video, analyze the motion trajectory and attitude information of the polishing demonstration object using an attitude estimation model;

[0058] In the specific implementation of this invention, the step of analyzing the motion trajectory and posture information of the polishing demonstration object using a posture estimation model based on the polishing demonstration video includes: extracting a polishing demonstration image sequence based on the polishing demonstration video, and inputting the polishing demonstration image sequence into the posture estimation model to analyze the motion trajectory and posture information of the polishing demonstration object. The posture estimation model adopts the Foundationpose posture estimation model.

[0059] Specifically, a polishing scene is set up, and the depth camera used is calibrated by hand and eye. The object to be polished can be a furnace frame. In the set-up polishing scene, a person holds the demonstration object and demonstrates the polishing motion trajectory at a certain speed according to the polishing requirements and polishing process from the perspective of the depth camera. The direction of the motion trajectory can be counterclockwise or clockwise. The depth camera records the demonstration process at a preset frame rate, which can be set to 15 frames per second, and saves the polishing demonstration video during the demonstration process. The process involves extracting a grinding demonstration image sequence from a grinding demonstration video. This involves extracting depth maps and red-green-blue (RGB) maps from all frames of the video to form a grinding demonstration image sequence. This sequence is then input into a pose estimation model to analyze the motion trajectory and pose information of the grinding demonstration object. The pose estimation model combines the current depth map with the estimated pose from the previous moment to generate the estimated pose for the current moment. Its output includes the pose matrix of the target object in the camera coordinate system, which generates the motion trajectory of the grinding demonstration object from the position and pose at each moment. The pose estimation model uses the Foundationpose pose estimation model, a unified foundational model for object pose estimation and tracking. This model requires no fine-tuning and only a small number of reference images for pose estimation and tracking.

[0060] S12: Generate the grinding contact point trajectory based on the motion trajectory and attitude information, and filter the grinding contact point trajectory to obtain the filtered grinding contact point trajectory.

[0061] In the specific implementation of this invention, the step of generating the grinding contact point trajectory based on motion trajectory and posture information, and filtering the grinding contact point trajectory to obtain the filtered grinding contact point trajectory, includes: analyzing the contact point based on the motion trajectory and posture information combined with the shape features of the object being ground, to obtain contact point position information and contact point posture information; generating the grinding contact point trajectory based on the contact point position information and contact point posture information; performing median filtering on the grinding contact point trajectory to obtain the median-filtered grinding contact point trajectory; and performing mean filtering on the median-filtered grinding contact point trajectory to obtain the filtered grinding contact point trajectory.

[0062] Furthermore, the expression for the median filtering process is as follows:

[0063] y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1}), 0≤i≤N1-k,

[0064] Among them, y i Here, denoted as median-filtered grinding contact point trajectory data, x represents the original grinding contact point trajectory data, median is the median filtering function, N1 is the length of the grinding contact point trajectory sequence used for filtering, and k is the size of the median filtering window.

[0065] The expression for the mean filtering process is:

[0066]

[0067] Among them, z i y represents the grinding contact point trajectory data after mean filtering, m is the mean filtering window size, y represents the grinding contact point trajectory data after median filtering, and N2 is the length of the grinding contact point trajectory sequence used for filtering.

[0068] Specifically, a polishing demonstration image is extracted from the polishing demonstration video. The polishing demonstration image includes the polishing demonstration object and the background. The background frame image and the polishing demonstration image are subtracted to obtain a foreground image that includes the polishing demonstration object and its shadow. The background frame image is a background image that does not contain the polishing demonstration object. Shadow removal is performed on the foreground image based on a shadow detection algorithm to obtain the region contour of the polishing demonstration object. The Hu moments and Fourier integral operators of the region contour are extracted. The Hu moments are image features with translation, rotation and scale invariance. The Hu moments can better describe the regional distribution characteristics of the polishing demonstration object. The Fourier integral operator can better describe the contour information of the polishing demonstration object. The shape features of the polishing demonstration object are determined by the Hu moments and Fourier integral operators. Contact point analysis is performed based on the shape features of the object being polished, combined with motion trajectory and posture information. The position and posture of the contact point between the workpiece and the object are determined according to the shape features, motion trajectory, and posture information of the object. The contact point position is represented by coordinates along the X, Y, and Z axes, and the contact point posture is represented by quaternions, thus obtaining the contact point position and posture information. A polishing contact point trajectory is generated based on the contact point position and posture information. For example, if the contact point position information is (x, y, z) and the contact point posture information is (Qw, Qx, Qy, Qz), then the motion trajectory information of the polishing contact point is represented as {x, y, z, Qw, Qx, Qy, Qz}. A median filter window size is set, and median filtering is applied to the polishing contact point trajectory to obtain the median-filtered polishing contact point trajectory. The expression for the median filtering is:

[0069] y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1}), 0≤i≤N1-k,

[0070] Among them, y i denoted as median-filtered grinding contact point trajectory data, x as the original grinding contact point trajectory data, median as the median filtering function, N1 as the length of the grinding contact point trajectory sequence used for filtering, and k as the median filtering window size.

[0071] Set the mean filtering window size, and apply mean filtering to the grinding contact point trajectory after median filtering to obtain the filtered grinding contact point trajectory. The expression for the mean filtering is:

[0072]

[0073] Among them, z i The data represents the grinding contact point trajectory data after mean filtering, where m is the mean filtering window size, y is the grinding contact point trajectory data after median filtering, and N² is the length of the grinding contact point trajectory sequence used for filtering. Median filtering and window mean filtering of the data in the grinding contact point trajectory can reduce interference caused by jitter or stuttering during manual demonstrations.

[0074] S13: The density-based clustering algorithm is used to extract the start and end points of the filtered grinding contact point trajectory, and the filtered grinding contact point trajectory is downsampled based on the start and end points to obtain the downsampled grinding contact point trajectory.

[0075] In the specific implementation of this invention, the step of using a density-based clustering algorithm to extract the start and end points of the filtered grinding contact point trajectory, and then downsampling the filtered grinding contact point trajectory based on the start and end points to obtain the downsampled grinding contact point trajectory includes: setting a first neighborhood radius and a minimum number of samples for the density-based clustering algorithm, clustering the trajectory points in the filtered grinding contact point trajectory based on the first neighborhood radius and the minimum number of samples to obtain clustering results, and determining the start and end points of the filtered grinding contact point trajectory based on the clustering results; generating several sets of key-value pairs based on the clustering results, and downsampling the filtered grinding contact point trajectory based on the several sets of key-value pairs combined with the start and end points to obtain the downsampled grinding contact point trajectory.

[0076] Specifically, a density-based clustering algorithm is set with a first neighborhood radius and a minimum number of samples. Based on the first neighborhood radius and the minimum number of samples, the trajectory points in the filtered grinding contact point trajectory are clustered. That is, if a trajectory point has points within its first neighborhood radius that reach the minimum number of samples or more, these points are classified into the same cluster point. This process continues until all trajectory points are clustered, resulting in several cluster points. Based on the clustering results, the start and end points of the filtered grinding contact point trajectory are determined, with the first cluster point as the start point and the last cluster point as the end point. Several key-value pairs are generated based on the clustering results. Corresponding key-value pairs are generated according to the cluster points and their labels. The grinding contact point trajectory after filtering based on several key-value pairs and the start and end point pairs is downsampled. Starting from the start point of the key-value pair, the distance between the selected trajectory point and the previous trajectory point is calculated sequentially from the filtered grinding contact point trajectory. If the distance is greater than or equal to a preset distance threshold, the selected trajectory point is taken as a valid point. If the distance is less than the preset distance threshold, the trajectory point is deleted. The above process is repeated until the end point is sampled, and the sampling process ends. The set of downsampled trajectory points is combined to form a new grinding contact point trajectory, thus obtaining the downsampled grinding contact point trajectory.

[0077] S14: Based on the hand-eye calibration matrix, the grinding trajectory in the coordinate system of the robot arm base is generated using the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling. The grinding trajectory is then coarsely optimized using an optimization function to obtain the coarsely optimized grinding trajectory.

[0078] In the specific implementation of this invention, the step of performing coarse optimization processing on the grinding trajectory using an optimization function to obtain a coarsely optimized grinding trajectory includes: setting a minimization function as the optimization function, and iteratively optimizing each trajectory point in the grinding trajectory based on the BFGS optimization method and the optimization function to obtain the coarsely optimized grinding trajectory. The expression of the optimization function is:

[0079]

[0080] Where f is the optimization function, n is the number of trajectory points, min is the minimization function, and p i Let c be a trajectory point, and c be a point in the point cloud data;

[0081] The expression for the iterative optimization process is:

[0082]

[0083] Where, x k+1 x is the translation amount of the trajectory point after iterative optimization. k Let α be the trajectory point to be iteratively optimized. kLet H be the step size, k be the number of iterations, and H be the number of iterations. k It is the inverse of the approximate Hessian matrix. For the objective function in x k The gradient at that point.

[0084] Specifically, although the shape of the downsampled grinding contact point trajectory is basically consistent with the surface of the workpiece to be ground, there is still a certain deviation in position. Therefore, it is necessary to perform coarse optimization on the grinding contact point trajectory to achieve overall trajectory correction. A 3D camera scans the workpiece to be ground to obtain the corresponding point cloud data. Based on the hand-eye calibration matrix, the grinding trajectory in the robot arm base coordinate system is generated using the point cloud data of the workpiece and the downsampled grinding contact point trajectory. The hand-eye calibration matrix can determine the accurate transformation relationship between the robot arm and the camera. Therefore, the trajectory data in the camera coordinate system can be converted to the trajectory data in the robot arm base coordinate system using the hand-eye calibration matrix. The effect diagram of the grinding trajectory in the robot arm base coordinate system is shown below. Figure 5 As shown. A minimization function is set as the optimization function, which represents minimizing the distance between the trajectory points and the point cloud data of the workpiece to be ground. Based on the BFGS (Broyden-Fletcher-Goldfarb-Shanno) optimization method, the optimization function is used to iteratively optimize each trajectory point in the grinding trajectory to obtain a coarsely optimized grinding trajectory. The BFGS optimization method is an iterative algorithm for nonlinear optimization problems. It reduces computational complexity by approximating the inverse of the Hessian matrix while maintaining a relatively fast convergence speed. The expression of the optimization function is:

[0085]

[0086] Where f is the optimization function, n is the number of trajectory points, min is the minimization function, and p i Let c be a trajectory point, and c be a point in the point cloud data;

[0087] The expression for the iterative optimization process is:

[0088]

[0089] Where, x k+1 x is the translation amount of the trajectory point after iterative optimization. k Let α be the trajectory point to be iteratively optimized. k Let H be the step size, k be the number of iterations, and H be the number of iterations. k It is the inverse of the approximate Hessian matrix. For the objective function in x kThe gradient at the point is calculated. In each iteration, the objective function and gradient are recalculated until the minimum value of the optimization function is reached. Based on the translation amount of the obtained trajectory points, all trajectory points are offset to better fit the surface of the workpiece to be ground, completing the coarse optimization of the grinding trajectory. The effect of the coarsely optimized grinding trajectory is shown in the figure below. Figure 6 As shown.

[0090] S15: Perform fine optimization on the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm.

[0091] In the specific implementation of this invention, the step of finely optimizing the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm includes: taking all trajectory points in the coarse-optimized grinding trajectory as points to be optimized, setting a second neighborhood radius, and taking the direction from all points in the point cloud data within the second neighborhood radius pointing to each point to be optimized as a reference direction; calculating the vector of each reference direction, and taking the sum of the vectors of all reference directions as the optimization direction; and optimizing each point to be optimized based on the optimization direction and a preset distance threshold to form the target grinding trajectory of the robotic arm.

[0092] Specifically, all trajectory points in the coarse-optimized grinding trajectory are taken as points to be optimized. A second neighborhood radius is set, and the directions pointing from all points in the point cloud data within the second neighborhood radius to each point to be optimized are taken as reference directions. That is, for each point to be optimized, this point is taken as the center, and the directions pointing from all points in the point cloud data within the second neighborhood radius of this center to this point are taken as reference directions. The vectors of each reference direction are calculated, and the sum of the vectors of all reference directions is taken as the optimization direction. Based on the optimization direction and a preset distance threshold, each point to be optimized is optimized. If the distance between the trajectory point of the coarse-optimized grinding trajectory and the point cloud data is greater than the preset distance threshold, the trajectory point is moved in the opposite direction of the optimization direction to reduce the distance. If the distance between the trajectory point and the point cloud data is less than the preset distance threshold, the trajectory point is moved in the positive direction of the optimization direction to increase the distance. After all the points to be optimized are optimized, all the optimized trajectory points are connected to form the final robotic arm running trajectory, that is, the target grinding trajectory of the robotic arm. The effect diagram of the fine-optimized grinding trajectory is shown in the figure. Figure 7 As shown.

[0093] In this embodiment of the invention, a posture estimation model is used to analyze the motion trajectory and posture information of the object being polished based on a polishing demonstration video to generate a polishing contact point trajectory. Filtering of the polishing contact point trajectory reduces interference caused by jitter or stuttering during manual demonstrations. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory for downsampling. Based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory can be used to quickly generate the polishing trajectory in the coordinate system of the robotic arm base. A coarse optimization function is combined with a fine optimization of the polishing trajectory to make the obtained target polishing trajectory of the robotic arm more reliable. When generating the polishing trajectory, a point on the demonstration object is mapped to the actual polishing tool used. This does not limit the object used for demonstration and is not dependent on the surface and edge features of the object's point cloud. That is, the generated trajectory is not limited to the object's edge and can generate complex trajectories applicable to different polishing scenarios and complex workpieces, improving the versatility of robotic arm polishing trajectory generation. Generating grinding trajectories from grinding demonstration videos is more efficient than teaching grinding trajectories via a teach pendant. This no-programming trajectory generation method eliminates the need for a teach pendant, reducing the difficulty of using the robotic arm and lowering the required level of expertise during operation.

[0094] Example 2

[0095] Please see Figure 2 , Figure 2 This is a flowchart illustrating a robotic arm grinding trajectory generation method according to another embodiment of the present invention, the method comprising:

[0096] S201: Analyze the motion trajectory and attitude information of the polishing demonstration object using an attitude estimation model based on the polishing demonstration video;

[0097] S202: Generate the grinding contact point trajectory based on the motion trajectory and attitude information, and filter the grinding contact point trajectory to obtain the filtered grinding contact point trajectory.

[0098] S203: Set the first neighborhood radius and minimum number of samples for the density-based clustering algorithm, and cluster the trajectory points in the filtered grinding contact point trajectory based on the first neighborhood radius and minimum number of samples to obtain the clustering results, and determine the start and end points of the filtered grinding contact point trajectory based on the clustering results.

[0099] S204: Generate several sets of key-value pairs based on the clustering results, and perform downsampling processing on the grinding contact point trajectory after filtering based on several sets of key-value pairs and the start and end point pairs to obtain the downsampled grinding contact point trajectory.

[0100] S205: Based on the hand-eye calibration matrix, the grinding trajectory in the coordinate system of the robot arm base is generated using the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling. The grinding trajectory is then coarsely optimized using an optimization function to obtain the coarsely optimized grinding trajectory.

[0101] S206: Take all trajectory points in the coarse optimization and polishing trajectory as points to be optimized, set a second neighborhood radius, and take the direction of the points in all point cloud data within the second neighborhood radius pointing to each point to be optimized as the reference direction;

[0102] S207: Calculate the vectors of each reference direction, and use the sum of the vectors of all reference directions as the optimization direction;

[0103] S208: Based on the optimization direction and preset distance threshold, each point to be optimized is optimized to form the target grinding trajectory of the robotic arm.

[0104] In this embodiment of the invention, a posture estimation model is used to analyze the motion trajectory and posture information of the object being polished based on a polishing demonstration video to generate a polishing contact point trajectory. Filtering of the polishing contact point trajectory reduces interference caused by jitter or stuttering during manual demonstrations. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory for downsampling. Based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory can be used to quickly generate the polishing trajectory in the coordinate system of the robotic arm base. A coarse optimization function is combined with a fine optimization process to further refine the polishing trajectory, making the obtained target polishing trajectory of the robotic arm more reliable. When generating the polishing trajectory, a point on the demonstration object is mapped to the actual polishing tool used. This does not limit the object used for demonstration and is independent of the surface and edge features of the object's point cloud. That is, the generated trajectory is not limited to the object's edge and can generate complex trajectories applicable to different polishing scenarios and complex workpieces, improving the versatility of robotic arm polishing trajectory generation. Generating grinding trajectories from grinding demonstration videos is more efficient than teaching grinding trajectories via a teach pendant. This no-programming trajectory generation method eliminates the need for a teach pendant, reducing the difficulty of using the robotic arm and lowering the required level of expertise during operation.

[0105] Example 3

[0106] Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of the robotic arm grinding trajectory generation device in an embodiment of the present invention. The device includes:

[0107] Trajectory and attitude analysis module 31: used to analyze the motion trajectory and attitude information of the polishing demonstration object based on the polishing demonstration video using an attitude estimation model;

[0108] Trajectory filtering module 32: used to generate grinding contact point trajectory based on motion trajectory and attitude information, and to filter the grinding contact point trajectory to obtain the filtered grinding contact point trajectory;

[0109] Trajectory downsampling module 33: used to extract the start and end points of the filtered grinding contact point trajectory using a density-based clustering algorithm, and to perform downsampling processing on the filtered grinding contact point trajectory based on the start and end points to obtain the downsampled grinding contact point trajectory.

[0110] The trajectory coarse optimization module 34 is used to generate a grinding trajectory in the coordinate system of the robot arm base based on the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling processing using the hand-eye calibration matrix, and to perform coarse optimization processing on the grinding trajectory using the optimization function to obtain the coarse optimized grinding trajectory.

[0111] Trajectory Fine Optimization Module 35: Used to fine optimize the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm.

[0112] In the specific implementation of this invention, the specific implementation of the device item can be referred to the implementation of the method item above, and will not be repeated here.

[0113] In this embodiment of the invention, a posture estimation model is used to analyze the motion trajectory and posture information of the object being polished based on a polishing demonstration video to generate a polishing contact point trajectory. Filtering of the polishing contact point trajectory reduces interference caused by jitter or stuttering during manual demonstrations. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory for downsampling. Based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory can be used to quickly generate the polishing trajectory in the coordinate system of the robotic arm base. A coarse optimization function is combined with a fine optimization process to further refine the polishing trajectory, making the obtained target polishing trajectory of the robotic arm more reliable. When generating the polishing trajectory, a point on the demonstration object is mapped to the actual polishing tool used. This does not limit the object used for demonstration and is independent of the surface and edge features of the object's point cloud. That is, the generated trajectory is not limited to the object's edge and can generate complex trajectories applicable to different polishing scenarios and complex workpieces, improving the versatility of robotic arm polishing trajectory generation. Generating grinding trajectories from grinding demonstration videos is more efficient than teaching grinding trajectories via a teach pendant. This no-programming trajectory generation method eliminates the need for a teach pendant, reducing the difficulty of using the robotic arm and lowering the required level of expertise during operation.

[0114] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the robotic arm grinding trajectory generation method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that can store or transmit information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0115] Example 4

[0116] Please see Figure 4 , Figure 4This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0117] This invention also provides an electronic device, such as... Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. Those skilled in the art will understand that... Figure 3 The illustrated electronic device does not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 41 can be used to store computer program 42 and various functional modules. Processor 43 runs the computer program 42 stored in memory 41, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. Processor 43 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or a processor 43, or any conventional processor, etc. The processors and memories disclosed in this invention include, but are not limited to, these types of processors and memories. The processors and memories disclosed in this invention are merely examples and not intended to be limiting.

[0118] As one embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to execute the robotic arm grinding trajectory generation method in any of the above embodiments. For specific implementation processes, please refer to the above embodiments, which will not be repeated here.

[0119] In this embodiment of the invention, a posture estimation model is used to analyze the motion trajectory and posture information of the object being polished based on a polishing demonstration video to generate a polishing contact point trajectory. Filtering of the polishing contact point trajectory reduces interference caused by jitter or stuttering during manual demonstrations. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory for downsampling. Based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory can be used to quickly generate the polishing trajectory in the coordinate system of the robotic arm base. A coarse optimization function is combined with a fine optimization process to further refine the polishing trajectory, making the obtained target polishing trajectory of the robotic arm more reliable. When generating the polishing trajectory, a point on the demonstration object is mapped to the actual polishing tool used. This does not limit the object used for demonstration and is independent of the surface and edge features of the object's point cloud. That is, the generated trajectory is not limited to the object's edge and can generate complex trajectories applicable to different polishing scenarios and complex workpieces, improving the versatility of robotic arm polishing trajectory generation. Generating grinding trajectories from grinding demonstration videos is more efficient than teaching grinding trajectories via a teach pendant. This no-programming trajectory generation method eliminates the need for a teach pendant, reducing the difficulty of using the robotic arm and lowering the required level of expertise during operation.

[0120] Furthermore, the above provides a detailed description of the robotic arm grinding trajectory generation method and related apparatus provided in the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for generating a grinding trajectory for a robotic arm, characterized in that, The method includes: Based on the polishing demonstration video, the motion trajectory and attitude information of the polishing demonstration object were analyzed using an attitude estimation model; The grinding contact point trajectory is generated based on the motion trajectory and attitude information, and the grinding contact point trajectory is filtered to obtain the filtered grinding contact point trajectory. A density-based clustering algorithm is used to extract the start and end points of the filtered grinding contact point trajectory, and the filtered grinding contact point trajectory is downsampled based on the start and end points to obtain the downsampled grinding contact point trajectory. Based on the hand-eye calibration matrix, the grinding trajectory in the coordinate system of the robot arm base is generated using the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling. The grinding trajectory is then coarsely optimized using an optimization function to obtain the coarsely optimized grinding trajectory. The coarse-optimized grinding trajectory is then fine-optimized to obtain the target grinding trajectory for the robotic arm.

2. The method for generating a grinding trajectory for a robotic arm according to claim 1, characterized in that, The analysis of the motion trajectory and attitude information of the object being polished using a posture estimation model based on the polishing demonstration video includes: The polishing demonstration image sequence is extracted from the polishing demonstration video, and then input into the attitude estimation model to analyze the motion trajectory and attitude information of the polishing demonstration object. The attitude estimation model adopts the Foundationpose attitude estimation model.

3. The method for generating a grinding trajectory for a robotic arm according to claim 1, characterized in that, The process of generating a grinding contact point trajectory based on motion trajectory and attitude information, and then filtering the grinding contact point trajectory to obtain a filtered grinding contact point trajectory includes: Contact point analysis is performed based on motion trajectory and posture information combined with the shape characteristics of the object being polished to obtain contact point location information and contact point posture information. The grinding contact point trajectory is generated based on the contact point location information and contact point posture information; The grinding contact point trajectory is processed by median filtering to obtain the grinding contact point trajectory after median filtering. The grinding contact point trajectory after median filtering is subjected to mean filtering to obtain the filtered grinding contact point trajectory.

4. The method for generating a grinding trajectory for a robotic arm according to claim 3, characterized in that, The expression for the median filtering process is: y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1 }),0≤i≤N1-k, Among them, y i Here, denoted as median-filtered grinding contact point trajectory data, x represents the original grinding contact point trajectory data, median is the median filtering function, N1 is the length of the grinding contact point trajectory sequence used for filtering, and k is the size of the median filtering window. The expression for the mean filtering process is: Among them, z i y represents the grinding contact point trajectory data after mean filtering, m is the mean filtering window size, y represents the grinding contact point trajectory data after median filtering, and N2 is the length of the grinding contact point trajectory sequence used for filtering.

5. The method for generating a grinding trajectory for a robotic arm according to claim 1, characterized in that, The process involves using a density-based clustering algorithm to extract the start and end points of the filtered grinding contact point trajectory, and then downsampling the filtered grinding contact point trajectory based on these start and end points to obtain the downsampled grinding contact point trajectory, including: Set the first neighborhood radius and minimum number of samples for the density-based clustering algorithm, and cluster the trajectory points in the filtered grinding contact point trajectory based on the first neighborhood radius and minimum number of samples to obtain the clustering results. Then, determine the start and end points of the filtered grinding contact point trajectory based on the clustering results. Several key-value pairs are generated based on the clustering results. Then, the grinding contact point trajectory after filtering by combining several key-value pairs with the start and end point pairs is downsampled to obtain the downsampled grinding contact point trajectory.

6. The method for generating a grinding trajectory for a robotic arm according to claim 1, characterized in that, The process of coarsely optimizing the grinding trajectory using an optimization function to obtain a coarsely optimized grinding trajectory includes: A minimization function is set as the optimization function. Based on the BFGS optimization method, the optimization function is used to iteratively optimize each trajectory point in the grinding trajectory to obtain a coarse-optimized grinding trajectory. The expression of the optimization function is: Where f is the optimization function, n is the number of trajectory points, min is the minimization function, and p i Let c be a trajectory point, and c be a point in the point cloud data; The expression for the iterative optimization process is: x k+1 =x k -α k H k ▽f(x k ), Where, x k+1 x is the translation amount of the trajectory point after iterative optimization. k Let α be the trajectory point to be iteratively optimized. k Let H be the step size, k be the number of iterations, and H be the number of iterations. k The inverse of the approximate Hessian matrix is ​​▽f(x) k ) is the objective function in x k The gradient at that point.

7. The method for generating a grinding trajectory for a robotic arm according to claim 1, characterized in that, The process of refining the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm includes: All trajectory points in the coarse optimization and polishing trajectory are taken as points to be optimized. A second neighborhood radius is set, and the direction from which all points in the point cloud data within the second neighborhood radius point to each point to be optimized is taken as the reference direction. Calculate the vectors for each reference direction, and use the sum of the vectors for all reference directions as the optimization direction; Based on the optimization direction and preset distance threshold, each point to be optimized is optimized to form the target grinding trajectory of the robotic arm.

8. A robotic arm grinding trajectory generation device, characterized in that, The device includes: Trajectory and attitude analysis module: used to analyze the motion trajectory and attitude information of the object being polished based on the polishing demonstration video using an attitude estimation model; Trajectory filtering module: used to generate grinding contact point trajectory based on motion trajectory and attitude information, and to filter the grinding contact point trajectory to obtain the filtered grinding contact point trajectory; Trajectory downsampling module: Used to extract the start and end points of the filtered grinding contact point trajectory using a density-based clustering algorithm, and to downsample the filtered grinding contact point trajectory based on the start and end points to obtain the downsampled grinding contact point trajectory. The trajectory coarse optimization module is used to generate a grinding trajectory in the coordinate system of the robot arm base based on the point cloud data of the workpiece to be ground and the grinding contact point trajectory after downsampling, using the hand-eye calibration matrix. The grinding trajectory is then coarsely optimized using an optimization function to obtain the coarsely optimized grinding trajectory. The trajectory fine optimization module is used to fine optimize the coarse-optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm.

9. An electronic device, the electronic device comprising a processor and a memory, characterized in that, The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the robotic arm grinding trajectory generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the robotic arm grinding trajectory generation method as described in any one of claims 1 to 7.

Images