Mechanical arm polishing track generation method and related device

Abstract

The invention discloses a mechanical arm polishing track generation method and a related device, and relates to the technical field of machine vision, the method comprises the following steps: analyzing the motion track and attitude information of a polishing demonstration object by using an attitude estimation model based on a polishing demonstration video; generating a polishing contact point track based on the motion track and the attitude information, and filtering the polishing contact point track; extracting starting and ending points of the polishing contact point track by adopting a density-based clustering algorithm, and performing down-sampling processing on the polishing contact point track based on the starting and ending points; on the basis of a hand-eye calibration matrix, the point cloud data of the to-be-polished workpiece and the polishing contact point track obtained after down-sampling processing are used for generating a polishing track under a mechanical arm base coordinate system, and rough optimization processing is conducted on the polishing track in combination with an optimization function; and fine optimization treatment is conducted on the rough optimization grinding track, and the target grinding track of the mechanical arm is obtained. According to the method, high precision and high efficiency of generating the polishing track of the robot are guaranteed, and meanwhile the professional requirement for generating the polishing track is lowered.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to a mechanical arm polishing trajectory generation method and related device. BACKGROUND

[0002] Robots are widely used in the field of intelligent manufacturing due to their high flexibility, strong collaboration ability, small space requirement and other advantages, and can replace humans to complete boring, repetitive, high-intensity or dangerous work. At present, industrial robots are mainly used for workpiece polishing operation in the form of online programming by a teach pendant or offline simulation. The teach pendant is used to drive the mechanical arm to generate a task path by manual operation, and the operation effect is intuitive and has strong on-site adaptability. However, the generation process of the motion trajectory often takes a long time, and the operation of the teach pendant has strong professionalism and certain difficulty in getting started. Offline simulation can generate a motion trajectory without using a real robot, thereby avoiding downtime operation, but the process still needs to be performed by a professional, and the professional requirement for the operation of the mechanical arm is also high, which is not conducive to popularization. Therefore, how to ensure high precision and high efficiency of the generated robot polishing trajectory while reducing the professional requirement for generating the polishing trajectory has become a key problem for enterprises to study. SUMMARY

[0003] The present application provides a mechanical arm polishing trajectory generation method and related device, which ensures high precision and high efficiency of the generated robot polishing trajectory while reducing the professional requirement for generating the polishing trajectory.

[0004] To solve the above technical problems, the present application provides a mechanical arm polishing trajectory generation method, which comprises:

[0005] Based on the polishing demonstration video, a pose estimation model is used to analyze the motion trajectory and pose information of the polishing demonstration object;

[0006] Based on the motion trajectory and pose information, a polishing contact point trajectory is generated, and the polishing contact point trajectory is filtered to obtain a filtered polishing contact point trajectory;

[0007] A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory, and the filtered polishing contact point trajectory is down-sampled based on the start and end points to obtain a down-sampled polishing contact point trajectory;

[0008] Based on a hand-eye calibration matrix, a polishing trajectory in a base coordinate system of a mechanical arm is generated by using point cloud data of a workpiece to be polished and the down-sampled polishing contact point trajectory, and a polishing trajectory is coarsely optimized by combining an optimization function to obtain a coarsely optimized polishing trajectory;

[0009] The coarse-optimized grinding trajectory is fine-optimized to obtain the target grinding trajectory of the robotic arm.

[0010] Optionally, analyzing the motion trajectory and posture information of the polishing demonstration object using a posture estimation model based on the polishing demonstration video includes:

[0011] A polishing demonstration image sequence is extracted based on the polishing demonstration video, and the polishing demonstration image sequence is input into a posture estimation model to analyze the motion trajectory and posture information of the polishing demonstration object. The posture estimation model adopts the Foundationpose posture estimation model.

[0012] Optionally, generating a polishing contact point trajectory based on the motion trajectory and the posture information, and filtering the polishing contact point trajectory to obtain the polishing contact point trajectory after filtering, includes:

[0013] Contact point analysis is performed based on the motion trajectory and posture information combined with the shape characteristics of the polishing demonstration object to obtain the contact point position information and contact point posture information;

[0014] Generate a polishing contact point trajectory based on the contact point position information and the contact point posture information;

[0015] Performing median filtering on the polishing contact point trajectory to obtain the polishing contact point trajectory after median filtering;

[0016] The polishing contact point trajectory after the median filtering is subjected to mean filtering to obtain the polishing contact point trajectory after the filtering.

[0017] Optionally, the expression for the median filtering process is:

[0018] y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1}),0≤i≤N1-k,

[0019] Among them, y i is the polishing contact point trajectory data after median filtering, x is the original polishing contact point trajectory data, median is the median filter function, N1 is the length of the polishing contact point trajectory sequence used for filtering, and k is the median filter window size;

[0020] The expression of the mean filtering process is:

[0021]

[0022] Among them, z iFor the mean filtering processed polishing contact point trajectory data, m is the mean filtering window size, y is the median filtering processed polishing contact point trajectory data, N2 is the length of the polishing contact point trajectory sequence for filtering.

[0023] Optionally, the beginning and end points of the filtering processed polishing contact point trajectory are extracted by using the density-based clustering algorithm, and the filtering processed polishing contact point trajectory is down-sampled based on the beginning and end points to obtain the down-sampled polishing contact point trajectory, comprising:

[0024] The first neighborhood radius and the minimum number of samples of the density-based clustering algorithm are set, and the trajectory points in the filtering processed polishing contact point trajectory are clustered based on the first neighborhood radius and the minimum number of samples to obtain a clustering result, and the beginning and end points of the filtering processed polishing contact point trajectory are determined based on the clustering result;

[0025] Based on the clustering result, a plurality of key-value pairs are generated, and the filtering processed polishing contact point trajectory is down-sampled based on the plurality of key-value pairs combined with the beginning and end points to obtain the down-sampled polishing contact point trajectory.

[0026] Optionally, the polishing trajectory is coarsely optimized by combining the optimization function to obtain a coarsely optimized polishing trajectory, comprising:

[0027] The minimization function is set as the optimization function, and each trajectory point in the polishing trajectory is iteratively optimized based on the BFGS optimization method combined with the optimization function to obtain the coarsely optimized polishing trajectory, and the expression of the optimization function is:

[0028]

[0029] Wherein, f is the optimization function, n is the number of trajectory points, min is the minimization function, p i is a point cloud data point;

[0030] The expression of the iterative optimization processing is:

[0031]

[0032] Wherein, x k+1 is the translation amount of the trajectory point after the iterative optimization processing, x k is the trajectory point to be iteratively optimized, a k is the step size, k is the iteration number, H k is the inverse matrix of the approximate Hessian matrix, is the gradient of the objective function at x k .

[0033] Optionally, the fine optimization processing on the coarse optimization polishing trajectory is performed to obtain the target polishing trajectory of the mechanical arm, comprising:

[0034] All trajectory points in the coarse optimization polishing trajectory are set as to-be-optimized points, a second neighborhood radius is set, and directions of points in all point cloud data within the second neighborhood radius are set as reference directions of the to-be-optimized points;

[0035] Vectors of the reference directions are calculated, and a sum of the vectors of all the reference directions is set as an optimization direction;

[0036] The to-be-optimized points are optimized based on the optimization direction and a preset distance threshold to form the target polishing trajectory of the mechanical arm.

[0037] In addition, the present application also provides a polishing trajectory generation device of a mechanical arm, the device comprising:

[0038] A trajectory posture analysis module is configured to analyze motion trajectory and posture information of a polishing demonstration object based on a polishing demonstration video by using a posture estimation model;

[0039] A trajectory filtering module is configured to generate a polishing contact point trajectory based on the motion trajectory and the posture information, and perform filtering processing on the polishing contact point trajectory to obtain a filtered polishing contact point trajectory;

[0040] A trajectory down-sampling module is configured to extract start and end points of the filtered polishing contact point trajectory by using a density-based clustering algorithm, and perform down-sampling processing on the filtered polishing contact point trajectory based on the start and end points to obtain a down-sampled polishing contact point trajectory;

[0041] A trajectory coarse optimization module is configured to generate a polishing trajectory in a base coordinate system of a mechanical arm by using point cloud data of a workpiece to be polished and the down-sampled polishing contact point trajectory based on a hand-eye calibration matrix, and perform coarse optimization processing on the polishing trajectory by using an optimization function to obtain a coarse optimization polishing trajectory;

[0042] A trajectory fine optimization module is configured to perform fine optimization processing on the coarse optimization polishing trajectory to obtain a target polishing trajectory of the mechanical arm.

[0043] In addition, the present application also provides an electronic device, which comprises a processor and a memory, the memory is configured to store instructions, and the processor is configured to call the instructions in the memory so that the electronic device performs the above-mentioned polishing trajectory generation method of a mechanical arm.

[0044] In addition, the present application also provides a computer readable storage medium, which stores computer instructions, when the computer instructions run on an electronic device, the electronic device performs the above-mentioned polishing trajectory generation method of a mechanical arm.

[0045] In the embodiment of the present application, the motion trajectory and posture information of the polishing demonstration object are analyzed based on the polishing demonstration video to generate a polishing contact point trajectory using a posture estimation model, and the polishing contact point trajectory is filtered to reduce the interference posture caused by shaking or lagging during manual demonstration. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory for downsampling processing of the filtered polishing contact point trajectory. The point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory are used to quickly generate a polishing trajectory in the base coordinate system of the robot arm based on the hand-eye calibration matrix. The polishing trajectory is coarsely optimized using an optimization function, and the coarsely optimized polishing trajectory is finely optimized, so that the target polishing trajectory of the robot arm obtained is more reliable. When generating the polishing trajectory, a certain point on the demonstration object is mapped to the coordinate of the polishing tool actually used, without limiting the object used for demonstration, and without relying on the surface and edge features of the object point cloud, i.e., the generated trajectory is not limited to the edge of the object, and a complex trajectory can be generated, which can be applied to different polishing scenes and complex workpieces, improving the universality of the polishing trajectory generation of the robot arm. Compared with the way of teaching the polishing trajectory by the teaching pendant, the polishing trajectory is generated from the polishing demonstration video, which is more efficient in polishing trajectory generation speed. The programming-free trajectory generation method does not require the participation of the teaching pendant, reducing the difficulty of using the robot arm and the requirement for professionalism in the operation process of the robot arm. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0047] Figure 1 is a flowchart of the polishing trajectory generation method of the robot arm in the embodiment of the present application;

[0048] Figure 2 is a flowchart of the polishing trajectory generation method of the robot arm in another embodiment of the present application;

[0049] Figure 3 is a structural composition diagram of the polishing trajectory generation device of the robot arm in the embodiment of the present application;

[0050] Figure 4 is a structural composition diagram of the electronic device in the embodiment of the present application;

[0051] Figure 5 is an effect diagram of the polishing trajectory in the base coordinate system of the robot arm in the embodiment of the present application;

[0052] Figure 6 is the effect diagram of the polished trajectory after coarse optimization in the embodiment of the present application;

[0053] Figure 7 is the effect diagram of the polished trajectory after fine optimization in the embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0055] Embodiment one

[0056] Please refer to Figure 1 , Figure 1 is the flowchart of the polishing trajectory generation method of the mechanical arm in the embodiment of the present application, and the method comprises:

[0057] S11: analyzing the motion trajectory and posture information of the polishing demonstration object based on the polishing demonstration video and using a posture estimation model;

[0058] In the specific implementation process of the present application, the analysis of the motion trajectory and posture information of the polishing demonstration object based on the polishing demonstration video and using a posture estimation model comprises: extracting a polishing demonstration image sequence based on the polishing demonstration video, and inputting the polishing demonstration image sequence into a posture estimation model to analyze the motion trajectory and posture information of the polishing demonstration object, wherein the posture estimation model adopts a Foundationpose posture estimation model.

[0059] Specifically, a grinding scene is set up, a hand-eye calibration is performed on a depth camera used, and a grinding object can be a furnace frame. In the set-up grinding scene, a demonstration object is manually held and demonstrated in a motion trajectory according to grinding requirements and a grinding process at a certain motion speed under a depth camera view. The motion direction of the demonstration trajectory can be counterclockwise or clockwise. The depth camera records the demonstration process at a preset frame rate, which can be set to 15 frames per second, and saves a grinding demonstration video in the demonstration process. Based on the grinding demonstration video, a grinding demonstration image sequence is extracted, that is, depth maps and red-green-blue (RGB) images of all frames are extracted from the grinding demonstration video to form the grinding demonstration image sequence. The grinding demonstration image sequence is input into a pose estimation model to analyze motion trajectory and pose information of the grinding demonstration object. The pose estimation model can generate an estimated pose at a current time based on a depth map at the current time and an estimated pose at a previous time. The output result of the pose estimation model includes a pose matrix of a position and a pose of the target object in a camera coordinate system. The motion trajectory of the grinding demonstration object is generated based on the position and the pose at each time. The pose estimation model adopts a Foundationpose pose estimation model. The Foundationpose pose estimation model is a unified base model for object pose estimation and tracking. The pose estimation model does not need fine tuning and can perform pose estimation and tracking only with a small amount of reference images.

[0060] S12: generating a grinding contact point trajectory based on the motion trajectory and the pose information, and performing filtering processing on the grinding contact point trajectory to obtain a filtered grinding contact point trajectory;

[0061] In the specific implementation process of the present application, the grinding contact point trajectory is generated based on the motion trajectory and the pose information, and the grinding contact point trajectory is filtered to obtain a filtered grinding contact point trajectory, which includes: based on the motion trajectory and the pose information, the contact point analysis is performed in combination with the shape features of the grinding demonstration object to obtain contact point position information and contact point pose information; the grinding contact point trajectory is generated based on the contact point position information and the contact point pose information; the median filtering processing is performed on the grinding contact point trajectory to obtain a median filtered grinding contact point trajectory; the mean filtering processing is performed on the median filtered grinding contact point trajectory to obtain the filtered grinding contact point trajectory.

[0062] Further, the expression of the median filtering processing is:

[0063] y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1}),0≤i≤N1-k,

[0064] wherein y i is the median filtering processed polishing contact point trajectory data, x is the original polishing contact point trajectory data, median is a median filtering function, N1 is a polishing contact point trajectory sequence length for filtering, and k is a median filtering window size;

[0065] The expression of the mean filtering processing is:

[0066]

[0067] wherein z i is the mean filtering processed polishing contact point trajectory data, m is a mean filtering window size, y is the median filtering processed polishing contact point trajectory data, and N2 is a polishing contact point trajectory sequence length for filtering.

[0068] Specifically, a polishing demonstration image is extracted from a polishing demonstration video, the polishing demonstration image including a polishing demonstration object and a background, a background frame image is differentially operated with the polishing demonstration image to obtain a foreground image containing the polishing demonstration object and its shadow, the background frame image is a background image not containing the polishing demonstration object, a shadow is removed from the foreground image based on a shadow detection algorithm to obtain a region contour of the polishing demonstration object, a Hu moment and a Fourier integral operator of the region contour are extracted, the Hu moment is an image feature with translation, rotation and scale invariance, the Hu moment can better describe the region distribution characteristics of the polishing demonstration object, the Fourier integral operator can better describe the contour information of the polishing demonstration object, and the shape feature of the polishing demonstration object is determined by the Hu moment and the Fourier integral operator. Contact point analysis is performed based on the motion trajectory and the posture information in combination with the shape feature of the polishing demonstration object, the position and the posture of a contact point between a workpiece and the polishing demonstration object are determined according to the shape feature of the polishing demonstration object in combination with the motion trajectory and the posture information of the polishing demonstration object, the position of the contact point is expressed by coordinates of X-axis, Y-axis and Z-axis, the posture of the contact point is expressed by a quaternion, the position information of the contact point and the posture information of the contact point are obtained. A polishing contact point trajectory is generated based on the position information of the contact point and the posture information of the contact point, if the position information of the contact point is (x, y, z) and the posture information of the contact point is (Qw, Qx, Qy, Qz), the motion trajectory information of the polishing contact point is represented as {x, y, z, Qw, Qx, Qy, Qz}. A median filtering window size is set, and the median filtering processing is performed on the polishing contact point trajectory to obtain the median filtering processed polishing contact point trajectory, and the expression of the median filtering processing is:

[0069] y i = median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1}, 0≤i≤N1-k,

[0070] wherein y i is the polished contact point trajectory data after median filtering processing, x is the original polished contact point trajectory data, median is a median filtering function, N1 is the length of the polished contact point trajectory sequence for filtering, and k is the median filtering window size.

[0071] A mean filtering window size is set, and the polished contact point trajectory after median filtering processing is subjected to mean filtering processing to obtain the polished contact point trajectory after filtering processing, and the expression of the mean filtering processing is as follows:

[0072]

[0073] wherein z i is the polished contact point trajectory data after mean filtering processing, m is the mean filtering window size, y is the polished contact point trajectory data after median filtering processing, and N2 is the length of the polished contact point trajectory sequence for filtering.

[0074] S13: The start and end points of the polished contact point trajectory after filtering processing are extracted by using a density-based clustering algorithm, and the polished contact point trajectory after filtering processing is subjected to down-sampling processing based on the start and end points to obtain the polished contact point trajectory after down-sampling processing.

[0075] In the specific implementation process of the present application, the start and end points of the polished contact point trajectory after filtering processing are extracted by using a density-based clustering algorithm, and the polished contact point trajectory after filtering processing is subjected to down-sampling processing based on the start and end points to obtain the polished contact point trajectory after down-sampling processing, which comprises: setting the first neighborhood radius and the minimum sample number of the density-based clustering algorithm, and clustering the trajectory points in the polished contact point trajectory after filtering processing based on the first neighborhood radius and the minimum sample number to obtain a clustering result, and determining the start and end points of the polished contact point trajectory after filtering processing based on the clustering result; generating a plurality of groups of key-value pairs based on the clustering result, and subjecting the polished contact point trajectory after filtering processing to down-sampling processing based on the plurality of groups of key-value pairs in combination with the start and end points to obtain the polished contact point trajectory after down-sampling processing.

[0076] Specifically, the first neighborhood radius and the minimum sample number of the density-based clustering algorithm are set, and the trajectory points in the polished contact point trajectory after filtering are clustered based on the first neighborhood radius and the minimum sample number, that is, if a trajectory point has a minimum sample number or more points in its first neighborhood radius, these points are classified as the same cluster point, and the process is repeated until all trajectory points are clustered, and a plurality of cluster points are obtained, that is, the clustering result is obtained, and the start and end points of the polished contact point trajectory after filtering are determined based on the clustering result, that is, the first cluster point is taken as the starting point, and the last cluster point is taken as the ending point. A plurality of key-value pairs are generated based on the clustering result, the corresponding key-value pairs are generated according to the cluster points and their labels, and the polished contact point trajectory after filtering is down-sampled based on the plurality of key-value pairs combined with the start and end points. Starting from the starting point of the key-value pair, the distance between the selected trajectory point and the previous trajectory point is calculated from the polished contact point trajectory after filtering, and if the distance is greater than or equal to the preset distance threshold, the selected trajectory point is taken as an effective point, and if the distance is less than the preset distance threshold, the trajectory point is deleted. Repeat the above process until the end point is sampled, and the sampling process is completed. The down-sampled trajectory point set forms a new polished contact point trajectory, that is, the polished contact point trajectory after down-sampling is obtained.

[0077] S14: generating a polishing trajectory in the base coordinate system of the robot arm based on the point cloud data of the workpiece to be polished and the down-sampled polishing contact point trajectory, and performing coarse optimization processing on the polishing trajectory combined with an optimization function to obtain a coarse optimized polishing trajectory;

[0078] In the specific implementation process of the present application, the coarse optimization processing on the polishing trajectory combined with the optimization function to obtain the coarse optimized polishing trajectory includes: setting a minimization function as the optimization function, and performing iterative optimization processing on each trajectory point in the polishing trajectory based on the BFGS optimization method combined with the optimization function to obtain the coarse optimized polishing trajectory. The expression of the optimization function is:

[0079]

[0080] Wherein, f is the optimization function, n is the number of trajectory points, min is the minimization function, p i is a trajectory point, and c is a point in the point cloud data;

[0081] The expression of the iterative optimization processing is:

[0082]

[0083] Wherein, x k+1 is the translation amount of the trajectory point after iterative optimization processing, x k is the trajectory point to be iteratively optimized, and a kis the step size, k is the iteration number, H k is the inverse matrix of the approximate Hessian matrix, is the gradient of the objective function at x k .

[0084] Specifically, although the polished contact point trajectory after downsampling processing is basically consistent with the surface of the workpiece to be polished in shape, there is still a certain deviation in position, so the polished contact point trajectory needs to be coarsely optimized to realize overall correction of the trajectory. The workpiece to be polished is scanned by a 3D camera to obtain corresponding point cloud data, and a polishing trajectory in the base coordinate system of the robot arm is generated based on the hand-eye calibration matrix using the point cloud data of the workpiece to be polished and the polished contact point trajectory after downsampling processing. The hand-eye calibration matrix can determine the accurate conversion relationship between the robot arm and the camera, so the trajectory data in the camera coordinate system can be converted to the trajectory data in the base coordinate system of the robot arm through the hand-eye calibration matrix. The effect diagram of the polishing trajectory in the base coordinate system of the robot arm is shown in Figure 5 . A minimization function is set as an optimization function, which represents the minimization of the distance between the trajectory points and the point cloud data of the workpiece to be polished. Based on the BFGS (Broyden-Fletcher-Goldfarb-Shanno) optimization method, each trajectory point in the polishing trajectory is iteratively optimized based on the optimization function to obtain a coarsely optimized polishing trajectory. The BFGS optimization method is an iterative algorithm for nonlinear optimization problems, which reduces the computational complexity by approximating the inverse matrix of the Hessian matrix while maintaining a fast convergence speed. The expression of the optimization function is:

[0085]

[0086] where f is the optimization function, n is the number of trajectory points, min is the minimization function, p i is the trajectory point, and c is the point in the point cloud data.

[0087] The expression of the iterative optimization processing is:

[0088]

[0089] where x k+1 is the translation amount of the trajectory point after iterative optimization processing, x k is the trajectory point to be iteratively optimized, a k is the step size, k is the iteration number, H k is the inverse matrix of the approximate Hessian matrix, is the gradient of the objective function at x kThe gradient of each iteration is recalculated until the minimum value of the optimization function is reached. According to the translation amount of the obtained trajectory points, all trajectory points are offset to better fit the surface of the workpiece to be polished, the rough optimization of the polishing trajectory is completed, and the effect diagram after the rough optimization of the polishing trajectory is as shown in FIG. 6. Figure 6

[0090] S15: performing fine optimization processing on the rough optimization polishing trajectory to obtain the target polishing trajectory of the mechanical arm.

[0091] In the specific implementation process of the present application, the fine optimization processing on the rough optimization polishing trajectory to obtain the target polishing trajectory of the mechanical arm includes: taking all trajectory points in the rough optimization polishing trajectory as optimization points, setting a second neighborhood radius, and taking the direction of all points in the point cloud data within the second neighborhood radius pointing to each optimization point as the reference direction; calculating the vector of each reference direction, and taking the sum of the vectors of all reference directions as the optimization direction; and performing optimization processing on each optimization point based on the optimization direction and the preset distance threshold to form the target polishing trajectory of the mechanical arm.

[0092] Specifically, all trajectory points in the rough optimization polishing trajectory are taken as optimization points, a second neighborhood radius is set, and the direction of all points in the point cloud data within the second neighborhood radius pointing to each optimization point is taken as the reference direction, i.e. for each optimization point, the optimization point is taken as the center, all points in the point cloud data within the second neighborhood radius of the center are taken, and the direction of these points pointing to the optimization point is taken as the reference direction. The vector of each reference direction is calculated, and the sum of the vectors of all reference directions is taken as the optimization direction. Each optimization point is optimized based on the optimization direction and the preset distance threshold. If the distance between the trajectory point of the rough optimization polishing trajectory and the point cloud data is greater than the preset distance threshold, the trajectory point is moved in the direction of reducing the distance in the opposite direction of the optimization direction, and if the distance between the trajectory point and the point cloud data is less than the preset distance threshold, the trajectory point is moved in the direction of increasing the distance in the positive direction of the optimization direction. After all optimization points are optimized, all optimized trajectory points are connected to form the final mechanical arm running trajectory, i.e. the target polishing trajectory of the mechanical arm, and the effect diagram of the fine optimization polishing trajectory is as shown in FIG. 7. Figure 7

[0093] ​​In the embodiment of the present application, the motion trajectory and posture information of the polishing demonstration object are analyzed based on the polishing demonstration video using the posture estimation model to generate the polishing contact point trajectory, and the polishing contact point trajectory is filtered to reduce the interference posture introduced by shaking or lagging during manual demonstration. The start and end points of the filtered polishing contact point trajectory are extracted using a density-based clustering algorithm for downsampling processing of the filtered polishing contact point trajectory. The polishing trajectory in the base coordinate system of the robot arm is quickly generated based on the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory. The polishing trajectory is coarsely optimized using an optimization function, and the coarsely optimized polishing trajectory is finely optimized, so that the target polishing trajectory of the robot arm obtained is more reliable. When generating the polishing trajectory, a certain point on the demonstration object is mapped with the polishing tool actually used, which does not limit the object used for demonstration and does not depend on the surface and edge features of the object point cloud, i.e., the generated trajectory is not limited to the edge of the object, and a complex trajectory can be generated, which can be applied to different polishing scenes and complex workpieces, improving the universality of the polishing trajectory generation of the robot arm. Compared with the way of teaching the polishing trajectory by the teaching pendant, the polishing trajectory is generated from the polishing demonstration video, which is more efficient in polishing trajectory generation speed. The programming-free trajectory generation method does not require the participation of the teaching pendant, reducing the difficulty of using the robot arm and the requirement for professionalism in the operation process of the robot arm.

[0094] Embodiment two

[0095] Please refer to Figure 2 , Figure 2 is a flowchart of a polishing trajectory generation method of a robot arm in another embodiment of the present application, which comprises:

[0096] S201: analyzing the motion trajectory and posture information of the polishing demonstration object based on the polishing demonstration video using the posture estimation model;

[0097] S202: generating the polishing contact point trajectory based on the motion trajectory and posture information, and filtering the polishing contact point trajectory to obtain the filtered polishing contact point trajectory;

[0098] S203: setting the first neighborhood radius and the minimum number of samples of the density-based clustering algorithm, clustering the trajectory points in the filtered polishing contact point trajectory based on the first neighborhood radius and the minimum number of samples to obtain the clustering result, and determining the start and end points of the filtered polishing contact point trajectory based on the clustering result;

[0099] S204: generating a plurality of key-value pairs based on the clustering result, and downsampling the filtered polishing contact point trajectory based on the plurality of key-value pairs and the start and end points to obtain the downsampled polishing contact point trajectory;

[0100] S205: generating a polishing trajectory in the base coordinate system of the mechanical arm based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the polishing contact point trajectory after the downsampling processing, and performing coarse optimization processing on the polishing trajectory in combination with an optimization function to obtain a coarse-optimized polishing trajectory;

[0101] S206: taking all trajectory points in the coarse-optimized polishing trajectory as to-be-optimized points, setting a second neighborhood radius, and taking the direction of each to-be-optimized point in all point cloud data within the second neighborhood radius as a reference direction;

[0102] S207: calculating the vector of each reference direction, and taking the sum of the vectors of all reference directions as an optimization direction;

[0103] S208: performing optimization processing on each to-be-optimized point based on the optimization direction and a preset distance threshold to form a target polishing trajectory of the mechanical arm.

[0104] In the embodiment of the application, the motion trajectory and posture information of the polishing demonstration object are analyzed based on the polishing demonstration video to generate a polishing contact point trajectory by using a posture estimation model, and the polishing contact point trajectory is filtered to reduce the interference posture introduced by shaking or lagging during manual demonstration. The start and end points of the filtered polishing contact point trajectory are extracted by using a density-based clustering algorithm to perform downsampling processing on the filtered polishing contact point trajectory, and the polishing trajectory in the base coordinate system of the mechanical arm can be quickly generated based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the polishing contact point trajectory after the downsampling processing. The polishing trajectory is coarsely optimized by using an optimization function, and the coarse-optimized polishing trajectory is finely optimized, so that the target polishing trajectory of the mechanical arm obtained is more reliable. When generating the polishing trajectory, a certain point on the demonstration object is mapped with the polishing tool actually used, the object used for demonstration is not limited, and the surface and edge features of the object point cloud are not relied on, that is, the generated trajectory is not limited to the edge of the object, a complex trajectory can be generated, which can be applied to different polishing scenes and complex workpieces, and the universality of the polishing trajectory generation of the mechanical arm is improved. Compared with the way of teaching the polishing trajectory by using the operation teach pendant, the polishing trajectory generation speed is more efficient. The programming-free trajectory generation mode does not need the participation of the teach pendant, reduces the difficulty of using the mechanical arm, and reduces the requirement for professionalism in the operation process of the mechanical arm.

[0105] Embodiment three

[0106] Please refer to Figure 3 , Figure 3 is a structural composition schematic diagram of the polishing trajectory generation device of the mechanical arm in the embodiment of the application, the device comprises:

[0107] The trajectory posture analysis module 31 is configured to analyze the motion trajectory and posture information of the polishing demonstration object based on the polishing demonstration video by using a posture estimation model;

[0108] The trajectory filtering module 32 is configured to generate a polishing contact point trajectory based on the motion trajectory and the posture information, and perform filtering processing on the polishing contact point trajectory to obtain a filtered polishing contact point trajectory.

[0109] The trajectory downsampling module 33 is configured to extract start and end points of the filtered polishing contact point trajectory by using a density-based clustering algorithm, and perform downsampling processing on the filtered polishing contact point trajectory based on the start and end points to obtain a downsampled polishing contact point trajectory.

[0110] The trajectory coarse optimization module 34 is configured to generate a polishing trajectory in a robot base coordinate system based on the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory by using a hand-eye calibration matrix, and perform coarse optimization processing on the polishing trajectory in combination with an optimization function to obtain a coarse-optimized polishing trajectory.

[0111] The trajectory fine optimization module 35 is configured to perform fine optimization processing on the coarse-optimized polishing trajectory to obtain a target polishing trajectory of the robot.

[0112] In the specific implementation process of the present application, the specific implementation of the device item can refer to the implementation of the above-mentioned method item, which will not be repeated here.

[0113] In the embodiment of the present application, the motion trajectory and posture information of the polishing demonstration object are analyzed based on the polishing demonstration video to generate a polishing contact point trajectory by using a posture estimation model, and the polishing contact point trajectory is filtered to reduce the interference posture caused by shaking or lagging during manual demonstration. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory to perform downsampling processing on the filtered polishing contact point trajectory. The polishing trajectory in the base coordinate system of the robot arm can be quickly generated based on the point cloud data of the workpiece to be polished and the downsampled polishing contact point trajectory. The polishing trajectory is coarsely optimized by using an optimization function, and the coarsely optimized polishing trajectory is finely optimized, so that the target polishing trajectory of the robot arm obtained is more reliable. When generating the polishing trajectory, a certain point on the demonstration object is mapped with the polishing tool actually used, so that the object used for demonstration is not limited, and the surface and edge features of the object point cloud are not dependent, that is, the generated trajectory is not limited to the edge of the object, and a complex trajectory can be generated, which can be applied to different polishing scenes and complex workpieces, and the universality of the polishing trajectory generation of the robot arm is improved. Compared with the way of teaching the polishing trajectory by using the teaching pendant, the polishing trajectory is generated from the polishing demonstration video, and the polishing trajectory generation speed is more efficient. The programming-free trajectory generation method does not need the participation of the teaching pendant, reduces the use difficulty of the robot arm, and reduces the requirement for professionalism in the operation process of the robot arm.

[0114] The computer readable storage medium provided in the embodiment of the present application stores a computer program, and the program is executed by a processor to realize the polishing trajectory generation method of the robot arm in any one of the above embodiments. The computer readable storage medium includes but is not limited to any type of disk (including a floppy disk, a hard disk, an optical disk, a CD-ROM, and a magneto-optical disk), a ROM (Read-Only Memory), a RAM (Random Access Memory), an EPROM (Erasable Programmable Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, a magnetic card or an optical card. That is, the storage device includes any medium that stores or transmits information in a form capable of being read by a device (for example, a computer, a mobile phone), and can be a read-only memory, a magnetic disk or an optical disk, etc.

[0115] Embodiment four

[0116] Please refer to Figure 4 , Figure 4It is a schematic diagram of the structure of an electronic device in an embodiment of the present invention.

[0117] The embodiment of the present invention further provides an electronic device, such as Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. It will be understood by those skilled in the art that Figure 3 The electronic devices shown do not constitute a limitation on all devices and may include more or fewer components than shown, or combinations of certain components. The memory 41 can be used to store the computer program 42 and various functional modules, and the processor 43 runs the computer program 42 stored in the memory 41, thereby executing various functional applications and data processing of the device. The memory can be internal memory or external memory, or include both internal memory and external memory. The internal memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. The external memory can include a hard disk, floppy disk, ZIP disk, USB flash drive, magnetic tape, etc. The processor 43 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, a single-chip microcomputer, or processor 43, or any conventional processor. The processor and memory disclosed in the present invention include but are not limited to these types of processors and memories. The processor and memory disclosed in the present invention are only examples and not limitations.

[0118] As an embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and are configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to execute the robot arm grinding trajectory generation method in any of the above-mentioned embodiments. For the specific implementation process, please refer to the above-mentioned embodiments and will not be repeated here.

[0119] In the embodiment of the present application, the motion trajectory and posture information of the polishing demonstration object are analyzed by using the posture estimation model based on the polishing demonstration video to generate the polishing contact point trajectory, and the polishing contact point trajectory is filtered to reduce the interference posture introduced by shaking or lagging during manual demonstration. A density-based clustering algorithm is used to extract the start and end points of the filtered polishing contact point trajectory to perform down-sampling processing on the filtered polishing contact point trajectory. The polishing trajectory in the base coordinate system of the robot arm is quickly generated based on the point cloud data of the workpiece to be polished and the down-sampled polishing contact point trajectory. The polishing trajectory is coarsely optimized by using an optimization function, and the coarsely optimized polishing trajectory is finely optimized, so that the target polishing trajectory of the robot arm obtained is more reliable. When generating the polishing trajectory, a certain point on the demonstration object is mapped with the polishing tool actually used, which does not limit the object used for demonstration and does not depend on the surface and edge features of the object point cloud, i.e., the generated trajectory is not limited to the edge of the object, and a complex trajectory can be generated, which can be applied to different polishing scenes and complex workpieces, and improves the universality of the polishing trajectory generation of the robot arm. The polishing trajectory is generated from the polishing demonstration video, and compared with the way of teaching the polishing trajectory by using the teaching pendant, the polishing trajectory generation speed is more efficient. The programming-free trajectory generation method does not require the participation of the teaching pendant, reduces the difficulty of using the robot arm, and reduces the requirement for professionalism in the operation process of the robot arm.

[0120] In addition, the above describes in detail a polishing trajectory generation method for a robot arm and related devices provided by the embodiment of the present application. The principles and implementation manners of the present application are described by using specific examples in this paper. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for generating a grinding trajectory of a robotic arm, characterized in that: The method comprises: Based on the polishing demonstration video, the posture estimation model is used to analyze the motion trajectory and posture information of the polishing demonstration object; Generate a polishing contact point trajectory based on the motion trajectory and posture information, and perform filtering on the polishing contact point trajectory to obtain a polishing contact point trajectory after filtering; A density-based clustering algorithm is used to extract the start and end points of the polishing contact point trajectory after filtering, and the polishing contact point trajectory after filtering is downsampled based on the start and end points to obtain the polishing contact point trajectory after downsampling; Based on the hand-eye calibration matrix, the point cloud data of the workpiece to be polished and the polishing contact point trajectory after downsampling are used to generate the polishing trajectory in the robot arm base coordinate system. The polishing trajectory is then roughly optimized using the optimization function to obtain the roughly optimized polishing trajectory. The coarse-optimized grinding trajectory is fine-optimized to obtain the target grinding trajectory of the robotic arm.

2. The method for generating a grinding trajectory of a robotic arm according to claim 1, wherein: The method of analyzing the motion trajectory and posture information of the polishing demonstration object using a posture estimation model based on the polishing demonstration video includes: A polishing demonstration image sequence is extracted based on the polishing demonstration video, and the polishing demonstration image sequence is input into a posture estimation model to analyze the motion trajectory and posture information of the polishing demonstration object. The posture estimation model adopts the Foundationpose posture estimation model.

3. The method for generating a grinding trajectory of a robotic arm according to claim 1, wherein: The step of generating a polishing contact point trajectory based on the motion trajectory and the posture information, and filtering the polishing contact point trajectory to obtain the polishing contact point trajectory after filtering includes: Contact point analysis is performed based on the motion trajectory and posture information combined with the shape characteristics of the polishing demonstration object to obtain the contact point position information and contact point posture information; Generate a polishing contact point trajectory based on the contact point position information and the contact point posture information; Performing median filtering on the polishing contact point trajectory to obtain the polishing contact point trajectory after median filtering; The polishing contact point trajectory after the median filtering is subjected to mean filtering to obtain the polishing contact point trajectory after the filtering.

4. The method for generating a grinding trajectory of a robotic arm according to claim 3, wherein: The expression of the median filtering process is: y i =median({x i ,x i+1 ,x i+2 ,x i+3 ,x i+k-1 }),0≤i≤N1-k, Among them, y i is the polishing contact point trajectory data after median filtering, x is the original polishing contact point trajectory data, median is the median filter function, N1 is the length of the polishing contact point trajectory sequence used for filtering, and k is the median filter window size; The expression of the mean filtering process is: Among them, z i is the polishing contact point trajectory data after mean filtering, m is the mean filtering window size, y is the polishing contact point trajectory data after median filtering, and N2 is the length of the polishing contact point trajectory sequence used for filtering.

5. The method for generating a grinding trajectory of a robotic arm according to claim 1, wherein: The method of extracting the starting and ending points of the polishing contact point trajectory after filtering by using a density-based clustering algorithm and performing downsampling processing on the polishing contact point trajectory after filtering based on the starting and ending points to obtain the polishing contact point trajectory after downsampling processing includes: Setting a first neighborhood radius and a minimum number of samples for a density-based clustering algorithm, clustering the trajectory points in the polishing contact point trajectory after filtering based on the first neighborhood radius and the minimum number of samples to obtain a clustering result, and determining the start and end points of the polishing contact point trajectory after filtering based on the clustering result; Based on the clustering results, several groups of key-value pairs are generated, and based on the several groups of key-value pairs combined with the start and end points, the polishing contact point trajectory after filtering is downsampled to obtain the polishing contact point trajectory after downsampling.

6. The method for generating a grinding trajectory of a robotic arm according to claim 1, wherein: The step of performing rough optimization processing on the grinding trajectory in combination with the optimization function to obtain a rough optimized grinding trajectory includes: The minimization function is set as the optimization function, and each trajectory point in the polishing trajectory is iteratively optimized based on the BFGS optimization method combined with the optimization function to obtain a rough optimized polishing trajectory. The expression of the optimization function is: Among them, f is the optimization function, n is the number of trajectory points, min is the minimization function, and p i is the trajectory point, c is the point in the point cloud data; The expression of the iterative optimization process is: x k+1 =x k -α k H k ▽f(x k ), Among them, x k+1 is the translation amount of the trajectory point after iterative optimization, x k is the trajectory point to be iteratively optimized, α k is the step size, k is the number of iterations, H k is the inverse matrix of the approximate Hessian matrix, ▽f(x k ) is the objective function at x k The gradient at .

7. The method for generating a grinding trajectory of a robotic arm according to claim 1, wherein: The fine optimization process of the coarse optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm includes: All trajectory points in the rough optimization polishing trajectory are taken as points to be optimized, a second neighborhood radius is set, and the directions of all points in the point cloud data within the second neighborhood radius pointing to the points to be optimized are taken as reference directions; Calculate the vector of each reference direction, and use the sum of the vectors of all reference directions as the optimization direction; Based on the optimization direction and the preset distance threshold, each point to be optimized is optimized to form the target grinding trajectory of the robotic arm.

8. A robot arm grinding trajectory generation device, characterized in that: The device comprises: Trajectory and posture analysis module: used to analyze the motion trajectory and posture information of the polishing demonstration object based on the polishing demonstration video using the posture estimation model; Trajectory filtering module: used to generate a polishing contact point trajectory based on the motion trajectory and posture information, and filter the polishing contact point trajectory to obtain the polishing contact point trajectory after filtering; Trajectory downsampling module: used to extract the start and end points of the polishing contact point trajectory after filtering using a density-based clustering algorithm, and downsample the polishing contact point trajectory after filtering based on the start and end points to obtain the polishing contact point trajectory after downsampling; Trajectory coarse optimization module: This module is used to generate the grinding trajectory in the robot arm base coordinate system using the point cloud data of the workpiece to be polished and the down-sampled grinding contact point trajectory based on the hand-eye calibration matrix, and to perform coarse optimization processing on the grinding trajectory in combination with the optimization function to obtain the coarse optimized grinding trajectory; Trajectory fine optimization module: used to perform fine optimization on the coarse optimized grinding trajectory to obtain the target grinding trajectory of the robotic arm.

9. An electronic device comprising a processor and a memory, characterized in that: The memory is used to store instructions, and the processor is used to call the instructions in the memory, so that the electronic device executes the robot arm grinding trajectory generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method for generating a robotic arm grinding trajectory according to any one of claims 1 to 7.

Images