A method for generating grasping data sets for multi-finger underactuated grippers

By establishing a jaw collision enclosure box model and PyBullet simulation scenario, a multi-finger under-driven jaw grab data set is generated, which solves the problem of high complexity in the data set generation in the existing technology, and achieves fast and efficient data set generation.

CN115042171BActive Publication Date: 2024-11-22SHANGHAI JIAOTONG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210619169.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-01
Publication Date
2024-11-22
Estimated Expiration
2042-06-01

AI Technical Summary

Technical Problem

The data-driven multi-finger grabbing method in the prior art lacks corresponding data sets, resulting in high complexity in the generation of multi-finger under-driven clamping data sets, making it difficult to quickly generate large-scale data sets.

Method used

By establishing a jaw collision enclosure box model based on forward kinematics, the jaw configuration is quickly obtained using matrix calculation and DH parameter matrix, and combined with PyBullet simulation scenarios, collect and transform data, generate a single object grabbing candidate model, and finally use it to The collision bounding box model and PyBullet realize the rapid generation of multi-finger under-driven jaw grabbing datasets.

Benefits of technology

The efficiency of jaw configuration calculation and scene collision detection is improved, and the rapid generation of multi-finger under-driven jaw grabbing data sets is realized, reducing the complexity of data set generation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115042171B_ABST
    Figure CN115042171B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for generating a data set for grasping a multi-finger underactuated gripper, which relates to the field of robot grasping and comprises the following steps: establishing a gripper collision bounding box model based on forward kinematics, using matrix calculation and DH parameter matrix to quickly obtain the gripper configuration, and completing scene collision detection; establishing a grasping simulation scene based on PyBullet, and collecting grasping scene data; establishing a single object grasping candidate model based on the collision bounding box model and GWS grasping index, the candidate model also including the gripper configuration and scoring factors for the gripper configuration; based on the model pose in the scene and the scene collision detection result, using coordinate transformation, transforming the single object grasping candidate model into a scene grasping candidate model, and completing data set generation. Based on the collision bounding box model, the present invention uses the parallel computing power of GPU to encapsulate the forward kinematics of the gripper, improves the efficiency of gripper configuration calculation and scene collision detection, and can quickly generate a large-scale grasping data set for a multi-finger underactuated gripper.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of robot grasping, and in particular to a method for generating a grasping data set of a multi-finger under-actuated gripper. Background Art

[0002] Robot grasping is an important research topic in the field of robot manipulation, involving perception, planning and control tasks, which is to find the appropriate robot actuator configuration related to the target state to achieve stable grasping. Since complex and unstructured environments increase the uncertainty of robot grasping, general object grasping in unstructured environments is crucial for robot manipulation.

[0003] Most current methods use a parallel two-finger gripper, but in order to achieve a more stable and smooth grasping, an underactuated multi-finger gripper is a feasible solution. Compared with a parallel two-finger gripper, the high search dimension of a multi-finger gripper increases the complexity of generating a multi-finger grasping dataset, which hinders the development of data-driven multi-finger grasping methods.

[0004] Therefore, technicians in this field are committed to developing a method for generating a grasping data set for a multi-finger under-actuated gripper. Summary of the invention

[0005] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is to provide a method for quickly generating a large-scale grasping data set for a multi-finger under-actuated gripper, thereby solving the problem of lack of corresponding data sets in the current data-driven multi-finger grasping field.

[0006] To achieve the above object, the present invention provides a method for generating a grasping data set of a multi-finger under-actuated gripper, characterized in that the method comprises the following steps:

[0007] S101: establishing a collision bounding box model of the gripper based on forward kinematics, using matrix calculation and DH parameter matrix to quickly obtain the gripper configuration, and completing scene collision detection;

[0008] S103: establishing a PyBullet-based grasping simulation scene, and collecting the grasping scene data, wherein the scene data includes camera position data and object model data in the scene;

[0009] S105: establishing a single object grasping candidate model based on the collision bounding box model and the GWS grasping index, wherein the candidate model also includes the gripper configuration and a scoring factor for the gripper configuration;

[0010] S107: Based on the model pose in the scene and the scene collision detection result, the single object grasping candidate model is transformed into a scene grasping candidate model by using coordinate transformation, thereby completing the data set generation;

[0011] In the method for generating a grasping dataset of a multi-finger under-actuated gripper based on forward kinematics, by combining the collision bounding box model and the single object grasping candidate model, and utilizing a grasping scene randomly generated based on PyBullet, the generation of a grasping dataset is rapidly realized.

[0012] Furthermore, in step S101, the matrix calculation is set to facilitate GPU acceleration, and the point set inside the bounding box can be quickly obtained through the matrix calculation. The matrix calculation meets the following requirements:

[0013] S=abs((unsqueeze(P)-C)R)≤E / 2

[0014] Among them, if and only if the last dimension of S is all True, the corresponding index point is located inside the i-th bounding box, abs is the absolute value operation of the matrix, unsqueeze is the operation of copying and expanding the matrix to satisfy the matrix operation dimension, shape is the operation of obtaining the matrix dimension, P represents the point set to be calculated, satisfying shape(P)=(N,3), C represents the center of the bounding box, satisfying shape(C)=(B,I,1,3), R represents the axial direction of the bounding box, satisfying shape(R)=(B,I,3,3), E represents the length of the bounding box along the axial direction, satisfying shape(E)=(B,I,1,3), B is the number of batches calculated in parallel, I is the number of bounding boxes participating in the calculation in each batch, and N is the number of calculated point sets.

[0015] Furthermore, in the step S101, the DH parameter matrix encapsulates the forward kinematics model of the gripper, and through matrix coordinate transformation, the bounding box model parameters of the gripper component are quickly calculated.

[0016] Furthermore, in step S101, the collision detection threshold is set to the ratio of the number of points inside the bounding box to the number of neighborhood points. The collision detection threshold can be used to quickly determine whether the gripper collides with the scene point cloud, as well as to perform contact detection between the gripper and the object model.

[0017] Furthermore, in the step S101, by means of contact detection based on the bounding box, it is possible to quickly regress multiple groups of the jaw joint angles under the posture of the specified jaw and the object model.

[0018] Furthermore, in step S103, a group of objects are randomly selected and randomly placed in different positions. By changing the camera position, multiple scene RGBD data are collected for the same object, and the point cloud data of the scene is calculated and obtained using the internal and external parameters of the camera and the RGBD information.

[0019] Further, in step S105, the single object grasping candidate model includes multiple sets of parameters, including grasping candidate points, grasping depth, grasping angle and multi-finger configuration, and the number of single object grasping candidates is set to be determined by the multiple sets of parameters.

[0020] Furthermore, in the step S105, the single object grasping candidates also need to exclude illegal candidates, and the illegal candidates include initial collisions between the gripper and the model based on the bounding box model.

[0021] Furthermore, in the step S107, the scene grabbing candidates also need to filter out illegal scene grabbing candidates, and the illegal scene grabbing candidates include illegal scene grabbing candidates of collision and grabbing candidates of illegal contact direction filtered out by desktop normal.

[0022] Furthermore, when filtering the grab candidates of illegal contact directions by using the desktop normal, a grab candidate when the angle between the contact direction and the desktop normal is greater than a predetermined angle is an illegal grab candidate, and the predetermined angle is 90 degrees.

[0023] In a preferred embodiment of the present invention, the present invention provides a method for generating a fast grasping data set in a multi-finger under-actuated gripper scenario, which is based on a collision bounding box model and utilizes the GPU parallel computing capability to encapsulate the forward kinematics of the gripper, thereby improving the efficiency of gripper configuration calculation and scene collision detection.

[0024] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of a method for generating a grasping data set of a multi-finger underactuated gripper according to a preferred embodiment of the present invention;

[0026] Figure 2 It is a simplified diagram of the steps of a method for generating a grasping data set of a multi-finger underactuated gripper according to a preferred embodiment of the present invention;

[0027] Figure 3 It is a schematic diagram of a multi-finger gripper bounding box model in different configurations of a preferred embodiment of the present invention.

[0028] Among them, 1-generate a gripper bounding box model, 2-generate a single object grasping candidate, 3-generate corresponding scene data, and 4-obtain a scene grasping candidate. DETAILED DESCRIPTION

[0029] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0030] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0031] like Figure 1 As shown, an embodiment of the present invention provides a method for generating a grasping data set of a multi-finger under-actuated gripper, the method comprising the following steps:

[0032] S101: establishing a collision bounding box model of the gripper based on forward kinematics, using matrix calculation and DH parameter matrix to quickly obtain the gripper configuration, and completing scene collision detection;

[0033] S103: establishing a PyBullet-based grasping simulation scene, and collecting the grasping scene data, wherein the scene data includes camera position data and object model data in the scene;

[0034] S105: establishing a single object grasping candidate model based on the collision bounding box model and the GWS grasping index, wherein the candidate model also includes the gripper configuration and a scoring factor for the gripper configuration;

[0035] S107: Based on the model pose in the scene and the scene collision detection result, the single object grasping candidate model is transformed into a scene grasping candidate model by using coordinate transformation, thereby completing the data set generation;

[0036] In the method for generating a grasping dataset of a multi-finger under-actuated gripper based on forward kinematics, by combining the collision bounding box model and the single object grasping candidate model, and utilizing a grasping scene randomly generated based on PyBullet, the generation of a grasping dataset is rapidly realized.

[0037] Through the above-mentioned method of the embodiment of the present invention, the present invention targets the grasping scenario of a multi-finger under-actuated gripper, based on the collision bounding box model, and utilizes the parallel computing capability of the GPU to encapsulate the forward kinematics of the gripper, thereby improving the efficiency of the gripper configuration calculation and scene collision detection, and can quickly complete the data set generation in the grasping scenario of a multi-finger under-actuated gripper.

[0038] The embodiment of the present invention further refines step S101, specifically including:

[0039] 1) The matrix calculation is set to facilitate GPU acceleration. The point set inside the bounding box can be quickly obtained through the matrix calculation. The matrix calculation meets the following requirements:

[0040] S=abs((unsqueeze(P)-C)R)≤E / 2

[0041] Among them, if and only if the last dimension of S is all True, the corresponding index point is located inside the i-th bounding box, abs is the absolute value operation of the matrix, unsqueeze is the operation of copying and expanding the matrix to satisfy the matrix operation dimension, shape is the operation of obtaining the matrix dimension, P represents the point set to be calculated, satisfying shape(P)=(N,3), C represents the center of the bounding box, satisfying shape(C)=(B,I,1,3), R represents the axial direction of the bounding box, satisfying shape(R)=(B,I,3,3), E represents the length of the bounding box along the axial direction, satisfying shape(E)=(B,I,1,3), B is the number of batches calculated in parallel, I is the number of bounding boxes participating in the calculation in each batch, and N is the number of calculated point sets.

[0042] 2) The DH parameter matrix encapsulates the forward kinematics model of the gripper, and quickly calculates the bounding box model parameters of the gripper component through matrix coordinate transformation.

[0043] 3) The collision detection threshold is set as the ratio of the number of points inside the bounding box to the number of neighborhood points. The collision detection threshold can be used to quickly determine whether the gripper collides with the scene point cloud, as well as the contact detection between the gripper and the object model.

[0044] 4) Through contact detection based on the bounding box, it is possible to quickly regress multiple sets of the jaw joint angles under the specified jaw and the object model posture.

[0045] For step S103, the embodiment of the present invention randomly selects a group of objects, randomly places the objects at different positions, collects multiple scene RGBD data for the same object by changing the camera position, and uses the internal and external parameters of the camera and the RGBD information to calculate and obtain the point cloud data of the scene.

[0046] For step S105, the single object grasping candidate model in the embodiment of the present invention includes multiple sets of parameters, including grasping candidate points, grasping depth, grasping angle and multi-finger configuration, and the number of single object grasping candidates is set to be determined by the multiple sets of parameters. The single object grasping candidate also needs to exclude illegal candidates, and the illegal candidates include the initial collision between the gripper and the model based on the bounding box model.

[0047] With respect to step S107, the scene grabbing candidates in the embodiment of the present invention also need to filter out illegal scene grabbing candidates, and the illegal scene grabbing candidates include illegal scene grabbing candidates of collision and grabbing candidates of illegal contact direction filtered out by desktop normal. When the grabbing candidates of illegal contact direction are filtered out by desktop normal, the grabbing candidate when the angle between the contact direction and the desktop normal is greater than a predetermined angle is an illegal grabbing candidate, and the predetermined angle is 90 degrees.

[0048] The present invention is described in detail below in conjunction with a preferred embodiment of the present invention.

[0049] like Figure 1 As shown, the purpose of the present invention is to provide a method for generating a fast grasping data set for a multi-finger under-actuated gripper, provide a training data set for a data-driven grasping method, and make full use of the parallel computing capability of the GPU based on the collision bounding box model matrix form and the DH parameter matrix to improve the efficiency of gripper configuration calculation and scene collision detection.

[0050] In step S101, the object model point cloud in the YCB data set is downsampled to a suitable size to avoid video memory overflow. Based on the motion characteristics of the underactuated gripper, a DH parameter matrix is ​​established to achieve a single calculation of the gripper model parameters. Subsequent parameter calculations are obtained through DH matrix transformation. The parameters mainly include:

[0051] Gripper point-surface model for display and accurate collision detection;

[0052] The parameters of the bounding box of the gripper component are used to quickly detect the inclusion relationship between the point set and the bounding box using the matrix form S=abs((unsqueeze(P)-C)R)≤E / 2.

[0053] Based on the downsampled object point cloud model and the gripper bounding box model, the gripper motion sequence is equally sampled, and with the help of bounding box model contact detection, the gripper configuration is obtained under given relative posture conditions between the gripper and the object.

[0054] In step S103, a grasping simulation scene is established based on PyBullet, and (3 to 8) objects are randomly selected and randomly placed at different positions on the desktop. By changing the camera position, n scene RGBD data are collected for the same object, and the scene point cloud data is calculated and obtained using the camera's internal and external parameters and RGBD information.

[0055] In step S105, first, based on the model obtained by downsampling in step S101, (1000-3000) grasping candidate points are sampled. Considering that the volume of the multi-finger gripper model and the grasping direction of general objects are all in the grasping point normal direction, it is assumed here that the approach direction of the grasping candidate point is the contact point normal direction. For each grasping point, (3-5) grasping depths are selected along the approaching direction, and for each discrete grasping depth, 24 grasping angles are rotated in the grasping plane. Then, the number of grasping candidates obtained by sampling for each object is: (1000-3000)×(3-5)×24. Considering that the multi-finger gripper also contains a key degree of freedom, that is, the relative position relationship between multiple fingers, a discrete multi-finger configuration variable Q_c is introduced here. In this example, the number of this variable is 4, so the number of grasping candidates for a single object is: (1000-3000)×(3-5)×24×4. Secondly, based on the bounding box model generated in step S101, illegal candidates of initial collision between the gripper and the model are excluded. For non-collision candidates, the contact joint angle and contact position under the current grasping posture are obtained by discrete gripper motion sequence and setting contact threshold. Based on the current contact position, the GWS grasping index is used to calculate the score of the current grasping candidate.

[0056] In step S107, PyBullet can be used to easily obtain the pose of objects in the scene. The single object grasping candidate obtained in step S105 is transformed into a scene grasping candidate using coordinate transformation. Based on the bounding box model in step S101, illegal scene grasping candidates that collide are excluded, and grasping candidates with illegal contact directions are screened through the desktop normal. Here, when the angle between the contact direction and the desktop normal is greater than 90 degrees, it is selected as an illegal grasping candidate, thereby obtaining the grasping candidates in the scene.

[0057] like Figure 2 As shown, in order to more intuitively describe the present invention, a preferred embodiment of the present invention provides a simplified diagram of the steps of a method for generating a grasping data set of a multi-finger underactuated gripper, wherein:

[0058] 1-Generate the bounding box model of the gripper: Generate the bounding box model of the gripper by using the input gripper and object model using forward kinematics;

[0059] 2-Generate single object grasping candidates: Use the gripper joint discrete generation of multiple sets of grasping states and bounding box collision detection algorithm to generate single object grasping candidates;

[0060] 3-Generate corresponding scene data: randomly select multiple objects through PyBullet to generate corresponding scene data, which includes RGBD data and the pose data of the object in the camera coordinate system;

[0061] 4- Obtain scene grasping candidates: The single object grasping candidates generated in process 2 are transferred to the scene camera coordinate system through coordinate transformation, and based on the gripper bounding box model, illegal grasping candidates that collide with the scene are eliminated to obtain scene grasping candidates.

[0062] like Figure 3 As shown, a schematic diagram of a clamping jaw bounding box model provided by a preferred embodiment of the present invention is shown, wherein the left figure shows a clamping jaw three-dimensional model of different configurations, and the right figure shows a clamping jaw bounding box model of a corresponding configuration. Since the amount of calculation required to directly use the clamping jaw three-dimensional model for collision detection is large, the points in the bounding box can be used for calculation, thereby quickly performing collision detection, reducing the amount of calculation and improving the calculation efficiency.

[0063] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for generating a multi-finger underactuated gripper grasping dataset, It is characterized in that The method comprises the following steps: S101: establishing a collision bounding box model of the gripper based on forward kinematics, using matrix calculation and DH parameter matrix to quickly obtain the gripper configuration, and completing scene collision detection; S103: establishing a grasping simulation scene based on PyBullet, and collecting grasping scene data, wherein the scene data includes camera position data and object model data in the scene; S105: establishing a single object grasping candidate model based on the collision bounding box model and the GWS grasping index, wherein the candidate model also includes the gripper configuration and a scoring factor for the gripper configuration; S107: Based on the model pose in the scene and the scene collision detection result, the single object grasping candidate model is transformed into a scene grasping candidate model by using coordinate transformation, thereby completing the data set generation; In the method for generating a grasping dataset of a multi-finger underactuated gripper based on forward kinematics, by combining the collision bounding box model and the single object grasping candidate model, and using a grasping scene randomly generated based on PyBullet, the generation of a grasping dataset is quickly realized; In step S101, the matrix calculation is set to facilitate GPU acceleration. The point set inside the bounding box can be quickly obtained through the matrix calculation. The matrix calculation meets the following requirements: S=abs((unsqueeze(P)-C)R)≤E / 2 Among them, if and only if the last dimension of S is all True, the corresponding index point is located inside the i-th bounding box, abs is the absolute value operation of the matrix, unsqueeze is the operation of copying and expanding the matrix to satisfy the matrix operation dimension, shape is the operation of obtaining the matrix dimension, P represents the point set to be calculated, satisfying shape(P)=(N,3), C represents the center of the bounding box, satisfying shape(C)=(B,I,1,3), R represents the axial direction of the bounding box, satisfying shape(R)=(B,I,3,3), E represents the length of the bounding box along the axial direction, satisfying shape(E)=(B,I,1,3), B is the number of batches calculated in parallel, I is the number of bounding boxes participating in the calculation in each batch, and N is the number of calculated point sets.

2. The method according to claim 1, It is characterized in that In the step S101, the DH parameter matrix encapsulates the forward kinematics model of the gripper, and quickly calculates the bounding box model parameters of the gripper through matrix coordinate transformation.

3. The method according to claim 2, It is characterized in that In step S101, the collision detection threshold is set to the ratio of the number of points inside the bounding box to the number of neighborhood points. The collision detection threshold can be used to quickly determine whether the gripper collides with the scene point cloud, and to perform contact detection between the gripper and the object model.

4. The method according to claim 3, It is characterized in that In the step S101, by using the bounding box-based contact detection, it is possible to quickly regress multiple groups of jaw joint angles under the specified positions of the jaw and the object model.

5. The method according to claim 1, It is characterized in that In step S103, a group of objects are randomly selected and randomly placed at different positions. By changing the camera position, multiple scene RGBD data are collected for the same object, and the point cloud data of the scene is calculated and obtained using the internal and external parameters of the camera and the RGBD information.

6. The method according to claim 1, It is characterized in that In the step S105, the single object grasping candidate model includes multiple sets of parameters, including grasping candidate points, grasping depth, grasping angle and multi-finger configuration, and the number of single object grasping candidates is set to be determined by the multiple sets of parameters.

7. The method according to claim 6, It is characterized in that In the step S105, the single object grasping candidate also needs to exclude illegal candidates, and the illegal candidates include the initial collision between the gripper and the single object grasping candidate model based on the bounding box model.

8. The method according to claim 1, It is characterized in that In the step S107, the scene grabbing candidates also need to filter out illegal scene grabbing candidates, and the illegal scene grabbing candidates include illegal scene grabbing candidates of collision and grabbing candidates of illegal contact direction filtered out by desktop normal.

9. The method according to claim 8, It is characterized in that When the grab candidates of illegal contact directions are screened out by the desktop normal, the grab candidates when the angle between the contact direction and the desktop normal is greater than a predetermined angle are illegal grab candidates, and the predetermined angle is 90 degrees.