GRIP INFORMATION GENERATION DEVICE, METHOD, AND PROGRAM
The gripping information generation device addresses inconsistency and calculation inefficiencies in existing methods by preprocessing and clustering gripping points, enabling efficient and consistent gripping information for both external and internal gripping scenarios, suitable for diverse robot applications.
Patent Information
- Application Number
- JP2021098120
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-06-11
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-06-11
AI Technical Summary
Existing methods for generating gripping information for robot hands, such as those described in Non-Patent Document 1, suffer from inconsistency due to parameter dependency, require specialized knowledge, involve lengthy calculations, and are limited to external gripping, failing to account for internal gripping scenarios.
A gripping information generation device that acquires and preprocesses three-dimensional object and gripper information, clusters and extracts gripping points based on conditions, and generates relative positional relationships using a non-parametric approach, incorporating simplification, smoothing, and optimization processes to reduce calculation time and ensure consistency.
The solution enables consistent gripping information generation with reduced calculation time, supporting both external and internal gripping, and is applicable to various robot applications, including logistics and industrial tasks, by automating the process and minimizing redundant or impractical grasping information.
Smart Images

Figure 0007585985000004 
Figure 0007585985000005 
Figure 0007585985000006
Abstract
Description
[Technical field]
[0001] The present invention relates to a gripping information generating device, a gripping information generating method, and a gripping information generating program. [Background technology]
[0002] In order to successfully perform robot operations such as pick and throw, pick and place, and pick and assembly of a workpiece by a robot hand, it is necessary for the robot hand to properly grasp the workpiece. In order to generate operation information for grasping the workpiece by such a robot hand, grasping information, which is the relative positional relationship between the robot hand and the workpiece, is prepared in advance.
[0003] As a technique for generating gripping information, for example, a technique for generating gripping information for industrial end effectors such as grippers and suction cups using a computer-aided design (CAD) model of the target object has been proposed (see Non-Patent Document 1). This technique preprocesses a mesh model, which is a model of the target object, by separating the mesh model into overlapping planes. This technique then uses the separated planes to identify contact points of a typical industrial end effector and synthesize a gripping pose. [Prior art documents] [Non-patent literature]
[0004] [Non-Patent Document 1] Weiwei Wan, Kensuke Harada, Fumio Kanehiro, "Planning Grasps with Suction Cups and Parallel Grippers Using Superimposed Segmentation of Object Meshes", IEEE Transactions on Robotics, Volume 37, Issue 1, 2021.02. Summary of the Invention [Problem to be solved by the invention]
[0005] However, the technology described in Non-Patent Document 1 has a problem of lack of consistency because it adopts a parametric-based analysis approach. That is, in the technology described in Non-Patent Document 1, many parameters need to be adjusted. Therefore, the grasp information finally generated depends heavily on the values selected as these parameters, and the selection of parameters affects the quality of the grasp information generated. In addition, when many parameters are adjusted, specialized knowledge on the parameter adjustment method is required, and it is difficult for a user without such specialized knowledge to generate reliable and consistent grasp information. Furthermore, in the technology described in Non-Patent Document 1, there is also a problem that the calculation time required to generate grasp information is long due to the adjustment of many parameters as described above, the amount of calculation such as mesh segmentation for generating a mesh model is enormous, and collision detection is performed twice to generate non-interfering grasp information. In addition, in the technology described in Non-Patent Document 1, since it is a parametric-based analysis approach, redundant grasp information such as grasp information about an impractical grasp posture may also be generated. Furthermore, the technology described in Non-Patent Document 1 only considers external grasping, in which an object is grasped by pinching it from the outside, and cannot be applied to internal grasping, in which, for example, a ring-shaped object is grasped by pushing it open from the inside.
[0006] The present invention has been made in consideration of the above points, and has an object to generate consistent grip information while reducing calculation time. [Means for solving the problem]
[0007] In order to achieve the above-mentioned object, the gripping information generating device of the present invention includes an acquisition unit that acquires object information indicating the three-dimensional shape of an object gripped by a gripping unit and gripping unit information related to the shape of the gripping unit, an extraction unit that clusters pairs of two points on the object that satisfy conditions based on the object information and the gripping unit information acquired by the acquisition unit and extracts a representative pair of each cluster as a gripping point, and a generation unit that generates gripping information indicating the relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the gripping points extracted by the extraction unit.
[0008] The object information may be information of a mesh structure formed by combining a plurality of polygonal planes.
[0009] In addition, the extraction unit may extract the gripping points based on the condition that the indexes representing the angle between the planes to which each of the two points belongs, the opening degree of the gripping portion, and the stability when the object is gripped by the gripping portion are each within a set value range.
[0010] The present invention also includes a pre-processing unit that performs at least one pre-processing on the object information acquired by the acquisition unit, including a simplification process to reduce the number of planes, a smoothing process to remove noise, an optimization process to reduce the variation in size of the planes to a threshold or less, and a symmetry detection process to detect a set of planes having a single rotation axis, and the extraction unit may extract the gripping point based on the object information after pre-processing by the pre-processing unit.
[0011] In addition, the pre-processing unit may evaluate the number of planes, the presence or absence of noise, and the variation in size of the planes in the object information acquired by the acquisition unit, and perform the pre-processing based on the evaluation results.
[0012] The extraction section may perform the clustering after removing one of the pairs that is similar to the other pairs.
[0013] The extraction section may extract a pair that is closest to a center of the cluster as a representative pair of the cluster.
[0014] In addition, the generation unit may generate the gripping information based on the positions and postures of the object and the gripping unit in each state in which the object is gripped at the gripping point by the gripping unit and the gripping unit is rotated relative to the object by a predetermined angle around a straight line passing through the gripping point as an axis.
[0015] The generation unit may also remove the gripping information in which the object and the gripping unit interfere with each other.
[0016] The generation unit may also assign an index indicating the stability of a gripping state to each of the gripping information.
[0017] Furthermore, a gripping information generating method according to the present invention is a method in which an acquisition unit acquires object information indicating the three-dimensional shape of an object gripped by a gripping unit and gripping unit information relating to the shape of the gripping unit, an extraction unit clusters pairs of two points on the object that satisfy conditions based on the object information and the gripping unit information acquired by the acquisition unit, and extracts representative pairs of each cluster as gripping points, and a generation unit generates gripping information indicating the relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the gripping points extracted by the extraction unit.
[0018] In addition, the gripping information generation program of the present invention is a program for causing a computer to function as an acquisition unit that acquires object information indicating the three-dimensional shape of an object gripped by a gripping unit and gripping unit information related to the shape of the gripping unit, an extraction unit that clusters pairs of two points on the object that satisfy conditions based on the object information and the gripping unit information acquired by the acquisition unit and extracts representative pairs of each cluster as gripping points, and a generation unit that generates gripping information indicating the relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the gripping points extracted by the extraction unit. Effect of the Invention
[0019] According to the gripping information generating device, method, and program of the present invention, it is possible to generate consistent gripping information while reducing calculation time. [Brief description of the drawings]
[0020] [Figure 1] FIG. 11 is a diagram for explaining grip information. [Diagram 2] FIG. 11 is a diagram for explaining grip information. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of the gripping information generating device. [Figure 4] 2 is a block diagram showing an example of a functional configuration of a gripping information generating device. FIG. [Diagram 5] FIG. 11 is a diagram showing an example of object information on which a simplification process has been executed. [Figure 6] FIG. 11 is a diagram showing an example of object information on which a smoothing process has been executed. [Figure 7] FIG. 11 is a diagram showing an example of object information on which optimization processing has been executed. [Figure 8] 1 is a diagram for explaining an angle θ between normal vectors of two planes. [Figure 9] 1 is a diagram for explaining an angle θ between normal vectors of two planes. [Figure 10]FIG. 11 is a diagram for explaining calculation of Ndistance and Hdistance. [Figure 11] FIG. 13 is a diagram for explaining conditions regarding stability when gripped. [Figure 12] FIG. 11 is a diagram for explaining calculation of an index regarding each condition. [Figure 13] FIG. 11 is a diagram for explaining calculation of an index regarding each condition. [Figure 14] FIG. 13 is a diagram for explaining extraction of pairs based on conditions. [Figure 15] FIG. 13 is a diagram for explaining removal of similar pairs. [Figure 16] FIG. 13 is a diagram for explaining gripping points based on pairs. [Figure 17] FIG. 13 is a diagram for explaining pair clustering. [Figure 18] FIG. 13 is a diagram showing an example of an array in each state rotated around a gripping point as an axis. [Figure 19] FIG. 13 is a diagram for explaining removal of an interfering state. [Figure 20] FIG. 11 is a diagram for explaining rankings of gripping information. [Figure 21] 13 is a flowchart showing a flow of a gripping information generating process. [Figure 22] FIG. 1 is a diagram for explaining an application example of a three-fingered robot hand. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0021] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensions and ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.
[0022] The gripping information generating device according to this embodiment generates gripping information that indicates multiple candidates for the relative positional relationship between a gripper and an object gripped by the gripper. With reference to Fig. 1, gripping information will be described using an example in which the gripper is a robot hand and the object is a workpiece. Note that in this embodiment, the gripper is described using an example in which the gripper is a robot hand with two opposing parallel fingers attached to the tip of a robot arm as shown in Fig. 1.
[0023] The gripping information is, for example, as shown in FIG. 1, information on the relative coordinates (x, y, z) and the relative posture (roll, pitch, yaw) (hereinafter referred to as "grip posture") of the reference position of the robot hand with respect to the workpiece when the robot hand grips the workpiece. The gripping information is a list of gripping information for a plurality of gripping postures. In the example of FIG. 1, three gripping postures (grip postures 1, 2, and 3) are shown, but the list includes gripping information for many more gripping postures (for example, several hundred patterns). Note that FIG. 1 shows an example of external gripping in which the workpiece (target object) is pinched and gripped from the outside by the robot hand (grip section). In this embodiment, as shown in FIG. 2, for example, internal gripping is also targeted in which the robot hand is inserted inside a ring-shaped workpiece and the robot hand is pushed out from the inside to grip the workpiece.
[0024] Fig. 3 is a block diagram showing the hardware configuration of a computer that functions as the gripping information generating device 10 according to this embodiment. As shown in Fig. 3, the gripping information generating device 10 has a CPU (Central Processing Unit) 12, a memory 14, a storage device 16, an input device 18, an output device 20, a storage medium reading device 22, and a communication I / F (Interface) 24. Each component is connected to each other via a bus 26 so as to be able to communicate with each other.
[0025] The storage device 16 stores a gripping information generating program for executing a gripping information generating process described below. The CPU 12 is a central processing unit, and executes various programs and controls each component. That is, the CPU 12 reads the program from the storage device 16 and executes the program using the memory 14 as a working area. The CPU 12 controls each of the components and performs various arithmetic processes according to the program stored in the storage device 16.
[0026] The memory 14 is made up of a RAM (Random Access Memory) and temporarily stores programs and data as a working area. The storage device 16 is made up of a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including an operating system, and various data.
[0027] The input device 18 is a device for performing various inputs, such as a keyboard, a mouse, etc. The output device 20 is a device for outputting various information, such as a display, a printer, etc. A touch panel display may be used as the output device 20 to function as the input device 18.
[0028] The storage medium reader 22 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray disc, and USB (Universal Serial Bus) memory, and writes data to the storage media, etc. The communication I / F 24 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark), for example.
[0029] Next, a functional configuration of the gripping information generating device 10 according to this embodiment will be described. Fig. 4 is a block diagram showing an example of the functional configuration of the gripping information generating device 10. As shown in Fig. 4, the gripping information generating device 10 includes, as its functional configuration, an acquisition unit 30, a pre-processing unit 40, an extraction unit 50, and a generation unit 60. Each functional configuration is realized by the CPU 12 reading out a gripping information generating program stored in the storage device 16, expanding it in the memory 14, and executing it.
[0030] The acquisition unit 30 acquires object information indicating the three-dimensional shape of the object gripped by the gripping unit, and gripping unit information relating to the shape of the gripping unit. The object information is information of a mesh structure formed by combining multiple polygonal (e.g., triangular) planes. The object information includes, for example, coordinate information of the vertices and center of each plane in the local coordinate system of the object. The gripping unit information includes information on the shape, movable range, etc. of the gripping unit. The acquisition unit 30 passes the acquired object information to the pre-processing unit 40, and passes the gripping unit information to the extraction unit 50.
[0031] The pre-processing unit 40 performs pre-processing on the object information passed from the acquisition unit 30. More specifically, the pre-processing unit 40 includes an evaluation unit 41, a simplification unit 42, a smoothing unit 43, an optimization unit 44, and a symmetry detection unit 45.
[0032] The evaluation unit 41 evaluates the number of planes constituting the object information, the presence or absence of noise in the object information, and the variation in the size of the planes constituting the object information. For example, when the object information input to the grip information generating device 10 is information prepared for the purpose of manufacturing an object using a three-dimensional printer or the like, the information may be more detailed than necessary for generating the grip information. In such a case, the information of the mesh structure, which is the object information, is also detailed, and the number of planes constituting the object information may be enormous. In this way, in order to evaluate whether the number of planes constituting the object information is equal to or greater than the number necessary for generating the grip information, for example, the evaluation unit 41 evaluates whether the ratio of the number of planes to the size of the object is within a threshold value.
[0033] Also, for example, the evaluation unit 41 determines whether or not there is noise in the object information based on information on the generation method of the object information. More specifically, when the object information is information generated by computer graphics such as CAD (Computer-Aided Design), the evaluation unit 41 may determine that the object information does not contain noise. Also, when the object information is information generated based on three-dimensional point cloud data obtained by measuring an object with a measuring device such as a three-dimensional scanner, the evaluation unit 41 may determine that the object information contains noise. Note that even in the case of three-dimensional point cloud data, the noise level may be low enough that the smoothing process described later is not necessary. Therefore, the evaluation unit 41 may evaluate the noise level of the object information by, for example, a method such as an entropy-based approach or a Gaussian-based approach. Based on the evaluation result, the evaluation unit 41 causes each of the simplification unit 42, the smoothing unit 43, and the optimization unit 44 to perform pre-processing.
[0034] When the evaluation unit 41 evaluates that the number of planes constituting the object information is more than necessary, the simplification unit 42 executes a simplification process to reduce the number of planes constituting the object information. Specifically, the simplification unit 42 reduces the number of planes by deleting edges, i.e., edges of planes, from the object information, where the change in the shape of the object before and after the deletion is within a threshold value. Fig. 5 shows an example of object information in which the number of planes has been reduced by the simplification process.
[0035] When the evaluation unit 41 evaluates that the object information contains noise, or when the noise level is evaluated to be equal to or higher than a predetermined threshold, the smoothing unit 43 executes a smoothing process to remove noise from the object information. Specifically, the smoothing unit 43 employs a noise removal approach to merge minute planes generated by noise into adjacent planes. FIG. 6 shows an example in which the unevenness of the outer shape of the object is smoothed by the smoothing process.
[0036] The optimization unit 44 executes an optimization process to make the variation in size of each of the multiple planes constituting the object information evaluated by the evaluation unit 41 equal to or less than a threshold value. A theoretically optimized mesh structure is formed by combining planes of a fixed size and a fixed shape. When the object information has such an optimized mesh structure, evenly distributed grasping information covering the entire task of grasping an object with a grasping unit can be generated in generating grasping information, which will be described later. Therefore, the optimization unit 44 specifically merges or divides the planes so that the variation in size of each of the multiple planes is equal to or less than a threshold value. FIG. 7 shows an example of object information on which the optimization process has been executed.
[0037] The symmetry detection unit 45 executes a symmetry detection process for detecting a set of planes (hereinafter referred to as "symmetric planes") having a single rotation axis for a plurality of planes constituting the object information. The grip information generated based on all grip points (described in detail later) related to a single symmetric plane represents the same grip posture. For this reason, for example, in the case where the object and the gripping unit interfere with each other in the grip information generated based on a certain grip point, or the approach angle is difficult, it is possible to easily avoid the interference or the difficulty of the approach angle by selecting another grip point on the same symmetric plane. Therefore, the symmetry detection unit 45 detects the symmetric plane for the purpose of simplifying the subsequent processing. Note that the difficulty of the approach angle indicates that the posture of the robot for realizing the grip posture indicated by the grip information is difficult due to the physical limitations of the robot, i.e., limitations on the degrees of freedom of the robot's joints.
[0038] The pre-processing unit 40 passes the pre-processed object information to the extraction unit 50.
[0039] The extraction unit 50 clusters pairs of two planes related to two points on the object that satisfy conditions based on the object information and the gripping part information, based on the object information after pre-processing by the pre-processing unit 40, and extracts gripping points based on the pairs that are representative of each cluster.
[0040] The analysis unit 51 extracts pairs of two planes that satisfy the conditions that the angle between the two planes, the opening degree of the gripping part, and the index representing the stability when gripping are within a predetermined range. Each condition will be specifically described below. In the following, the k-th plane will be referred to as "F k ”, plane F k The center of "c k " Note that k is 1, 2, . . . , max, and max is the number of planes that make up the object information.
[0041] First, the condition regarding the angle between the two planes will be described. The analysis unit 51 calculates c i A plane F passing through i Normal vector N i → (In the figures and formulas, N i (→) above the vector notation, and the same applies below) and c j A plane F passing through j Normal vector N j → The angle θ i,j Normal vector N i → and N j → are for the directions towards the inside and the outside of the plane, respectively.
[0042]
number
[0043] The analysis unit 51 determines θ i,j If is within a given range, then plane F i and plane F j For example, the analysis unit 51 determines that the pair of θ i,j When is 178°~180°, plane F i and plane F jThe pair is determined to satisfy the condition regarding the angle between the two planes. Note that the left diagram in FIG. 8 shows two planes that can be internally grasped, and the right diagram shows two planes that can be externally grasped. As shown in FIG. 9, i,j When the angle is close to 0° or is smaller than 178°, the object cannot be grasped by the grasping part, and the analysis part 51 cannot grasp the object by the plane F. i and plane F j It is determined that the pair does not satisfy the condition regarding the angle between the two planes. i,j is an example, and the value of this angle depends on the physical characteristics of the gripping portion.
[0044] Next, the conditions regarding the opening degree of the gripping portion will be described. The analysis unit 51 calculates the opening degree of the gripping portion on the plane F i Center of c i From plane F j Center of c j Vector D heading towards ij → The analysis unit 51 also calculates c i From plane F j A vector pointing to the plane F i Normal vector N i → Vector N parallel to ij → Find the vector N ij → and plane F j The intersection point with plane F j c against i The vertical projection point of c i * The analysis unit 51 calculates N ij → The size of point c i and c i * Distance from N distance In the following, the plane F i and plane F j With N distance "N i,j In the case of external gripping as shown in the right figure of Figure 8 or Figure 10, N i,j<0, and in the case of internal gripping as shown in the left figure of Figure 8, N i,j >0.
[0045]
number
[0046] The analysis unit 51 is i,j If the size of is within the range of the opening of the gripper, the plane F i and plane F j It is determined that the pair of the gripper satisfies the condition regarding the gripper opening. In this embodiment, the gripper opening is the width of the opening between the two fingers of the robot hand. max is 50 mm, for example, analysis unit 51 sets the following formula (4) as the condition regarding the opening degree of the gripping portion. 1<|N i,j | <Gripper max =50mm (4) In the above formula (4), as an example, Gripper max = 50mm, but Gripper max The value of is dependent on the physical characteristics of the gripping portion.
[0047] Next, the condition regarding the stability when gripping will be described. The analysis unit 51 calculates the stability of the gripping position by the following formula (5) as shown in FIG. j and c i * Distance from H distance is calculated as an index of stability during gripping. distance The larger H is, the greater the degree of momentum that will be generated when a two-fingered robotic hand grasps an object. Therefore, H distance This can indicate the stability when gripped.
[0048]
number
[0049] For example, if the angle between two planes is set to 180°, the H value corresponding to this angle is distance The threshold H max The threshold H max For example, when the plane F is 3 mm, the analysis unit 51 sets the following formula (6) as the condition for stability when gripping. i and plane F j Sex with distance "H i,j " should be written. 0≦H i,j <H max =3mm (6) In the above formula (6), as an example, max = 3mm, but H max The value of is dependent on the physical characteristics of the gripping portion.
[0050] In addition, even if the condition of the above formula (6) is not satisfied, the analysis unit 51 determines that the point c i * is the plane F j If the point is included in the point c, it may be determined that the condition for stability during grasping is satisfied. i * is the plane F j For example, as shown in Figure 11, i * And, F j The sum of the areas of the three triangles formed by the lines connecting each of the vertices of S1 and S2 (S3) is F. j The determination may be made based on whether the area is equal to or not.
[0051] The analysis unit 51 calculates the above θ for all pairs of two planes as shown in FIG. 12 and FIG. i,j , N i,j , and H i,j Calculate the pairs that satisfy all the conditions. ijBy extracting pairs of two planes that satisfy the above conditions, it is possible to extract a reduced number of pairs from the total number of pairs of two planes, as shown in Fig. 14. In Fig. 14, solid arrows indicate extracted pairs, and dotted arrows indicate pairs that were not extracted, i.e., deleted pairs. The same applies to Fig. 15, which will be described later. The analysis unit 51 passes the extracted pairs to the similarity removal unit 52.
[0052] The similarity removal unit 52 removes one of the similar pairs from the pairs transferred from the analysis unit 51. Specifically, the similarity removal unit 52 removes one of the similar pairs from the plane F i Pairs related to ij Among them, N unique distance Every H distance We may keep the pair for which ,is ,smallest, and remove the remaining pairs. For example, the pair 1,2 and Pair 1,4 About N 1,2 =N 1,4 , and H 1,2 >H 1,4 In this case, as shown in FIG. 15, the similarity removal unit 52 1,4 Leave the pair 1,2 Similarly, the pair 1,7 and Pair 1,max About N 1,7 =N 1,max , and H 1,7 >H 1,max In this case, as shown in FIG. 15, the similarity removal unit 52 1,max Leave the pair 1,7 The similarity removal unit 52 performs this process on each plane F i The similarity removal unit 52 passes the remaining pairs that have not been removed to the learning unit 53.
[0053] The learning unit 53 performs clustering on the pairs passed from the similarity removal unit 52. Specifically, the learning unit 53 performs clustering on the pairs. ij A feature vector V ij =(x i ,y i ,z i ,x j ,y j ,zj ,N i,j ) is generated. (x i ,y i ,z i ) is c i The coordinates of (x j ,y j ,z j ) is c j The learning unit 53 calculates the coordinates of each feature vector V ij Based on the position of the pair ij The learning unit 53 clusters the feature vector V by using a method such as the MeanShift method. The learning unit 53 selects a feature vector to be a representative point from each cluster. For example, the learning unit 53 may obtain the center of each cluster and select the feature vector closest to the center as the representative point. The learning unit 53 clusters the selected feature vector V by using a method such as the MeanShift method. ij The corresponding pair ij For example, as shown in FIG. 16, the selected feature vector V ij The corresponding pair ij c i * is the plane F j If it exists on the pair (c i , c i * ) is extracted as the gripping point. On the other hand, the selected feature vector V ik The corresponding pair ik c i * is the plane F k If not present, the learning unit 53 selects the pair (c i , c k ) is extracted as the grasp point.
[0054] Figure 17 shows the feature vector V ij 17 shows a schematic diagram of the clustering process of the feature vector V. In FIG. 17, the feature vector space is shown in two dimensions for ease of explanation. In FIG. 17, each circle represents each feature vector V ij The dashed ellipses represent the clusters, and the cross marks represent the centers of the clusters. Among the circles, the shaded circles represent the feature vectors V ijThe extraction unit 50 transfers the grasping points extracted by the learning unit 53 to the generation unit 60.
[0055] The generating unit 60 generates gripping information based on the gripping points received from the extracting unit 50. More specifically, the generating unit 60 includes an array generating unit 61, an interference removing unit 62, and a ranking unit 63.
[0056] The array generating unit 61 generates an array (arrangement) in a state where the gripping unit rotates the gripping unit by a predetermined angle relative to the object with the gripping unit gripping the object at the gripping point and the gripping unit rotates the gripping unit by a predetermined angle relative to the object with the straight line passing through the two gripping points as an axis. The array includes information on the position and posture of the object and the gripping unit in each state indicated by the array. FIG. 18 shows an example of the array. As shown in FIG. 18, when the straight line passing through the two gripping points is set as the y axis, the x axis and the z axis are also assumed as the rotation axis. However, these will result in the same gripping posture between arrays based on the detection result of the symmetry detecting unit 45, and duplicate gripping information will be generated, so in this embodiment, the y axis is used as the rotation axis. The array generating unit 61 passes the generated array to the interference removing unit 62.
[0057] The interference removal unit 62 removes from the array, among the states indicated by the array transferred from the array generation unit 61, states in which the object and the gripping unit interfere with each other. Specifically, as shown in FIG. 19, the interference removal unit 62 removes states in which there is an overlapping area between the shape of the object indicated by the object information and the shape of the gripping unit indicated by the gripping unit information in each state of the array. FIG. 19 shows an example in which the state indicated by the dotted line is removed from the array. The interference removal unit 62 generates gripping information from the information on the position and posture of the object and the gripping unit for each state of the array from which the states in which the object and the gripping unit interfere with each other have been removed, and transfers the information to the ranking unit 63.
[0058] The ranking unit 63 assigns an index indicating the stability of the gripping state to each piece of gripping information. Specifically, the ranking unit 63 calculates the contact area between the object and the gripping part in the gripping state based on the shape of the object indicated by the object information, the shape of the gripping part indicated by the gripping part information, and the gripping information. The ranking unit 63 then assigns an index indicating that the larger the contact area is, the higher the stability is. Therefore, the gripping information to which an index indicating high stability is assigned has a higher ranking. For example, in the examples (A) to (F) of FIG. 20, the rankings are higher in the order of (A) to (F). Also, in the examples (G) and (H) of FIG. 20, (G) is ranked higher than (H). By assigning this index, when calculating the robot posture using the gripping information, it is possible to efficiently select gripping information with high stability from multiple candidates of gripping information, such as selecting a gripping posture with a high ranking. Note that the index indicating the stability of the gripping state is not limited to the above example of the contact area. For example, instead of or in addition to the contact area, the distance between the center of gravity of the object and the center of gravity of the contact area between the object and the gripping portion in the gripped state may be used.
[0059] The generating unit 60 outputs the gripping information to which the index is assigned, and stores it in a gripping information DB (database) 70.
[0060] Next, the operation of the gripping information generating device 10 according to this embodiment will be described. Fig. 21 is a flowchart showing the flow of gripping information generating processing executed by the CPU 12 of the gripping information generating device 10. The gripping information generating processing is an example of a gripping information generating method according to the present invention. The CPU 12 reads out a gripping information generating program from the storage device 16, expands it in the memory 14, and executes it, whereby the CPU 12 functions as each functional component of the gripping information generating device 10, and the gripping information generating processing shown in Fig. 21 is executed.
[0061] In step S10 , the acquisition unit 30 acquires the object information and the gripping part information, passes the acquired object information to the pre-processing unit 40 , and passes the gripping part information to the extraction unit 50 .
[0062] Next, in step S12, the evaluation unit 41 of the pre-processing unit 40 evaluates the object information, and causes the simplification unit 42, the smoothing unit 43, and the optimization unit 44 to execute pre-processing based on the evaluation result. In addition, the symmetry detection unit 45 executes a symmetry detection process. The pre-processing unit 40 passes the object information after the pre-processing to the extraction unit 50.
[0063] Next, in step S14, the analysis unit 51 of the extraction unit 50 calculates the θ i,j , N i,j , and H i,j Calculate.
[0064] Next, in step S16, the analysis unit 51 calculates θ i,j Conditions for the angle between two planes using N i,j Conditions for the opening degree of the gripper using H i,j A pair of two planes that satisfies the conditions for the judgement when grasping using ij Next, in step S18, the similarity removal unit 52 of the extraction unit 50 extracts the plane F i Pairs related to ij Among them, N unique distance Every H distance The pair for which is the smallest is kept, and the remaining pairs are removed, thereby removing similar pairs.
[0065] Next, in step S20, the learning unit 53 of the extraction unit 50 extracts the pairs remaining in the process of step S18. ij A feature vector V ij =(x i ,y i ,z i ,x j ,y j ,z j ,N i,j The learning unit 53 generates V ij Based on the position of the pair ij Then, the learning unit 53 finds the center of each cluster and selects the feature vector V ij is selected as the representative point, and the selected feature vector V ij The corresponding pairij Based on this, the gripping point is extracted.
[0066] Next, in step S22, the array generation unit 61 of the generation unit 60 generates an array based on the gripping points. Then, the interference removal unit 62 of the generation unit 60 removes from the array, among the states included in the array, those in which the object and the gripping unit interfere with each other, and generates gripping information for each remaining state of the array from the information on the position and posture of the object and the gripping unit. Furthermore, the ranking unit 63 of the generation unit 60 assigns an index indicating the stability of the gripping state to each piece of gripping information. Then, the generation unit 60 outputs the gripping information with the assigned index, stores it in the gripping information DB 70, and the gripping information generation process ends.
[0067] As described above, the grip information generating device according to the present embodiment acquires object information indicating the three-dimensional shape of an object gripped by a gripping unit, and gripping unit information regarding the shape of the gripping unit. The grip information generating device then clusters pairs of two points on the object that satisfy conditions based on the acquired object information and gripping unit information, and extracts a representative pair of each cluster as a gripping point. Furthermore, the grip information generating device generates grip information indicating a relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the extracted gripping points. In this way, by clustering pairs of two points on the object, that is, by extracting gripping points by a non-parametric learning method, it is possible to generate consistent grip information while suppressing calculation time.
[0068] For example, the technique described in Non-Patent Document 1 requires the adjustment of eight parameters. On the other hand, in the grip information generating device according to the present embodiment, the parameters that need to be adjusted are the average shift radius during clustering, the threshold value H max , and the rotation angle when generating the array. The average shift radius is a parameter for adjusting the number of pieces of grip information to be generated.
[0069] Furthermore, the gripping information generating device according to this embodiment narrows down pairs according to conditions based on the object information and gripping part information before clustering pairs of two planes. Furthermore, the gripping information generating device deletes similar pairs. This makes it possible to suppress the generation of duplicate gripping information and impractical gripping information, and also to reduce calculation time.
[0070] In addition, the grip information generating device according to this embodiment uses object information with a mesh structure in which multiple polygonal planes are combined, and performs pre-processing such as simplification, smoothing, optimization, and symmetry detection on this object information. This makes it possible to further reduce the time required to generate grip information. For example, when the object information contains approximately 9,000 planes, the grip information generating device according to this embodiment can generate grip information 10 times faster than the technology described in Non-Patent Document 1.
[0071] By automatically generating grip information using the grip information generating device according to this embodiment, it is possible to reduce the time required for pre-setting in robot path planning for pick-and-throw, pick-and-place, pick-and-assembly, etc.
[0072] The disclosed technology can be applied as an online service that provides automatic grip information registration of CAD files, as an add-on function to existing grip information registration software, as an add-on function to a 3D simulation-based grip planning system, and the like.
[0073] The technology of grip planning for robots is an important technology for a wide range of robot applications, such as robots used in logistics, robots used in services such as hospitals, offices, and restaurants, and industrial robots such as bulk picking and assembly robots. A technology that can generate consistent grip information while suppressing calculation time, as in the present embodiment, is extremely useful for applications that handle various objects such as those described above.
[0074] In the above embodiment, a robot hand with two fingers has been described as an example, but the disclosed technology can also be applied to a robot hand with three or more fingers. In this case, the center point of the force acting at the contact point between each of the multiple fingers and the object can be regarded as the point where one of the fingers in the above embodiment contacts the object. For example, as shown in FIG. 21, in the case of a three-fingered robot hand having two fingers and one finger arranged parallel to each other, the point of action where the force of each of the two fingers on one side acts on the object can be regarded as c i1 and c i2 Let the force acting on each point of application be P ci1 and P ci2 In this case, P ci1 and P ci2 Center of action c according to the ratio of i rep P ci rep For the other finger, the force of the finger acts on the object at the point of application c j The force acting on P cj In the case of c j The center of the action point c j rep Let c j rep P cj rep (=P cj ) force acting on the object.
[0075] Furthermore, the grip information generating device according to this embodiment can be applied to generating grip information for external gripping as shown in FIG. 1, and to generating grip information for internal gripping as shown in FIG.
[0076] In addition, the grip information generation process executed by the CPU by reading the software (program) in the above embodiment may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. The grip information generation process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). The hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.
[0077] In the above embodiment, the grip information generating program is stored (installed) in advance in the storage device, but the present invention is not limited to this. The program may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, a Blu-ray disc, or a USB memory. The program may be downloaded from an external device via a network. [Explanation of symbols]
[0078] 10 Grip information generation device 12 CPU 14. Memory 16 Storage device 18 Input Devices 20 Output Devices 22 Storage media reader 24 Communication I / F 26 Bus 30 Acquisition Department 40 Pre-processing section 41 Evaluation Department 42 Simplification part 43 Smoothing section 44 Optimization Division 45 Symmetry detector 50 Extraction part 51 Analysis Department 52 Similarity removal part 53 Learning Department 60 Generation part 61 Array Generation Unit 62 Interference removal section 63 Ranking Section 70 Gripping information DB
Claims
1. an acquisition unit that acquires object information indicating a three-dimensional shape of an object gripped by a grip unit and grip unit information relating to a shape of the grip unit; an extraction unit that clusters pairs of two points on the object that satisfy a condition based on the object information and the gripping unit information acquired by the acquisition unit, and extracts a representative pair of each cluster as a gripping point; a generating unit that generates gripping information indicating a relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the gripping points extracted by the extracting unit; A gripping information generating device comprising:
2. The gripping information generating device according to claim 1 , wherein the object information is information of a mesh structure in which a plurality of polygonal planes are joined together.
3. The gripping information generating device according to claim 2, wherein the extraction unit extracts the gripping points under the condition that the indices representing the angle between the planes to which each of the two points belongs, the opening degree of the gripping portion, and the stability when the object is gripped by the gripping portion are each within a range of a set value.
4. a pre-processing unit that performs at least one pre-processing on the object information acquired by the acquisition unit, the pre-processing being a simplification process for reducing the number of planes, a smoothing process for removing noise, an optimization process for reducing the variation in size of the planes to a threshold or less, and a symmetry detection process for detecting a set of planes having a single rotation axis; The extraction unit extracts the gripping point based on the object information after the pre-processing by the pre-processing unit. The grip information generating device according to claim 2 or 3.
5. The gripping information generating device according to claim 4, wherein the pre-processing unit evaluates the number of planes, the presence or absence of noise, and the variation in size of the planes in the object information acquired by the acquisition unit, and performs the pre-processing based on the evaluation results.
6. The grip information generating device according to any one of claims 1 to 5, wherein the extraction unit performs the clustering after removing one of the pairs that is similar to the other pairs.
7. The gripping information generating device according to any one of claims 1 to 6, wherein the extraction unit extracts a pair that is closest to a center of the cluster as a representative pair of the cluster.
8. The gripping information generating device according to any one of claims 1 to 7, wherein the generation unit generates the gripping information based on the positions and postures of the object and the gripping unit in each state in which the object is gripped at the gripping point by the gripping unit and the gripping unit is rotated relative to the object by a predetermined angle around a straight line passing through the gripping point as an axis.
9. The gripping information generating device according to any one of claims 1 to 8, wherein the generating unit removes the gripping information in which the object and the gripping unit interfere with each other.
10. The gripping information generating device according to any one of claims 1 to 9, wherein the generating unit assigns an index indicating the stability of a gripping state to each of the gripping information.
11. The acquisition unit acquires object information indicating a three-dimensional shape of an object gripped by the gripping unit and gripping unit information relating to a shape of the gripping unit; an extraction unit performs clustering on pairs of two points on the object that satisfy a condition based on the object information and the gripping unit information acquired by the acquisition unit, and extracts a representative pair of each cluster as a gripping point; A generating unit generates gripping information indicating a relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the gripping points extracted by the extracting unit. Grip information generation method.
12. Computer, an acquisition unit that acquires object information indicating a three-dimensional shape of an object gripped by the grip unit and grip unit information relating to the shape of the grip unit; an extraction unit that clusters pairs of two points on the object that satisfy a condition based on the object information and the gripping unit information acquired by the acquisition unit, and extracts a representative pair of each cluster as a gripping point; and a generating unit that generates gripping information indicating a relative positional relationship between the object and the gripping unit when the object is gripped by the gripping unit, based on the gripping points extracted by the extracting unit; A grip information generation program for functioning as a grip information generating program.
Citation Information
Patent Citations
Gripping robot, control device, control method, and program
JP2013111696A
Gripping plan decision program, gripping plan determination method, and engineering support device
JP2016093879A
Setting device, setting method and program
JP2021079464A
