Robot control device and robot control method
By generating gripping points, the robot calculates the shape deformation information of the object and determines the gripping points, thus solving the gripping failure problem when the robot control device grips soft, irregularly shaped objects, achieving high success rate gripping and efficient production.
Patent Information
- Application Number
- CN202180060765.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2020-10-19
- Filing Date
- 2021-10-04
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-10-04
AI Technical Summary
Existing robot control devices are prone to failure when grasping soft, irregularly shaped objects due to deformation, resulting in a decrease in the success rate of picking operations and a drop in production efficiency.
By generating a gripping point, the robot calculates the shape deformation information of the object and determines the gripping point. Using the deformation evaluation unit and the gripping point determination unit, the robot's gripping action is optimized to improve the gripping success rate.
It improved the success rate of grasping soft objects, shortened the production cycle time, and increased production efficiency.
Smart Images

Figure CN116194255B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a robot control device and a robot control method for issuing motion commands to a robot and an end effector located at the fingertips of the robot in order to grasp an object and prevent it from falling. Background Technology
[0002] In existing robot control devices, there is a method in which, in order to determine the gripping point of the end effector (robot hand) relative to the object, the shape and weight of the object are measured based on the object's measurement information, the center of gravity of the object is estimated, and the point passing near the center of gravity is set as the gripping point (for example, see Patent Document 1).
[0003] Patent Document 1: Japanese Patent Application Publication No. 2008-49459 (pp. 9-10) Figure 2 )
[0004] Non-Patent Literature 1: "Deformation Model of Rheological Object for Shape Control" Shinichi Tokumoto, Shinichi Hirai: Journal of the Robotics Society of Japan, Vol. 18, No. 8 (P1143, Equation (1)~ (3)) Summary of the Invention
[0006] In existing robot control devices, when performing picking operations with two or more irregularly shaped objects as the target items, interference may prevent the robot from grasping the object at a position close to the center of gravity. Sometimes, due to deformation during grasping, the object may fall from the robot's hand. As a result, the success rate of picking operations decreases, leading to reduced production efficiency.
[0007] The present invention was proposed to solve the above-mentioned problems and to obtain a robot control device that can grasp objects with a high success rate and shorten production cycle time in picking operations where soft objects or irregularly shaped objects are selected.
[0008] The robot control device of the present invention controls the robot and its robotic hand for grasping an object. The robot control device has a grasping point generation unit that generates a grasping point of the object grasped by the robotic hand. The grasping point generation unit has: a deformation evaluation unit that calculates shape deformation information when the shape of the object is deformed due to the grasping action of the robotic hand; and a grasping point determination unit that determines the grasping point of the object based on the shape deformation information.
[0009] The effects of the invention
[0010] According to the present invention, in the picking operation where soft objects or irregularly shaped objects are selected, objects can be grasped with a high success rate, and production cycle time can be shortened while maintaining high production efficiency. Attached Figure Description
[0011] Figure 1 This is an overall diagram of the robot system involved in Implementation Method 1.
[0012] Figure 2 This is a block diagram showing the structure of the robot control device according to Embodiment 1.
[0013] Figure 3 This is a block diagram showing the structure of the gripping point generation unit involved in Embodiment 1.
[0014] Figure 4 This is a diagram showing the positional relationship between the finger and the object involved in Embodiment 1.
[0015] Figure 5 This is a diagram showing the positional relationship between the finger and the object in the case where the object involved in Embodiment 1 is movable.
[0016] Figure 6 This is a flowchart illustrating the operation of the robot control device according to Embodiment 1.
[0017] Figure 7 This is a diagram showing the physical property model after combining the spring element and damping element involved in Implementation Method 2.
[0018] Figure 8 This is a block diagram showing the structure of the gripping point generation unit involved in Embodiment 3.
[0019] Figure 9 This is a flowchart illustrating the operation of the robot control device involved in Embodiment 3.
[0020] Figure 10 This is a diagram showing the positional relationship between the finger and the object involved in Embodiment 4.
[0021] Figure 11 This is a diagram showing the positional relationship between the finger and the object involved in Embodiment 5.
[0022] Figure 12 This is a block diagram showing the structure of the gripping point generation unit involved in Embodiment 6.
[0023] Figure 13 This is a block diagram showing the structure of other gripping point generating units involved in Embodiment 6.
[0024] Figure 14This is a block diagram showing the structure of the gripping point generation unit according to Embodiment 7.
[0025] Figure 15 This is a schematic diagram of the object to be grasped according to Embodiment 9.
[0026] Figure 16 This is a diagram showing the hardware structure of the robot control device involved in embodiments 1 to 9. Detailed Implementation
[0027] Implementation method 1.
[0028] Figure 1 This is an overall diagram of the robot system according to Embodiment 1 of the present invention. The robot system is based on a robot and a robot control device that controls the robot to make it move. The robot 10 sometimes performs operations called material handling, such as grasping an object. In this case, a structure with a measuring device 60 and a robot hand 20 (end effector) is added. The measuring device 60 measures information such as the shape of the object 70 in order to obtain position and shape information of the object 70, and the robot hand 20 is used to grasp the object 70. The information of the object 70 measured by the measuring device 60 is processed in the measuring device controller 50, and the information of the object 70 is input to the robot control device 30.
[0029] The robot control device 30 controls the robot 10 and its robotic hand 20 to grasp the object 70, and includes a grasping point generation unit 31 that generates the grasping points of the object 70 grasped by the robotic hand 20. To allow the robotic hand 20 to grasp the object 70 based on input information about the object 70, the robot control device 30 calculates the position command value to the robot 10 at the position of the grasped object 70 and the opening position of the fingers (fingertips) of the robotic hand 20. The robot control device 30 controls the joints of the robot 10's arm and the fingers of the robotic hand 20 respectively, so that the fingers of the robotic hand 20 are controlled in the desired position. Next, the robot control device 30 controls at least one of the joints of the robot 10's arm and the fingers of the robotic hand 20, so that the fingers of the robotic hand 20 move to an appropriate position. Position information and shape information of the object 70 are illustrated as information about the object 70.
[0030] Furthermore, the robot control device 30 outputs calculated position command values and position command values for the open and closed states of the robot hand 20's fingers to the robot 10. The robot control device 30 determines the timing for executing the position command values of the robot hand 20 relative to the position command values of the robot 10, and sends these as position command values at each time t to the robot 10. This enables the robot hand 20 to be brought close to the object 70 with its fingers in the open state, and then closed at the gripping point of the object 70. Here, unless otherwise specified, the position command values of the robot 10 are set to 6 degrees of freedom (3 degrees of translation and 3 degrees of rotation). Regarding the position command values of the robot hand 20's fingers, these depend on the type of hand, but if it is a linkage structure, they are defined by the position of the fingertips or the opening width. Additionally, sometimes the position command values refer to the position command values of each drive unit; here, this is not specifically limited to the hand structure, but broadly refers to any position command value that can be specified. Furthermore, if the actuators of the fingers of the robotic hand 20 are capable of pressure control, force control, or torque control, the gripping force can be controlled. Below, when a gripping force is specified, it is assumed that the gripping force command value is assigned to the gripping point candidate.
[0031] "Grasping point" refers to the position and orientation of the fingertips of the robot hand 20 that enable it to grasp the object 70. In actual robot control, as mentioned earlier, in addition to the position and orientation of the gripping point, position command values at each time t are also required. However, the target position values of each joint of the robot 10, such as the position and orientation of the robot hand 20 to reach the gripping point of the object 70, are calculated separately. Furthermore, the information that can be used to calculate the gripping point of the object 70 is not limited to the position and shape information of the object 70. That is, in addition to direct information such as position and shape information, the position and shape information of the object 70 can also be inferred from indirect information such as temperature, distance, and color information.
[0032] Figure 2 This is a block diagram showing the structure of the robot control device 30. For example... Figure 2 As shown, the robot control device 30 mainly consists of a gripping point generation unit 31 and a command value generation unit 39. Figure 2 As shown, the robot control device 30 calculates the position of the gripping point that the robot 10 should move to, and controls the robot 10 so that the robot hand 20 moves to the gripping point and the robot 10 performs a gripping action. The gripping point generation unit 31 uses the shape information of the object to be gripped by the robot 10, i.e., the object 70, to output the gripping point of the object 70.
[0033] Regarding object shape information, specifically, it is obtained by calculating the image information or distance information of the object 70 obtained by the vision sensor, which serves as the measuring device 60, as a point group. Alternatively, the object 70 can be temporarily grasped by the fingers of the robot hand 20, and the shape information can be obtained based on the position information of the fingers during grasping. Furthermore, a distance measurement sensor can be used as the measuring device 60 to obtain shape information based on the cross-sectional shape of the object. Additionally, a temperature sensor can be used as the measuring device 60 to obtain shape information based on the approximate position and shape of the object 70. As described above, the measuring device 60 is not limited to a vision sensor. Position information, shape information, temperature information, distance information, color information, etc., can also be obtained from sources other than the measuring device 60.
[0034] Figure 3 This is a block diagram showing the structure of the gripping point generation unit 31. For example... Figure 3 As shown, the gripping point generation unit 31 consists of a gripping point candidate generation unit 32, a deformation evaluation unit 33, and a gripping point determination unit 36. The deformation evaluation unit 33 calculates the shape deformation information of the object 70 when its shape is deformed due to the gripping action of the robot hand 20. Furthermore, the gripping point determination unit 36 determines the gripping point of the object based on the deformation amount of the object contained in the shape deformation information and the geometric constraints of the object after deformation. The various structural units will be described below.
[0035] The gripping point candidate generation unit 32 generates gripping point candidates that the robot hand 20 mounted on the robot 10 can grasp, based on the object shape information input to the robot control device 30. At this time, the gripping point candidate generation method can be performed by exploring any two points around the entire circumference based on the travel (opening width) of the fingers of the robot hand 20 and the object shape information. For example, the case of a two-finger gripper will be described later. Figure 4 As shown.
[0036] The exploration of the object involves selecting any two points on the outer periphery of the elliptical object. The deformation evaluation unit 33 moves the finger to the inside of the object relative to the selected two points, grasps it, and performs the deformation evaluation described later. Since the finger acts as a constraint, its movable direction and range are defined. Therefore, the exploration itself is not a complete exploration. Methods can be implemented that compare the distance between the two candidate points with the opening / closing distance L0 as a constraint, without limiting the exploration method itself.
[0037] Next, in the deformation evaluation unit 33, the individual situations of the multiple gripping point candidates generated by the gripping point candidate generation unit 32 are analyzed, such as... Figure 4 The output shows the evaluation of the desired shape deformation information. This shape deformation information includes the deformed shape information. To evaluate the shape deformation information, assuming gripping via point contact of each finger, the desired shape deformation information can be calculated using models that deform relative to the object 70.
[0038] Furthermore, regarding the point contact points existing in relation to the number of fingers possessed by the robotic hand 20, an appropriate fixed gripping force Fi is assumed for each point i (i = 1, 2, 3...), and the mechanical relationship of deformation relative to the force can be calculated at each point contact point. In order to calculate the amount of deformation relative to the force, the object 70 is treated as a homogeneous shape, the spring constant is set to K, and the damping coefficient is set to C. Relative to the direction of the force generating the fingers, the object's characteristics are approximated as a rigid body, an elastic body, or a rheological object, and the shape deformation information can be evaluated. For example, if it is a rheological object, as described in Non-Patent Document 1, the relationship between displacement and force is known to hold. As described above, by specifying the gripping points and causing shape deformation under appropriate conditions, the desired shape deformation information can be obtained.
[0039] Finally, in the gripping point determination unit 36, processing is performed to determine the gripping point based on the shape deformation information generated by the deformation evaluation unit 33. In the gripping point determination unit 36, for each shape deformation information generated for multiple gripping point candidates, points with geometric constraints are extracted and used as gripping points. Specifically, Figure 4 This is an example of constraint. Figure 5 The scenario shown represents an unconstrained instance. Here, Figure 4 This is a diagram showing the positional relationship between the fingers of the robot hand 20 and the object 70. Figure 4 (a) shows the robotic hand 20 grasping the object 70 before its fingers are in contact with it. Figure 4 (b) and Figure 4 (c) shows the positional relationship of the robotic hand 20 after its fingers grasp the object 70. Figure 5 This is a diagram showing the positional relationship between the finger and the object 70 when the object 70 is movable. Figure 4 and Figure 5 In this design, the direction in which the fingers of the robot hand 20 open and close is defined as the X-axis, and the direction perpendicular to the direction along the fingers and perpendicular to the X-axis is defined as the Y-axis.
[0040] Figure 4 and Figure 5 The black arrow pointing outwards from the object 70 indicates the direction in which an external force is attempting to move the object 70. In this case, if... Figure 4The gripping method, based on the deformed shape information of the object 70, geometrically constrains the object even when external forces are applied in the X and Y directions, thus easily maintaining the gripping state. This is because the geometric constraint is established according to the positional relationship between the deformed shape information of the object 70 and the gripping fingers. This embodiment focuses on this point and extracts candidate gripping points for achieving stable gripping. On the other hand, in Figure 5 If we observe the deformed shape of object 70 and the positional relationship of the fingers, then in the X direction, the gripping stability achieved through geometric constraints is applied, and the object remains immobile. However, in the negative Y direction, no geometric constraints are applied, resulting in low gripping stability and allowing the object to move.
[0041] The following is an example of a method by which the gripping point determination unit 36 evaluates gripping stability. For example... Figure 4 As shown in (c), the deformation evaluation unit 33 outputs multiple discrete points DP1, DP2, ... as shape deformation information of the object 70. The discrete points DP1, DP2, ... are set based on the contour of the shape expected by the model that deforms relative to the object 70. As an example, the gripping point determination unit 36 determines whether the geometric constraints are valid based on the relationship between the positions of the discrete points DP1, DP2, ... and the positions of the fingers FP1, FP2, thereby evaluating the gripping stability.
[0042] As another example, the gripping point determination unit 36 calculates a first approximate curve for multiple discrete points near the finger position FP1. Based on the outline of the finger at position FP1, the gripping point determination unit 36 sets multiple discrete points (not shown) related to the finger and calculates a second approximate curve for these discrete points. Based on the first and second approximate curves, the gripping point determination unit 36 compares the shape (convexity / concavity information, etc.) of the object 70 near the finger position FP1 with the shape of the finger at position FP1 (arc, rectangle, etc.) to determine whether geometric constraints are valid, thereby evaluating gripping stability. As comparison objects, the relationship between the curvature of the object 70 and the finger shape, and the difference in elevation between the maximum and minimum points of the first approximate curve are cited. Similarly, the gripping point determination unit 36 calculates approximate curves for multiple discrete points near the finger position FP2 and multiple discrete points (not shown) related to the finger at position FP2, and evaluates gripping stability using the same method as described above.
[0043] As another example, the gripping point determination unit 36 verifies the position coordinates of discrete points DP1, DP2, ... when a hypothetical force Fvir is applied to the object 70, and determines whether the change in position coordinates since the application is less than or equal to a predetermined value, thereby determining whether the geometric constraint is valid and evaluating the gripping stability. The gripping point determination unit 36 can determine that the gripping stability is low even if the change in one of the multiple discrete points exceeds the predetermined value, or if the change in a portion of the multiple discrete points exceeds the predetermined value. The hypothetical force Fvir is assumed to be applied to the object 70 from any direction.
[0044] The above three methods are cited as methods for evaluating grip stability by the grip point determination unit 36, but are not limited to these.
[0045] When multiple gripping points are extracted, it is possible to select the gripping point closest to the center of gravity. When the distance between the center of gravity and the gripping point is short, it is desirable to minimize the torque when the gripping force near the gripping point fluctuates due to disturbances, etc.
[0046] Here, the operation of the robot control device 30 will be explained. Figure 6 This is a flowchart illustrating the actions of the robot control device. First, in step S101, object shape information is input. Next, in step S102, the gripping point candidate generation unit 32 generates gripping point candidates that the robot hand 20 can grasp based on the input object shape information. Next, in step S103, the deformation evaluation unit 33 evaluates the shape deformation information for each of the multiple gripping point candidates and outputs it. Furthermore, in step S104, the gripping point determination unit 36 determines the gripping point based on the shape deformation information.
[0047] As described above, in the robot control device 30 with the gripping point generation unit 31, the gripping point generation unit 31 specifically includes a deformation evaluation unit 33 that calculates shape deformation information when the shape of an object is deformed by the gripping action of a hand, and a gripping point determination unit 36 that determines the gripping point of the object based on the shape deformation information. As a result, for objects such as soft objects that are amorphous objects, by gripping at the selected gripping point, the gripping failure is greatly reduced, and the object can be gripped with a high success rate, resulting in the special effect of shortening the production cycle time and maintaining high production efficiency.
[0048] Furthermore, production efficiency refers to the speed of operations such as picking. For example, as an example of production efficiency, it refers to cycle time. If a 1-second operation is tested 100 times and all 100 tests are successful, the average cycle time is 1 second per operation. If the same operation is tested 100 times and only 50 tests are successful, the average cycle time is 2 seconds per operation. Therefore, fewer failures mean higher production efficiency.
[0049] Implementation method 2.
[0050] In this embodiment, the deformation evaluation unit 33 differs from that in Embodiment 1 in that it further includes a structure for evaluating whether the force exceeds an upper limit calculated based on the relationship between the force applied to the object 70 and the displacement of the object 70. In the robot control device 30 described in Embodiment 1, multiple gripping point candidates are extracted in the gripping point candidate generation unit 32 based on the object satisfying geometric constraints. The characteristic of this embodiment is that the deformation evaluation unit 33 evaluates whether the object 70 exceeds the allowable deformation amount based on the gripping force F(t) (the value that changes at time t) exhibited in the time series from these gripping point candidates and applies constraint conditions accordingly.
[0051] The deformation evaluation unit 33 calculates the upper limit of the gripping force applied to the object 70 based on the relationship between the gripping force applied to the object 70 and the displacement of the object 70, as well as the allowable amount of deformation of the object 70. Furthermore, the deformation evaluation unit 33 evaluates whether the gripping force applied from the robot hand 20 to the object 70 exceeds the upper limit. Additionally, as part of the shape deformation information, the deformation evaluation unit 33 outputs time-series information of the gripping force calculated based on the relationship between the gripping force applied to the object 70 and the displacement of the object 70 to the gripping point determination unit 36.
[0052] Generally, soft objects can be classified into three categories based on their deformation. Deformation includes elastic bodies that recover their shape after the gripping force is removed, rheotypes that do not completely recover their shape, and plastic bodies that deform with the amount of applied force. On the other hand, there is an upper limit to the permissible deformation of soft objects. Exceeding the deformation limit will result in damage to the object or impairment of its commercial value.
[0053] Furthermore, the amount of deformation is calculated using the force and the time for which the force is applied. The relationship between force and deformation can be expressed mathematically, for example, as described in Non-Patent Document 1. For instance, in rheological or plastic bodies, since they do not return to their original shape, a material property model can be simulated if the elastic and damping elements are connected in series, as in the Maxwell model.
[0054] Figure 7 It is a physical property model that combines spring and damping elements. For example... Figure 7 As shown, a physical property model is constructed by combining spring elements and damping elements. Figure 7 In this diagram, the spring constant of the spring element is represented by K1, and the damping coefficients of the damping element are represented by C1 and C2. By connecting multiple elements in series and parallel, it is possible to exhibit, for example, [the desired effect]. Figure 7 The physical property model shown here. Figure 7 Taking the "Maxwell model" shown in (b) as an example, the calculation of force and deformation is explained.
[0055] First, a fixed point is designated as P1, and the point where the force is applied is designated as P2. A physical property model is established between P1 and P2, containing a spring element with a spring constant K1 and a damping element with a damping coefficient C2. The coefficients of the spring element and the damping element are pre-input based on the soft material to be applied. Furthermore, regarding each coefficient, as a physical property of the object 70, positional information obtained by inputting known forces (such as 1N, 2N, 3N, etc.) over a time series is used, thereby allowing estimation of the coefficients of the spring element and the damping element. Additionally, in cases where measurement is difficult, coefficients related to similar soft materials obtained beforehand can be used based on the physical properties of the object 70.
[0056] When using two fingers, assign two gripping points to the fingers. Designate these as gripping points PG1 and PG2. At this point, ensure that gripping point PG2 aligns with the point of force application, P2. Furthermore, set the vectors (P1P2) and (PG1PG2) to be parallel. Regarding displacement, define the displacement at the junction of the spring element and the damping element using x1, and define the displacement of gripping point PG2 using x2. The origins of both displacements x1 and x2 can be defined for their natural lengths. As the initial positional relationship, set the length of the spring element (k1) to X10 and the length of the damping element to X20.
[0057] Under the conditions described above, when the time series data F(t) of the externally applied holding force is obtained, the displacements x1 and x2 can be calculated by performing calculations on the equation of motion. As described in Non-Patent Document 1, by making the damping coefficient C2 nonlinear, the characteristics of rheological objects (with residual displacement) can be simulated. However, the definition of the physical property model is not limited to this; by changing the coefficients and structure, it can be applied to rigid bodies, elastic bodies, rheological objects, and plastic bodies.
[0058] As a result of the above calculations, the change in position of displacement x2 can be obtained, and thus the deformation that occurred can be determined in conjunction with the time series data F(t) of the appropriate gripping force. In particular, when damping elements are included, the position after unloading sometimes does not return to its original position (x1 = 0 and x2 = 0).
[0059] In reality, there are various models that involve connecting elastic and damping elements in series, and then adding damping elements in parallel with the elastic elements, resulting in models that vary depending on the characteristics of the object. Therefore, the form of the material property model in this embodiment is not particularly limited.
[0060] As explained above, according to this embodiment, the deformation evaluation unit 33 can extract only the gripping points that take into account the allowable deformation of the gripping object 70. Therefore, the proportion of gripping points that will fail to grip is reduced, resulting in the special effect of improving production efficiency. Furthermore, when the robot arm 20 grips a soft, amorphous object, i.e., the object 70, based on the deformed shape of the object 70 and the time sequence information of the force up to that point, gripping points with high gripping stability can be selected without damaging the object 70. Specifically, when the robot arm 20 grips the object 70 with a relatively large gripping force, this includes cases where deformation of the object 70 can be tolerated if the gripping force is unloaded within a predetermined time. Therefore, failures are reduced, and the object 70 can be gripped with a high success rate, resulting in the special effect of shortening production cycle time and maintaining high production efficiency.
[0061] Next, other variations of this embodiment will be described. When considering the gripping of food as the object 70, from an aesthetic point of view, there is sometimes an acceptable amount of deformation for the reason that it would damage the value of the product. In this case, if the determination is based solely on whether a certain upper limit is exceeded, as previously mentioned, the number of potential gripping points sometimes becomes very small. In this case, if the deformation amount only exceeds a small time limit, it will converge to an acceptable range.
[0062] That is, in this embodiment, the deformation evaluation unit 33, as part of the shape deformation information, outputs the magnitude of the force acting on the gripping point, i.e., the gripping force F(t), and the time t for which a force greater than or equal to the allowable load is applied. In this case, evaluation can be performed based on the gripping force F(t), which shows whether the allowable amount of deformation is ultimately achieved through a time series. For example, the gripping point determination unit 36, as part of the shape deformation information, uses the relationship between the magnitude of the force and time thresholds to determine whether the amount of deformation is within the allowable range using the gripping point, gripping force, and gripping time. Thus, the gripping point determination unit 36 can obtain the gripping point and gripping force that achieve the state of maintaining the shape of the food within a certain range. In this structure, the gripping point information includes information on the position of the gripping point and the gripping force (applied force) at the gripping point.
[0063] Furthermore, the thresholds related to gripping force and time can be calculated by converting the allowable range of deformation of the object 70 into gripping force and gripping time. The relationship between displacement and force can be referred to the description in Non-Patent Document 1. However, the determination of whether the deformation of the object 70 is within the allowable range can be made by calculating the deformation using the gripping point, gripping force, and gripping time, and setting an upper limit value based on the deformation. The upper limit value of the deformation of the object 70 is provided in advance by the user of this system for each food product. In this case, the robot control device 30 can also extract only the gripping point that takes into account the allowable deformation of the object being gripped as the object 70, thus reducing the proportion of gripping points that will fail to grip, enabling gripping of the object with a high success rate, and achieving the special effect of shortening the production cycle time and maintaining high production efficiency.
[0064] Implementation method 3.
[0065] In this embodiment, the difference from embodiment 2 is that it also includes a gripping stability calculation unit, which further evaluates the gripping stability by considering the balance of forces after deformation near the gripping point and the mechanical stability of a predetermined external force. Figure 8 This is a block diagram showing the structure of the gripping point generation unit according to Embodiment 3. Figure 3 Based on the structure of the gripping point generation unit 31 shown, the gripping point generation unit 31a is further composed of a gripping stability calculation unit 34 and a result DB (result database) 35.
[0066] The gripping stability calculation unit 34 evaluates the mechanical stability against a predetermined external force by considering the force balance of the deformed object 70 near the gripping point of the object 70. Furthermore, the gripping stability calculation unit 34 evaluates the force balance of the deformed object 70 near the gripping point of the object 70 and extracts the gripping point of the robot hand 20 where the gripping force on the object 70 becomes minimal.
[0067] Shape deformation information is input to the gripping stability calculation unit 34. First, the gripping stability calculation unit 34 calculates the force vector generated on the gripped object after deformation, based on the force vector generated at each point of the robot hand 20's fingers and the gripped object. Next, the gripping stability calculation unit 34 evaluates whether the object 70 has not moved based on the force balance at the gripping point of the object 70. At this time, when the deformed object 70 and the robot hand 20's fingers are geometrically constrained (immobile), the object 70 and the fingers are pushed down by forces other than the gripping force of the robot hand 20's fingers, which is considered a stable state.
[0068] The gripping stability calculation unit 34 determines whether a "stable state" can be maintained. The stable state (stability) is explained. A predetermined external force is defined as Fdis, and the state in which the object 70 deforms in response to the force from the fingers of the robot hand 20 is defined as the state where external force Fdis = 0. The stability in the state where Fdis = 0 can be evaluated based on the balance of forces in mechanics, assessing whether a couple or acceleration does not occur on the object 70. If a couple or acceleration occurs on the object 70, and the method to prevent movement of the object 70 in the direction of the couple or acceleration is constituted by the fingers of the robot hand 20 and the deformed shape of the object, then the geometric constraint is satisfied, the couple and acceleration are canceled out, and the gripping of the object at the designated gripping point is determined to be a "stable state."
[0069] Furthermore, the gripping stability calculation unit 34 determines whether a "stable state" can be maintained when the external force Fdis is not set to 0. When the external force Fdis is applied, the deformation is further calculated by adding the gripping force F(t) and the external force Fdis based on the shape deformation information. The deformation is obtained by using the aforementioned physical property model through the relationship between displacement and force. The "stable state" is determined based on this shape deformation information.
[0070] Furthermore, the gripping stability calculation unit 34 can also determine the "stable state" of the robot 10 under conditions of acceleration and deceleration. First, if acceleration or deceleration is performed while gripping an object, an inertial force is generated on the object. In the case of the inertial force, the external force Fdis(t) generated by the inertial force can also be expressed as Equation 1 using the mass m of the object 70 and the acceleration α_obj(t) of the object 70. The acceleration α_obj(t) of the object 70 is a function of time t, but it is basically calculated based on the command value related to the pointer of the robot 10.
[0071] Fdis(t)=m·α_obj(t) (Equation 1)
[0072] Corresponding to the physical properties of the object 70 (elastic coefficient K and damping coefficient C), the phenomenon of the object 70 slipping from the fingers of the robot hand 20 is determined for the inertial force Finr. That is, the upper limit of the constraint force Flim used to eliminate geometric constraints is exceeded. When the upper limit of the constraint force Flim is exceeded, the geometric constraints are eliminated, and the state is not stable. While maintaining a "stable state," the gripping stability calculation unit 34 sets a high evaluation value for gripping stability and outputs it as a stability evaluation result to the result DB 35. Conversely, when the state is not "stable," the gripping stability calculation unit 34 sets a low evaluation value for gripping stability and outputs it as a stability evaluation result to the result DB 35.
[0073] Furthermore, the upper limit of the constraint force at which the stable state disappears can also be defined by the friction coefficient μ of the object 70 and the robot hand 20. If the pushing force at the gripping point of the robot hand 20 is set as Fi, the upper limit of the constraint force Flim can also be defined according to Equation 2.
[0074] Flim=μ·Fi (Equation 2)
[0075] In this case, for example, if the "stable state" of the gripping point i of the finger of the robot hand 20 is set as the gripping stability Si, the gripping stability Si can also be defined according to Equation 3.
[0076] Si=(Flim-max(Fdis(t)))(Formula 3)
[0077] By using the gripping stability Si, the gripping points can be compared with each other. The gripping stability calculation unit 34 outputs this as a stability evaluation result to the result DB 35. The method for obtaining the stability evaluation result is not limited to this method.
[0078] Based on the above rules, the gripping stability calculation unit 34 outputs the gripping point candidates and stability evaluation results calculated by the result DB 35 to the gripping point determination unit 36. The gripping point determination unit 36, based on storing multiple gripping point candidates and stability evaluation results, can select the gripping point with the highest stability evaluation result.
[0079] Here, the operation of the robot control device 30 will be explained. Figure 9 This is a flowchart illustrating the actions of the robot's control device. Figure 9 Steps S101 to S103 and Figure 6 The process is the same, therefore the explanation is omitted. In step S201, the gripping stability calculation unit 34 determines whether a "stable state" can be maintained. If a "stable state" can be maintained, the process proceeds to step S202, where the gripping stability calculation unit 34 sets a high evaluation value for gripping stability. If a "stable state" is not maintained, the process proceeds to step S203, where the gripping stability calculation unit 34 sets a low evaluation value for gripping stability. Furthermore, in step S204, the gripping point determination unit 36 selects the gripping point with the highest stability evaluation result based on multiple stored gripping point candidates and stability evaluation results, and determines the gripping point.
[0080] By using the structure described above, considering the drop caused by shape deformation during the transport of the object 70 by the robot arm 20, it is possible to extract gripping points where the object 70 is unlikely to fall. In this case, it is also possible to extract only gripping points that take into account the allowable deformation of the gripping object 70, thus reducing the proportion of gripping points that will fail to grip, and achieving the special effect of improving production efficiency.
[0081] Next, other variations of this embodiment will be described. In the deformation evaluation unit 33, simulations are performed to simulate various changes in the gripping force Fi(t) at each gripping point. When the gripping force Fi(t) decreases, according to Equation 2, the upper limit of the constraint force Flim decreases, and as a result, the gripping stability Si tends to decrease. On the other hand, if the gripping force Fi(t) at each gripping point decreases, the amount of deformation output as shape deformation information calculated by the deformation evaluation unit 33 decreases. In this case, the gripping stability Si can also be defined according to Equation 4 as an index that includes "minimum deformation" as an index other than "stable state" as Equation 3.
[0082] Si=w1*(Flim-max(Fdis(t)))+w2 / max(Fi(t))(Formula 4)
[0083] Here, w1 and w2 are appropriate weighting coefficients. The weighting coefficients are designed by the user based on whether to evaluate the ease of maintaining a stable state or to emphasize gripping through the minimum gripping force.
[0084] The gripping stability calculation unit 34 evaluates gripping stability based on gripping stability Si. Therefore, when the robotic arm 20 grips a soft, amorphous object, it can select gripping points with high gripping stability based on the shape of the deformed object 70 without damaging it. As a result, gripping failures are reduced, and objects can be gripped with a high success rate, resulting in the special effect of shortening production cycle time and improving production efficiency.
[0085] Implementation method 4.
[0086] In this embodiment, the gripping stability calculation unit 34 evaluates the difficulty of geometric offset based on the shape and shape deformation information of the fingers of the robot hand 20, and outputs the evaluation result as the gripping stability. In embodiment 3, the gripping point is represented by a point, but in this embodiment, the gripping point is given a geometric shape. In this case, multiple contact points are generated for one finger. The gripping stability calculation unit 34 evaluates the difficulty of geometric offset of the object 70 relative to the robot hand 20 based on the shape and shape deformation information of the fingers of the robot hand 20.
[0087] Figure 10 This is a diagram showing the positional relationship between the fingers of the robot hand 20 and the object 70. By importing the physical property model for the case where there are multiple contact points associated with the gripping of the object 70, it is possible to replace Equation 2, which defines the upper limit of the constraint force Flim related to geometric constraints, with Equation 5.
[0088] Flim=μ·A·Fi (Equation 5)
[0089] Here, A is the effective contact area between the fingers of the robot hand 20 and the object 70. The effective contact area is as follows: Figure 10 The diagram shows the contact area when the fingers are in surface contact with the object, rather than point contact. Generally, the coefficient of friction for surface contact is greater than that for point contact. This embodiment uses a model to reflect this. The amount of increased contact area of the fingers is shown as the object 70 deforms. If a deformation greater than or equal to a certain amount occurs, A = 1. In the case of a small gripping force Fi(t) when the object 70 is being held lightly, the property model is 0 < A < 1.
[0090] That is, the contact amount between the deformed shape of the object 70 and the fingers of the robot hand 20 is equivalent to the case that causes a change in the effective coefficient of friction. As described above, the gripping stability calculation unit 34 defines the coefficient of friction based on the effective contact area, and the gripping stability is calculated based on this coefficient of friction, which is a feature of this embodiment.
[0091] In structures incorporating the aforementioned physical property model, a weaker gripping force Fi(t) results in a smaller effective contact area A. Consequently, the upper limit of the constraint force Flim tends to change accordingly with the gripping force Fi(t). As a result, the simulation accuracy of the deformation evaluation and gripping stability calculation in the actual gripping state is improved.
[0092] According to this embodiment, when the robotic arm 20 grasps a soft, amorphous object, the accuracy of the grasping stability calculated based on the deformed shape is improved. Compared to the past, it is possible to select a grasping point with high grasping stability without damaging the object. As a result, failures are reduced, and objects can be grasped with a high success rate, resulting in the special effect of shortening production cycle time and improving production efficiency.
[0093] Implementation method 5.
[0094] The difference between this embodiment and embodiment 3 is that shape deformation information is output after a certain period of time has elapsed since the application of the gripping force and the gripping force has been removed. The deformation evaluation unit 33 outputs shape deformation information after the gripping force is unloaded a certain period of time after the gripping force is applied to the object 70 by the robot hand 20. Furthermore, the gripping stability calculation unit 34 calculates the difference between the original shape of the object 70 and the shape of the object 70 after unloading, and evaluates the difference by comparing it with a predetermined allowable deformation value.
[0095] When the robotic arm 20 continuously applies a gripping force to the object 70, the gripping force can be set as Fi(t) = F0 (constant). In this case, the shape deformation information is expected to converge to a certain shape. On the other hand, unless it is a complete plastic deformation, if the gripping force is unloaded, the shape will change further.
[0096] Here, for example, consider the case where a gripping force F0 is applied during the period from 0 seconds to t0 seconds, and then the force is unloaded after t0 seconds. After unloading, Fi(t+t0) = 0. In this embodiment, the following structure is added: the shape after a sufficient time has elapsed after unloading is taken as "shape deformation information after unloading" and output as part of the shape deformation information from the deformation evaluation unit.
[0097] The shape deformation information under the applied state of the gripping force Fi(t) = F0 output from the deformation evaluation unit 33 is called the first shape deformation information. The shape deformation information after the gripping force is unloaded for a certain time t0 is called the second shape deformation information. At this time, the feature is that the difference between the original shape of the object 70 and the shape in the second shape deformation information is calculated. Regarding the difference, a predetermined allowable value of deformation and its magnitude are compared. If the deformation exceeds the allowable value, the gripping stability is evaluated as low, and if it does not exceed the allowable value, the gripping stability is evaluated as high.
[0098] Figure 11 This is a diagram showing the positional relationship between the finger and the object 70. Below, a method for obtaining the difference in shape before and after applying the gripping force will be described using... Figure 11 Here is an example. The gripping stability calculation unit 34 calculates the difference between the curvature of the original shape of the object 70 and the curvature of the object 70 after unloading, and evaluates it by comparing the difference in curvature with a predetermined allowable value for deformation. The difference in curvature can be calculated as follows. Regarding the first shape deformation information and the second shape deformation information, they are overlapped based on points without deformation (points far from the gripping point). After overlap, two points are selected after the point of deformation becomes a point without deformation. That is, the line segment of the curve whose position changes before and after deformation was initially without deformation, and therefore is the overlapping point (in Figure 11 In this case, equivalent to discrete points DP3 and DP4, the length of the curve between these two points is calculated. The curve between the two points consists of two parts: the curve of length L1 from discrete point DP3 through discrete point DP5 to discrete point DP4, and the curve of length L2 from discrete point DP3 through discrete point DP1 to discrete point DP4. Based on each length L1 and length L2, corresponding points are defined for a certain ratio. For example, in each curve, points corresponding to 0.25 × L1 and 0.25 × L2 from the endpoints are defined as corresponding points, and the distance between each corresponding point is calculated. This distance is calculated separately, and the maximum value is defined as the "difference component of curvature". Figure 11 In this case, the curvature difference component is the distance DC1 between discrete points DP1 and DP5. The deformation evaluation unit 33 evaluates whether this difference component is greater than or less than a pre-determined "deformation allowable value" and outputs the evaluation result as part of the shape deformation information. Furthermore, in Figure 11 In the case of 2 points, discrete points of 3 points are set on the curve between the 2 points, but it is not limited to 3 points; for example, it can also be 9 points. When there are 9 discrete points, in each curve, the points corresponding to 0.1×L1 and 0.1×L2 from the endpoints are defined as the corresponding points.
[0099] In this embodiment, the gripping stability calculation unit 34 uses shape deformation information to output a tag as a candidate for rejection of the gripping point when the deformation exceeds the "permissible value".
[0100] According to this embodiment, when the robotic arm 20 grasps a soft, amorphous object, the shape after unloading, i.e., the final shape after the operation, is evaluated to determine whether the shape is acceptable. As a result, grasping forces or grasping points selected as failures can be excluded from the object being retrieved. Consequently, failures are reduced, and objects can be grasped with a high success rate, resulting in the special effect of shortening production cycle time and improving production efficiency.
[0101] Implementation method 6.
[0102] In this embodiment, the difference from embodiment 1 is that the gripping point generation unit has a gripping point candidate learning unit. This gripping point candidate learning unit constructs a neural network that learns from the input of the result label (success or failure label) of the simulation or actual operation performed on the gripping point, as well as the shape deformation information, gripping force, gripping point, and physical properties of the object at that time. It can output the gripping point by inputting the shape of the object.
[0103] Figure 12 This is a block diagram showing the structure of the gripping point generation unit 31b according to Embodiment 6. Figure 3 Based on the structure of the gripping point generation unit 31 shown, the gripping point generation unit 31b further comprises a gripping stability calculation unit 34 and a result DB (result database) 35. Additionally, the robot control device 30 has a gripping point candidate learning unit 37 and a learning DB (learning database) 38. The gripping point candidate learning unit 37 has a neural network 40. The gripping point candidate learning unit 37 learns the relationship between the gripping point candidates output from the gripping stability calculation unit 34, the result data as stability evaluation results, and the result labels obtained through actual operation, and outputs the gripping point candidates of the object based on shape deformation information. Figure 12 As shown, an example illustrates how a network learns by taking the object's shape information (before deformation) as input and outputting the gripping point, gripping force, and gripping stability.
[0104] Regarding the gripping point candidate learning unit 37, an example of effectively using simulation and actual machine experiments will be explained. Considering the effective use of simulation (numerical calculation processing), gripping point candidates are determined based on the shape information of the object 70, which is the object to be gripped, obtained before gripping, and gripping operations are actually tested. Based on the structure described up to Embodiment 5, if an actual machine operation is performed, a high probability of successful gripping is expected. However, since it cannot be modeled in the case of actual machine operation, gripping failure is anticipated.
[0105] In this case, for all gripping point candidates designed by the deformation evaluation unit 33 and the gripping point determination unit 36, there are success and failure result labels. However, it is generally difficult to formulate the causal relationship between success, the shape of the object (before and after deformation), the gripping point, and the gripping force. Therefore, by using a neural network framework, for example, to learn nonlinear relationships, it is possible to obtain the relational nature.
[0106] The success and failure labels associated with multiple trials, gripping points, gripping forces, physical properties of the object, deformed shape of the object (shape before and after deformation information), and gripping stability are prepared as success and failure labels for each trial, and then processed for learning by the neural network.
[0107] Here, the gripping point generation unit 31b has a physical property model definition unit (not shown) that models the relationship between the force acting on the object 70 and the displacement of the object 70 using a model of spring constant and damping coefficient. The physical property model definition unit applies a force that varies over time to the object 70 and, for the applied force, estimates a physical property model (spring constant K and damping coefficient C) based on time-series information of the displacement generated by the deformation of the object 70. Furthermore, for the predetermined spring constant K and damping coefficient C, it is also possible to update the spring constant K and damping coefficient C based on deformation results obtained through actual machine operation. Alternatively, without assuming the spring constant K and damping coefficient C, deformation information and time-series information of the relationship between force and displacement can be obtained through learning. For example, an example is a neural network that constructs a model that outputs displacement information based on the time-series information of the gripping force, incorporating deformation information and the time-series information of the gripping force.
[0108] Additionally, the gripping point candidate learning unit 37 includes a property model learning unit (not shown) that learns by modeling the relationship between the force acting on the object 7 and the displacement of the object 70 through a neural network. The property model learning unit applies a force that varies over time to the object 70 and learns from the neural network 40, which is set based on the time-series information of the displacement caused by the deformation of the object, for the applied force.
[0109] The gripping point candidate learning unit 37 performs learning processing based on the gripping point candidates and stability evaluation results stored in the result DB 35. For example, the learning of neural network 40 is illustrated. Neural network 40 contains a learning unit and an inference unit (not shown). Using the learning parameters from the learning unit, a neural network 41 reflecting the learning parameters is added to the inference unit. Thus, taking object shape information as input, it is possible to output gripping points and gripping forces. The learning parameters are coefficients defined for the network construction of the neural network.
[0110] Figure 13 This is a block diagram illustrating the structure of other gripping point generation units 31c involved in Embodiment 6. For example... Figure 13 As shown, the neural network 41 obtained in the above process is applied as a gripping point candidate generation unit 32a. If object shape information is input, the gripping point candidate generation unit 32a generates multiple gripping point candidates and gripping stability, and outputs them to the gripping point determination unit 36. The gripping point determination unit 36 uses the gripping stability to select one gripping point candidate and outputs it.
[0111] According to this embodiment, a gripping point generation algorithm can be obtained by learning to correct the modeling errors obtained through actual operations. As a result, the computational cost of calculating gripping point candidates is reduced, and the time for calculating gripping points is shortened, thus achieving the special effect of improved production efficiency.
[0112] Next, other variations of this embodiment will be described. A method for processing the gripping point candidates obtained through simulation in the same manner as described above will be explained. Considering the effective use of the simulation (numerical calculation processing) described previously, gripping point candidates are determined based on the shape information of the gripping object obtained before gripping, and the gripping operation is actually tested. At this time, the shape information of the gripping object is also generated through simulation. Furthermore, the gripping test of the object 70 itself is simulated using physical contact phenomena, and the shape is implemented in a finely observed physical simulation. Therefore, when using a physical simulation without interference or uncertainties, the gripping point with the highest success rate is known, and it is expected that all successful gripping points will be successfully applied.
[0113] For the experiment of pasting multiple successful labels, the gripping point, gripping force, physical properties of object 70, deformation shape of object 70 (shape before deformation and shape after deformation information), and gripping stability are prepared as success and failure labels related to each experiment, and then processed for learning by the neural network.
[0114] Although based on a physical simulation model, just as with using a real machine, an example is shown of learning a network that takes shape information (before deformation) as input and outputs gripping point, gripping force, and gripping stability.
[0115] The neural network 41 obtained in the above process is used as a gripping point candidate generation unit 32a. If object shape information is input, multiple gripping point candidates and gripping stability are generated and output to the gripping point determination unit 36. The gripping point determination unit 36 selects one gripping point candidate using the gripping stability and outputs it.
[0116] According to this embodiment, when the robotic arm 20 grasps a soft, amorphous object 70, only shape information is input, and one candidate point with the highest grasping stability can be extracted, reducing grasping failures. Furthermore, physical property information related to the deformation of the object 70 can be obtained from the actual object, improving the deformation simulation accuracy of the simulation-based deformation evaluation unit. As described above, regarding the grasping points and stable grasping points determined based on complex physical simulation models, a grasping point generation algorithm is learned that automatically outputs grasping points if the object shape is input. As a result, the computational cost of calculating grasping point candidates is reduced, and the calculation time for grasping points is shortened, thus achieving the special effect of shortening production cycle time and improving production efficiency.
[0117] Implementation method 7.
[0118] In this embodiment, the difference from embodiment 3 is that the gripping point candidate generation unit defines a first gripping force, evaluates the conditions under which a gripping point is gripped by the first gripping force, and after extracting a valid gripping point, grips the object by a second gripping force smaller than the first gripping force, thereby enabling efficient exploration of gripping points.
[0119] Figure 14 This is a block diagram showing the structure of the gripping point generation unit 31d according to Embodiment 7. (Regarding...) Figure 8 The structure of the gripping point generation unit 31a shown is such that gripping point candidates are input into the gripping point candidate generation unit 32 from the result DB 35. For example... Figure 14 As shown, the database of results related to the gripping point candidates and stability evaluation results output by the gripping stability calculation unit 34 is input again into the gripping point candidate generation unit 32. Moreover, the gripping point candidate generation unit 32 extracts a finite number of gripping points with high stability evaluations as these finite number of extracted gripping point candidates, and defines the second gripping force (wherein, it is less than the first gripping force).
[0120] The gripping point generation unit 31d includes: a result database 35 that stores multiple gripping point candidates; and a gripping point candidate generation unit 32 that defines a first gripping force output by the robot hand 20 to the object 70, and outputs the first gripping point candidates specified by the first gripping force to the deformation evaluation unit. The gripping stability calculation unit 34 calculates the stability evaluation result relative to the first gripping point candidates, and outputs the first gripping point candidates and the stability evaluation result to the result database 35. The gripping point candidate generation unit 32 extracts multiple gripping point candidates from the first gripping point candidates stored in the result database 35 based on the stability evaluation result, defines a second gripping force for the multiple gripping point candidates, and outputs it again to the deformation evaluation unit 33.
[0121] Furthermore, the gripping point candidate generation unit 32 can repeat the same process three times or more. For example, it can repeat the third gripping force, the fourth gripping force, ..., the kth gripping force, continuously reducing the gripping force being explored. This allows for the extraction of gripping points that effectively deform the object 70 with the minimum gripping force. As a result, it can efficiently explore points that can be stably gripped with the minimum gripping force, and can extract candidate points that are less likely to fail to grip in a short time. Consequently, the operation time of each robot action is shortened, achieving the special effect of reducing production cycle time and improving production efficiency.
[0122] Implementation method 8.
[0123] In this embodiment, the difference from Embodiment 1 is that the gripping stability calculation unit 34 calculates a combination of candidate gripping points for stably gripping the object based on the object's contour information. The gripping stability calculation unit 34 obtains the contour information of the object 70 from the coordinates of a group of points on the object's contour and selects a combination of candidate gripping points from the object's contour. Furthermore, for each combination, the gripping stability calculation unit 34 calculates an evaluation value for when the robot hand 20 grips the object 70 with a predetermined gripping force, and calculates a combination of candidate gripping points for stably gripping the object 70 based on the evaluation value.
[0124] An evaluation method for exploring candidate combinations of gripping points for stably gripping the object 70 is described. The gripping stability calculation unit 34 derives combinations of stable gripping points based on an evaluation of the minimum gripping force required to grip the object 70. The minimum gripping force required to grip the object 70 is the minimum fingertip force of the robot hand 20 required to counteract the gravity acting on the object 70. From the viewpoint of minimizing damage to the object 70, this value is preferred to be small. The evaluation is performed using the value of the fingertip force calculated based on the gripping force and friction, and combinations of stable gripping points are explored in the following order.
[0125] First, using the point coordinates of the object 70's outline, the points arranged in the two-dimensional plane are smoothly connected through spline interpolation to obtain the outline information of the object 70. Then, candidate gripping points are obtained on the outline of the object 70, and all combinations of two points are saved.
[0126] Next, the gripping force is reduced to a certain value, and an evaluation value is calculated for all candidate gripping points. Based on the results, a stable combination of gripping points under that gripping force is obtained. Furthermore, the gripping force is changed, and an evaluation value is calculated for all candidate gripping points. Based on the results, a stable combination of gripping points under that gripping force is obtained. This process is repeated to obtain the optimal combination of stable gripping points under the optimal gripping force.
[0127] As described above, the gripping stability calculation unit 34 calculates a combination of candidate gripping points for the object 70 to be gripped stably. As a result, for objects that are amorphous objects such as soft objects, gripping failures are greatly reduced by gripping them at the selected gripping points. This enables gripping objects with a high success rate and achieves the special effect of shortening production cycle time and improving production efficiency.
[0128] Implementation method 9.
[0129] In this embodiment, the gripping stability calculation unit 34 calculates the evaluation value based not only on the shape deformation information of the object 70 after gripping, but also on the shape information of the object 70 before gripping. Figure 15 This is a schematic diagram of the object 70 before grasping according to Embodiment 9. Specifically, the deformation evaluation unit 33 outputs multiple discrete points DPB1, DPB2, ... as shape information of the object 70 before grasping. The multiple discrete points DPB1, DPB2, ... are set based on the contour of the object 70. The grasping stability calculation unit 34 quantitatively evaluates the degree of indentation of the object based on the positional relationship of the multiple discrete points DPB1, DPB2, ... and outputs it as an evaluation value. Figure 15 In the case where a depression occurs at discrete points DPB1 and DPB2, the gripping stability calculation unit 34 can output discrete points DPB1 and DPB2 as candidates for gripping points with high gripping stability.
[0130] As described above, the gripping stability calculation unit 34 calculates an evaluation value based on the shape information of the object 70 before gripping, thereby enabling the selection of the gripping point with high precision. This allows for gripping of the object with a high success rate, resulting in the significant effect of shortening production cycle time and improving production efficiency.
[0131] Here, the hardware structure of the robot control device 30 is described. The various functions of the robot control device 30 are implemented through processing circuitry. The processing circuitry has at least one processor and at least one memory.
[0132] Figure 16 This is a diagram illustrating the hardware structure of the robot control device according to embodiments 1 to 9. The robot control device 30 can... Figure 16 The control circuit shown in (a) is implemented by processor 81 and memory 82. Examples of processor 81 are CPU (also known as Central Processing Unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, DSP (Digital Signal Processor)) or system LSI (Large Scale Integration) circuits.
[0133] The memory 82 includes, for example, non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), disks, floppy disks, optical disks, compact disks, mini disks, or DVDs (Digital Versatile Disk).
[0134] The robot control device 30 is implemented by the processor 81 reading and executing the program stored in the memory 82 for performing the actions of the robot control device 30. Alternatively, this program can be described as the sequence or method by which the computer executes the robot control device 30. The program executed by the robot control device 30 includes a gripping point generation unit 31 and an instruction value generation unit 39, which are downloaded to the main storage device and generated thereon.
[0135] The memory 82 stores obstacle information, object shape information, shape deformation information, etc. The memory 82 is also used as temporary memory when the processor 81 performs various processes.
[0136] The program executed by processor 81 can be provided as a computer program product by storing it in a computer-readable storage medium in an installable or executable form. Alternatively, the program executed by processor 81 can also be provided to robot control device 30 via a network such as the Internet.
[0137] Alternatively, the robot control device 30 can be implemented using dedicated hardware. Furthermore, regarding the functions of the robot control device 30, some can be implemented using dedicated hardware, while others can be implemented using software or firmware.
[0138] In addition, the robot control device 30 can also be... Figure 16 The dedicated processing circuit 83 shown in (b) implements this. At least a portion of the gripping point generation unit 31 and the instruction value generation unit 39 can be implemented by the processing circuit 83. The processing circuit 83 is dedicated hardware. The processing circuit 83 can be, for example, a single circuit, a composite circuit, a programmable processor, a parallel programmable processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof. A portion of the functionality of the robot control device 30 can be implemented by software or firmware, and the remainder can be implemented by dedicated hardware.
[0139] Explanation of the label
[0140] 10 Robot, 20 Robot Hand, 30 Robot Control Device, 31, 31a, 31b, 31c, 31d Grip Point Generation Unit, 32, 32a Grip Point Candidate Generation Unit, 33 Deformation Evaluation Unit, 34 Grip Stability Calculation Unit, 35 Result DB, 36 Grip Point Determination Unit, 37 Grip Point Candidate Learning Unit, 38 Learning DB, 39 Instruction Value Generation Unit, 50 Measuring Device Controller, 60 Measuring Device, 70 Object, 81 Processor, 82 Memory, 83 Processing Circuit, FP1~FP6 Pointer Positions, DP1~DP5, DPB1, DPB2 Discrete Points.
Claims
1. A robot control device for controlling a robot and its robotic hand in order to grasp an object. The robot control device is characterized by the following features: It has a gripping point generation unit that generates gripping points for the object to be gripped by the robotic arm. The gripping point generating unit has: The deformation evaluation unit calculates shape deformation information when the object's shape deforms due to the gripping action of the robot hand; and The gripping point determination unit determines the gripping point of the object based on the deformation amount of the object contained in the shape deformation information and the geometric constraints of the object after deformation. The gripping point generation unit includes a gripping stability calculation unit, which evaluates the ease of geometric displacement of the object relative to the robot hand based on the shape of the robot hand's fingertips and the shape deformation information. The deformation evaluation unit outputs multiple discrete points as the shape deformation information. The gripping point determination unit determines the geometric constraints based on the change in the positions of the plurality of discrete points when an imaginary force is applied to the object.
2. The robot control device according to claim 1, characterized in that, The deformation evaluation unit calculates the upper limit of the gripping force applied to the object based on the relationship between the gripping force applied to the object and the displacement of the object, as well as the allowable deformation of the object, and evaluates whether the gripping force applied from the robot hand to the object exceeds the upper limit.
3. The robot control device according to claim 1, characterized in that, The deformation evaluation unit outputs time-series information of the gripping force, calculated based on the relationship between the gripping force applied to the object and the displacement of the object, as part of the shape deformation information.
4. The robot control device according to claim 1, characterized in that, The gripping point generation unit has a gripping stability calculation unit, which evaluates the mechanical stability relative to a predetermined external force based on the force balance of the object after deformation near the gripping point of the object.
5. The robot control device according to claim 4, characterized in that, The gripping stability calculation unit evaluates the force balance of the object after deformation near the gripping point of the object, and extracts the gripping point of the object where the gripping force of the robot hand on the object becomes the minimum.
6. The robot control device according to claim 4, characterized in that, The deformation evaluation unit outputs shape deformation information after a certain period of time has elapsed since the gripping force applied by the robotic arm to the object was unloaded. The gripping stability calculation unit calculates the difference between the original shape of the object and the shape of the object after unloading, and evaluates the difference by comparing it with a predetermined allowable value for variation.
7. The robot control device according to claim 4, characterized in that, The deformation evaluation unit outputs shape deformation information after a certain period of time has elapsed since the gripping force applied by the robotic arm to the object was unloaded. The gripping stability calculation unit calculates the difference between the curvature of the original shape of the object and the curvature of the shape of the object after unloading, and evaluates the difference in curvature by comparing it with a predetermined allowable value for variation.
8. The robot control device according to claim 1, characterized in that, The gripping point generation unit includes a gripping stability calculation unit. This gripping stability calculation unit obtains the contour information of the object from the point group coordinates of the object's contour, selects a combination of gripping point candidates from the object's contour, calculates an evaluation value for each combination of gripping point candidates when the robot hand grips the object with a specified gripping force, and calculates a combination of gripping point candidates that stably grips the object based on the evaluation value.
9. The robot control device according to claim 4, characterized in that, The gripping point generating unit has: The results database stores multiple candidate gripping points; and The gripping point candidate generation unit defines the first gripping force output by the robotic arm to the object, and outputs the first gripping point candidate, which specifies gripping with the first gripping force, to the deformation evaluation unit. The gripping stability calculation unit calculates the stability evaluation results for the first gripping point candidate, and outputs the first gripping point candidate and the stability evaluation results to the result database. Based on the stability evaluation results, the gripping point candidate generation unit extracts multiple gripping point candidates from the first gripping point candidates stored in the result database, defines a second gripping force for the multiple gripping point candidates, and outputs it to the deformation evaluation unit again.
10. The robot control device according to claim 4, characterized in that, The gripping point generation unit has a gripping point candidate learning unit, which learns the relationship between the result data output from the gripping stability calculation unit and the result labels obtained through actual operation, and outputs the gripping point candidates of the object based on the shape deformation information.
11. The robot control device according to claim 1, characterized in that, The gripping point generation unit has a physical property model definition unit, which models the relationship between the force acting on the object and the displacement of the object by using a model of spring constant and damping coefficient. The physical property model definition unit applies a force that varies with time to the object, and estimates the spring constant and damping coefficient of the model based on the time series information of the displacement generated by the deformation of the object.
12. The robot control device according to claim 1, characterized in that, The gripping point generation unit includes a property model learning unit, which models the relationship between the force acting on the object and the displacement of the object using a neural network. The material property model learning unit applies a force that varies over time to the object and learns a neural network based on time-series information of displacements caused by deformation of the object in response to this force.
13. The robot control device according to claim 8, characterized in that, The deformation evaluation unit further outputs multiple discrete points as shape information of the object before grasping. The gripping stability calculation unit evaluates the indentation of the object based on the positional relationship between the plurality of discrete points and uses it as an indentation evaluation value, and outputs the gripping point candidates based on the indentation evaluation value.
14. A robot control method for controlling a robot and its robotic hand in order to grasp an object. The robot control method is characterized by including the following steps: As shape deformation information of the object due to the grasping action of the robotic hand, multiple discrete points are output; and Based on the shape of the robot hand's fingertips and the shape deformation information, the ease of geometric offset of the object relative to the robot hand is evaluated. Based on the change in the position of the multiple discrete points when an imaginary force is applied to the object, the geometric constraints are determined. Based on the deformation amount of the object contained in the shape deformation information and the geometric constraints of the object after deformation, the gripping point of the object is determined, and the gripping point of the object held by the robot hand is generated.
Citation Information
Patent Citations
System, method and program for controlling manipulator
JP2008049459A
Information processor, information processing method, and program
JP2016196077A
Gripping force-setting system, gripping force-setting method and gripping force-estimating system
WO2018092254A1
