Holding position determination device, determination system, determination method, and recording medium

Abstract

The present invention aims to provide a gripping position decision device, a gripping position decision system, a gripping position decision method, and a recording medium that can determine a gripping point considering operation. The gripping position decision device is a gripping position decision device of a multi-joint multi-fingered robot hand, including: a friction force distribution calculation section that, based on predicted control of gripping force when an object is gripped by at least two fingers of the multi-fingered robot hand, estimates friction force between one of the fingers that grips the object and the object, and calculates a friction force distribution on the surface of the object on which the object can be gripped, based on a value related to the friction force calculated using the estimated friction force; a grippable region selection section that selects at least one or more grippable regions based on the friction force distribution; and a gripping position calculation section that calculates a gripping position at which the object can be stably gripped from among the selected grippable regions.

Description

Technical Field

[0001] This invention relates to a grip position determination device, a grip position determination system, a grip position determination method, and a recording medium. Background Technology

[0002] A control device for enabling a robot to grasp an object is proposed. As such a control device, the following method is proposed: after temporarily determining a gripping point that can be physically contacted, the gripping force is determined by considering the balance of forces, and after evaluating the quality of the gripping force as the volume of the gripping force's envelope, a gripping point with higher quality is searched (for example, see Patent Document 1).

[0003] Furthermore, in the technology described in Patent Document 2, before actually holding the object, the fingers are slid across the surface of the object with a predetermined pressing force, and the pressing force and frictional force at this time are measured. An estimated coefficient of friction is then calculated based on the ratio of the pressing force to the frictional force. Moreover, in the technology described in Patent Document 2, the calculated estimated coefficient of friction is used to maintain the grip force to prevent slippage, and is appropriately controlled to prevent the grip force from becoming excessive. Thus, an appropriate grip force is used to stably hold the object.

[0004] Figure 13 This diagram illustrates an example of a robot using two fingers (upper finger 901 and lower finger 902) to grasp an object (a coin-shaped object Obj) in conventional technology. In the first state g901, the deviation of the upper finger 901 and lower finger 902 along the long side of the coin Obj, i.e., the finger gap, is 0.0 (m). In the second state g902, the finger gap between the upper finger 901 and lower finger 902 is 0.0015 (m). Graph g903 is a graph showing the relationship between the finger gap and the lateral force (sum of squares) in both the first and second states, with the horizontal axis representing the finger gap (m) and the vertical axis representing the lateral force. Furthermore, the lateral force refers to the lateral force (friction) exerted on the grasped object. Figure 13 As shown, when the gap between the fingers increases, the lateral force increases, and the grip becomes unstable.

[0005] Figure 14This diagram illustrates an example of a robot using three fingers (first finger 911, second finger 912, and third finger 913) to grasp an object (a spherical object Obj) in conventional technology. Figures g911 to g914 represent the first to fourth states. In figures g911 to g914, the vertical and horizontal axes represent the distance between the fingers (m). As shown in figure g911, in the first state, the angles of the three fingers are 10 degrees, 170 degrees, and 270 degrees, respectively, and the lateral force is 6.21e-23. As shown in figure g912, in the second state, the angles of the three fingers are 10 degrees, 170 degrees, and 230 degrees, respectively, and the lateral force is 9.81e-2. As shown in figures g911 and g912, the lateral force is minimized when the fingers are positioned opposite to gravity. Furthermore, as shown in Figure g913, in the third state, the angles of the three fingers are 30 degrees, 150 degrees, and 270 degrees, respectively, and the lateral force is 6.29e-23. As shown in Figure g914, in the fourth state, the angles of the three fingers are 70 degrees, 150 degrees, and 270 degrees, respectively, and the lateral force is 5.72e-07. As shown in Figures g913 and g914, the lateral force is minimized when the fingers are in symmetrical positions.

[0006] Figure 15 This diagram illustrates an example of the index ε (refer to Non-Patent Document 1) when a robot uses two fingers (upper finger 901 and lower finger 902) to grasp an object (a coin-shaped object Obj) with a finger gap of 0.0 (m) in a prior art. Graph g921 is a graph representing the finger gap. Graph g922 is a two-dimensional graph representing the index ε, with force on the horizontal axis and torque on the vertical axis. The larger the area enclosed by the chain lines, the more stable the grip. Thus, the index ε when the fingers are relatively aligned in the thickness direction of the object (finger gap 0.0 (m)) is approximately 5.69e-06. Graph g923 is a graph representing the frictional push and force vector of the upper finger, and graph g924 is a graph representing the frictional push and force vector of the lower finger. In this state, Gcfc = -Fe holds, providing sufficient grip margin. Furthermore, Fe is the external force, fc is the force the robot can exert during gripping, and Gc is the coefficient of friction. In addition, the condition for force closure is that Gcfc = -Fe holds true for any external force Fe ∈ R6.

[0007] Figure 16This diagram illustrates an example of the index ε (refer to Non-Patent Document 1) when a robot uses two fingers (upper finger 901 and lower finger 902) to grasp an object (a coin-shaped object Obj) with a finger gap of 0.0015 (m) in a prior art. Graph g931 is a graph representing the finger gap. Graph g932 is a two-dimensional graph representing the index ε. Thus, the index ε when the finger gap deviates from 0.0015 (m) is approximately 1.21e-05. Graph g933 is a graph representing the frictional push and force vector of the upper finger, and graph g934 is a graph representing the frictional push and force vector of the lower finger. In this state, Gcfc = -Fe holds, but the gripping margin is small.

[0008] [Existing Technical Documents]

[0009] [Patent Literature]

[0010] [Patent Document 1] Japanese Patent No. 6476358

[0011] [Patent Document 2] Japanese Patent Application Publication No. 2005-144573

[0012] [Non-patent literature]

[0013] [Non-Patent Literature 1] Andrew T, Miller Peter K, Allen, “GraspIt!: A Multifunctional Simulator for Grasping Analysis”, IEEE Journal of Robotics and Automation (Vol. 11, No. 4, December 2004), 2004, pp. 110-122. Summary of the Invention

[0014] [The problem the invention aims to solve]

[0015] However, in previous technologies, regarding the index ε and volume v, which are general grip characteristics, when only a specific finger moves along the surface of an object under the contact of the same number of fingers, the magnitude of the index may not actually be directly related to the difficulty of finger sliding, and finger slippage may occur. Therefore, in the evaluation of index ε, such as Figure 16 In that case, there is a possibility that even in an unstable state, it may be mistaken for a stable state.

[0016] Furthermore, in the technology described in Patent Document 2, the coefficient of friction is estimated by sliding the fingers on the surface of the object with a predetermined pressing force before actually holding the object. However, due to position correction, the contact position of the fingertips is constrained and lacks freedom. In the technology described in Reference Document 2, for example, the coefficient of friction cannot be estimated when the surface of the object is curved.

[0017] Thus, in previous technologies, it was difficult to determine the grip point based on the operation.

[0018] The present invention was made in view of the aforementioned problems, and its object is to provide a grip position determining device, grip position determining system, grip position determining method, and procedure capable of determining the grip position of the grip point taking into account the operation.

[0019] [Technical means to solve the problem]

[0020] (1) In order to achieve the above objective, a gripping position determination device according to an embodiment of the present invention is a gripping position determination device for a multi-joint, multi-finger robotic hand. The gripping position determination device includes: a friction force distribution calculation unit, which estimates the friction force between one of the gripping fingers and the object based on the predicted gripping force when at least two of the multi-finger gripping the object, and calculates the friction force distribution on the object surface that allows the object to be gripped based on a friction force-related value calculated using the estimated friction force; a gripable area selection unit, which selects at least one gripable area based on the friction force distribution; and a gripping position calculation unit, which calculates a gripping position from the selected gripable area that allows the object to be gripped stably.

[0021] (2) Furthermore, in a grip position determination device of an embodiment of the present invention, the friction force distribution calculation unit calculates the friction force distribution within a specified range from the initial grip position of one of the fingers gripping the object, and the grip position calculation unit searches within the friction force distribution from the initial grip position to calculate a grip position that can be stably gripped relative to an external force.

[0022] (3) Furthermore, in a gripping position determination device of an embodiment of the present invention, the gripping position calculation unit may select the gripable area that can maintain the posture or activity of the object in a specific state, and calculate the gripping position in the selected gripable area that can maintain the posture or activity of the object in a specific state.

[0023] (4) Furthermore, in a gripping position determination device of an embodiment of the present invention, the friction force distribution calculation unit may calculate the force that the robotic arm can exert, and infer the friction force by solving the balance equation between the calculated force that the robotic arm can exert and the predicted external force when gripping the object.

[0024] (5) Furthermore, in a grip position determination device of an embodiment of the present invention, when the object is gripped at the grip position, the grip position calculation unit calculates the state of each joint of the fingers that are gripping.

[0025] (6) Furthermore, in a grip position determination device of an embodiment of the present invention, the grip position calculation unit may take the initial grip position as a starting point and search for the most stable grip position among discrete positions from the starting point to the end point at a predetermined position of the gripable area.

[0026] (7) Furthermore, in a grip position determination device of an embodiment of the present invention, the grip position calculation unit may take the initial grip position as a starting point and search for the position that can be gripped most stably in each position separated from the starting point by a predetermined interval.

[0027] (8) In order to achieve the above objective, a gripping position determination system according to an embodiment of the present invention includes: the gripping position determination device according to any one of (1) to (7); a robotic hand having multiple fingers with multiple joints; an environmental sensor disposed in the robot including the robotic hand or the surrounding environment of the robot to detect the robot's environmental sensor value; and a control unit that controls the activity of the robotic hand to grip the object based on the gripping position determined by the gripping position determination device.

[0028] (9) In order to achieve the above objective, a gripping position determination method according to an embodiment of the present invention is a gripping position determination method for a multi-joint, multi-finger robotic hand, wherein the friction force distribution calculation unit estimates the friction force between one of the gripping fingers and the object based on the predicted gripping force when at least two of the multi-finger fingers are used to grip the object, calculates the friction force distribution on the object surface that allows the object to be gripped based on the friction force-related value calculated using the estimated friction force, selects at least one gripping area based on the friction force distribution, and calculates the gripping position calculation unit from the selected gripping area that allows the object to be gripped stably.

[0029] (10) In order to achieve the above objective, a recording medium of an embodiment of the present invention stores a program that enables a computer of a gripping position determination device for a multi-jointed, multi-finger robotic hand to infer the frictional force between one of the gripping fingers and the object based on a predicted control of the gripping force when at least two of the multi-finger fingers are used to grip the object, calculate the frictional force distribution on the surface of the object that allows the object to be gripped based on a frictional force-related value calculated using the inferred frictional force, select at least one grippable area based on the frictional force distribution, and calculate a gripping position that allows the object to be gripped stably from the selected grippable area.

[0030] [The effects of the invention]

[0031] According to (1) to (10), the grip point that takes into account operation can be determined. Moreover, according to (1) to (10), even if there are errors in shape or control, a robust grip position can be determined.

[0032] Moreover, according to (3), it is possible to determine more specifically the positions that are difficult to slide.

[0033] Moreover, according to (5), the robotic arm can be made to hold the gripping point with high precision, taking into account the operation.

[0034] Moreover, according to (6) and (7), it is possible to reduce the computational cost of determining the grip point that takes into account the operation. Attached Figure Description

[0035] Figure 1 This is a diagram illustrating an example of a robot's gripping part using fingers to hold an object in an embodiment.

[0036] Figure 2 This is a block diagram illustrating a structural example of the grip position determination system in the first embodiment.

[0037] Figure 3 This diagram illustrates the grip position determination process in the first embodiment.

[0038] Figure 4 This is a flowchart of the process for determining the gripping position in the first embodiment.

[0039] Figure 5 This is a diagram used to illustrate the method for calculating the L2 norm in the implementation method.

[0040] Figure 6 This is a diagram illustrating the initial grip position, friction distribution, and corrected position when holding a smartphone with curved edges.

[0041] Figure 7 It is a map of the search from the initial gripping position to the corrected position.

[0042] Figure 8 This is a diagram showing an example of a track and waypoint.

[0043] Figure 9 This is a block diagram illustrating a structural example of the grip position determination system in the second embodiment.

[0044] Figure 10 This is a diagram used to illustrate an example of determining the gripping position in the second embodiment.

[0045] Figure 11 This is a diagram illustrating an example of the friction force distribution in the second embodiment.

[0046] Figure 12 This is a flowchart of the process for determining the gripping position in the second embodiment.

[0047] Figure 13 This diagram illustrates an example of how robots use two fingers to grasp objects in conventional technology.

[0048] Figure 14 This diagram illustrates an example of how robots use three fingers to grasp objects in conventional technology.

[0049] Figure 15 This is a diagram illustrating an example of the index ε used in conventional technology when a robot uses two fingers to grasp an object with a gap of 0.0 (m) between the fingers.

[0050] Figure 16 This is a diagram illustrating an example of the index ε used in conventional technology when a robot uses two fingers to grasp an object with a gap of 0.0015 (m) between the fingers.

[0051] [Explanation of Symbols]

[0052] 1.1A: Grip Position Determination System

[0053] 2: Robot

[0054] 3: Environmental sensors

[0055] 21, 21A: Grip position determining device

[0056] 23: Control Department

[0057] 24: Storage Department

[0058] 25: Hand

[0059] 211: Information Acquisition Department

[0060] 212: Object Speculation Section

[0061] 213, 213A: Friction Force Distribution Calculation Section

[0062] 214, 214A: Grip area selection section

[0063] 215: Grip Position Calculation Unit

[0064] 26: Sensors

[0065] 27: Drive Unit

[0066] 28: Grip section

[0067] 281: Finger part

[0068] 281a: Thumb

[0069] 282b: Index finger

[0070] 282c: Middle finger

[0071] 282d: Ring finger

[0072] 282e: little finger

[0073] 31: Filming device

[0074] 32: Sensor

[0075] 33: Object Position Detection Unit

[0076] 34: Ministry of Communications Detailed Implementation

[0077] The following is a reference to the appendix. Figure 1 The embodiments of the present invention will be described below. Additionally, the scale of each component in the accompanying drawings used in the following description has been appropriately altered to make each component a recognizable size.

[0078] [summary]

[0079] Figure 1 This is a diagram illustrating an example of how the gripping part 28 of a robot in an embodiment uses the finger part 281 to grip the object Obj.

[0080] The robot includes a gripping part 28, which includes at least two fingers 281. In this embodiment, when the robot's gripping part 28 grips an object Obj, the magnitude of the frictional force at the gripping position is directly incorporated into an evaluation formula to assess the error of the gripping position during actual gripping. Thus, even if there are errors in shape or control, a stable gripping position can be determined.

[0081] Furthermore, the robot selects the gripping area based on friction and gravity, and infers the position and posture of the gripping part (the posture of the fingers or hand joints) that can be stably gripped within the gripping area.

[0082] <First Implementation>

[0083] Figure 1 This is a block diagram illustrating a structural example of the grip position determination system in this embodiment. For example... Figure 2 As shown, the grip position determination system 1 includes a robot 2 and an environmental sensor 3.

[0084] Robot 2 includes a gripping position determination device 21, a control unit 23, a storage unit 24, and a hand 25 (mechanical arm). Additionally, robot 2 may also include legs, a head, a torso, a waist, etc., besides the hand 25. Furthermore, robot 2 includes a power source (not shown). The power source supplies power to all parts of robot 2. The power source may include, for example, a rechargeable battery or a charging circuit.

[0085] The grip position determination device 21 includes an information acquisition unit 211, an object estimation unit 212, a friction force distribution calculation unit 213, a gripable area selection unit 214, and a grip position calculation unit 215.

[0086] The hand 25 includes a sensor 26, a drive unit 27, and a grip unit 28.

[0087] The gripping part 28 includes a finger part 281. In addition, the finger part 281 includes at least two fingers. Figure 1 In the example, the finger portion 281 includes five fingers (thumb 281a, index finger 282b, middle finger 282c, ring finger 282d, little finger 282e). The hand 25 of this embodiment includes multiple fingers with multiple joints.

[0088] The environmental sensor 3 includes an imaging device 31, a sensor 32, an object position detection unit 33, and a communication unit 34.

[0089] Additionally, robot 2 and environmental sensor 3 can be connected, for example, via a wireless or wired network. Robot 2 and environmental sensor 3 can also be connected directly without a network.

[0090] [Functions of the grip control system]

[0091] Environmental sensor 3 is positioned, for example, at a location capable of capturing and detecting the object being manipulated. Furthermore, environmental sensor 3 can be included in robot 2 or mounted on robot 2. Alternatively, multiple environmental sensors 3 can be used, and they can be positioned in the working environment and also mounted on robot 2. Environmental sensor 3 detects the object's position information based on the captured image and the detection results obtained by the sensor, and sends the detected object position information (environmental sensor value) to robot 2.

[0092] The imaging device 31 is, for example, a red-green-blue (RGB) camera. The imaging device 31 outputs the captured image to the object position detection unit 33. In addition, the positional relationship between the imaging device 31 and the sensor 32 in the environmental sensor 3 is known.

[0093] Sensor 32 is, for example, a depth sensor. Sensor 72 outputs the detection result to object position detection unit 33.

[0094] The object position detection unit 33 detects the three-dimensional position, size, and shape of objects in the captured image using well-known methods, based on the captured image and the detection results obtained by the sensor. Referring to pattern matching models stored in the object position detection unit 33, the object position detection unit 33 performs image processing (edge ​​detection, binarization, feature extraction, image enhancement, image extraction, pattern matching, etc.) on the image captured by the imaging device 31 to infer the object's position. Furthermore, when multiple objects are detected in the captured image, the object position detection unit 33 detects the position of each object. The object position detection unit 33 transmits the detected object position information (environment sensor values) to the robot remote operation control device 30 via the communication unit 34.

[0095] The communication unit 34 sends the object's position information to robot 2.

[0096] Robot 2 controls the gripping based on the gripping position information determined by gripping position determination device 21.

[0097] The control unit 23 controls the drive unit 27 based on the grip position information output by the grip position determination device 21.

[0098] The storage unit 24 stores, for example, the control unit 23, the program, threshold, etc. used for control.

[0099] Sensor 26 may be, for example, an encoder for each joint. Furthermore, sensor 26 may be mounted on each joint of robot 2. Sensor 26 outputs the detected results to grip position determination device 21 and control unit 23.

[0100] The drive unit 27 drives the gripping part 28 of the robot 2 under the control of the control unit 23. The drive unit 27 may include, for example, an actuator, gears, artificial muscles, etc.

[0101] The information acquisition unit 211 acquires object position information (environment sensor value) from the environmental sensor 3. The information acquisition unit 211 outputs the acquired object position information to the object estimation unit 212 and the control unit 23.

[0102] The object estimation unit 212 uses object position information and well-known methods to estimate the position, size, shape, tilt, etc. of the held object. In addition, the object estimation unit 212 can also estimate the type or name of the object by using methods such as pattern matching.

[0103] The friction force distribution calculation unit 213 calculates the friction force distribution of at least one finger based on the external force (including gravity) acting on the object, the position of the contact point between the object Obj and a representative part of the finger (e.g., the tip or pad of the finger) in the object coordinate system, and the direction (including the number) of each contact point on the object Obj in the object coordinate system. In other words, the friction force distribution calculation unit 213 calculates the force that should be applied at each contact point within the range of the control target value required for the robot 2 to hold the object Obj relative to the external force. The friction force distribution calculation unit 213 infers the lateral force (friction) within a predetermined range from the initial gripping position, and uses the inferred lateral force (friction) to calculate the value related to the friction force, thereby calculating the friction force distribution on the object surface. In other words, the friction force distribution calculation unit 213 infers the friction force between the object and the finger based on the predictive control of the gripping force.

[0104] Additionally, a value related to friction is, for example, the square root of the sum of the squares of the predicted friction forces, i.e., the L2 norm (=√(Fx)). 2 +Fy 2 )).

[0105] Alternatively, the value related to friction is, for example, the sum of the squares of the estimated friction forces (Fx). 2 +Fy 2 When the estimated frictional force is small, if the square root is not taken, the variation can sometimes be large but easy to assess.

[0106] Alternatively, the value related to friction is, for example, the sum of the squares of the estimated frictional forces divided by the value of the normal force ([Fx)). 2 +Fy 2 ] / Fz 2 Frictional force varies with Fz. For example, if Fz is large, the sum of the squares of the frictional forces tends to be large as well. Thus, the calculated values ​​related to frictional force can vary greatly and are easy to assess.

[0107] Furthermore, in the following implementation methods, the use of the L2 norm is illustrated as an example of a value related to friction, but it is not limited to this.

[0108] The gripable area selection unit 214 selects at least one gripable area within the region of frictional force distribution based on the L2 norm. For example, the gripable area selection unit 214 selects areas with an L2 norm below a predetermined value as gripable areas within the region of frictional force distribution.

[0109] The grip position calculation unit 215 searches the grip area and calculates the grip position (the contact point of the fingers) that best resists external forces. In addition, the grip position calculated by the grip position calculation unit 215 is the position where the friction between the object and the fingers is minimal.

[0110] [Holding position determines processing example]

[0111] Next, the grip position determination process of this embodiment will be explained. In this embodiment, the grip position determination device 21 determines the grip position of the finger portion 281 of the grip portion 28 before gripping the object.

[0112] Figure 3 This diagram illustrates the grip position determination process in this embodiment. Figure 3 The example shown is of holding object Obj using three fingers (thumb 281a, index finger 281b, and middle finger 281c). Furthermore, Figure 3 The example is an example of predicting and determining a grip position that further reduces friction and makes the grip state more stable, while keeping the grip positions of the index finger 281b and middle finger 281c unchanged and moving the thumb 281a.

[0113] Arrow g11 represents the force exerted by the index finger 282b on the surface of object Obj. Arrow g12 represents the force exerted by the middle finger 282c on the surface of object Obj. Arrow g13 represents the force exerted by the thumb 282a in its initial position on the surface of object Obj.

[0114] Arrow g21 represents the z-axis component of the action of the thumb 282a at the initial position. Arrow g22 represents the x-axis component of the action of the thumb 282a at the initial position. Arrow g23 represents the y-axis component of the action of the thumb 282a at the initial position. Furthermore, x, y, and z are the coordinates in the contact coordinate system.

[0115] Figure 3 In the diagram, each point g32 represents the L2 norm, with higher values ​​indicating denser areas and lower values ​​indicating lighter areas. This L2 norm distribution corresponds to the frictional force distribution g31. Furthermore, the region enclosed by the chain line g51 represents the grippable area.

[0116] Figure 3 In the case of holding object Obj, the gripping position of the thumb 281a, which is in a stable gripping state, is not the initial position but a corrected position g42 with the minimum L2 norm. Arrow g42 represents the force exerted by the thumb 282a at the corrected position on the surface of object Obj.

[0117] Here, the description of the prior art is used as a reference. Figure 15 and Figure 16 This explains why, when holding the object with fingers facing each other while maintaining the object's posture, the grip position calculated by the grip position calculation unit 215 is the position where the friction between the object and the fingers is minimized.

[0118] When using two fingers to hold a coin-shaped object, such as Figure 15 As shown, when the two fingers of a hand are facing each other across an object, they can be held together with minimal friction. On the other hand, as... Figure 16 As shown, when the two fingers of a grasp are facing each other without being separated by an object and there is a gap between them, if a ratio is not used... Figure 15 Too much friction makes it impossible to hold.

[0119] Figure 15 Charts g923 and g924 Figure 16 In charts g933 and g934, the size of the V indicates the limit it can achieve. Furthermore, the lines shown between the lines on both sides of the V (g925, g926, g935, g936) represent the forces the robot can exert.

[0120] Figure 15 In charts g923 and g924, the lines g925 and g926, which represent the forces that can be exerted, are located approximately in the middle of the two lines of the V.

[0121] In contrast, Figure 16 In chart g933, the line g935 representing the force that can be exerted roughly coincides with the line on the right side of the V, indicating that the force to be exerted must reach its limit. Furthermore, in chart g934, the line g936 representing the force that can be exerted is close to the line on the right side of the V, indicating that the force to be exerted must reach its limit.

[0122] In addition, the angle difference between the lines (g925, g926, g935, g936) representing the forces that can be exerted and the center of the V is equivalent to the lateral force.

[0123] Thus, when holding the object with fingers facing each other while maintaining its posture, the position with less friction becomes the gripping position.

[0124] [Example of the process for determining the grip position]

[0125] The following example illustrates the process for determining the grip position. Figure 4 This is a flowchart of the process for determining the grip position in this embodiment.

[0126] (Step S1) The information acquisition unit 211 acquires object position information (environment sensor value) from the environmental sensor 3.

[0127] (Step S2) The object estimation unit 212 uses the object position information and well-known methods to estimate the position, size, shape, tilt, etc. of the object being held.

[0128] (Step S3) The friction distribution calculation unit 213 predicts the contact points between the object and each finger when holding the object based on the inferred information related to the object.

[0129] (Step S4) The friction force distribution calculation unit 213 calculates the force that can be used when holding the object based on the specifications of the robot 2 and the inferred information related to the object.

[0130] (Step S5) The friction force distribution calculation unit 213 calculates the equilibrium equation G for a single finger within a specified range from the initial gripping position. c f c =-F e Solve for the lateral force of each candidate grip position obtained during the solution process.

[0131] (Step S6) The friction force distribution calculation unit 213 calculates the square root of the sum of squares of the obtained transverse forces, i.e., the L2 norm, and thus calculates the friction force distribution on the surface of the object.

[0132] (Step S7) The gripable area selection unit 214 selects, for example, an area where the L2 norm is below a specified value within the area of ​​friction distribution as a gripable area.

[0133] (Step S8) The grip position calculation unit 215 searches the grip area and calculates the grip position that can best resist external forces and information such as the angle of the finger joints.

[0134] (Step S9) The control unit 23 generates control commands based on the calculated grip position and the angle of the finger joints.

[0135] (Step S10) Based on the generated control commands, the control unit 23 controls the drive unit 27 to drive the gripping part 28 of the robot 2.

[0136] Additionally, using Figure 4 The processing flow described is an example and is not limited to this.

[0137] [Methods for calculating L2 norm]

[0138] Here, we will further explain how to calculate the L2 norm.

[0139] Figure 5 This is a diagram used to illustrate the method for calculating the L2 norm in the implementation method. Figure 5 The example is an example of using three fingers to hold a spherical object.

[0140] Friction distribution calculation unit 213 first calculates the force G that can be exerted when gripping. c f c .

[0141] Next, the friction force distribution calculation unit 213 calculates the equilibrium equation G for a finger, within a specified range from the contact point. c fc =-F e Solve to obtain the lateral force at each contact point obtained during the solution process. Additionally, F e This refers to external forces. Furthermore, the so-called lateral force refers to the force parallel to the surface of object Obj, which is the force in the lateral direction (x-axis, y-axis) of the contact point coordinate system. The sum of these lateral forces is the L2 norm.

[0142] (Modified example)

[0143] Here, use Figures 6-8 This example illustrates how to use information related to the shape of an object to search for the corrected grip position (corrected position).

[0144] Figure 6 This is a diagram illustrating the initial grip position, friction distribution, and corrected position when holding a smartphone with curved edges. Figure 6 An example is shown below: When robot 2 is holding a smartphone using its thumb 281a, index finger 282b, and middle finger 282c, the gripping position of thumb 281a is adjusted to achieve a more stable grip. Arrow g101 represents the force exerted by index finger 282b on the surface of object Obj. Arrow g102 represents the force exerted by middle finger 282c on the surface of object Obj. Arrow g103 represents the force exerted by thumb 282a in its initial position on the surface of object Obj.

[0145] At this time, the grip force distribution g103 is, for example, a defined range centered on the thumb 282a in the initial position, extending to the left, right, and upper limit of the side of the smartphone. Furthermore, the grip position calculation unit 215 searches for a corrected position g121 based on the L2 norm within the grippable area g113.

[0146] Figure 7 This is a mapping of the search from the initial gripping position to the corrected position. In this embodiment, instead of actually gripping the object and moving the fingers on the surface of the object as in conventional techniques, a search is performed from the initial position g151 in a direction that improves cost (smaller L2 norm) (search direction g153) through calculation. Furthermore, the dot matrix g153 represents the trajectory of the fingers as they move due to the search. Point g161 is a waypoint. The trajectory is, for example, the path that the tip of the finger should traverse from the initial position (starting point g151) to the ending point g152. The gripping position calculation unit 215 solves the equation of motion (equilibrium equation) of the fingertip contact point within the waypoints of this search path (via position) to calculate the optimal gripping position.

[0147] Furthermore, the search path for waypoints is set to bypass locations on the object that the fingers must avoid (such as locations with protrusions). Moreover, waypoints and paths can be set sequentially, for example, by moving the candidate gripping position by a predetermined amount each time. At this point, the initial search position is determined, but the endpoint is not yet determined.

[0148] Furthermore, the spacing between waypoints is preferably equal, but it does not have to be equal.

[0149] According to this embodiment, by performing the search in this way, the computational cost of determining the grip point that takes into account the operation can be reduced.

[0150] Figure 8 This is a diagram showing an example of a track and waypoints. Additionally, Figure 8 This is an example of using the tips of your fingers to grasp the surface of an object. Figure 8 In the model, model g201 represents a model of a hand 25 containing one finger. The finger is modeled using a first joint g202 connecting the base g203 to the palm and a second joint g204 connecting the distal phalanx g205 to the base g203.

[0151] During the search, the grip position calculation unit 215 defines discrete waypoints (e.g., point g213) along the trajectory that the fingertip should traverse, from the initial position (starting point g211) to the ending point g212. Additionally, as... Figure 7 As shown, the angle of the finger joints (and consequently, the angle of the joints of the hand 25) changes depending on where the finger is used to touch the object (the tip, the fingertip, etc.). Therefore, while searching for the grip position, the grip position calculation unit 215 also calculates the angles of the joints of the hand 25 and the finger joints, and outputs the results to the control unit 23. Furthermore, the number of waypoints can be, for example, 100, but can be less or more than 100. This limits the path that the fingertip must traverse from the initial contact point to the corrected position for adjusting the grip position within the grippable area.

[0152] Furthermore, in this embodiment, discrete key points can be defined on the track. By using waypoints, the motion equations of the fingertip grip position (the motion equations related to the contact between the curved surface and the surface of the object relative to a certain curved surface from the tip of the fingertip to the fingertip) are searched in the defined solution space to evaluate whether the sum of the frictional forces of all fingers is within an acceptable range when the fingertip contact point (e.g., the tip or the fingertip) is changed. Control is performed while correcting the contact point of the fingertip tip, which can be controlled within an appropriate range.

[0153] Furthermore, in this embodiment, an evaluation index ε is incorporated into the equation of motion, and the gripping area is determined from the friction force distribution based on the results of the obtained equilibrium equation, and the optimal position within the gripping area is determined.

[0154] As described above, in this embodiment, the predicted frictional force is obtained based on the predictive control of the gripping force, and the frictional force distribution of a specific finger is calculated on the surface of the object based on the L2 norm of the frictional force of all the fingers obtained. Furthermore, in this embodiment, at least one grippable area is determined based on the frictional force distribution, and the contact point that best resists external forces is calculated from the grippable area. Moreover, in this embodiment, the frictional force is not measured by actual contact, but rather by determining the gripping position that can stably maintain the state of the object.

[0155] Therefore, according to this embodiment, when operating an object, if the fingertip shape is curved, the entire trajectory of the grip position moving along the curved surface is evaluated, thereby enabling the determination of the grip point that takes into account the operation.

[0156] Furthermore, in the method described in prior art patent document 2, when the surface of the object is curved, even if the finger in contact with the object is moved, it is impossible to obtain adequate friction. In contrast, in this embodiment, instead of actually moving the finger in contact with the object while obtaining friction, the friction of the already gripped state is predicted, and the contact point that best resists external forces is calculated. Therefore, the gripping position that can be stably gripped can be appropriately determined.

[0157] <Second Implementation>

[0158] In this embodiment, in order to further maintain the posture or movement of the object in a specific state, a gripping position reflecting specific conditions (such as gravity) is calculated. At this time, the gripping position is not necessarily limited to the position with the least friction. In this embodiment, it also includes states that can be gripped when gravitational balance is achieved, and states that can be gripped even when changes such as tilting the object's posture are made.

[0159] Figure 9 This is a block diagram illustrating a structural example of the grip position determination system in this embodiment. For example... Figure 9 As shown, the grip position determination system 1A includes a robot 2A and an environmental sensor 3.

[0160] The robot 2A includes a gripping position determination device 21A, a control unit 23, a storage unit 24, and a hand 25.

[0161] The grip position determination device 21A includes an information acquisition unit 211, an object estimation unit 212, a friction force distribution calculation unit 213A, a gripable area selection unit 214A, and a grip position calculation unit 215.

[0162] The friction distribution calculation unit 213A determines whether there is a grippable area that can be held in a specific state based on the constraints.

[0163] The grippable area selection unit 214A selects at least one grippable area within the area of ​​frictional force distribution based on constraints and the L2 norm. Constraints include, for example, the force that the hand 25 can exert, the force exerted by each finger in an equal and minimal manner, and whether a posture is maintained.

[0164] Figure 10 This is a diagram used to illustrate an example of determining the grip position in this embodiment. Figure 10 In the text, object Obj is a cylindrical object. Furthermore, Figure 10 The example is one that uses four fingers (thumb, index finger, middle finger, and ring finger) to hold from above.

[0165] Position g301 is the gripping position of the index finger 282b. Position g302 is the gripping position of the middle finger 282c. Position g303 is the gripping position of the ring finger 282d. Position g304 is the initial gripping position of the thumb 282a. Additionally, each arrow indicates the force applied to object Obj when gripping with each finger.

[0166] Here, in a manner that does not disrupt the gravitational balance of object Obj, i.e., the object does not tilt, the initial gripping position of the thumb 282a is opposite to the gripping positions of the index finger 282b, middle finger 282c, and ring finger 282d. At this time, the friction force distribution calculation unit 213 calculates the gripping force distribution g311 within a predetermined range in the xy direction, i.e., the circumferential direction, of object Obj. Furthermore, the gripping position determination device 21A calculates a corrected gripping position g312 to determine a position that can be stably maintained.

[0167] Next, while holding the object in a manner that could also disrupt the gravitational balance of the object Obj, i.e., the object could also be tilted, the grip position determination device 21A searches on the surface of the object Obj (search path g321) for the position where the L2 norm is minimized, and calculates the grip position g322, which can be held stably by including a specified range in the z-axis direction of the object Obj.

[0168] [Example of friction force distribution]

[0169] Next, an example of the friction force distribution in this embodiment will be described.

[0170] Figure 11 This is a diagram illustrating an example of the friction force distribution in this embodiment. Figure 11 In the text, object Obj is a spherical object. Furthermore, Figure 11The example is a grip using two fingers (thumb and index finger) from the side.

[0171] Position g401 represents the gripping position of the index finger 282b. Position 402 represents the initial gripping position of the thumb 282a. Additionally, each arrow indicates the force exerted on object Obj when gripping with each finger.

[0172] At this point, the frictional force distribution g411 is distributed not only in the xy direction of object Obj, but also in the z-axis direction.

[0173] The first grippable region g421 is an L2-norm-based region. Within the first grippable region g421, a stable gripping position is designated as position g422. Position g422 is the position facing the index finger, chosen, for example, when the influence of gravity is weak.

[0174] The second grippable region g431 is the region where the ability to hold the object stably under gravity is also taken into account as a limiting condition. Within the second grippable region g431, the gripping position where stable holding is possible is position g432. Position g432 is a position slightly lower than position g422 that is also stable in the direction of gravity, for example, when variations in the object's posture are permissible.

[0175] Additionally, as indicated by arrow g441, the area that can be held is defined within the range of the holdable area.

[0176] In this embodiment, at least one grippable area is determined based on the friction distribution, and the contact point that best resists gravity or external forces is calculated from it. Furthermore, in this embodiment, starting from at least one grippable area, the initial gripping position of one finger is moved to search for and correct positions where all fingers will generate friction against the object in the lateral direction as little as possible.

[0177] [Example of the process for determining the grip position]

[0178] The following example illustrates the process for determining the grip position. Figure 12 This is a flowchart of the process for determining the grip position in this embodiment.

[0179] (Steps S1 to S5) The grip position determination device 21A performs the processing of steps S1 to S5.

[0180] (Step S101) The friction force distribution calculation unit 213 calculates the square root of the sum of squares of the obtained transverse forces, i.e., the L2 norm, and thus calculates the friction force distribution on the surface of the object.

[0181] (Steps S7 to S8) The grip position determination device 21A performs the processing of steps S7 to S8.

[0182] (Step S102) The friction distribution calculation unit 213A determines, based on the constraints, whether there exists a grippable area that can be held in a specific state. If the friction distribution calculation unit 213A determines that a grippable area exists (Step S102; Yes), it proceeds to step S103. If the friction distribution calculation unit 213A determines that there is no grippable area that can be held in a specific state (Step S102; No), it proceeds to step S104.

[0183] (Step S103) The grip position calculation unit 215 searches for a grip area and calculates other grip positions that can best resist external forces, as well as information such as the angle of the finger joints.

[0184] (Steps S9 to S10) The control unit 23 performs the processing of steps S9 to S10.

[0185] In addition, the friction distribution calculation unit 213A performs steps S102 to S103 for the griptable area that can be held in all specific states.

[0186] Thus, in this embodiment, when there are multiple gripping methods that balance external force or gravity, the gripping position is calculated for each selected gripping area by selecting one of the multiple gripping areas sequentially. The gripping position calculation unit 215 selects one gripping position from the multiple gripping positions calculated in this way, based on the state of the object or limiting conditions, etc.

[0187] Therefore, multiple grip positions that can be stably held can be determined. Thus, according to this embodiment, the evaluation taking into account the error of the grip position can be considered as an expected value taking into account the probability distribution. By treating the shape elements on the fingertip side or the wear of the skin as a distribution, multiple factors can be considered and the evaluation can be performed without tuning their weighting parameters.

[0188] In addition, regarding the two or more gripping positions, when the posture of the object changes (e.g., during transport), the gripping position selected from the calculated multiple gripping positions can be switched according to the posture of the object.

[0189] When manipulating an object, if the fingertip shape is curved, the grip point moves while rolling. As described above, according to the first and second embodiments, by evaluating the entire trajectory, it is possible to determine the grip point taking into account the operation. Furthermore, the evaluation considering the error in the grip position can be considered as an expected value taking into account the probability distribution. Therefore, as described above, according to the first and second embodiments, by treating the shape elements or skin wear on the fingertip side as a distribution, it is possible to consider multiple factors and perform evaluation without tuning their weighting parameters.

[0190] Furthermore, according to the first and second embodiments, unlike the prior art which evaluates the gripping performance of all fingers, information on the friction of all fingers is incorporated into the evaluation of the goodness of the contact point of a single finger, and the contact point is determined on the object surface based on the distribution described above. Therefore, according to the first and second embodiments, it is possible to more specifically determine locations where slippage is unlikely.

[0191] Furthermore, according to the first and second embodiments, by evaluating the error of the gripping position when actually gripping in the direction that makes it easy to stay inside the gripable ellipse, a stable gripping position can be determined even if there are errors in shape or control.

[0192] Alternatively, all or part of the program used to implement the functions of the grip position determination device 21 (or 21A) in this invention can be recorded on a computer-readable recording medium, allowing the computer system to read and execute the program recorded on the recording medium, thereby performing all or part of the processing performed by the grip position determination device 21 (or 21A). Furthermore, the term "computer system" as used herein includes hardware such as an operating system (OS) or peripheral machines. Moreover, "computer system" also includes systems built on a local area network or systems built in the cloud. Furthermore, the term "computer-readable recording medium" refers to removable media such as floppy disks, optical disks, read-only memory (ROM), and compact disc read-only memory (CD-ROM), and storage devices such as hard drives built into a computer system. Furthermore, "computer-readable recording medium" also includes servers that send programs via networks such as the Internet or communication lines such as telephone lines, or memory that retains programs for a certain period of time, such as volatile memory (random access memory) within a computer system acting as a client.

[0193] Furthermore, the program can be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium, or via transmission waves in the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium with information transmission capabilities, such as a network (communication network) like the Internet or a communication line (communication line) like a telephone line. Moreover, the program can also be a component used to implement the aforementioned functions. Furthermore, it can be a so-called differential file (differential program) capable of implementing the aforementioned functions through combination with a program already recorded in the computer system.

[0194] The above describes the specific implementation method, but the present invention is not limited to this implementation method in any way, and various modifications and substitutions can be made without departing from the spirit of the present invention.

Claims

1. A gripping position determining device, which is a gripping position determining device for a multi-joint, multi-fingered robotic hand, the gripping position determining device comprising: The friction force distribution calculation unit calculates the friction force distribution that one of the fingers can hold the object when at least two of the multiple fingers are used to hold the object, based on the predetermined external force on the object, the position of the contact point on the object with the representative part of the finger in the object coordinate system, and the direction of each position of the contact point on the object in the object coordinate system. The gripable area selection unit selects at least one gripable area based on the friction distribution. as well as The grip position calculation unit calculates the contact point of the fingers that can stably grip the object from the selected gripable area as the grip position, wherein... The friction force distribution calculation unit estimates the lateral force as friction within a specified range from the initial gripping position, and uses the estimated lateral force to calculate the value related to the friction force, thereby calculating the friction force distribution on the object surface. The values ​​associated with the frictional force include the square root of the sum of the squares of the estimated frictional forces, i.e., the L2 norm. The grippable area selection unit selects the area with an L2 norm below a predetermined value as the grippable area within the area of ​​friction distribution, and The grip position calculation unit searches the grippable area and calculates the contact point of the finger that can best resist external force as the grip position.

2. The grip position determining device according to claim 1, wherein... The gripping position calculation unit selects the gripping area that can maintain the posture or activity of the object in a specific state, and calculates the gripping position in the selected gripping area that can maintain the posture or activity of the object in a specific state.

3. The grip position determining device according to claim 1, wherein... The friction force distribution calculation unit calculates the force that the robotic arm can exert, and infers the friction force by solving the balance equation between the calculated force that the robotic arm can exert and the predicted external force when holding the object.

4. The grip position determining device according to claim 1, wherein... When the object is held in the gripping position, the gripping position calculation unit calculates the state of each joint of the fingers that are gripping.

5. The grip position determining device according to claim 1, wherein... The grip position calculation unit takes the initial grip position as the starting point and searches for the most stable grip position among discrete positions from the starting point to the end point at a predetermined position in the gripable area.

6. The grip position determining device according to claim 1, wherein... The grip position calculation unit takes the initial grip position as a starting point and searches for the most stable grip position in each position separated from the starting point by a predetermined interval.

7. A grip position determination system, comprising: The grip position determining device according to any one of claims 1 to 6; A robotic hand with multiple joints and multiple fingers; An environmental sensor is installed on the robot, including the robotic arm, or in the robot's surrounding environment to detect the robot's environmental sensor values. as well as The control unit controls the movement of the robotic arm to grip the object based on the gripping position determined by the gripping position determining device.

8. A method for determining the grip position, which is a method for determining the grip position of a multi-joint, multi-fingered robotic hand, wherein... When at least two fingers from the multiple fingers are used to grasp an object, the friction force distribution calculation unit calculates the friction force distribution that one of the fingers can use to grasp the object, based on the predetermined external force on the object, the position of the contact point between the object and the representative portion of the finger in the object coordinate system, and the direction of each position of the contact point on the object in the object coordinate system. The grippable area selection unit selects at least one grippable area based on the friction force distribution. The grip position calculation unit calculates the contact point of the fingers that can stably grip the object from the selected gripable area as the grip position, wherein... The friction force distribution calculation unit estimates the lateral force as friction within a specified range from the initial gripping position, and uses the estimated lateral force to calculate the value related to the friction force, thereby calculating the friction force distribution on the object surface. The values ​​associated with the frictional force include the square root of the sum of the squares of the estimated frictional forces, i.e., the L2 norm. The grippable area selection unit selects the area with an L2 norm below a predetermined value as the grippable area within the area of ​​friction distribution, and The grip position calculation unit searches the grippable area and calculates the contact point of the finger that can best resist external force as the grip position.

9. A recording medium storing a program that enables a computer for determining the gripping position of a multi-jointed, multi-fingered robotic hand. When using at least two fingers of the multiple fingers to grasp an object, the frictional force distribution that one of the fingers can use to grasp the object is calculated based on the predetermined external force on the object, the position of the contact point on the object with the representative portion of the finger in the object coordinate system, and the direction of each position of the contact point on the object in the object coordinate system. Select at least one grippable area based on the friction distribution. The contact point of the fingers that can stably grip the object is calculated from the selected griptable area as the gripping position, wherein... Within a specified range from the initial gripping position, a lateral force, which is considered as friction, is estimated. Using this estimated lateral force, a value related to the friction force is calculated, thereby determining the friction force distribution on the object's surface. The values ​​associated with the frictional force include the square root of the sum of the squares of the estimated frictional forces, i.e., the L2 norm. Within the area of ​​the friction distribution, the region where the L2 norm is below a predetermined value is selected as the grippable region. Search the grippable area and calculate the contact point of the finger that can best resist external force as the gripping position.

Images