Gripping force estimating device and gripping force estimating method

Abstract

A gripping force estimating device (3) estimates a gripping force of a gripping tool (1) that has a kirigami structure and that includes attachment portions (12) to which a pulling implement is attached, and two object gripping portions (13), which are parts for gripping an object (2), the gripping tool (1) deforming such that the two object gripping portions (13) approach one another when the attachment portions (12) are pulled up by the pulling implement, and the gripping force estimating device (3) being configured to comprise: a sensing data acquiring unit (31) that acquires sensing data from a sensor for observing the deformation of the gripping tool (1); and a gripping force estimating unit (32) that estimates the gripping force of the gripping tool (1) with respect to the object (2) on the basis of the sensing data acquired by the sensing data acquiring unit (31).

Description

Grip force estimation device and grip force estimation method

[0001] The present disclosure relates to a grip force estimation device and a grip force estimation method.

[0002] There is a grip force estimation device that estimates the grip force of a gripping tool that is gripping an object. For example, Patent Document 1 discloses an estimation observer that estimates the grip force of a gripping device based on the drive current of a motor that drives the gripping device and the rotation speed of the motor.

[0003] Japanese Patent Application Laid-Open No. 2002-178281

[0004] A gripping tool that is not necessarily driven by a motor includes a gripping tool with a kirigami structure. The gripping tool has an attachment portion to which a pulling tool is attached. Examples of pulling tools include a manual magic wand, tongs, or a robotic hand. The gripping tool is deformed when the attachment portion is pulled up by the pulling tool. The deformation of the gripping tool causes two object gripping sections, which are part of the gripping tool, to approach each other, thereby enabling the two object gripping sections to grasp the object. As described above, the gripping tool has a novel gripping mechanism called a kirigami structure, which differs from existing gripping mechanisms. Because the gripping tool does not necessarily need to be driven by a motor, there is a problem in that no technology exists to detect the gripping force. The gripping force estimation technology disclosed in Patent Document 1 requires the gripping device to be driven by a motor, and is therefore not suitable for detecting the gripping force of a gripping mechanism with a kirigami structure.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a gripping force estimation device that can estimate the gripping force of a gripping tool having a kirigami structure that is not necessarily driven by a motor.

[0006] The grip force estimation device according to the present disclosure estimates the grip force of a gripping tool having a kirigami structure, which includes an attachment section to which a tensile tool is attached and two object gripping sections that grip an object, and in which the attachment section is pulled up by the tensile tool, causing the two object gripping sections to deform so as to approach each other. The grip force estimation device includes a sensing data acquisition section that acquires sensing data from a sensor that observes deformation of the gripping tool, and a grip force estimation section that estimates the grip force of the gripping tool with respect to the object based on the sensing data acquired by the sensing data acquisition section.

[0007] According to the present disclosure, it is possible to estimate the gripping force of a gripping tool having a sawtooth structure that is not necessarily driven by a motor.

[0008] 11A is a configuration diagram showing a gripping tool 1 to which a gripping force estimation device 3 according to embodiment 1 is applied. FIG. 11B is an explanatory diagram showing a state in which the gripping tool 1 is gripping an object 2. FIG. 11C is a configuration diagram showing a gripping force estimation device 3 according to embodiment 1. FIG. 11B is a hardware configuration diagram showing the hardware of the gripping force estimation device 3 according to embodiment 1. FIG. 11C is a hardware configuration diagram showing the hardware of the gripping force estimation device 3 according to embodiment 1. FIG. 11A is an explanatory diagram showing a state before the gripping tool 1 grips the object 2. FIG. 11B is an explanatory diagram showing a state in which the object gripping unit 13 lightly touches the object 2. FIG. 11C is an explanatory diagram showing a state in which the gripping tool 1 has gripped the object 2. 14A is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 is in a state before gripping the object 2, FIG. 14B is an explanatory diagram showing the position of the marker 4 when the object gripping section 13 is in a state where it is lightly touching the object 2, and FIG. 14C is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 has gripped the object 2. FIG. 14B is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 has gripped the object 2. FIG. 14C is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 has gripped the object 2. FIG. 14A is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 has gripped the object 2. FIG. 14C is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 has gripped the object 2. FIG. 14B is an explanatory diagram showing the position of the marker 4 when the gripping tool 1 has gripped the object 2.

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1. Fig. 1 is a configuration diagram showing a gripping tool 1 to which a gripping force estimation device 3 according to embodiment 1 is applied. Fig. 2 is an explanatory diagram showing the gripping tool 1 gripping an object 2. The gripping tool 1 shown in Fig. 1 has a cutout portion 11, two attachment portions 12, and two object gripping portions 13. The gripping tool 1 shown in Fig. 1 has two attachment portions 12. However, this is merely an example, and the gripping tool 1 may have one attachment portion 12, or three or more attachment portions 12. The gripping tool 1 is realized by, for example, an elastic body. Examples of elastic bodies include rubber or silicone. The gripping tool 1 has a kirigami structure.

[0011] The cutout portion 11 is a cutout provided approximately in the center of the gripping tool 1 to form a kirigami structure. The two mounting portions 12 are locations to which pulling devices are attached. The pulling devices are not limited to those driven by a motor, and examples of the pulling devices include manual magic hands or tongs. The pulling devices may also be motor-driven robot hands, etc. The two object gripping portions 13 are locations that grip the object 2. By providing a cutout approximately in the center, when a pulling device is attached to each of the two mounting portions 12 and the pulling devices pull up the respective mounting portions 12, the two object gripping portions 13 approach each other. As a result, the two object gripping portions 13 are able to grip the object 2 as shown in FIG. 2 . The object 2 may be any object, but examples of the object 2 include fried chicken, fruit, electronic components, and medical instruments. 2 shows an example in which the object 2 is fried chicken, and the pulling device is a robot hand.

[0012] FIG. 3 is a configuration diagram showing the gripping force estimation device 3 according to the first embodiment. FIG. 4 is a hardware configuration diagram showing the hardware of the gripping force estimation device 3 according to the first embodiment. In FIG. 3 , the hollow body 21 is realized by, for example, an air pouch or an elastic material such as rubber. The hollow body 21 is attached to the object gripping portion 13, which is one of the multiple portions of the gripping tool 1 and which grips the object 2. The internal pressure of the hollow body 21 changes with the deformation of the gripping tool 1. Specifically, the internal pressure of the hollow body 21 when the tensioning tool is pulling up the attachment portion 12 is higher than the internal pressure of the hollow body 21 before the tensioning tool pulled up the attachment portion 12. Furthermore, the closer the tensioning tool moves the two attachment portions 12 to each other, the higher the internal pressure of the hollow body 21. One end of the hollow body 21 is closed, and the other end of the hollow body 21 is connected to one end of the tube 22. 3, the hollow body 21 is attached so as to cover almost the entire object gripping portion 13. This is because if the hollow body 21 is attached so as to cover only a small part of the object gripping portion 13, the internal pressure of the hollow body 21 does not change significantly even if the gripping tool 1 is deformed.

[0013] The tube 22 is realized by, for example, rubber piping. One end of the tube 22 is connected to the other end of the hollow body 21. The other end of the tube 22 is connected to a pressure sensor 23. When the internal pressure of the hollow body 21 increases, air that was present inside the hollow body 21 enters the inside of the tube 22, and the internal pressure of the tube 22 increases.

[0014] The pressure sensor 23 is a sensor that observes the deformation of the gripping tool 1 that accompanies the gripping of the object 2 by the gripping tool 1. Specifically, the pressure sensor 23 is a sensor that observes the internal pressure of the hollow body 21 that changes in accordance with the deformation of the gripping tool 1. The pressure sensor 23 outputs sensing data indicating the internal pressure of the hollow body 21 to the gripping force estimation device 3.

[0015] The grip force estimation device 3 shown in Fig. 3 includes a sensing data acquisition unit 31 and a grip force estimation unit 32. The sensing data acquisition unit 31 is realized by, for example, a sensing data acquisition circuit 41 shown in Fig. 4. The sensing data acquisition unit 31 acquires sensing data from the pressure sensor 23. The sensing data acquisition unit 31 outputs the sensing data from the pressure sensor 23 to the grip force estimation unit 32.

[0016] The grip force estimation unit 32 is realized, for example, by a grip force estimation circuit 42 shown in FIG. 4 . The grip force estimation unit 32 acquires sensing data of the pressure sensor 23 from the sensing data acquisition unit 31. The grip force estimation unit 32 estimates the grip force of the gripping tool 1 on the object 2 based on the sensing data of the pressure sensor 23. Specifically, a table indicating the correspondence between the internal pressure of the hollow body 21 and the grip force of the gripping tool 1 on the object 2 is stored in the internal memory of the grip force estimation unit 32. The grip force estimation unit 32 estimates the grip force of the gripping tool 1 by acquiring information indicating the grip force of the gripping tool 1 corresponding to the internal pressure indicated by the sensing data from the table. The grip force estimation unit 32 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot. In the grip force estimation device 3 shown in FIG. 3 , the table is stored in the internal memory of the grip force estimation unit 32. However, this is merely an example, and the table may be stored in an external memory of the grip force estimating unit 32 .

[0017] 3, it is assumed that the sensing data acquisition unit 31 and the grip force estimation unit 32, which are components of the grip force estimation device 3, are each realized by dedicated hardware as shown in FIG. 4. That is, it is assumed that the grip force estimation device 3 is realized by a sensing data acquisition circuit 41 and a grip force estimation circuit 42. Each of the sensing data acquisition circuit 41 and the grip force estimation circuit 42 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0018] The components of the grip force estimation device 3 are not limited to those realized by dedicated hardware, and the grip force estimation device 3 may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the memory of a computer. The computer refers to hardware that executes a program, and includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0019] 5 is a hardware configuration diagram of a computer when the grip force estimation device 3 is realized by software, firmware, etc. When the grip force estimation device 3 is realized by software, firmware, etc., a program for causing a computer to execute the respective processing procedures of the sensing data acquisition unit 31 and the grip force estimation unit 32 is stored in a memory 51. A processor 52 of the computer then executes the program stored in the memory 51.

[0020] 4 shows an example in which each of the components of the grip force estimation device 3 is realized by dedicated hardware, and FIG. 5 shows an example in which the grip force estimation device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the grip force estimation device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0021] Next, the operation of the gripping force estimation device 3 shown in Fig. 3 will be described. Fig. 6 is a flowchart showing a gripping force estimation method, which is a processing procedure of the gripping force estimation device 3. When the tensile tool pulls up the two attachment parts 12 of the gripping tool 1, the two object gripping parts 13 move closer to each other. As a result, the two object gripping parts 13 become able to grip the object 2, as shown in Fig. 2.

[0022] The hollow body 21 is deformed by the pulling tool pulling up the attachment parts 12. As the hollow body 21 deforms, the internal pressure of the hollow body 21 increases. In other words, as the pulling tool pulls up the attachment parts 12, the internal pressure of the hollow body 21 increases. The closer the pulling tool moves the two attachment parts 12 to each other, the higher the internal pressure of the hollow body 21 becomes. As the internal pressure of the hollow body 21 increases, air that was present inside the hollow body 21 enters the inside of the tube 22, and the internal pressure of the tube 22 increases.

[0023] The pressure sensor 23 observes the internal pressure of the tube 22 as the internal pressure of the hollow body 21. The pressure sensor 23 outputs sensing data indicating the internal pressure of the hollow body 21 to the gripping force estimation device 3.

[0024] The sensing data acquisition unit 31 of the gripping force estimation device 3 acquires sensing data from the pressure sensor 23 (step ST1 in FIG. 6 ). The sensing data acquisition unit 31 outputs the sensing data from the pressure sensor 23 to the gripping force estimation unit 32.

[0025] The gripping force estimating unit 32 acquires sensing data of the pressure sensor 23 from the sensing data acquiring unit 31. The gripping force estimating unit 32 estimates the gripping force of the gripping tool 1 by acquiring, from a table stored in an internal memory, information indicating the gripping force of the gripping tool 1 corresponding to the internal pressure of the hollow body 21 indicated by the sensing data (step ST2 in FIG. 6 ). The gripping force estimating unit 32 outputs the information indicating the gripping force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0026] In the first embodiment described above, the gripping force estimation device 3 is configured to estimate the gripping force of a gripping tool 1 having a kirigami structure, which includes an attachment portion 12 to which a pulling tool is attached and two object gripping portions 13 that grip an object 2, and in which the two object gripping portions 13 are deformed so as to approach each other when the attachment portion 12 is pulled up by the pulling tool. The gripping force estimation device 3 includes a sensing data acquisition unit 31 that acquires sensing data from a sensor that observes the deformation of the gripping tool 1, and a gripping force estimation unit 32 that estimates the gripping force of the gripping tool 1 with respect to the object 2 based on the sensing data acquired by the sensing data acquisition unit 31. Therefore, the gripping force estimation device 3 can estimate the gripping force of a gripping tool 1 having a kirigami structure that is not necessarily driven by a motor.

[0027] The grip force estimation device 3 shown in Fig. 3 is assumed to be a device separate from the control device that controls the pulling device. However, this is merely an example, and the grip force estimation device 3 shown in Fig. 3 may be implemented in the control device.

[0028] 2, the hollow body 21 is attached to the object gripping portion 13, which is a portion of the gripping device 1 that grips the object 2. The portion to which the hollow body 21 is attached may be any portion that changes in accordance with the deformation of the gripping device 1, and may be attached to the cutout portion 11 or the attachment portion 12, for example.

[0029] In the gripping tool 1 shown in FIG. 2 , a hollow body 21 is attached to the object gripping unit 13, and a pressure sensor 23 that monitors the internal pressure of the hollow body 21 is provided as a sensor for monitoring the deformation of the gripping tool 1. However, the sensor for monitoring the deformation of the gripping tool 1 is not limited to the pressure sensor 23 that monitors the internal pressure of the hollow body 21. For example, it may be a strain sensor that monitors the deformation of the object gripping unit 13. The strain sensor detects the strain associated with the deformation as an electrical signal when the object gripping unit 13 deforms. When a strain sensor is used as the sensor for monitoring the deformation of the gripping tool 1, a table indicating the correspondence between the electrical signal corresponding to the strain associated with the deformation and the gripping force of the gripping tool 1 on the object 2 is stored in the internal memory of the gripping force estimating unit 32. The gripping force estimating unit 32 estimates the gripping force of the gripping tool 1 by obtaining, from the table stored in the internal memory, information indicating the gripping force of the gripping tool 1 corresponding to the electrical signal indicated by the sensing data.

[0030] Embodiment 2 In embodiment 2, a grip force estimation device 3 will be described in which a grip force estimation unit 33 provides sensing data acquired by a sensing data acquisition unit 31 to a first learning model 24 and acquires, from the first learning model 24, information indicating a grip force corresponding to the internal pressure indicated by the sensing data.

[0031] FIG. 7 is a configuration diagram showing a grip force estimation device 3 according to a second embodiment. In FIG. 7, the same reference numerals as those in FIG. 3 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. FIG. 8 is a hardware configuration diagram showing the hardware of the grip force estimation device 3 according to the second embodiment. In FIG. 8, the same reference numerals as those in FIG. 4 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. The first learning model 24 is realized, for example, by a neural network. During learning, the first learning model 24 is provided with pressure data indicating the internal pressure of the hollow body 21 and training data indicating the grip force of the grip tool 1 on the object 2, and learns the grip force of the grip tool 1 corresponding to the internal pressure indicated by the pressure data. During inference, the first learning model 24 is provided with sensing data and outputs information indicating the grip force corresponding to the internal pressure indicated by the sensing data. In the grip force estimation device 3 shown in FIG. 7, the first learning model 24 is provided outside the grip force estimation device 3. However, this is merely an example, and the first learning model 24 may be provided inside the grip force estimation device 3.

[0032] The grip force estimation unit 33 is realized, for example, by a grip force estimation circuit 43 shown in FIG. 8 . The grip force estimation unit 33 acquires sensing data of the pressure sensor 23 from the sensing data acquisition unit 31. The grip force estimation unit 33 estimates the grip force of the gripping tool 1 with respect to the object 2 based on the sensing data of the pressure sensor 23. Specifically, the grip force estimation unit 33 provides the sensing data to the first learning model 24 and acquires, from the first learning model 24, information indicating the grip force corresponding to the internal pressure indicated by the sensing data, thereby estimating the grip force of the gripping tool 1. The grip force estimation unit 33 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0033] 7, it is assumed that the sensing data acquisition unit 31 and the grip force estimation unit 33, which are components of the grip force estimation device 3, are each realized by dedicated hardware as shown in FIG. 8. That is, it is assumed that the grip force estimation device 3 is realized by a sensing data acquisition circuit 41 and a grip force estimation circuit 43. Each of the sensing data acquisition circuit 41 and the grip force estimation circuit 43 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0034] The components of the grip force estimation device 3 are not limited to those realized by dedicated hardware, and the grip force estimation device 3 may be realized by software, firmware, or a combination of software and firmware. When the grip force estimation device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the sensing data acquisition unit 31 and the grip force estimation unit 33 is stored in a memory 51 shown in Fig. 5. Then, a processor 52 shown in Fig. 5 executes the program stored in the memory 51.

[0035] 8 shows an example in which each of the components of the grip force estimation device 3 is realized by dedicated hardware, while Fig. 5 shows an example in which the grip force estimation device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the grip force estimation device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0036] Next, the operation of the grip force estimation device 3 shown in Figure 7 will be described. Except for the grip force estimation unit 33, the grip force estimation device 3 is the same as that shown in Figure 3. Therefore, only the operation of the grip force estimation unit 33 will be described here. The grip force estimation unit 33 acquires sensing data from the pressure sensor 23 from the sensing data acquisition unit 31. The grip force estimation unit 33 provides the sensing data to the first learning model 24. The grip force estimation unit 33 then estimates the grip force of the gripping tool 1 by acquiring information indicating the grip force corresponding to the internal pressure indicated by the sensing data from the first learning model 24. The grip force estimation unit 33 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0037] In the second embodiment described above, pressure data indicating the internal pressure of the hollow body 21 and teacher data indicating the gripping force of the gripping tool 1 on the object 2 are provided, and a first learning model 24 is provided in which learning of the gripping force corresponding to the internal pressure of the hollow body 21 has been completed. The gripping force estimation device 3 is configured so that the gripping force estimation unit 33 provides the sensing data acquired by the sensing data acquisition unit 31 to the first learning model 24 and acquires information indicating the gripping force corresponding to the sensing data from the first learning model 24. Therefore, the gripping force estimation device 3 can estimate the gripping force of a gripping tool 1 having a kirigami structure that is not necessarily driven by a motor.

[0038] Embodiment 3 In embodiment 3, a gripping force estimation device 3 including a gripping force estimation unit 35 that estimates the gripping force of the gripping tool 1 with respect to the object 2 based on a captured image that is the image capture result of a visual sensor 25 that captures an image of the gripping tool 1 will be described.

[0039] FIG. 9 is a configuration diagram showing a gripping force estimation device 3 according to embodiment 3. In FIG. 9, the same reference numerals as those in FIGS. 3 and 7 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. FIG. 10 is a hardware configuration diagram showing the hardware of the gripping force estimation device 3 according to embodiment 3. In FIG. 10, the same reference numerals as those in FIGS. 4 and 8 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. In FIG. 9, the visual sensor 25 is a sensor that observes the deformation of the gripping tool 1. Specifically, the visual sensor 25 is a sensor that captures an image of the gripping tool 1 that is deformed by gripping the object 2. The visual sensor 25 outputs sensing data indicating the captured image of the gripping tool 1 to the gripping force estimation device 3. The captured image of the gripping tool 1 may be one or more still images or a video.

[0040] The second learning model 26 is realized by, for example, a neural network. When the second learning model 26 is provided with a captured image of the gripping tool 1 and training data indicating the gripping force of the gripping tool 1 on the object 2 during learning, the second learning model 26 learns the gripping force of the gripping tool 1 corresponding to the captured image of the gripping tool 1. The captured image used during learning may be one or more still images or may be a video. When the second learning model 26 is provided with sensing data during inference, the second learning model 26 outputs information indicating the gripping force corresponding to the captured image indicated by the sensing data. In the gripping force estimation device 3 shown in FIG. 9 , the second learning model 26 is provided outside the gripping force estimation device 3. However, this is merely an example, and the second learning model 26 may be provided inside the gripping force estimation device 3.

[0041] The grip force estimation device 3 shown in Fig. 9 includes a sensing data acquisition unit 34 and a grip force estimation unit 35. The sensing data acquisition unit 34 is realized by, for example, a sensing data acquisition circuit 44 shown in Fig. 10. The sensing data acquisition unit 34 acquires sensing data from the visual sensor 25. The sensing data acquisition unit 34 outputs the sensing data from the visual sensor 25 to the grip force estimation unit 35.

[0042] The grip force estimation unit 35 is realized, for example, by a grip force estimation circuit 45 shown in FIG. 10 . The grip force estimation unit 35 acquires sensing data of the visual sensor 25 from the sensing data acquisition unit 34. The grip force estimation unit 35 estimates the grip force of the gripping tool 1 with respect to the object 2 based on the sensing data of the visual sensor 25. Specifically, the grip force estimation unit 35 provides the sensing data to the second learning model 26 and acquires, from the second learning model 26, information indicating the grip force corresponding to the captured image indicated by the sensing data, thereby estimating the grip force of the gripping tool 1. The grip force estimation unit 35 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0043] 9, the grip force estimation unit 35 provides sensing data to the second learning model 26 and acquires information indicating the grip force corresponding to the captured image indicated by the sensing data from the second learning model 26. However, this is merely an example, and the grip force estimation unit 35 may also estimate the grip force corresponding to the captured image indicated by the sensing data based on a rule base.

[0044] 9, it is assumed that the sensing data acquisition unit 34 and the grip force estimation unit 35, which are components of the grip force estimation device 3, are each realized by dedicated hardware as shown in Fig. 10. That is, it is assumed that the grip force estimation device 3 is realized by a sensing data acquisition circuit 44 and a grip force estimation circuit 45. Each of the sensing data acquisition circuit 44 and the grip force estimation circuit 45 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0045] The components of the grip force estimation device 3 are not limited to those realized by dedicated hardware, and the grip force estimation device 3 may be realized by software, firmware, or a combination of software and firmware. When the grip force estimation device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the sensing data acquisition unit 34 and the grip force estimation unit 35 is stored in a memory 51 shown in Fig. 5. Then, a processor 52 shown in Fig. 5 executes the program stored in the memory 51.

[0046] 10 shows an example in which each of the components of the grip force estimation device 3 is realized by dedicated hardware, while Fig. 5 shows an example in which the grip force estimation device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the grip force estimation device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0047] Next, the operation of the gripping force estimation device 3 shown in FIG. 9 will be described. When the pulling tool pulls up the two attachment portions 12 of the gripping tool 1, the two object gripping portions 13 move closer to each other. As a result, the two object gripping portions 13 are able to grip the object 2, as shown in FIG. 2. As the pulling tool moves the two attachment portions 12 closer to each other, the opening degree of the notch 11 decreases, as shown in FIG. 11, and the gripping force of the gripping tool 1 increases. FIG. 11 is an explanatory diagram showing different opening degrees of the notch 11. FIG. 11 shows an example in which the object 2 is a mandarin orange. FIG. 11A is an explanatory diagram showing a state before the gripping tool 1 grips the object 2, with the notch 11 wide open. FIG. 11B is an explanatory diagram showing a state in which the object gripping portion 13 lightly touches the object 2, with the notch 11 narrower than before gripping the object 2. FIG. 11C is an explanatory diagram showing a state in which the gripping tool 1 grips the object 2, in which the notch 11 is narrower than in the state in which the object gripping portion 13 is lightly touching the object 2.

[0048] The visual sensor 25 captures an image of the gripping tool 1 that is deformed when the pulling instrument pulls up the attachment portion 12. The visual sensor 25 outputs sensing data that indicates the captured image of the gripping tool 1 to the gripping force estimation device 3. The sensing data acquisition unit 34 of the gripping force estimation device 3 acquires the sensing data of the visual sensor 25. The sensing data acquisition unit 34 outputs the sensing data of the visual sensor 25 to the gripping force estimation unit 35.

[0049] The grip force estimation unit 35 acquires sensing data of the visual sensor 25 from the sensing data acquisition unit 34. The grip force estimation unit 35 provides the sensing data to the second learning model 26. The grip force estimation unit 35 then estimates the grip force of the gripping tool 1 by acquiring information indicating the grip force corresponding to the captured image indicated by the sensing data from the second learning model 26. The grip force estimation unit 35 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0050] In the third embodiment described above, the sensor is a visual sensor 25 that captures an image of the gripping tool 1. The gripping force estimation device 3 is configured so that the sensing data acquisition unit 34 acquires sensing data indicating the captured image that is the image capture result of the visual sensor 25, and the gripping force estimation unit 35 estimates the gripping force of the gripping tool 1 with respect to the object 2 based on the captured image indicated by the sensing data. Therefore, the gripping force estimation device 3 can estimate the gripping force of a gripping tool 1 that has a kirigami structure and is not necessarily driven by a motor.

[0051] Fourth Embodiment In a fourth embodiment, a marker 4 is attached to each of a plurality of locations on a gripping tool 1. A gripping force estimation device 3 will be described, which includes a gripping force estimation unit 36 ​​that provides a captured image, which is an imaging result of a visual sensor 25, to a third learning model 27 and acquires, from the third learning model 27, information indicating gripping forces corresponding to the positions of the plurality of markers 4 shown in the captured image.

[0052] Fig. 12 is a configuration diagram showing a grip force estimation device 3 according to embodiment 4. In Fig. 12, the same reference numerals as those in Figs. 3, 7, and 9 indicate the same or corresponding parts, and detailed description thereof will be omitted. Fig. 13 is a hardware configuration diagram showing the hardware of the grip force estimation device 3 according to embodiment 4. In Fig. 13, the same reference numerals as those in Figs. 4, 8, and 10 indicate the same or corresponding parts, and detailed description thereof will be omitted. In embodiment 4, as shown in Fig. 14 described later, markers 4 are attached to each of a plurality of locations on the gripping tool 1.

[0053] The third learning model 27 is realized by, for example, a neural network. During learning, when position data indicating the positions of multiple markers 4 shown in the captured image and teacher data indicating the gripping force of the gripping tool 1 with respect to the object 2 are provided, the third learning model 27 learns the gripping force of the gripping tool 1 corresponding to the positions of the multiple markers 4. During inference, when sensing data is provided, the third learning model 27 outputs information indicating the gripping force corresponding to the positions of the multiple markers 4 shown in the captured image indicated by the sensing data. In the gripping force estimation device 3 shown in FIG. 12 , the third learning model 27 is provided outside the gripping force estimation device 3. However, this is merely an example, and the third learning model 27 may be provided inside the gripping force estimation device 3.

[0054] The grip force estimation device 3 shown in FIG. 12 includes a sensing data acquisition unit 34 and a grip force estimation unit 36. The grip force estimation unit 36 ​​is realized, for example, by a grip force estimation circuit 46 shown in FIG. 13. The grip force estimation unit 36 ​​acquires sensing data of the visual sensor 25 from the sensing data acquisition unit 34. The grip force estimation unit 36 ​​estimates the grip force of the gripping tool 1 with respect to the object 2 based on the sensing data of the visual sensor 25. Specifically, the grip force estimation unit 36 ​​provides the captured image indicated by the sensing data to a third learning model 27 and acquires, from the third learning model 27, information indicating the grip forces corresponding to the positions of multiple markers 4 shown in the captured image, thereby estimating the grip force of the gripping tool 1. The grip force estimation unit 36 ​​outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0055] 12, the grip force estimation unit 36 ​​provides the captured image indicated by the sensing data to the third learning model 27, and acquires information indicating the grip forces corresponding to the positions of the multiple markers 4 shown in the captured image from the third learning model 27. However, this is merely an example, and the grip force estimation unit 36 ​​may also estimate the grip forces corresponding to the positions of the multiple markers 4 shown in the captured image based on a rule base.

[0056] 12, it is assumed that the sensing data acquisition unit 34 and the grip force estimation unit 36, which are components of the grip force estimation device 3, are each realized by dedicated hardware as shown in FIG. 13. That is, it is assumed that the grip force estimation device 3 is realized by a sensing data acquisition circuit 44 and a grip force estimation circuit 46. Each of the sensing data acquisition circuit 44 and the grip force estimation circuit 46 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0057] The components of the grip force estimation device 3 are not limited to those realized by dedicated hardware, and the grip force estimation device 3 may be realized by software, firmware, or a combination of software and firmware. When the grip force estimation device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the sensing data acquisition unit 34 and the grip force estimation unit 36 ​​is stored in a memory 51 shown in Fig. 5. Then, a processor 52 shown in Fig. 5 executes the program stored in the memory 51.

[0058] 13 shows an example in which each of the components of the grip force estimation device 3 is realized by dedicated hardware, while Fig. 5 shows an example in which the grip force estimation device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the grip force estimation device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0059] Next, the operation of the gripping force estimation device 3 shown in FIG. 12 will be described. When the tensile tool pulls up the two attachment portions 12 of the gripping tool 1, the two object gripping portions 13 move closer to each other. As a result, the two object gripping portions 13 are able to grip the object 2, as shown in FIG. 2. When the distance between the two attachment portions 12 changes, the positions of the markers 4 attached to each portion of the gripping tool 1 change, as shown in FIG. 14. Furthermore, when the distance between the two attachment portions 12 changes, the gripping force of the gripping tool 1 changes. FIG. 14 is an explanatory diagram showing the difference in the positions of the markers 4 attached to each portion of the gripping tool 1. FIG. 14A is an explanatory diagram showing the positions of the markers 4 when the gripping tool 1 is in a state before gripping the object 2, FIG. 14B is an explanatory diagram showing the positions of the markers 4 when the object gripping portions 13 are lightly touching the object 2, and FIG. 14C is an explanatory diagram showing the positions of the markers 4 when the gripping tool 1 has gripped the object 2. The position of the marker 4 varies depending on the state.

[0060] The visual sensor 25 captures an image of the gripping tool 1 that is deformed when the pulling instrument pulls up the attachment portion 12. The visual sensor 25 outputs sensing data indicating the captured image of the gripping tool 1 to the gripping force estimation device 3. The sensing data acquisition unit 34 of the gripping force estimation device 3 acquires the sensing data of the visual sensor 25. The sensing data acquisition unit 34 outputs the sensing data of the visual sensor 25 to the gripping force estimation unit 36.

[0061] The grip force estimation unit 36 ​​acquires sensing data of the visual sensor 25 from the sensing data acquisition unit 34. The grip force estimation unit 36 ​​provides the captured image indicated by the sensing data to the third learning model 27. The grip force estimation unit 36 ​​then estimates the grip force of the gripping tool 1 by acquiring information indicating the grip forces corresponding to the positions of the multiple markers 4 shown in the captured image from the third learning model 27. The grip force estimation unit 36 ​​outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0062] In the fourth embodiment described above, the third learning model 27 is provided with position data indicating the positions of the plurality of markers 4 shown in the captured image and teacher data indicating the gripping force of the gripping tool 1 with respect to the object 2, and has already learned the gripping forces corresponding to the positions of the plurality of markers 4. The gripping force estimation device 3 is configured so that the gripping force estimation unit 36 ​​provides the captured image, which is the image capture result of the visual sensor 25, to the third learning model 27 and acquires information indicating the gripping forces corresponding to the positions of the plurality of markers 4 shown in the captured image from the third learning model 27. Therefore, the gripping force estimation device 3 can estimate the gripping force of a gripping tool 1 having a kirigami structure that is not necessarily driven by a motor.

[0063] Fifth Embodiment In the fifth embodiment, a gripping force estimation device 3 will be described in which a gripping force estimation unit 38 estimates the gripping force of a gripping tool 1 with respect to an object 2 based on the internal pressure observed by a pressure sensor 23 and a captured image that is an imaging result of a visual sensor 25.

[0064] Fig. 15 is a configuration diagram showing a grip force estimation device 3 according to embodiment 5. In Fig. 15, the same reference numerals as those in Figs. 3, 7, 9, and 12 indicate the same or corresponding parts, and detailed descriptions thereof will be omitted. Fig. 16 is a hardware configuration diagram showing the hardware of the grip force estimation device 3 according to embodiment 5. In Fig. 16, the same reference numerals as those in Figs. 4, 8, 10, and 13 indicate the same or corresponding parts, and detailed descriptions thereof will be omitted.

[0065] The fourth learning model 28 is realized, for example, by a neural network. During learning, the fourth learning model 28 receives pressure data indicating the internal pressure of the hollow body 21, a photographed image of the gripping tool 1, and training data indicating the gripping force of the gripping tool 1 on the object 2. The fourth learning model 28 learns the gripping force of the gripping tool 1 corresponding to the internal pressure indicated by the pressure data and the photographed image of the gripping tool 1. During inference, the fourth learning model 28 receives sensing data from the pressure sensor 23 and sensing data from the visual sensor 25. The fourth learning model 28 outputs information indicating the internal pressure indicated by the sensing data and the gripping force corresponding to the photographed image indicated by the sensing data. In the gripping force estimation device 3 shown in FIG. 15 , the fourth learning model 28 is provided outside the gripping force estimation device 3. However, this is merely an example, and the fourth learning model 28 may be provided inside the gripping force estimation device 3.

[0066] The grip force estimation device 3 shown in Fig. 15 includes a sensing data acquisition unit 37 and a grip force estimation unit 38. The sensing data acquisition unit 37 is realized by, for example, a sensing data acquisition circuit 47 shown in Fig. 16. The sensing data acquisition unit 37 acquires sensing data from the pressure sensor 23 and sensing data from the visual sensor 25. The sensing data acquisition unit 31 outputs each of the sensing data from the pressure sensor 23 and the sensing data from the visual sensor 25 to the grip force estimation unit 38.

[0067] The grip force estimation unit 38 is realized, for example, by a grip force estimation circuit 48 shown in FIG. 16 . The grip force estimation unit 38 acquires the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25 from the sensing data acquisition unit 31. The grip force estimation unit 38 estimates the grip force of the gripping tool 1 with respect to the object 2 based on the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25. Specifically, the grip force estimation unit 38 provides the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25 to a fourth learning model 28, and estimates the grip force of the gripping tool 1 by acquiring, from the fourth learning model 28, information indicating the grip force of the gripping tool 1 corresponding to the internal pressure of the hollow body 21 and the captured image of the gripping tool 1. The grip force estimation unit 38 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0068] 15, the grip force estimation unit 38 provides the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25 to the fourth learning model 28, and acquires information indicating the grip force of the gripping tool 1 corresponding to the internal pressure of the hollow body 21 and the captured image of the gripping tool 1 from the fourth learning model 28. However, this is merely an example, and the grip force estimation unit 38 may also estimate the grip force corresponding to the internal pressure of the hollow body 21 and the captured image of the gripping tool 1 based on a rule base.

[0069] 15, it is assumed that the sensing data acquisition unit 37 and the grip force estimation unit 38, which are components of the grip force estimation device 3, are each realized by dedicated hardware such as that shown in Fig. 16. That is, it is assumed that the grip force estimation device 3 is realized by a sensing data acquisition circuit 47 and a grip force estimation circuit 48. Each of the sensing data acquisition circuit 47 and the grip force estimation circuit 48 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0070] The components of the grip force estimation device 3 are not limited to those realized by dedicated hardware, and the grip force estimation device 3 may be realized by software, firmware, or a combination of software and firmware. When the grip force estimation device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the sensing data acquisition unit 37 and the grip force estimation unit 38 is stored in a memory 51 shown in Fig. 5. Then, a processor 52 shown in Fig. 5 executes the program stored in the memory 51.

[0071] 16 shows an example in which each of the components of the grip force estimation device 3 is realized by dedicated hardware, while Fig. 5 shows an example in which the grip force estimation device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the grip force estimation device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0072] Next, the operation of the gripping force estimation device 3 shown in Figure 15 will be described. The pressure sensor 23 observes the internal pressure of the tube 22 as the internal pressure of the hollow body 21. The pressure sensor 23 outputs sensing data indicating the internal pressure of the hollow body 21 to the gripping force estimation device 3. The visual sensor 25 photographs the gripping tool 1 that is deformed when the tensile instrument pulls up the attachment part 12. The visual sensor 25 outputs sensing data indicating the photographed image of the gripping tool 1 to the gripping force estimation device 3.

[0073] The sensing data acquisition unit 37 of the grip force estimation device 3 acquires the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25. The sensing data acquisition unit 37 outputs each of the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25 to the grip force estimation unit 38.

[0074] The grip force estimation unit 38 acquires the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25 from the sensing data acquisition unit 37. The grip force estimation unit 38 provides the sensing data of the pressure sensor 23 and the sensing data of the visual sensor 25 to the fourth learning model 28. The grip force estimation unit 38 then estimates the grip force of the gripping tool 1 by acquiring, from the fourth learning model 28, information indicating the grip force of the gripping tool 1 that corresponds to the internal pressure of the hollow body 21 indicated by the sensing data of the pressure sensor 23 and the captured image indicated by the sensing data of the visual sensor 25. The grip force estimation unit 38 outputs the information indicating the grip force of the gripping tool 1 to, for example, a control device (not shown) of the robot.

[0075] In the fifth embodiment described above, the grip force estimation device 3 shown in FIG. 15 is configured such that the sensing data acquisition unit 37 acquires sensing data indicating the internal pressure observed by the pressure sensor 23 and sensing data indicating the captured image, which is the image capture result of the visual sensor 25, and the grip force estimation unit 38 estimates the grip force of the gripping tool 1 with respect to the object 2 based on the internal pressure observed by the pressure sensor 23 and the captured image. Therefore, the grip force estimation device 3 shown in FIG. 15 can estimate the grip force of a gripping tool 1 having a kirigami structure that is not necessarily driven by a motor. Furthermore, the grip force estimation device 3 shown in FIG. 15 can improve the accuracy of grip force estimation compared to the grip force estimation device 3 shown in FIG. 7 or the grip force estimation device 3 shown in FIG. 9.

[0076] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0077] The present disclosure is suitable for a grip force estimation device and a grip force estimation method.

[0078] 1 Grip tool, 2 Object, 3 Grip force estimation device, 4 Marker, 11 Cutout portion, 12 Mounting portion, 13 Object gripping portion, 21 Hollow body, 22 Tube, 23 Pressure sensor, 24 First learning model, 25 Visual sensor, 26 Second learning model, 27 Third learning model, 28 Fourth learning model, 31 Sensing data acquisition unit, 32, 33 Grip force estimation unit, 34 Sensing data acquisition unit, 35, 36 Grip force estimation unit, 37 Sensing data acquisition unit, 38 Grip force estimation unit, 41 Sensing data acquisition circuit, 42, 43 Grip force estimation circuit, 44 Sensing data acquisition circuit, 45, 46 Grip force estimation circuit, 47 Sensing data acquisition circuit, 48 Grip force estimation circuit, 51 Memory, 52 Processor.

Claims

1. A gripping force estimation device for estimating the gripping force of a gripping tool having a kirigami structure, which has an attachment part to which a pulling tool is attached and two object gripping parts that grip an object, and in which the two object gripping parts deform so as to approach each other when the attachment part is pulled up by the pulling tool, the gripping force estimation device comprising: a sensing data acquisition part that acquires sensing data from a sensor that observes the deformation of the gripping tool; and a gripping force estimation part that estimates the gripping force of the gripping tool with respect to the object based on the sensing data acquired by the sensing data acquisition part.

2. A gripping force estimation device as described in claim 1, characterized in that a hollow body is attached to an object gripping portion which is a portion of the gripping tool that grips the object, the sensor is a pressure sensor which observes the internal pressure of the hollow body which changes with deformation of the gripping tool, the sensing data acquisition unit acquires sensing data indicating the internal pressure observed by the pressure sensor, and the gripping force estimation unit estimates the gripping force of the gripping tool with respect to the object based on the internal pressure indicated by the sensing data acquired by the sensing data acquisition unit.

3. A gripping force estimation device as described in claim 2, characterized in that there is a table indicating the correspondence between the internal pressure of the hollow body and the gripping force of the gripping tool on the object, and the gripping force estimation unit obtains from the table information indicating the gripping force of the gripping tool corresponding to the internal pressure indicated by the sensing data.

4. A gripping force estimation device as described in claim 2, characterized in that there is a first learning model that is given pressure data indicating the internal pressure of the hollow body and teacher data indicating the gripping force of the gripping tool against the object, and that has already learned the gripping force corresponding to the internal pressure of the hollow body, and the gripping force estimation unit provides the sensing data acquired by the sensing data acquisition unit to the first learning model and acquires from the first learning model information indicating the gripping force corresponding to the internal pressure indicated by the sensing data.

5. The gripping force estimation device described in claim 1, characterized in that the sensor is a visual sensor that photographs the gripping tool, the sensing data acquisition unit acquires sensing data indicating a captured image that is the image capture result of the visual sensor, and the gripping force estimation unit estimates the gripping force of the gripping tool against the object based on the captured image indicated by the sensing data acquired by the sensing data acquisition unit.

6. A gripping force estimation device as described in claim 5, characterized in that a second learning model is provided that is given a captured image that is the image capture result of the visual sensor and teacher data indicating the gripping force of the gripping tool against the object, and that has already learned the gripping force corresponding to the captured image, and the gripping force estimation unit provides the sensing data acquired by the sensing data acquisition unit to the second learning model and acquires from the second learning model information indicating the gripping force corresponding to the captured image shown by the sensing data.

7. A gripping force estimation device as described in claim 5, characterized in that a marker is attached to each of multiple locations on the gripping tool, and a third learning model is provided that is given position data indicating the positions of the multiple markers shown in the captured image which is the image capture result of the visual sensor and teacher data indicating the gripping force of the gripping tool against the object, and has learned the gripping force corresponding to the positions of the multiple markers, and the gripping force estimation unit provides the sensing data acquired by the sensing data acquisition unit to the third learning model and acquires from the third learning model information indicating the gripping force corresponding to the positions of the multiple markers shown in the captured image shown by the sensing data.

8. A gripping force estimation device as described in claim 1, characterized in that a hollow body is attached to an object gripping portion which is a portion of the gripping tool that grips the object, among multiple portions of the gripping tool, the sensor includes a pressure sensor which observes the internal pressure of the hollow body which changes with deformation of the gripping tool, and a visual sensor which photographs the gripping tool, the sensing data acquisition unit acquires sensing data indicating the internal pressure observed by the pressure sensor and sensing data indicating a captured image which is the imaging result of the visual sensor, and the gripping force estimation unit estimates the gripping force of the gripping tool with respect to the object based on the internal pressure indicated by the sensing data acquired by the sensing data acquisition unit and the captured image indicated by the sensing data acquired by the sensing data acquisition unit.

9. A gripping force estimation method for estimating the gripping force of a gripping tool having a kirigami structure, which has an attachment part to which a pulling tool is attached and two object gripping parts that grip an object, and in which the two object gripping parts deform so as to approach each other when the attachment part is pulled up by the pulling tool, the method comprising: a sensing data acquisition unit acquires sensing data from a sensor that observes the deformation of the gripping tool; and a gripping force estimation unit estimates the gripping force of the gripping tool with respect to the object based on the sensing data acquired by the sensing data acquisition unit.

Images