Gripping device, control method, and program

Abstract

The invention discloses a gripping device, a control method, and a program, the gripping device includes: a gripping portion module having a first surface and a second surface; an arm part for changing the position of the gripping part module; an imaging unit that is provided at a position that moves together with the gripping unit module and that images at least a part of the object; and a control unit capable of comparing specific amount information indicating a contact state when the first surface is in contact with a specific amount of the object with information indicating a contact state captured by the imaging unit when the gripping unit module holds at least a portion of the object, and determining a specific amount of the object based on a result of the comparison. And a control unit that outputs drive information for driving the grip unit module or the arm unit, and controls at least one of the grip unit module and the arm unit such that the amount of the object to be gripped approaches the specific amount. The present invention can provide a gripping device, a control method, and a program capable of gripping a specific amount of a soft object to be gripped.

Description

[0001] Related applications This application is a divisional application of Chinese invention patent application No. 201980102943.6, filed on December 16, 2019, entitled "Holding device, control method and procedure". Technical Field

[0002] This invention relates to a gripping device, a control method, and a procedure. Background Technology

[0003] Previously, it was known that a robotic arm was controlled based on image information captured by a camera unit located on the upper part of the object being held, thereby holding the object being held (for example, see Patent Document 1).

[0004] Existing technical documents Patent documents Patent Document 1: Japanese Patent Publication No. 2019-198950 Summary of the Invention Here, it is desired to use a robotic arm to grasp a specific quantity of a soft object, such as a side dish, and move it to a destination. However, when using a robotic arm with existing technology to grasp a specific quantity of a soft object, there is a problem that the internal state of the soft object cannot be identified based on information captured by a camera mounted on top of the object.

[0005] That is, according to existing methods, there is a problem that it is difficult to estimate a specific amount in a soft, graspable object.

[0006] The present invention was made in view of this situation, and its object is to provide a gripping device, control method and procedure capable of gripping a specific amount in a soft gripping object.

[0007] One aspect of the present invention relates to a holding device comprising: a holding module having a first surface and a second surface, holding an object between the first surface and the second surface; an arm that changes the position of the holding module; a shooting unit disposed at a position that moves together with the holding module to shoot at least a portion of the object; and a control unit capable of controlling at least one of the holding module or the arm, based on specific quantity information indicating a contact state when the first surface is in contact with a specific amount of the object, and information indicating a contact state when the holding module holds at least a portion of the object and the shooting unit takes a picture, so that the amount of the object held approaches the specific quantity.

[0008] In addition, in the holding device according to one embodiment of the present application, the photographing section is provided on the back side of the first face and is capable of photographing the object.

[0009] In addition, in the holding device according to one embodiment of the present application, the control section is capable of controlling at least one of the holding section module or the arm section based on the specific amount information, that is, information indicating a proportion of a portion representing a background to a portion representing the object, and information indicating a contact state photographed by the photographing section when the holding section module holds at least a portion of the object.

[0010] In addition, in the holding device according to one embodiment of the present application, the control section is capable of controlling at least one of the holding section module or the arm section based on the specific amount information, that is, information indicating a proportion of a portion in which a luminance of the object changes greatly, and information indicating a contact state photographed by the photographing section when the holding section module holds at least a portion of the object.

[0011] In addition, in the holding device according to one embodiment of the present application, the control section is capable of controlling at least one of the holding section module or the arm section based on the specific amount information, that is, information indicating a color of the object, and information indicating a contact state photographed by the photographing section when the holding section module holds at least a portion of the object.

[0012] In addition, the control method according to one embodiment of the present application is a control method of a holding device including a holding section module having a first face and a second face, holding an object between the first face and the second face, an arm section changing a position of the holding section module, and a photographing section provided at a position moving together with the holding section module, photographing at least a portion of the object, the control method including a control step of controlling at least one of the holding section module or the arm section based on specific amount information indicating a contact state when the first face contacts the object in a specific amount, and information indicating a contact state photographed by the photographing section when the holding section module holds at least a portion of the object, so that an amount of the held object approaches the specific amount.

[0013] In addition, a program according to one embodiment of the present application causes a computer connected to a holding device to execute a control step, the holding device including a holding portion module having a first surface and a second surface, holding an object between the first surface and the second surface, an arm portion that changes a position of the holding portion module, and a photographing portion provided at a position that moves together with the holding portion module, photographing at least a portion of the object. The control step controls at least one of the holding portion module or the arm portion based on specific amount information indicating a contact state when the first surface contacts the object by a specific amount and information indicating a contact state when the photographing portion photographs at least a portion of the object held by the holding portion module, so that the amount of the held object approaches the specific amount.

[0014] According to the present application, it is possible to provide a holding device, a control method, and a program that can hold a specific amount of a soft holding object. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 FIG. 1 is a diagram illustrating one example of a holding device in an embodiment.

[0016] Figure 2 FIG. 2 is a diagram illustrating one example of a case where a holding device in an embodiment holds a soft holding object.

[0017] Figure 3 FIG. 3 is a diagram illustrating one example of a holding portion module in an embodiment.

[0018] Figure 4 FIG. 4 is a diagram illustrating one example of a cross-sectional view of a holding portion in an embodiment.

[0019] Figure 5 FIG. 5 is a diagram illustrating one example of a functional structure of a holding device in an embodiment.

[0020] Figure 6 FIG. 6 is a diagram for explaining an action of a holding portion on a holding object in an embodiment.

[0021] Figure 7 FIG. 7 is a diagram illustrating one example of a series of actions of a holding device in an embodiment.

[0022] Figure 8 FIG. 8 is a diagram illustrating one example of control based on a ratio of an object to a background in an embodiment.

[0023] Figure 9 FIG. 9 is a diagram illustrating one example of control based on a ratio of an edge in an embodiment.

[0024] Figure 10is a diagram showing one example of image-based color control in the embodiment.

[0025] Reference Signs List 1 … gripping portion, 2 … support portion, 10 … gripping portion module, 11 … connecting portion, 100 … gripping device, 20 … container, 21 … gripping object, 22 … specific amount, 110 … front end portion, 120 … upper arm portion, 130 … joint portion, 140 … lower arm portion, 150 … main horizontal axis portion, 160 … main vertical axis portion, 170 … base portion, 180 … arm portion, 90 … gripping device control portion, 30 … imaging portion, 41 … contact surface, 42 … support surface, 43 … contact state detection portion, 45 … marker, 50 … frame, 70 … hard layer, 92 … input portion, 93 … output portion, 84 … storage portion DETAILED DESCRIPTION [Structure of Gripping Device 100] Hereinafter, an embodiment of the present application will be described with reference to the drawings.

[0026] Figure 1 is a diagram showing one example of the structure of the gripping device 100 in the embodiment. The posture of the gripping device 100 is represented by a three-dimensional rectangular coordinate system of x-axis, y-axis, and z-axis. The gripping device 100 in the present embodiment detects the gripping state by contacting the gripping object and grips the specific amount in the gripping object.

[0027] In the present embodiment, the gripping device 100 includes the gripping portion module 10, the gripping device control portion 90 (or control portion, hereinafter the same), the front end portion 110, the upper arm portion 120, the joint portion 130, the lower arm portion 140, the main horizontal axis portion 150, the main vertical axis portion 160, and the base portion 170. In the case where the front end portion 110, the upper arm portion 120, the joint portion 130, the lower arm portion 140, the main horizontal axis portion 150, and the main vertical axis portion 160 are not distinguished, they are described as the arm portion 180.

[0028] The base portion 170 is a site connected with the main vertical axis portion 160.

[0029] The main vertical axis portion 160 is a site connecting the main horizontal axis portion 150 and the base portion 170. The main vertical axis portion 160 is controlled by the gripping device control portion 90 to displace the main horizontal axis portion 150 around the axis of the main vertical axis portion 160.

[0030] The main horizontal axis portion 150 is a site connecting the lower arm portion 140 and the main vertical axis portion 160. The main horizontal axis portion 150 is controlled by the gripping device control portion 90 to displace the lower arm portion 140 around the axis of the main horizontal axis portion 150.

[0031] The lower arm portion 140 is a site connecting the joint portion 130 and the main horizontal axis portion 150.

[0032] The joint portion 130 is a portion that connects the upper arm portion 120 and the lower arm portion 140. The joint portion 130 is controlled by the gripping device control portion 90 to displace the upper arm portion 120 around an axis of the joint portion 130.

[0033] The upper arm portion 120 is a portion that connects the front end portion 110 and the joint portion 130.

[0034] The front end portion 110 is connected to the gripping portion module 10. The posture (e.g., position and direction) of the front end portion 110 is controlled by the gripping device control portion 90. The posture of the gripping portion module 10 changes in accordance with the change in the posture of the front end portion 110.

[0035] Further, the arm portion 180 can change the position of the gripping portion module 10 in any one of the x-axis, y-axis, or z-axis, and is not limited to this example. In this example, the arm portion 180 has at least six degrees of freedom, and further includes one degree of freedom of the interval between the gripping portion 1 and the support portion 2 of the gripping portion module 10, and has seven degrees of freedom.

[0036] The gripping portion module 10 detects the contact state of the gripping object, and outputs information indicating the detected contact state of the gripping object to the gripping device control portion 90.

[0037] The gripping device control portion 90 acquires the information output from the gripping portion module 10. The gripping device control portion 90 drives the arm portion 180 to change the position of the gripping portion module 10. The gripping device control portion 90 controls the change in the position of the gripping portion module 10 on the basis of the information acquired from the gripping portion module 10.

[0038] Figure 2 is a diagram indicating one example of the case where the gripping device 100 of the embodiment grips the soft gripping object 21. Figure 2 (A) indicates one example of the case where the gripping device 100 contacts the soft gripping object 21 and acquires information indicating the contact state, Figure 2 (B) indicates one example of the case where the gripping device 100 grips the specific amount 22 that is a part of the soft gripping object 21. The posture of the gripping device 100 is indicated by a three-dimensional rectangular coordinate system of the x-axis, y-axis, and z-axis.

[0039] In this example, the soft gripping object 21 is accommodated in the container 20.

[0040] Referring to Figure 2 (A), one example of the case where the gripping device 100 contacts the soft gripping object 21 and acquires information indicating the contact state is described.

[0041] The gripping device control section 90 drives the arm section 180 so as to change the position of the gripping section module 10 with respect to the gripping object 21. In this figure, the gripping section module 10 is in contact with the gripping object 21. The gripping section module 10 acquires information indicating the contact state of the gripping section module 10 with the gripping object 21, and provides it to the gripping device control section 90. The gripping device control section 90 controls the arm section 180 on the basis of the acquired information indicating the contact state of the gripping section module 10 with the gripping object 21.

[0042] Referring to Figure 2 (B), an example of a case in which the gripping device 100 grips a certain amount 22 that is a part of the soft gripping object 21 will be described.

[0043] The gripping device control section 90 determines whether or not the gripping section module 10 is gripping the certain amount 22 on the basis of the information indicating the contact state of the gripping section module 10 with the gripping object 21. The information indicating the certain amount 22 can be stored in the gripping device control section 90, or in an external storage section 84.

[0044] The gripping device control section 90 compares the information indicating the certain amount 22 and the information indicating the contact state of the gripping section module 10 with the gripping object 21, and in the case where the certain amount 22 is small, controls at least one of the arm section 180 and the gripping section module 10 so as to perform an operation for gripping a smaller amount. The method of such control is not particularly limited. For example, the amount of the gripping object can be reduced by causing the gripping section module 10 to operate so as to change the opening and closing amount of the gripping section 1 and the support section 2, or by causing the arm section 180 to rise, or by appropriately combining the operation of the gripping section module 10 and the operation of the arm section 180. By performing such operation control, the information indicating the contact state of the gripping section module 10 with the gripping object 21 is controlled so as to approach the information indicating the certain amount 22, and thus the amount of the gripping object gripped by the gripping section module 10 approaches the certain amount 22.

[0045] The holding device control section 90 compares the information indicating the specified amount 22 and the information indicating the contact state of the holding section module 10 with the holding object 21, and in the case where the specified amount 22 is large, controls at least one of the arm section 180 and the holding section module 10 so as to perform an operation for holding a larger amount. Such a control method is not particularly limited. For example, the amount of the holding object can be increased by causing the holding section module 10 to operate so as to change the opening and closing amount of the holding section 1 and the support section 2, the amount of the holding object can be increased by causing the arm section 180 to descend, or the amount of the holding object can be increased by appropriately combining the above operation of the holding section module 10 and the operation of the arm section 180. By performing such operation control, the information indicating the contact state of the holding section module 10 with the holding object 21 is controlled so as to approach the information indicating the specified amount 22, and thus the amount of the holding object held by the holding section module 10 approaches the specified amount 22.

[0046] The holding device control section 90 compares the information indicating the specified amount 22 and the information indicating the contact state of the holding section module 10 with the holding object 21, and in the case where the information indicating the contact state of the holding section module 10 with the holding object 21 is within the range of the information indicating the specified amount 22, moves the specified amount 22 from the holding object 21 to the target location.

[0047] For example, the holding device 100 moves the specified amount 22 of the holding object 21 accommodated in the container 20 to the target location located outside the container 20.

[0048] Figure 3 is a view illustrating one example of the holding section module 10 in the embodiment. The structure of the holding section module 10 will be described with reference to this view. The posture of the holding section module 10 is represented by a three-dimensional rectangular coordinate system of an x-axis, a y-axis, and a z-axis.

[0049] The holding section module 10 includes a connection section 11, a holding section 1, and a support section 2.

[0050] The connection section 11 is a portion that connects the front end section 110, the holding section 1, and the support section 2.

[0051] The holding section 1 is connected to the connection section 11. The holding section 1 includes a contact surface 41 that contacts at least a portion of the holding object 21. Hereinafter, the contact surface 41 will also be referred to as a first surface.

[0052] The support section 2 is connected to the connection section 11. The support section 2 includes a support surface 42 that contacts at least a portion of the holding object 21. Hereinafter, the support surface 42 will also be referred to as a second surface.

[0053] In this example, the holding portion 1 and the support portion 2 are disposed at positions at which the contact surface 41 and the support surface 42 face each other. The connecting portion 11 is provided with a drive device not shown, which displaces at least the holding portion 1 in the y-axis direction based on a holding instruction from the holding device control portion 90. The holding portion module 10 holds the holding target object 21 located between the holding portion 1 and the support portion 2 by driving the holding portion 1 at least in the y-axis direction.

[0054] In this example, the holding portion 1 is displaced in the y-axis direction by the connecting portion 11, but the support portion 2 can also be displaced in the y-axis direction.

[0055] Further, in this example, the holding portion module 10 holds the holding target object 21 by the holding portion 1 and the support portion 2, but the structure of the holding portion module 10 is not limited to this example. For example, the holding portion module 10 can also be composed of a plurality of holding portions 1, a plurality of support portions 2, or a combination of a plurality of holding portions 1 and a plurality of support portions 2. In this case, the support surface 42 (second surface) is disposed at a position at which the normals of the support surfaces 42 (second surfaces) intersect the contact surface 41 (first surface). For example, the support surface 42 (second surface) is disposed at a position opposite the contact surface 41 (first surface).

[0056] The holding portion 1 holds at least a portion of the prescribed amount of the holding target object 21 with the contact surface 41 (first surface) and the support surface 42 (second surface) of the support portion 2.

[0057] Figure 4 is a drawing showing one example of a cross-sectional view of the holding portion 1 in the embodiment. Figure 4 shows Figure 3 a cross-sectional view in the x-y plane of the holding portion 1 in . The directions of the holding portion module 10 are represented by a three-dimensional rectangular coordinate system of an x-axis, a y-axis, and a z-axis.

[0058] Figure 4 (A) is a drawing showing one example of a cross-sectional view of the holding portion 1A in the embodiment. Figure 4 (B) is a drawing showing one example of a cross-sectional view of the holding portion 1B, which is a modification of the holding portion 1 in the embodiment.

[0059] Further, in the case where the support portion 2 is configured to be drivable, the support portion 2 can also be configured to have the same structure as the holding portion 1.

[0060] Referring to Figure 4 (A) explains one example of the structure of the holding portion 1A. In this example, the holding portion 1A detects the contact state of the holding target object 21 with the contact surface 41 by imaging an image of the holding target object 21.

[0061] The holding portion 1A in this example is one example of the holding portion 1. The holding portion 1 includes the photographing portion 30, the contact state detecting portion 43, the hard layer 70, and the frame 50.

[0062] The frame 50 holds the photographing portion 30, the contact state detecting portion 43, and the hard layer 70.

[0063] The contact state detecting portion 43 is provided with a contact surface 41. The contact surface 41 is a surface that comes into contact with the holding object 21. The contact state detecting portion 43 detects a contact state between the holding object 21 that comes into contact with the contact surface 41 and the contact surface 41.

[0064] For example, in a case where the contact state of the holding object 21 that comes into contact with the contact surface 41 and the contact surface 41 is detected by an image, the contact state detecting portion 43 can also be composed of a transparent material that transmits light.

[0065] The hard layer 70 is provided at a position that comes into contact with the contact state detecting portion 43. For example, the hard layer 70 can be made of a material such as transparent hard acrylic.

[0066] Further, although the contact state detecting portion 43 and the hard layer 70 are described as separate components in the present embodiment, the contact state detecting portion 43 and the hard layer 70 can be composed of the same component, thereby being configured not to include the hard layer 70.

[0067] The photographing portion 30 is disposed at a back side of the contact surface 41 that can come into contact with the holding object 21, among the front and back surfaces of the contact state detecting portion 43. Here, with respect to the contact state detecting portion, the contact surface 41 is set as a surface, the side of the contact surface 41 is set as a surface side, the opposite surface of the contact surface 41 is set as a back surface, and the opposite side of the contact surface 41 is set as a back surface side. More specifically, a position at which the holding object 21 can be photographed from the back surface of the contact surface 41 (first surface) that comes into contact with the holding object 21 is provided.

[0068] Further, in this example, the photographing portion 30 is described as being disposed inside the holding portion 1, but the position at which the photographing portion 30 is disposed is not limited to this example. The photographing portion 30 can be disposed at the front end of the arm portion 180 and can move together with the holding portion module 10. In addition, the photographing portion 30 can be disposed at a position at which an image that indicates a contact state of at least a portion of the holding object 21 with the contact surface 41 (first surface) is photographed.

[0069] The photographing portion 30 photographs an image of the holding object 21 that comes into contact with the contact surface 41 and outputs information of the photographed image to the holding device control portion 90.

[0070] Reference Signs List Figure 4(B), an example of the structure of the holding portion 1B is described. In this example, the holding portion 1B detects the contact state of the holding object 21 with the contact surface 41B by displacement of the mark 45.

[0071] The holding portion 1B in this example is an example of the holding portion 1. For the same structure as the holding portion 1A, the same reference numerals are annotated and the description is omitted. Figure 4 (A) is the same, and the description is omitted.

[0072] The holding portion 1B is different from the holding portion 1A in that the contact state detection portion 43B has the mark 45. The contact state detection portion 43B has the contact surface 41B. The contact state detection portion 43B is a variation of the contact state detection portion 43, and the contact surface 41B is a variation of the contact surface 41.

[0073] The mark 45 is configured with a plurality of positions at a predetermined position of the contact state detection portion 43B. In this example, the mark 45 refers to an opaque member configured at positions of lattice points divided at equal intervals inside the contact state detection portion 43B. Furthermore, the mark 45 is configured inside the contact state detection portion 43B, but is not limited thereto, and can be provided to the contact surface 41B or the back surface of the contact surface 41B. In addition, the mark 45 is described as being discretely configured at the positions of the lattice points, but is not limited thereto. The mark 45 can also be a lattice pattern or other continuous pattern.

[0074] In this example, at least a portion of the contact state detection portion 43B is deformed along the shape of the holding object 21 in contact with the contact surface 40 as a contact surface. For example, in the case where the contact state detection portion 43B is composed of a transparent material, as a specific material of the contact state detection portion 43B, there is a silicon material having a thickness of 2 millimeters and a transmittance of about 94%.

[0075] At least a portion of the contact state detection portion 43B is deformed along the shape of the holding object 21, whereby the position of the mark 45 also displaces. The holding device control portion 90 detects the contact state between the holding object 21 and the contact surface 41B based on the displacement of the mark 45 photographed by the photographing portion 30.

[0076] Furthermore, the pattern of the mark 45 can also be an irregular pattern to make it easy to detect the contact state of the holding object 21 with the contact surface 41B. In addition, the mark 45 is described as an opaque member, but is not limited thereto, and can be a translucent member or a transparent member as long as the displacement when the holding object contacts can be optically recognized.

[0077] Further, the contact state detecting section 43 is not limited to the example shown in the figure. The contact state detecting section 43 can be configured to detect the deformation of the contact state detecting section 43 by refraction of light, for example, as long as it can detect the contact state of the held object 21. At this time, the contact state detecting section 43 can be configured such that light does not pass through the contact surface 41 to the back surface of the contact surface 41.

[0078] Figure 5 FIG. 1 is a diagram showing one example of a functional configuration of a holding device 100 in an embodiment.

[0079] The holding device 100 has, as a functional configuration, a holding device control section 90, an input section 92, an output section 93, a photographing section 30, an arm section 180, and a holding section 1.

[0080] The holding device control section 90 has a microcomputer, a RAM (Random access memory), a ROM (Read only memory), and the like, which are not shown, a communication section, and the like, which communicates with an external device.

[0081] The input section 92 acquires information from sensors such as a pressure sensor, a position sensor, a temperature sensor, and an acceleration sensor, a camera, a microphone, and the like, which are not shown. In a case where the information output from the input section 92 and the information acquired by the holding device control section 90 are not distinguished, it is described as input information IIN.

[0082] The output section 93 outputs a drive signal to a motor, which is not shown, for driving a robot, and the like. In a case where the information output from the holding device control section 90 to the output section 93 is not distinguished, it is described as output information IOUT.

[0083] The holding device control section 90 outputs the output information IOUT to the output section 93 on the basis of the input information IIN acquired from the input section 92. Hereinafter, the holding device control section 90 is also described as a drive control section.

[0084] The photographing section 30 outputs information of a photographed image to the holding device control section 90 as image information IIMG.

[0085] The gripping device control section 90 outputs drive information I DRV containing information of the drive arm section 180 and the gripping section 1 connected to the front end of the arm section 180, based on the acquired image information I IMG. Specifically, the gripping device control section 90 drives at least one of the gripping section module 10 and the arm section 180, based on the specific amount information ITGT indicating the contact state when the contact surface 41 (the first surface) is in contact with the specific amount of the gripping object 21, and information indicating the contact state photographed by the photographing section 30 when the gripping section module 10 is gripping at least a part of the gripping object 21, so that the amount of the gripping object 21 being gripped approaches the information indicated by the specific amount information ITGT. More specifically, the gripping device control section 90 compares the information indicated by the specific amount information ITGT and the information indicated by the image information I IMG, outputs the drive information I DRV based on the result of the comparison, and drives at least one of the gripping section module 10 and the arm section 180.

[0086] Here, the specific amount information ITGT can also be stored in the storage section 84. Specifically, the specific amount information ITGT is information indicating the contact state when the contact surface 41 (the first surface) is in contact with the specific amount of the gripping object 21. The specific amount refers to the amount that is the target of gripping by the gripping device 100. The specific amount can be weight, or volume.

[0087] The storage section 84 can also be provided on the gripping device 100.

[0088] Further, the gripping device 100 can also be configured to update the specific amount information ITGT stored in the storage section 84 by outputting specific amount update information IFB to the storage section 84, in a case where it is learned that the result of the plurality of gripping is that there is an error between the specific amount information ITGT and the actual specific amount.

[0089] Figure 6 is a diagram for explaining the action of the gripping section 1 on the gripping object 21 in the embodiment.

[0090] Figure 6 (A) is a cross-sectional view of the x-z plane of the gripping device 100 in a case where the gripping section 1 is in contact with the gripping object 21. Figure 6 (B) is a diagram indicating one example of the image I10 photographed by the photographing section 30 provided to the gripping section 1. Figure 6 (C) is a diagram indicating one example of the image I20 photographed by the photographing section 30 provided to the support section 2.

[0091] Reference Signs List Figure 6 (A) explains the action of the gripping section 1 on the gripping object 21.

[0092] The gripping device control section 90 controls the arm section 180 so as to be able to displace in the x-axis direction, the y-axis direction, and the z-axis direction, and so as to be able to rotate around each axis. In addition, the gripping device control section 90 controls the distance Ll between the contact surface 41 of the gripping section 1 and the support surface 42 of the support section 2. That is, the gripping section 1 has six degrees of freedom plus one degree of freedom, that is, seven degrees of freedom.

[0093] The gripping device control section 90 adjusts the amount of gripping to be a certain amount by changing the position of the gripping section 1 with respect to the gripping object 21 The imaging section 30 provided to the gripping section 1 images the contact surface 41.

[0094] Figure 6 (B) is an example of an image I10 of the gripping object 21 imaged by the imaging section 30 provided to the gripping section 1. The image I10 is represented by a two-dimensional orthogonal coordinate system of a yc-axis and a zc-axis. The yc-zc plane represents an image surface imaged in the y-z plane in (A). Figure 6 The image surface imaged in the y-z plane in (A).

[0095] In the image I10, the portion occupied by the gripping object 21 is set as an object image I11. In the image I10, the portion occupied by the background is set as a background image I12. The background refers to a portion in which the gripping object 21 is not imaged.

[0096] In addition, the support section 2 can have the same structure as the gripping section 1. In this example, a case in which the support section 2 has the same structure as the gripping section 1 is described. In the case in which the support section 2 has the same structure as the gripping section 1, the support section 2 has the imaging section 30.

[0097] Figure 6 (C) is an image I20 of the gripping object 21 imaged by the imaging section 30 provided to the support section 2. The image I20 is represented by a two-dimensional orthogonal coordinate system of a yc-axis and a zc-axis. The yc-zc plane represents an image surface imaged in the y-z plane in (A). Figure 6 The image surface imaged in the y-z plane in (A).

[0098] In the image I20, the portion occupied by the gripping object 21 is set as an object image I21. In the image I20, the portion occupied by the background is set as a background image I22. The background refers to a portion in which the gripping object 21 is not imaged.

[0099] Figure 7 is a diagram illustrating an example of a series of actions of the gripping device 100 in the embodiment. With reference to this diagram, the series of actions of the gripping device 100 is described.

[0100] (Step S10) The holding device control section 90 acquires the specific amount information ITGT indicating the specific amount as the holding target from the storage section 84. The storage section 84 storing the specific amount information ITGT can be provided in the holding device 100 or can not be provided in the holding device 100. In the case where the storage section 84 is not provided in the holding device 100, the holding device control section 90 communicates with the storage section 84 through a not-shown communication section, thereby acquiring the specific amount information ITGT. After the specific amount information ITGT is acquired, the holding device control section 90 causes the process to proceed to step Sll.

[0101] (Step Sll) The holding device control section 90 drives the arm section 180 provided in the holding section 1, and causes the holding section 1 to approach the holding target object 21. The holding device control section 90 causes the process to proceed to step S12.

[0102] (Step S12) The photographing section 30 provided in the holding section 1 photographs the holding target object 21. The holding device control section 90 acquires the image information IIMG from the photographing section 30. The holding device control section 90 causes the process to proceed to step S13.

[0103] (Step S13) The holding device control section 90 compares the information indicating the contact state shown by the image information IIMG acquired from the holding section 1 with the information indicated by the specific amount information ITGT acquired from the storage section 84. Here, the information indicating the contact state can be, for example, the ratio between the image of the object shown by the image information IIMG and the image of the background. In the case where the image information IIMG is the image II0, the information indicating the contact state can be the ratio of the area of the object image II1 to the area of the background image II2.

[0104] In the case where the information indicating the contact state shown by the image information IIMG is within the range indicated by the specific amount information ITGT (i.e., step S13; YES), the process is caused to proceed to step S19. In the case where the information indicating the contact state shown by the image information IIMG is not within the range indicated by the specific amount information ITGT (i.e., step S13; NO), the process is caused to proceed to step S14.

[0105] (Step S14) The holding device control section 90 extracts the difference between the information indicating the contact state shown by the image information IIMG acquired from the holding section 1 and the information indicated by the specific amount information ITGT. In the case where the information indicating the contact state shown by the image information IIMG acquired from the holding section 1 is smaller than the information indicated by the specific amount information ITGT (i.e., step S14; YES), the process is caused to proceed to step S16. In the case where the information indicating the contact state shown by the image information IIMG acquired from the holding section 1 is larger than the information indicated by the specific amount information ITGT (i.e., step S14; NO), the process is caused to proceed to step S18.

[0106] (Step S16) The holding device control section 90 controls at least one of the arm section 180 or the holding section module 10, thereby driving to a position where the difference is further taken. That is, an action of taking (holding) the holding target object more is performed. Specifically, regarding the arm section 180, for example, if the downward direction is staggered, the contact area of the holding section module 10 with the holding target object increases, and the holding amount increases. Regarding the holding section module 10, for example, if the interval between the holding section 1 and the support section 2 is made narrower than the state where the holding section module 10 is inserted into the stack of the holding target objects, the holding amount increases. Further, both the arm section 180 and the holding section module 10 can be caused to perform the above respective actions to hold the holding target object more. After the control of driving the arm section 180 and the holding section module 10 as described above is performed, the holding device control section 90 causes the process to proceed to step S12.

[0107] (Step S18) The holding device control section 90 controls at least one of the arm section 180 or the holding section module 10, thereby driving to a position where the difference is discarded. That is, an action of taking (holding) the holding target object less is performed. Specifically, regarding the arm section 180, for example, if the upward direction is staggered, the contact area of the holding section module 10 with the holding target object decreases, and the holding amount decreases. Regarding the holding section module 10, for example, if the interval between the holding section 1 and the support section 2 is expanded to return to the state where the holding section module 10 is inserted into the stack of the holding target objects, the holding amount decreases. Further, both the arm section 180 and the holding section module 10 can be caused to perform the above respective actions to hold the holding target object less. After the control of driving the arm section 180 and the holding section module 10 as described above is performed, the holding device control section 90 causes the process to proceed to step S12.

[0108] (Step S19) The holding device control section 90 moves the held holding target object 21 to a prescribed position while maintaining the holding state of the holding section 1. The maintenance of the holding state means, for example, maintenance of the distance LI between the contact surface 41 of the holding section 1 and the support surface 42 of the support section 2.

[0109] [One example of control based on the ratio of the object to the background] Figure 8 is a view that shows one example of control based on the ratio of the object to the background in the embodiment.

[0110] Here, as one example of comparison of the information indicating the contact state shown by the specific amount information ITGT and the image information IIMG, the holding device control section 90 can also perform control based on the ratio of the object to the background. With reference to this view, one example of control based on the ratio of the object to the background is described.

[0111] In this graph, the horizontal axis corresponds to "the ratio of the object to the background", and the vertical axis corresponds to the "action of the arm 180", and is expressed in the relational expression W1.

[0112] The grip device control section 90 calculates the ratio of the object to the background shown in the acquired image. In this example, the "ratio of the object to the background" is explained as the ratio of the object to the background. The grip device control section 90 determines the action in the case where the calculated value is between the symbol P13 and the symbol P14. In the case where the calculated value is smaller than the symbol P13, the grip device control section 90 drives the arm 180 to perform the action of taking out the grip object 21 more. In the case where the calculated value is larger than the symbol P14, the grip device control section 90 drives the arm 180 to perform the action of discarding the grip object 21.

[0113] That is, in this example, the grip device control section 90 controls the arm 180 based on the information showing the ratio of the portion representing the background to the portion representing the grip object shown by the specific quantity information, and the image representing the contact state captured by the imaging section 30.

[0114] Further, in this example, the relational expression W1 is explained as a linear function, but the relational expression W1 is not limited to this example. The relational expression W1 can also be configured to be calculated by a regression model of a neural network, a support vector machine, or the like.

[0115] [One example of the control based on the ratio of the edge] Figure 9 is a graph showing one example of the control based on the ratio of the edge in the embodiment.

[0116] Here, as one example of the comparison of the information representing the contact state shown by the specific quantity information ITGT and the image information IIMG, the grip device control section 90 can also perform the control based on the ratio of the edge shown by the image information IIMG.

[0117] Specifically, there is a case where the weight indicated by the specific amount information ITGT has a positive correlation with the proportion of the edge indicated by the image information IIMG. For example, in a case where the focal position of the imaging section 30 is adjusted to the contact surface 41, the more the held object 21 approaches (sticks to) the contact surface 41, the more clearly the held object 21 is imaged in the image information IIMG, and thus the proportion of the edge indicated by the image information IIMG increases. On the other hand, the more the held object 21 is away from the contact surface 41 (the degree of sticking decreases), the more the blur amount increases, and thus the proportion of the edge indicated by the image information IIMG decreases. That is, the proportion of the edge indicated by the image information IIMG has a correlation with the distance between the held object 21 and the contact surface 41. The more the held object 21 sticks to the contact surface 41, the more the weight is, and thus there is a case where the weight indicated by the specific amount information ITGT has a positive correlation with the proportion of the edge indicated by the image information IIMG.

[0118] With reference to the figure, one example of control based on the proportion of the edge indicated by the image information IIMG will be described.

[0119] In the figure, the horizontal axis corresponds to the "proportion of the edge", and the vertical axis corresponds to the "action of the arm section 180", and indicates the relationship in the relationship W2.

[0120] The holding device control section 90 calculates the edge of the acquired image. For example, the holding device control section 90 calculates the edge of the image indicated by the image information IIMG by applying image processing such as a differential filter. In this example, the edge refers to a portion where the luminance of the image indicated by the image information IIMG changes greatly. The holding device control section 90 determines the action in a case where the calculated value is between the sign P23 and the sign P24. In a case where the calculated value is less than the sign P23, the holding device control section 90 drives the arm section 180 to perform the action of taking out the held object 21 more. In a case where the calculated value is greater than the sign P24, the holding device control section 90 drives the arm section 180 to perform the action of discarding the held object 21.

[0121] That is, in this example, the holding device control section 90 controls the arm section 180 based on the information indicating the proportion of the portion where the luminance of the held object 21 changes greatly, which is indicated by the specific amount information, and the image indicating the contact state imaged by the imaging section 30.

[0122] Further, in this example, a case where the relationship W2 is a linear function is described, but the relationship W2 is not limited to this example. The relationship W2 can also be configured to be calculated by a regression model such as a neural network, a support vector machine, or the like.

[0123] [One example of control based on the color of the image] Figure 10is a graph showing one example of the control of the color of the image in the embodiment.

[0124] Here, as one example of the comparison of the information indicating the contact state shown by the specific quantity information ITGT and the image information IIMG, the holding device control section 90 can also perform the control of the color of the image shown by the image information IIMG. With reference to the graph, one example of the control of the color of the image shown by the image information IIMG is described.

[0125] In the graph, the horizontal axis corresponds to "R" and the vertical axis corresponds to the "action" of the arm section 180, and indicates the relational expression W3. In this example, the case where the color is R (red) is described, but a relational expression based on the three axes of RGB can also be used.

[0126] The holding device control section 90 calculates the color of the acquired image. For example, the holding device control section 90 calculates the average value of the color of the image shown by the image information IIMG. In this example, the color refers to the average color of the image shown by the image information IIMG, but can also be the color of the portion occupied by the object, and can also be configured to perform the action based on the color of each point of the image information IIMG.

[0127] The holding device control section 90 determines the action in the case where the calculated value is between the symbols P33 and P34. In the case where the calculated value is less than the symbol P33, the holding device control section 90 drives the arm section 180 to perform the action of taking out the holding object 21 more. In the case where the calculated value is greater than the symbol P34, the holding device control section 90 drives the arm section 180 to perform the action of discarding the holding object 21.

[0128] That is, in this example, the holding device control section 90 controls the arm section 180 based on the information indicating the color of the holding object 21 shown by the specific quantity information and the image indicating the contact state photographed by the photographing section 30.

[0129] Further, in this example, the case where the relational expression W3 is a linear function is described, but the relational expression W3 is not limited to this example. The relational expression W3 can also be configured to be obtained by a regression model such as a neural network, a support vector machine, or the like.

[0130] [Summary of Effects of Embodiment] As described above, the holding device 100 of the present embodiment has the holding section 1 and the arm section 180, thereby holding the specific quantity in the holding object 21. The holding device control section 90 controls the arm section 180 based on the image acquired by the photographing section 30 provided to the holding section 1. The holding device control section 90 compares the information acquired from the storage section 84 with the information shown by the image acquired by the photographing section 30, and controls the arm section 180.

[0131] Therefore, the holding device 100 controls the arm portion 180 based on the image acquired by the imaging portion 30 provided to the holding portion 1, and thereby can hold the specific amount.

[0132] In addition, the holding device 100 of the present embodiment is provided with the imaging portion 30 at a position from which the holding target 21 can be imaged from the back of the contact surface 41 of the holding portion 1.

[0133] For example, compared to a case in which a camera is provided on a ceiling as in the related art, the holding device 100 of the present embodiment can acquire more information about the holding target 21.

[0134] Therefore, according to the present embodiment, the holding device 100 can hold the specific amount more accurately from the holding target 21.

[0135] In addition, the holding device 100 of the present embodiment controls the arm portion 180 based on information showing the proportion of a portion representing the background to a portion representing the holding target.

[0136] Therefore, the holding device control portion 90 can estimate how much the holding portion 1 holds, and thereby can hold the specific amount.

[0137] In addition, the holding device 100 of the present embodiment controls the arm portion 180 based on the proportion of a portion in which the brightness of the holding target 21 changes greatly.

[0138] Here, the holding target 21 held by the holding device 100 of the present embodiment is sometimes a soft holding target. In the case where the holding target 21 is soft and in the case where it is hard, there are cases in which the weight differs.

[0139] Therefore, according to the holding device 100 of the present embodiment, by controlling the arm portion 180 based on the proportion of a portion in which the brightness of the holding target 21 changes greatly, the specific amount can be estimated according to the state of the holding target 21 (for example, the softness of the holding target 21).

[0140] In addition, the holding device 100 of the present embodiment controls the arm portion 180 based on the color of the holding target 21.

[0141] Here, the holding target 21 held by the holding device 100 of the present embodiment is sometimes different in weight according to the color. For example, there can be cases in which yellow-green cabbage is light and dark green cabbage is heavy.

[0142] Therefore, according to the holding device 100 of the present embodiment, by controlling the arm portion 180 based on the color of the holding target 21, the specific amount can be estimated more accurately.

[0143] Moreover, the entire or a part of the functions of the respective sections of the holding device 100 in the embodiments can be realized by recording a program for realizing the functions in a computer-readable recording medium and causing a computer system to read in and execute the program recorded in the recording medium. Here, the "computer system" includes an OS and hardware such as a peripheral device.

[0144] Further, the "computer-readable recording medium" refers to a portable medium such as a magneto-optical disk, a ROM, a CD-ROM, and the like, or a storage section such as a hard disk included in a computer system. Further, the "computer-readable recording medium" refers to a medium that can include a medium that dynamically retains a program for a short time, such as a communication line in the case where a program is transmitted through a network such as the Internet, and a medium that retains a program for a certain time, such as a volatile memory in the computer system of a server or a client in this case. Further, the above program can be a program for realizing a part of the above functions, and can also be a program that realizes the above functions by a combination with a program already recorded in the computer system.

[0145] The above describes the mode for implementing the present application using the embodiments, but the present application is not limited to any of the embodiments, and various modifications and substitutions can be made within the scope of the gist of the present application.

Claims

1. A gripping device comprising: a gripping module having a first surface and a second surface for gripping an object between the first surface and the second surface; an arm for changing the position of the gripping module; a shooting unit disposed at a position that moves together with the gripping module and shooting at least a portion of the object; and a control unit capable of comparing specific quantity information indicating a contact state when the first surface is in contact with a specific amount of the object with information indicating a contact state captured by the shooting unit when the gripping module grips at least a portion of the object, and based on the comparison result, outputting driving information for driving the gripping module or the arm, and controlling at least one of the gripping module or the arm to bring the amount of the gripped object close to the specific quantity.

2. The gripping device according to claim 1, wherein, The camera unit is located on the back side of the first surface and is capable of capturing the position of the object.

3. The gripping device according to claim 1 or 2, wherein, The control unit can control at least one of the gripping module or the arm based on the specific quantitative information, namely, information showing the ratio of the portion representing the background to the portion representing the object, and information showing the contact state of the shooting unit when the gripping module holds at least a portion of the object.

4. The gripping device according to any one of claims 1-3, wherein, The control unit can control at least one of the gripping module or the arm based on the specific quantity information, namely the information indicating the proportion of the portion of the object with large brightness changes, and the information indicating the contact state of the shooting unit when the gripping module holds at least a portion of the object.

5. The gripping device according to any one of claims 1-4, wherein, The control unit can control at least one of the gripping module or the arm based on the specific quantity information, namely, information indicating the color of the object, and information indicating the contact state of the shooting unit when the gripping module holds at least a part of the object.

6. A control method for a holding device. The holding device includes: a holding section module having a first surface and a second surface, which holds an object between the first surface and the second surface; The arm portion, which causes the position of the gripping module to change; and The imaging unit is positioned to move together with the holding module and captures images of at least a portion of the object. The control method includes: The system can compare specific quantity information indicating the contact state when the first surface is in contact with a specific amount of the object with information indicating the contact state captured by the shooting unit when the holding module holds at least a part of the object. Based on the comparison result, it outputs driving information for driving the holding module or the arm, and controls at least one of the holding module or the arm to make the amount of the object held approach the specific quantity.

7. A program that causes a computer connected to a holding device to perform control steps. The holding device includes: a holding section module having a first surface and a second surface, which holds an object between the first surface and the second surface; The arm portion, which causes the position of the gripping module to change; and The imaging unit is positioned to move together with the holding module and captures images of at least a portion of the object. The control step can compare specific quantity information indicating the contact state when the first surface is in contact with a specific amount of the object with information indicating the contact state captured by the shooting unit when the holding module holds at least a part of the object. Based on the comparison result, drive information for driving the holding module or the arm is output, and at least one of the holding module or the arm is controlled so that the amount of the object held is close to the specific quantity.

Images