Holding system and program

Abstract

Based on the state of the object, the area to be retained is determined. [Solution] The holding system 1 comprises an articulated robot 30, a housing state detection sensor 41, and a control device 70. The articulated robot 30 performs holding on an object present in the target area. The housing state detection sensor 41 detects the state of the object using waves. The control device 70 controls the articulated robot 30. Based on the parameters relating to the height of the object in each of the multiple candidate target areas detected by the housing state detection sensor 41, the control device 70 determines the target area to be actually held from among the multiple candidate target areas.

Description

Technical Field

[0001] The present invention relates to a holding system and a program.

Background Art

[0002] In recent years, work has been carried out by robots to process various objects. For example, there are robots that hold an object in a storage container in which the object is stored and release the held object into another container.

[0003] Techniques related to such robots are disclosed in, for example, Patent Document 1. In the technique disclosed in Patent Document 1, a container in which a holding object is stored is divided into a plurality of regions. And it is disclosed that holding is repeated by circulating through these plurality of regions in order, such as clockwise or counterclockwise.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When holding is repeated by the method as disclosed in Patent Document 1 described above, the state of the ingredients stored along with the holding gradually changes. However, in the prior art disclosed in Patent Document 1 and the like, sufficient consideration has not been given to such changes in the state of the ingredients. Therefore, it is desirable to determine a region to be held based on the state of the object.

[0006] Furthermore, these challenges are not limited to cases where the object is food, but are common to various fields in which robots perform processing, such as in the industrial sector. Moreover, the processing performed by robots is not limited to cases where the object is held by grasping. For example, these challenges are also common when the object is held by methods such as suction.

[0007] The object to be addressed by this invention is to determine the area to be held based on the state of the object. [Means for solving the problem]

[0008] To solve the above problems, a holding system according to one embodiment of the present invention is A robot that holds objects in the target area, A detection means for detecting the state of the object by wave motion, Control means for controlling the robot, Equipped with, The control means is Based on the parameters relating to the height of the object in each of the multiple candidate target areas detected by the detection means, the target area to be actually held is determined from the multiple candidate target areas. It is characterized by the following: [Effects of the Invention]

[0009] According to the present invention, the area to be held can be determined based on the state of the object. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram illustrating the configuration of the holding system 1 according to the present invention. [Figure 2] This is a perspective view showing the configuration of the gripping mechanism 31. [Figure 3] This is a front view showing the gripping mechanism 31 in the open state (i.e., the state during the release operation). [Figure 4]It is a front view showing the closed state (i.e., the state during the gripping operation) of the gripping mechanism 31. [Figure 5] It is a schematic diagram showing the hardware configuration of the control device 70. [Figure 6] It is a block diagram showing the functional configuration of the control device 70. [Figure 7] It is a schematic diagram showing the state of the holding system 1 when performing a gripping operation or a releasing operation. [Figure 8] It is a schematic diagram showing the state of the holding system 1 when performing a gripping operation or a releasing operation. [Figure 9] It is a schematic diagram showing the state of the holding system 1 when performing a gripping operation or a releasing operation. [Figure 10] It is a schematic diagram showing a method for determining a target area to be gripped by the gripping area determination unit 154. [Figure 11] It is a schematic diagram showing a method for determining a target area to be gripped by the gripping area determination unit 154. [Figure 12] It is a schematic diagram showing a method for determining a target area to be gripped by the gripping area determination unit 154. [Figure 13] It is a flowchart showing the flow of the ingredient loading process executed by the holding system 1. [Figure 14] It is a diagram showing the setting of a plurality of candidate areas C in the second modification. [Figure 15] It is a schematic diagram showing the determination of the shape of the candidate area C in the third modification. [Figure 16] It is a schematic diagram showing how to determine the shape of the candidate area C based on a range different from the movable range of the gripping member 314 in the fourth modification. [Figure 17] It is a diagram showing the configuration of the gripping mechanism 31a in the sixth modification.

Embodiments of the Invention

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [Embodiment] [Overall structure] Figure 1 is a schematic diagram illustrating the configuration of the holding system 1 according to the present invention. Here, the holding system 1 is intended to be a system that grasps and releases food as an object, and the present invention is intended to be applied to this system. In the following description, an example will be given in which the holding system 1 grasps ingredients such as those in prepared foods and releases these grasped ingredients to place them in a container.

[0012] However, this is merely an example for illustrative purposes and is not intended to limit the scope of application of the present invention. The present invention is applicable to any system for holding an object. For example, it can be applied to systems that hold uncooked vegetables (such as shredded cabbage, carrots, or bean sprouts) rather than prepared foods. It can also be applied to systems that hold industrial products such as electronic devices (such as screws, bolts, or product components) as objects. Furthermore, the method of holding the object is not limited to gripping it. For example, holding may be achieved by methods other than gripping, such as suction. Alternatively, holding may be achieved by wrapping the object, such as noodles, around a fork-shaped member, or by inserting a skewer-shaped member into an object, such as meat, fish, or vegetables, to pierce it. Furthermore, the method of releasing the object into the container is not limited to releasing it directly into the container. For example, a robot may release the object into the discharge chute's input port, and then supply the object to the container from the discharge port of the discharge chute for filling. In other words, the present invention can be implemented in a wide range of holding systems, regardless of the object to be gripped, the specific method of holding, or the field in which it is applied.

[0013] As shown in Figure 1, the holding system 1 comprises a food container 10, a container supply device 20, an articulated robot 30, a food storage state detection sensor 41, a serving state detection sensor 42, a first weighing scale 51, a second weighing scale 52, a base 60, a control device 70, and a first transport path T1. Of these components, the container supply device 20, the articulated robot 30, the storage state detection sensor 41, the serving state detection sensor 42, the first weighing scale 51, the second weighing scale 52, the first transport path T1, and the control device 70 are connected to each other by wired or wireless communication and are able to communicate with one another.

[0014] Adjacent to the holding system 1, a belt conveyor 2 is installed to automatically transport the prepared food containers from upstream to downstream. The belt conveyor 2 has a second transport path T2, which is a transport surface for transporting the containers. The containers, which have been filled with ingredients by the articulated robot 30, are then transported downstream while placed on this second transport path T2. In Figure 1, as indicated by the arrows representing the second transport direction, the left side of the page represents the upstream of transport in the second transport path T2, and the right side of the page represents the downstream of transport. A location for subsequent operations (e.g., closing the container lids) is provided downstream of the second transport path T2.

[0015] Although Figure 1 shows only one holding system 1, it is not limited to this. In this embodiment, multiple holding systems 1 are installed along the second transport direction of a single belt conveyor 2, and it is assumed that multiple articulated robots 30 work together.

[0016] The ingredient container 10 has a storage space for storing ingredients such as side dishes that are to be served in the holding system 1. The ingredient container 10 can be implemented using a general-purpose container such as a large tray or tub. The storage space of the ingredient container 10 can hold various ingredients that can be grasped by the articulated robot 30, such as paste salads like potato salad (i.e., side dishes containing ingredients with viscosity or stickiness), okara (soy pulp), dried daikon radish, namasu (pickled daikon radish and carrots), hijiki seaweed, boiled beans, fiddlehead ferns, buttered corn, noodles, croquettes, and fried chicken. In this embodiment, the ingredient container 10 is assumed to contain multiple servings (for example, several dozen to several hundred servings) of one type of ingredient. When the amount of ingredients stored in the ingredient container 10 becomes low, it can be replaced manually by an operator or automatically by the articulated robot 30.

[0017] The container supply device 20 is a device that supplies containers for the articulated robot 30 to put ingredients into. Inside the container supply device 20, a large number of containers are housed vertically. When the holding system 1 starts operating, the container supply device 20 supplies containers by discharging them one by one into the first transport path T1.

[0018] The first transport path T1 transports the containers supplied by the container supply device 20 to the serving position 62, where the ingredients are placed, by the articulated robot 30. The first transport path T1 is installed on the top surface of the base 60 and includes a transport mechanism for transporting the containers to the serving position 62 and an extrusion mechanism for pushing the containers from the serving position 62 to the second transport path T2. The conveying mechanism may be a mechanism that conveys the container by contacting the bottom surface of the container with the conveying surface, such as a belt conveyor 2, or it may be a mechanism that conveys the container by having a component contact the side or back of the container and pushing it out. When the container is transported to the serving position 62 by the transport mechanism, the articulated robot 30 releases the ingredients, and the released ingredients are placed into the container supplied to the serving position 62. Then, the extrusion mechanism pushes the filled container onto the second transport path T2 of the belt conveyor 2. As a result, the filled container is transported downstream on the second transport path T2.

[0019] The articulated robot 30 is composed of, for example, a horizontal articulated robot or a vertical articulated robot, and includes a gripping mechanism 31 capable of gripping an object, a robot arm 32 for moving the gripping mechanism 31 to an arbitrary position, and a robot base end 33.

[0020] The gripping mechanism 31 is attached to the tip of the robot arm 32 and is supported by the robot arm 32. The gripping mechanism 31 can then be moved to any desired position in accordance with the movement of the robot arm 32 based on the control of the control device 70. Furthermore, the joint that holds the gripping mechanism 31 is equipped with an axis that rotates the gripping mechanism 31 in a twisting direction relative to the robot arm 32. Therefore, when the gripping mechanism 31 grips an ingredient, the direction in which the gripping mechanism 31 grips can be adjusted by changing the orientation of the gripping mechanism 31. As a result, when the gripping mechanism 31 reaches near the inner wall surface of the ingredient container 10, it becomes possible to change the orientation of the gripping mechanism 31 to a direction parallel to the inner wall surface of the ingredient container 10, making it easier to grip ingredients near the inner wall surface of the container. The robot base portion 33 is the base part that supports the robot arm 32 to which the gripping mechanism 31 is attached. The robot base portion 33 is installed on the frame 63, which will be described later.

[0021] The contents state detection sensor 41 is a sensor that detects the contents state of the contents contained in the contents container 10. The contents state detection sensor 41 is implemented, for example, by a depth camera capable of detecting the distance to a subject. In this case, the field of view of the contents state detection sensor 41 is set to be such that it can capture the entire opening surface of the contents container 10. To this end, the contents state detection sensor 41 is positioned, for example, vertically above the center of the contents container 10 in the horizontal plane. The control device 70 can determine the amount of ingredients remaining in each area of ​​the ingredient container 10, the degree of surface roughness (unevenness), etc., by analyzing the distance information detected by the storage state detection sensor 41 (i.e., depth information for each pixel across the entire opening surface of the ingredient container 10).

[0022] The serving state detection sensor 42 is a sensor that detects the serving state of the container in which the ingredients are placed and the operating status of the gripping mechanism 31. Similar to the storage state detection sensor 41, the serving state detection sensor 42 is implemented, for example, by a depth camera capable of detecting the distance to the subject. In this case, the field of view of the serving state detection sensor 42 is set to capture, for example, the entire opening surface of the container in which the ingredients are placed and the operating status of the gripping mechanism 31 that places the ingredients into the container. To this end, the serving state detection sensor 42 is positioned, for example, vertically above the center of the serving position 62 in the horizontal plane. The control device 70 can identify the arrangement state of the ingredients in each area of ​​the container where the ingredients are placed, the operating status of the gripping mechanism 31, etc., by analyzing the distance information detected by the food arrangement state detection sensor 42 (i.e., depth information for each pixel across the entire opening surface of the container where the ingredients are placed, and depth information of the gripping mechanism 31 regarding the behavior of the gripping mechanism 31).

[0023] The first weighing scale 51 and the second weighing scale 52 are both devices for detecting the weight of an object. The first weighing scale 51 and the second weighing scale 52 are implemented, for example, by weighing scales that measure weight using strain gauges. Here, the first weighing scale 51 is positioned where the ingredient container 10 will be placed. The first weighing scale 51 then detects the weight of the ingredient container 10 itself, as well as the weight of the ingredients contained in the ingredient container 10. Based on the values ​​detected by the first weighing scale 51, the control device 70 can determine the total weight of the ingredients contained (i.e., the remaining amount) and the increase or decrease in the total weight due to gripping or other actions of the ingredients contained (i.e., the change in the remaining amount).

[0024] The second weighing scale 52 is positioned in the holding system 1 at a predetermined serving position 62 where the ingredients are placed. The second weighing scale 52 detects the weight of the container itself supplied to this serving position 62, and the weight of the ingredients that are released into the container and placed inside. The control device 70 can determine the total weight of the ingredients that have been released into the container and placed inside, based on the detection value of the second weighing scale 52. The control device 70 can also determine, based on the detection value of the second weighing scale 52, whether the container has been supplied to the serving position 62 and whether the filled container has been pushed out onto the conveying surface of the belt conveyor 2.

[0025] The base 60 is a base for installing the ingredient container 10, the container supply device 20, the first transport path T1, and the articulated robot 30, etc. These articulated robots 30, etc., have a combined weight of several hundred kg (for example, more than 300 kg). Therefore, the base 60 has a structure with sufficient rigidity to support these components even when they are placed on its top surface.

[0026] Such a base portion 60 is equipped with casters 61, a serving position 62, and a stand 63. The caster 61 is a caster mounted on the base 60. The base 60 supports the articulated robot 30, etc., by contacting the ground with this caster 61. Furthermore, because this caster 61 functions as a wheel, the base 60 can be moved by human power. As described above, the serving position 62 is the position where the ingredients are placed by the articulated robot 30. The support frame 63 is provided vertically above the first transport path T1 and is a structure that supports the object to be supported vertically above the first transport path T1. In this embodiment, the robot base end portion 33 of the articulated robot 30 is installed as the object to be supported. As a result, in this embodiment, the first transport path T1 and the articulated robot 30 are arranged to overlap vertically. In this case, the legs of the support frame 63 are configured to straddle the first transport path T1. Therefore, they do not obstruct the transport of containers along the first transport path T1. In this holding system 1, instead of arranging the container transport path and the robot in close proximity and adjacent positions as in conventional technology, they are arranged to overlap vertically. This makes it possible to reduce the installation area of ​​the holding system 1 and to facilitate worker access to each component.

[0027] Furthermore, in the holding system 1, the main components of the holding system 1, such as the ingredient container 10, the container supply device 20, the articulated robot 30, and the first transport path T1, are all installed on a portable base 60. By using casters 61, these components can be moved as a single unit by human power. Therefore, it becomes possible to easily transport the holding system 1 and change its layout in food factories and other similar facilities.

[0028] The control device 70 is composed of an information processing device such as a PC (Personal Computer) or a programmable controller, and controls the entire holding system 1 by executing various programs. For example, the control device 70 controls the operation of the container supply device 20, such as supplying containers or pushing out containers that have already been filled, and the operation of the articulated robot 30, such as grasping ingredients from the ingredient container 10 and releasing them into the container to fill it with ingredients.

[0029] For example, the control device 70 controls the drive of the robot arm 32 to move the gripping mechanism 31 to a predetermined position along a predetermined route and at a predetermined speed, and controls the drive of the actuator of the gripping mechanism 31 to perform actions such as gripping and releasing the material using the gripping mechanism.

[0030] The components constituting the holding system 1 have been described above. In addition to these components, plate-like members may be further arranged to surround or above the locations where each component is installed. This plate-like member shields each component from the external space, preventing materials gripped or released by the articulated robot 30 from scattering from the internal space to the external space. It also prevents workers from coming into contact with the articulated robot 30 while it is in operation, thereby ensuring worker safety and preventing malfunctions of the articulated robot 30. In this case, the plate-like member should be made of a transparent material such as glass or resin, so that the operating status of the holding system 1 can be visually observed from the outside space. Furthermore, this plate-like member may be used to provide an additional, openable and closable door on a portion of the side wall. This allows workers to open the door and perform various tasks such as replacing or replenishing the ingredient container 10, adding containers to the container supply device 20, or performing maintenance on the holding system 1.

[0031] [Configuration of the gripping mechanism 31] Figure 2 is a perspective view showing the configuration of the gripping mechanism 31. As shown in Figure 2, the gripping mechanism 31 comprises a coupling member 311, two air cylinders 312, two connecting members 313, and two gripping members 314. In the following description, the front, back, left side, and right side of the gripping mechanism 31 are defined as indicated by arrows in the figure.

[0032] The connecting member 311 is a member that connects the gripping mechanism 31 and the robot arm 32. The gripping mechanism 31 is supported by the robot arm 32 when connected to the robot arm 32 by the connecting member 311, and moves in accordance with the movement of the robot arm 32. Two air cylinders 312 are also arranged inside the connecting member 311.

[0033] The two air cylinders 312 are a drive mechanism that can move back and forth in a straight line in the horizontal direction (in this case, left-right direction). In this case, the two air cylinders 312 are arranged so that their directions of movement are opposite to each other. The piston rods (i.e., the reciprocating parts) of the two air cylinders 312 are each connected to a connecting member 313 corresponding to itself.

[0034] The two connecting members 313 are members that connect the corresponding air cylinder 312 and the corresponding gripping member 314. The piston rod of the corresponding air cylinder 312 is connected to the upper end of the connecting member 313, and the upper end of the corresponding gripping member 314 is connected to the lower end of the connecting member 313. This forms a pair of sets consisting of the air cylinder 312, the connecting member 313, and the gripping member 314. When this pair of sets is driven, the gripping and releasing operations of the gripping mechanism 31 are realized.

[0035] The two gripping members 314 are the parts that grip the ingredients by contacting them. The two gripping members 314 are positioned so that their opening surfaces face each other. The shape of the two gripping members 314 is such that when their opening surfaces come close together and come into contact, they form a gripping space for gripping the ingredients. The gripping member 314 of this shape is composed of a plate-like member having multiple surfaces, and is provided with slits (i.e., narrow gaps) of a size that prevents the gripped ingredients from falling out. By providing these slits, it is possible to suppress the adhesion of sticky ingredients to the gripping member 360 (for example, the ingredients sticking to it). Therefore, it becomes possible to suppress the occurrence of situations where the attached ingredients fall off and to easily release the ingredients.

[0036] Figure 3 is a front view showing the gripping mechanism 31 in the open state (i.e., the state during the release operation). Figure 4 is a front view showing the gripping mechanism 31 in the closed state (i.e., the state during the gripping operation). As shown in Figures 3 and 4, the air cylinder 312 and the connecting member 313 are fastened together at the first point P1 in a rotatable manner using screws or the like. Furthermore, in the two connecting members 313, one connecting member 313 and the other connecting member overlap and intersect near the center, and at the intersection point, the second point P2, they are also fastened together at the same point in a rotatable manner using screws or the like. The lower end of the gripping member 314 is designated as the third point P3.

[0037] With this structure, the gripping mechanism 31 as a whole achieves a mechanism similar to that of a typical pair of scissors or forceps. In this case, the first point P1 functions as the point of force application, the second point P2 functions as the fulcrum, and the third point P3 functions as the point of application. The specific operation in this configuration will now be explained. In Figures 3 and 4, the forward and backward directions of the piston rods of each air cylinder are indicated by white arrows.

[0038] First, when transitioning to the open state, as shown in Figure 3, the piston rods of each of the two air cylinders 312 are driven to extend outwards from the gripping mechanism 31 (in this case, outwards in the left-right direction). In other words, the piston rods are driven to move away from each other. As a result, the first point P1, which is the point of force application, moves away from each other, and the third point P3, which is the point of application, also moves away from each other via the second point P2, which is the fulcrum. In this way, the gripping mechanism 31 transitions to the open state. Consequently, the gripping mechanism 31 can release the gripped material from the opening surface of the gripping member 314.

[0039] In contrast, when transitioning to the closed state, as shown in Figure 3, the piston rods of each of the two air cylinders 312 are driven to retract toward the inside of the gripping mechanism 31 (in this case, toward the inside in the left-right direction). That is, they are driven to move closer to each other. As a result, the first point P1, which is the point of force application, moves closer together, and the third points P3, which are the points of application, also move closer together via the second point P2, which is the fulcrum. In this way, the gripping mechanism 31 transitions to the closed state. Consequently, the edges of the opening surfaces of the gripping members 314 come into contact with each other, and a gripping space is formed on the inner surface of the gripping members 314. The gripping mechanism 31 can then grip the object by enclosing it in this gripping space.

[0040] Furthermore, because the gripping mechanism 31 as a whole has a scissor-like shape, the force driving the two air cylinders 312 horizontally is converted into a force that opens and closes the gripping member 314 diagonally (i.e., a combination of horizontal and vertical directions). This makes it easier to insert the gripping member 314 into the contents compared to when the gripping member 314 is directly connected to the air cylinders 312 and simply opens and closes horizontally. In other words, the gripping mechanism 31 performs an action that gathers the contained ingredients from the left-right and up-down directions toward the center of the gripping space using the two gripping members 314, making it possible to grip the ingredients more efficiently.

[0041] Note that one or more such gripping mechanisms 31 may be attached to a single robot arm 32. For example, as shown in Figure 1, two gripping mechanisms 31 may be arranged side by side on a single robot arm 32 so that their opening and closing directions are parallel.

[0042] [Hardware configuration of control device 70] Figure 5 is a schematic diagram showing the hardware configuration of the control device 70. As shown in Figure 5, the control device 70 includes a CPU (Central Processing Unit) 711, a ROM (Read Only Memory) 712, a RAM (Random Access Memory) 713, a bus 714, an input unit 715, an output unit 716, a storage unit 717, a communication unit 718, and a drive 719.

[0043] The CPU 711 executes various processes according to the program recorded in the ROM 712 or the program loaded into the RAM 713 from the storage unit 717. RAM713 also stores data necessary for CPU711 to perform various processes.

[0044] The CPU 711, ROM 712, and RAM 713 are interconnected via a bus 714. The input unit 715, output unit 716, storage unit 717, communication unit 718, and drive 719 are connected to the bus 714.

[0045] The input unit 715 is equipped with an input device such as a mouse or keyboard and accepts various types of information for input to the control device 70. Alternatively, the input unit 715 may be equipped with a microphone and accept various types of information via voice input from the operator. The output unit 716 consists of a display, speakers, etc., and outputs images and sound. The memory unit 717 consists of an SSD (Solid State Drive), HDD (Hard Disk Drive), or DRAM (Dynamic Random Access Memory), and stores various types of data managed by each server. The communications unit 718 controls communication with other devices via the network.

[0046] The drive 719 is appropriately equipped with removable media 731, which may consist of a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. Programs read from the removable media 731 by the drive 719 are installed in the storage unit 717 as needed. The above hardware configuration is the basic configuration of the control device 70, and it is possible to omit some hardware, add additional hardware, or change the hardware implementation.

[0047] [Functional configuration] Next, the functional configuration of the control device 70 will be described. Figure 6 is a block diagram showing the functional configuration of the control device 70. As shown in Figure 6, by executing a program to control the operation of the holding system 1, the CPU 711 of the control device 70 functions as follows: information acquisition unit 151, articulated robot control unit 152, container supply control unit 153, and recording control unit 155. In addition, the storage unit 717 is configured with a parameter storage unit 171 and a history database (history DB) 172.

[0048] The parameter storage unit 171 stores various parameters used when the holding system 1 operates. For example, the parameter storage unit 171 stores data that serves as control criteria for the gripping operation, such as the position and shape of the ingredient container 10, the position and shape of the containers supplied from the container supply device 20, the position of the area within the container where the ingredients are provided and placed, the weight per unit volume of the ingredients (i.e., the density of the ingredients), and the target amount of ingredients to be grasped and released, as well as parameters that define the operation pattern of the articulated robot 30.

[0049] The learning model memory unit 172 stores a learning model constructed to determine the region to be grasped. This learning model is constructed by the grasping region determination unit 154 through prior machine learning.

[0050] The history DB173 stores control parameters acquired when the holding system 1 is operating, as well as measurement data of the weight of the ingredients placed by the holding system 1, as operation history.

[0051] The information acquisition unit 151 acquires information detected by various sensors and weighing scales installed in the holding system 1. For example, the information acquisition unit 151 sequentially acquires distance information detected by the storage state detection sensor 41 and the serving state detection sensor 42. In addition, for example, the information acquisition unit 151 sequentially acquires weight values ​​detected by the first weighing scale 51 and the second weighing scale 52. In this way, the various types of information acquired by the information acquisition unit 151 are used as appropriate by other functional blocks provided by the control device 70.

[0052] The articulated robot control unit 152 controls the movement of the articulated robot 30 and causes the articulated robot 30 to perform a series of operations for serving ingredients based on the operation patterns defined in the holding system 1 and various information acquired by the information acquisition unit 151. For example, the articulated robot control unit 152 causes the articulated robot 30 to perform operations such as gripping the ingredients with the gripping mechanism 31 (gripping operation), releasing the ingredients gripped by the gripping mechanism 31 (release operation), and removing ingredients attached to the gripping mechanism 31 (removal operation).

[0053] For example, when performing a gripping operation, the articulated robot control unit 152 determines the surface height of the ingredients at each position within the ingredient container 10 in advance, based on the distance information detected by the storage state detection sensor 41. Then, the articulated robot control unit 152 inserts the gripping member 314 into the group of ingredients to a predetermined depth from this height and performs the gripping operation. This predetermined depth can be set in advance based on the density of the ingredients, the characteristics of the ingredients (viscosity, etc.), the number and shape of the gripping members 314 to be used for gripping, and the target amount (in this case, the target weight) to be targeted for gripping and releasing. By setting the insertion depth in advance in this way, the gripping member 314 can grip ingredients of the same weight as the target amount, or a weight close to the target amount.

[0054] Furthermore, for example, when performing a release operation, the articulated robot control unit 152 pre-determines the position and shape of the container on which the ingredients are placed, as well as the position of the area within the container where the ingredients are placed, based on data that serves as the control standard for the release operation and is stored in the parameter storage unit 171. Therefore, the articulated robot control unit 152 moves the gripping member 314 to an appropriate position based on this determined position and then performs the release operation. Note that this appropriate position includes not only the position in the horizontal plane, but also the position in the vertical direction (i.e., height).

[0055] In this embodiment, the weight of the ingredients that were actually grasped and the weight of the ingredients that were actually released can be determined based on the weight values ​​detected by the first weighing scale 51 and the second weighing scale 52. The articulated robot control unit 152 then performs various adjustment operations if these weights are far from the target amount. For example, if the weight of the grasped ingredients is too much or too little from the target amount, the grasped ingredients are released from the ingredient container 10 and the grasping is repeated to grasp only the target amount. Alternatively, if the weight of the released ingredients is too much or too little from the target amount, the robot returns to the ingredient container 10, grasps the missing weight and releases it into the container, or grasps the excess weight from the container and returns it to the ingredient container 10.

[0056] Furthermore, in this embodiment, the state of the ingredient container 10 and the ingredients in the container (for example, the height and unevenness of the ingredient surface) can be identified based on the distance information detected by the storage state detection sensor 41 and the serving state detection sensor 42. The articulated robot control unit 152 then performs various adjustment operations based on the state of these ingredients. For example, it adjusts the position for gripping and releasing based on the state of these ingredients. Alternatively, it shapes the serving state of the ingredients by leveling the surface of the released ingredients based on the state of the ingredients released into the container.

[0057] In addition, for example, when performing a removal operation, the articulated robot control unit 152 controls the removal operation based on parameters that define the operation pattern of the articulated robot 30, which are stored in the parameter storage unit 171. As a prerequisite for the removal operation, when a gripping operation or release operation is performed, some of the ingredients may adhere to the gripping member 360 due to the characteristics of the ingredients. In particular, ingredients with characteristics such as viscosity or stickiness, ingredients with a lot of oil or moisture, and long, easily tangled ingredients are prone to adhesion. Therefore, it is preferable for the articulated robot control unit 152 to perform a removal operation by moving the gripping member 314 up and down or vibrating it to remove the adhered ingredients and drop them into the ingredient storage container 10, etc.

[0058] The container supply control unit 153 controls the container supply device 20 to supply containers for serving ingredients to be placed in the holding system 1 to the first transport path T1. The container supply control unit 153 also controls the first transport path T1 to transport the containers supplied by the container supply device 20 to the serving position 62, or to push containers with ingredients already placed on them onto the transport surface.

[0059] The gripping area determination unit 154 determines the target area to be actually held from among multiple target area candidates, based on the state of the group of ingredients in the ingredient container 10 detected by the storage state detection sensor 41. The gripping area determination unit 154 also constructs a learning model to determine the area to be gripped by performing machine learning in advance. The constructed learning model is then stored in the learning model storage unit 172. The specific details of the processing performed by the gripping area determination unit 154 will be described later with reference to Figures 10 to 12.

[0060] The recording control unit 155 stores control parameters acquired when the holding system 1 performs gripping operations, etc., and measurement data of the weight of the ingredients placed by the holding system 1 in the history DB 173. This data is used by the administrator of the holding system 1, etc., as log data for analyzing the operation of the holding system 1.

[0061] [Control during operation of the articulated robot 30] Next, we will explain how the articulated robot control unit 152 controls the articulated robot 30 during its operation. Figures 7 to 9 are schematic diagrams showing the state of the holding system 1 during gripping and releasing operations. Figures 7 to 9 illustrate the holding system 1 as viewed from above, looking downwards. Furthermore, a large quantity of ingredients is contained within the storage space of the ingredient container 10. These large quantities of ingredients will be referred to as "groups of ingredients" in the following explanation and diagrams.

[0062] First, referring to Figure 7(a), the positional relationship between the placement of each sensor in the holding system 1 and the standby position of the articulated robot 30 will be explained. In the holding system 1, each sensor is placed in a position that makes it easy to detect the object to be detected.

[0063] Specifically, the contents containment detection sensor 41 is positioned vertically above the center of the ingredient containment container 10 on the horizontal plane, and the shooting direction (i.e., the measurement direction) is vertically downward. Similarly, the serving state detection sensor 42 is positioned vertically above the center of the serving position 62 on the horizontal plane, and the shooting direction (i.e., the measurement direction) is vertically downward.

[0064] Here, each sensor detects the state of the ingredients by optical means. Therefore, if there is an obstacle between each sensor and the ingredients, it will obstruct the optical path of each sensor, making proper detection impossible. In this respect, in the holding system 1, the gripping mechanism 31 and robot arm 32 of the articulated robot 30, which move to perform gripping and releasing operations, can become such obstacles.

[0065] Therefore, the gripping mechanism 31 and the robot arm 32 define a "standby position" as the region where the optical path between the ingredient container 10 and the opening surface of the ingredient container 62 does not overlap vertically with the detection of the containment state detection sensor 41 and the serving state detection sensor 42 (in this case, the region between the ingredient container 10 and the serving position 62 in the horizontal plane). When gripping or releasing operations are not being performed, the gripping mechanism 31 and the robot arm 32 are moved to the standby position. This allows for highly accurate detection of the ingredients' condition without interfering with the optical detection methods of each sensor. Note that in Figures 7(b) to 9, the contents detection sensor 41 and the serving state detection sensor 42 are omitted from the illustration in order to clearly show the state of the container and ingredients.

[0066] Specifically, when the holding system 1 starts operating, as shown in Figure 7(a), the gripping mechanism 31 and the robot arm 32 first move to a standby position and wait. Next, as shown in Figure 7(b), the container supply device 20 supplies containers by discharging them one by one into the first transport path T1. The containers are then transported in the first transport direction toward the serving position 62 along the first transport path T1. At this time, the containers move vertically below the articulated robot 30 installed on the frame 63. Here, as mentioned above, the legs of the frame 63 are configured to straddle the first transport path T1. Therefore, the presence of the articulated robot 30 does not obstruct the transport of containers along the first transport path T1. In Figure 7(b), only one container is transported by the first transport path T1. However, as shown in Figure 1, multiple containers may be supplied sequentially onto the first transport path T1 in an adjacent arrangement, and these multiple containers may be transported sequentially to the serving position 62.

[0067] Then, as shown in Figure 8(c), the container reaches the serving position 62. In this case, a mechanism may be provided to prevent the container from moving by contacting the side of the container or the upper surface of the rim of the container at the serving position 62.

[0068] Next, as shown in Figure 8(d), the gripping mechanism 31 grips the ingredients from the ingredient container 10. Then, after being moved to the serving position 62 by the robot arm 32, the gripping mechanism 31 releases the ingredients from the container at the serving position 62. As a result, the container is filled with ingredients, as shown in Figure 9(d). Once the gripping and releasing operations are completed in this manner, the gripping mechanism 31 and the robot arm 32 return to their standby positions. Then, as shown in Figure 9(f), the containers filled with ingredients are pushed from the first transport path T1 to the second transport path T2 of the belt conveyor 2. The containers filled with ingredients are then transported downstream by the second transport path T2. This completes the series of serving operations of the holding system 1, including the gripping and releasing operations.

[0069] [Determining the area to be grasped] The above describes the control flow of the articulated robot 30 by the articulated robot control unit 152. As shown in Figure 8(d), the gripping mechanism 31 of the articulated robot 30 grips an ingredient from any area within the ingredient container 10. To do this, the gripping area determination unit 154 determines the target area to be held from among several candidate target areas, based on the state of the group of ingredients in the ingredient container 10 detected by the storage state detection sensor 41. The specific details of this process will now be explained.

[0070] Figures 10 to 12 are schematic diagrams illustrating the method by which the gripping area determination unit 154 determines the target area to be gripped. As shown in Figure 10(A) and Figure 7(A) above, the contents state detection sensor 41 is positioned vertically above the center of the contents container 10 in the horizontal plane. Therefore, the information of the distance from the contents state detection sensor 41 to the contents, detected by the contents state detection sensor 41, represents the height of the contents (i.e., the height of the contents from the bottom surface to the opening surface of the contents container 10).

[0071] The contents state detection sensor 41 generates a height map of the contents container 10, as shown in Figure 10(B), by imaging the depth information (height information) for each pixel across the entire opening surface of the contents container 10 that it has detected. This height map represents information indicating the position of each pixel in the horizontal plane (coordinate values ​​of the x and y axes) and information indicating the height (coordinate value of the z axis). In this example, higher positions on the contents surface (i.e., positions with a large amount of contents remaining) are represented by lighter pixels, and lower positions on the contents surface (i.e., positions with a small amount of contents remaining) are represented by darker pixels.

[0072] The gripping area determination unit 154 obtains a height map of the ingredient container 10 from the container state detection sensor 41. The gripping area determination unit 154 also sets multiple candidate target areas (hereinafter referred to as "candidate areas C") based on this height map.

[0073] To this end, the gripping region determination unit 154 first determines the shape of the candidate region C in the horizontal plane based on the range of motion in the horizontal plane during the holding operation of the gripping mechanism 31. Figure 11 is a schematic diagram illustrating the determination of the shape of the candidate region C. First, as shown in Figure 11(A), the gripping mechanism 31 is observed from the front. The distance between the lower ends of the two gripping members 314 when they are separated and in an open state is defined as the "opening width". On the other hand, as shown in Figure 11(B), the gripping mechanism 31 is observed from the side. The length of the horizontal plane of the gripping member 314 in this case is defined as the "depth width". Based on the opening width, which is the range of motion in the horizontal plane during the holding operation of the gripping mechanism 31, and the depth width, the shape of the candidate region C in the horizontal plane is determined, as shown in Figure 11(C).

[0074] Next, the gripping area determination unit 154 sets candidate areas C based on the acquired height map and the shape of the determined candidate areas C. Then, it determines the target area to be actually held from among the multiple candidate areas C. Figure 12 is a schematic diagram showing the setting of candidate areas C and the determination of the target area to be actually held. Here, the gripping area determination unit 154 performs the processing using either the "average height-based method," which is based on the average height, one of the parameters related to the height of the ingredients, or the "surface roughness-based method," which is based on the surface roughness, another parameter related to the height of the ingredients. These methods will be explained in order below.

[0075] <Method based on average height> As shown in Figure 12, the gripping area determination unit 154 extracts the highest point with the highest height value from the acquired height map. Here, we assume that the highest point H1 in the figure is extracted as the highest point. Furthermore, the gripping region determination unit 154 sets candidate region C1 by arranging the shape of the determined candidate region C so that the highest point H1 is at the center. Then, it calculates the average height of candidate region C1 (i.e., average height) by averaging the height values ​​of all pixels present within candidate region C1.

[0076] Next, the gripping area determination unit 154 extracts the new highest point with the highest height value from the "area excluding candidate area C1" of the acquired height map. Here, let's assume that the highest point H2 in the figure is extracted as the new highest point. The gripping area determination unit 154 then sets candidate area C2 by arranging the shape of the determined candidate area C so that this highest point H2 is at the center. Then, it calculates the average height of candidate area C2 based on the height values ​​of all pixels present within candidate area C2. Similarly, the gripping area determination unit 154 extracts the new highest point H3 with the highest height value from the acquired height map "area excluding candidate area C1 and candidate area C2", and calculates the average height of candidate area C3.

[0077] The gripping region determination unit 154 repeats this process n times (where n is an integer of 2 or more) to extract n candidate regions C and calculate the average height of each. Then, it determines the candidate region C with the highest average height (i.e., the largest average height value) as the target region to be actually held. Subsequently, the articulated robot control unit 152 controls the gripping mechanism 31 to grip the material from the determined target area. Specifically, the highest point H, which is the center of the determined target area, is used as the center position for gripping. That is, during the gripping operation, gripping is performed such that the center position of the horizontal plane when the two gripping members 314 of the gripping mechanism 31 are in the closed state coincides with the position of the highest point H (coordinate values ​​of the x and y axes). In addition, the gripping members 314 are inserted into the material according to the target amount to be gripped, and the height of the surface of the material is considered to be the same as the height of the highest point H (coordinate value of the z axis), which serves as the basis for determining the insertion depth. This allows the gripping operation to be performed on the area with the highest average height, that is, the area where the amount of remaining ingredients is greatest and can be gripped most efficiently.

[0078] Furthermore, a threshold may be set for the average height of the area to be gripped. If the ingredient container 10 contains only areas below this threshold (i.e., areas where the surface height does not meet the standard), an alert sound or the like may be emitted to prompt the worker to replace the ingredient container 10.

[0079] <Methods based on surface roughness> In the surface roughness-based method, similar to the average height-based method described above, the gripping area determination unit 154 extracts the highest point H1 with the highest height value from the acquired height map. Then, by arranging the shape of the determined candidate area C so that this highest point H1 is at the center, candidate area C1 is set. However, in this method, the average height of the extracted candidate area C is not calculated.

[0080] The gripping region determination unit 154 then repeats this process n times (where n is an integer of 2 or more) to set n candidate regions C. Furthermore, the gripping region determination unit 154 sequentially inputs the images of each of the n candidate regions C into the learning model stored in the learning model storage unit 172. The learning model then outputs a value representing the surface roughness of each of the n candidate regions C (i.e., the variation in the height values ​​of each pixel in each candidate region C). The candidate region C with the least surface roughness (i.e., the smallest variation value) is then determined as the target region to be actually held.

[0081] Subsequently, in a manner similar to the average height-based method described above, the articulated robot control unit 152 controls the gripping mechanism 31 to grasp the material from the determined target area. This allows the gripping operation to be performed on the area with the least surface roughness, that is, the area where the material can be gripped with the highest precision depending on the insertion depth.

[0082] To implement this method, it is necessary to pre-construct a learning model that takes an image of candidate region C as input and outputs a value indicating surface roughness. In this case, for example, the gripping region determination unit 154 can generate training data by assigning labeled ground truth values ​​(in this case, values ​​indicating surface roughness) to images of candidate region C for training, and then construct a learning model by performing supervised learning using this training data. As an example, a suitable learning model can be constructed by performing machine learning using an algorithm such as a convolutional neural network (CNN), which is excellent at image recognition. Alternatively, instead of using an image of candidate region C as input, the system may use the height value of each pixel in candidate region C (in this case, the depth value of each pixel detected by the grasping region determination unit 154) as input to build a learning model and perform inference.

[0083] Furthermore, a threshold may be set for the surface roughness of the area to be gripped. If only areas exceeding this threshold (i.e., areas with surface roughness above the standard) exist, the gripping member 314 of the gripping mechanism 31 may be brought into contact with the contained material on its inner or outer surface, and while maintaining this state, the gripping member 314 may be moved horizontally to smooth the surface of the material. After smoothing, the area to be gripped may be determined again by performing the surface roughness-based processing described above.

[0084] The gripping area determination unit 154 may use either the method based on average height or the method based on surface roughness as described above, or it may appropriately select between these two methods depending on the characteristics of the material.

[0085] As explained above, according to the holding system 1, the gripping area determination unit 154 sets candidate areas C based on the height map acquired from the housing state detection sensor 41 and the shape of the determined candidate areas C. Then, from the multiple candidate areas C, it determines the target area to be actually held. This results in many advantageous effects as described above. In other words, the holding system 1 of this embodiment can solve the problem that the present invention aims to solve, which is "determining the area to be held based on the state of the object."

[0086] [Overall Operation] Next, we will explain the overall operation of the holding system 1. Figure 13 is a flowchart showing the flow of the ingredient plating process performed by the holding system 1. The ingredient plating process is initiated, for example, when an operator initiates the ingredient plating process.

[0087] When the ingredient placement process begins, in step S11, the articulated robot control unit 152 reads operation data (for example, operation pattern data, position and shape data of the ingredient container 10, etc.) from the parameter storage unit 171 to perform a series of operations in the ingredient placement process. This prepares the robot for performing the gripping and releasing operations described above. In step S12, the articulated robot control unit 152 moves the robot arm 32 and the gripping mechanism 31 to the standby position.

[0088] In step S13, the container supply control unit 153 transports the container to the serving position 62 via the first transport path T1. Note that the process in step S13 may be performed after the processes in steps S14 to S18 described later, or it may be performed in parallel with these processes.

[0089] In step S14, the storage state detection sensor 41 detects the state of the ingredients inside the ingredient storage container 10 and generates a height map. In step S15, the gripping region determination unit 154 sets multiple candidate regions C based on the generated height map.

[0090] In step S16, the gripping area determination unit 154 determines the target area to be actually held from among multiple candidate areas C using the method based on average height or the method based on surface roughness as described above. In step S17, the articulated robot control unit 152 causes the gripping mechanism 31 to grip the material from the target area determined in step S16.

[0091] In step S18, the articulated robot control unit 152 instructs the gripping mechanism 31 to release the gripped material into the container. In step S19, the articulated robot control unit 152 moves the robot arm 32 and the gripping mechanism 31 to the standby position.

[0092] In step S20, the container supply control unit 153 causes the first transport path T1 to push the container with the ingredients into the second transport path T2. Then, the container is transported downstream via the second transport path T2.

[0093] In step S21, the recording control unit 155 stores the control parameters acquired during the ingredient plating process and the measured weight data (history data) of the plated ingredients in the history DB 173.

[0094] In step S22, the articulated robot control unit 152 determines whether the conditions for terminating the ingredient placement process have been met. In this case, the conditions for terminating the ingredient placement process are that the ingredients have been placed in the planned number of containers, or that the operator has performed an operation to terminate the ingredient placement process. If the conditions for terminating the ingredient plating process are not met, the result is determined as No in step S22, and the process returns to step S13, a new container is supplied, and the process is repeated for this new container. On the other hand, if the conditions for terminating the ingredient plating process are met, the result is determined as Yes in step S22, and the ingredient plating process is terminated.

[0095] As described above, the ingredient plating process yields various advantageous effects as shown in Figures 10 to 12, etc.

[0096] [Differentiation] Although embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other forms without departing from the spirit of the invention, and various modifications such as omissions and substitutions can be made. For example, it is possible not only to apply any of the modifications described below to the embodiments of the present invention, but also to combine some or all of the modifications described below as appropriate and apply them to the embodiments of the present invention.

[0097] In the above-described embodiment, the storage state detection sensor 41 and the serving state detection sensor 42 were assumed to be sensors that detect the state of the ingredients using optical means (i.e., means that use light such as visible light or infrared light). The sensors are not limited to these examples; they may also use electromagnetic waves other than light, such as radio waves or X-rays, to detect the state of the ingredients, or they may use sound waves such as ultrasound to detect the state of the ingredients. In other words, the functions of the food storage state detection sensor 41 and the food preparation state detection sensor 42 can be realized using any sensor that uses waves, such as electromagnetic waves or sound waves. In this case, the term "angle of view" in the description of the above embodiment may be appropriately replaced with "detection range."

[0098] [Differentiation 2] In the embodiment described above, multiple candidate regions C were all set to the same orientation. However, the system is not limited to this, and each candidate region C may be set to have a different orientation. Figure 14 shows the setting of multiple candidate regions C in this modified example. As shown in the figure, for example, candidate regions C2, C4, and Cn all have different orientations in the horizontal plane. After setting multiple candidate regions C in this way, the target region to be actually held may be determined from among the multiple candidate regions C. Then, when actually holding the target region to be held, the gripping mechanism 31 may be rotated to have the same orientation as the target region in the horizontal plane before gripping is performed.

[0099] This modified version allows for more flexible setting of candidate region C. This modified version is particularly suitable when it is difficult to set an appropriate candidate region C, such as when the amount of ingredients remaining is low or when the surface roughness of the ingredients falls below a certain standard.

[0100] [Difference 3] In the embodiment described above, the shape of the candidate region C in the horizontal plane was determined based on the range of motion in the horizontal plane during the holding operation of the gripping mechanism 31. That is, as shown in Figure 11, the shape of the candidate region C was determined based on the opening width when the two gripping members 314 of the gripping mechanism 31 move away from each other and open. However, the shape of the candidate region C may also be determined by considering, for example, the insertion depth during the gripping operation.

[0101] Figure 15 is a schematic diagram illustrating the determination of the shape of candidate region C in this modified example. Figure 15 shows the gripping member 314 inserted into the group of materials, as observed from the front. Figure 15(A1) shows the gripping members 314 of the gripping mechanism 31 inserted sufficiently deep to grip a large amount of ingredients. The insertion depth in this case is called the first insertion depth, and the width of the ingredients to be cut by the gripping mechanism 31 in this case is called the first cutting width. The cutting width here is the width on the surface of the ingredients when they are inserted, and is the width that is gripped by the gripping action of the two gripping members 314. In this case, the shape of the candidate region C in the horizontal plane is determined based on this first cutting width and depth width, as shown in Figure 15(A2).

[0102] In contrast, Figure 15(B1) shows a state in which the gripping member 314 of the gripping mechanism 31 is inserted shallowly in order to grip a relatively small amount of material. The insertion depth in this case is called the second insertion depth, and the material cutting width of the gripping mechanism 31 in this case is called the second cutting width. In this case, the shape of the candidate region C in the horizontal plane is determined based on this second cutting width and depth, as shown in Figure 15(B2).

[0103] As shown above, the length of the cut width changes significantly depending on the insertion depth. Therefore, in this modified example, as illustrated, multiple candidate regions C are set according to the length of the cut width, which changes depending on the amount of material to be gripped, and the target region to be actually held is determined from among these. This determines the shape of the candidate region C in the horizontal plane, which corresponds to the range of motion during the actual gripping operation. Therefore, the target region to be held can be determined more appropriately.

[0104] [Differentiation Example 4] In the embodiment described above, the shape of the candidate region C in the horizontal plane was determined based on the range of motion in the horizontal plane during the holding operation of the gripping mechanism 31. That is, as shown in Figure 11, the shape of the candidate region C was determined based on the opening width when the two gripping members 314 of the gripping mechanism 31 move away from each other and become open. However, the shape of the candidate region C may be determined based on a range different from the range of motion. Figure 16 is a schematic diagram showing how the shape of the candidate region C is determined based on a range different from the movable range of the gripping member 314 in this modified example.

[0105] For example, when determining the shape of candidate region C using the average height-based method described above, as shown in Figure 16(A), the highest point Hn was extracted, and the shape of candidate region Cn was set based on this point according to the opening width of the actual gripping member 314. Then, based on the average height of this candidate region Cn, it was determined as the target region to be actually held. However, in this case, even if there is a valley-like area that is clearly recessed (i.e., significantly lower in height) than the surrounding area, as shown in Figure 16(A), if the height of the surrounding area is high, the average height will be high as a result. Therefore, in some cases, the candidate area Cn based on the actual opening width may be determined as the target area to be held. However, in this case, even if the insertion depth is determined based on the height of the highest point Hn and gripping is performed, the low height of this valley-like recess may result in the inability to grip a smaller amount of material than expected.

[0106] Therefore, in this modified example, as shown in Figure 16(B), the candidate region Cn based on the detection width is set to a shorter length than the candidate region Cn based on the actual aperture width. In this case, even if the height of the highest point Hn is high, the influence of the low depth of valley-like depressions becomes large, and the average height does not become a large value. Therefore, the candidate region Cn shown in Figure 16(B) is not actually selected as the target region to be held. As a result, the problem of being able to grasp less material than expected, as explained with reference to Figure 16(A), can be prevented.

[0107] Furthermore, if a candidate region Cn is set based on the detection width in this manner, and this candidate region Cn is determined to be the target region to be held, then when gripping is performed, the gripping member 314 may be opened only by the actual opening width, rather than the detection width, to perform the gripping. Alternatively, if a candidate region Cn detected by the detection width has a narrow area but a clearly higher average height than other areas, making it suitable for gripping, this candidate region Cn may be determined as the target area to be actually held, and when gripping is performed, the gripping member 314 may be opened only by the detection width, rather than the actual opening width, to perform the gripping.

[0108] [Difference 5] In the embodiment described above, the gripping area determination unit 154 determined the area to be actually held when performing a grip on any area of ​​the ingredient container 10, using a method based on average height or a method based on surface roughness. However, it is not limited to this, and the area to be actually held may be determined using these methods in other situations as well.

[0109] For example, as described above, if the amount of ingredients released and placed in the container is in excess of the target amount, the articulated robot control unit 152 performs an adjustment operation to hold the excess amount from the container and release it into the ingredient container 10 (i.e., return it to the ingredient container 10). In this case, the gripping area determination unit 154 may determine the target area within the container to be held for the excess ingredients using a method based on average height or a method based on surface roughness. In other words, the target area may be determined when grasping the ingredients from the container 10, rather than when grasping the ingredients from the ingredient container 10. In this case, the processing may be performed in the same manner as in the above embodiment, based on the detection result of the serving state detection sensor 42 instead of the detection result of the containment state detection sensor 41.

[0110] [Modification 6] The gripping mechanism 31 of the above-described embodiment shown in Figures 2 to 4 may be modified. Figure 17 shows the configuration of the gripping mechanism 31a in this modified example. Figure 17(A) shows a front view of the gripping mechanism 31a, and Figure 17(B) shows a left side view of the gripping mechanism 31a.

[0111] In the gripping mechanism 31 of the above-described embodiment, the plate-like member is composed of multiple surfaces and is provided with slits (i.e., narrow gaps) of a size that prevents the gripped material from falling out. By providing such slits, it is possible to suppress the adhesion of adhesive or other easily adhering materials to the gripping member 360 (for example, the material sticking to it). However, in the case of ingredients that are finely packed, such as hijiki seaweed, or ingredients that deform, such as potato salad or okara (soy pulp), there is a risk that these ingredients may protrude from the slits and spill out. Therefore, the gripping mechanism 31a is designed to have the same overall shape as the gripping mechanism 31, but without the slits. This prevents ingredients with the characteristics described above from spilling, keeping the surrounding area clean and allowing for precise control over the amount of ingredients served.

[0112] Furthermore, the shape of the gripping member 314 of the gripping mechanism 31 may also be of a different shape. For example, a gripping member 314 can be a gripping member that comprises multiple members such as wire-like linear members, plate-like members, or rod-like members, and grips an ingredient by sandwiching it between these multiple members. In this case, the multiple members may each have the same shape or they may have different shapes.

[0113] [Difference 7] In the embodiment described above, the gripping and releasing operations were performed by comparing the target amount with the weight of the ingredients. However, the system is not limited to this, and the determination may also be made by comparing the target amount with the amount of ingredients from other perspectives. For example, the determination may be made by comparing the target amount with the volume of the ingredients. Alternatively, the determination may be made by comparing the target amount with the number of ingredients. The volume of the serving and the number of ingredients can be determined, for example, by performing image analysis on the detection results of the storage state detection sensor 41 and the serving state detection sensor 42.

[0114] [Differentiation 8] In the embodiment described above, the gripping member 314 identified the weight of the ingredients it gripped or the ingredients it served based on the weight changes detected by the first weighing scale 51 and the second weighing scale 52, and performed control based on this. However, it is not limited to this, and load cells or force sensors may be placed on the gripping member 314. Then, the gripping member 314 may identify the weight of the ingredients it gripped or the ingredients it serves based on the weight changes detected by the load cells or force sensors, and perform control based on this. This makes it possible to omit the first weighing scale 51 and the second weighing scale 52. In addition, this increases the degree of freedom in the placement of the ingredient container 10 and the container itself.

[0115] [Example Configuration] As described above, the holding system 1 in this embodiment comprises a multi-joint robot 30, a housing state detection sensor 41, and a control device 70. The articulated robot 30 performs holding on objects that are present in the target area. The storage state detection sensor 41 detects the state of the object using waves. The control device 70 controls the articulated robot 30. The control device 70 determines the target area to be actually held from among the multiple target area candidates, based on the parameters relating to the height of the object in each of the multiple target area candidates detected by the storage state detection sensor 41.

[0116] The articulated robot 30 is equipped with a gripping member 314 for holding an object. The control device 70 determines the shape of the horizontal plane of each of the multiple candidate target areas based on the range of motion in the horizontal plane during the holding operation of the gripping member 314.

[0117] The control device 70 is The insertion depth of the gripping member 314 into the object is determined according to the amount of object to be held by the articulated robot 30, and the shape of the horizontal plane of each of the multiple candidate object regions is determined based on the range of motion in the horizontal plane during the gripping operation of the gripping member 314, according to the determined insertion depth.

[0118] The control device 70 determines the shape of the horizontal plane of each of the multiple candidate target areas based on a range narrower than the range of motion in the horizontal plane during the holding operation of the gripping member 314. When the control device 70 performs a holding operation, it performs the holding operation without changing the range of motion of the gripping member 314 in the horizontal plane during the holding operation.

[0119] The parameter related to the object's height is the average height for each of the multiple candidate target areas.

[0120] The parameter related to the height of the object is the surface roughness of each of the multiple candidate target areas.

[0121] The control device 70 is The shape of the target area on the horizontal plane is defined, and one of several candidate target areas is determined based on parameters related to the height of the object when the shape is positioned in a first direction on the horizontal plane. Additionally, one of the other candidate target areas is determined based on parameters related to the height of the object when the shape is positioned in a second direction intersecting the first direction on the horizontal plane.

[0122] The control device 70 modifies the shape of the candidate target area on the horizontal plane according to the parameter relating to the height of the object, and determines the target area to be actually held from among a plurality of candidate target areas identified based on the modified shape.

[0123] The embodiments and modifications described above are merely examples of embodiments of the present invention, and various embodiments that realize the functions of the present invention are included within the scope of the present invention. For example, in the embodiments and modifications described above, the present invention was explained using the application of the present invention to a holding system for serving prepared foods as an example, but the present invention can be applied to systems for gripping various objects. For example, the present invention can be applied to systems for gripping materials with high viscosity or adhesiveness, such as mixed mortar, concrete, plaster, and clay. The present invention is suitable for gripping objects having a viscosity of medium viscosity or higher (5000 mPa·s) or higher at working temperature or room temperature. Furthermore, the present invention can be implemented by appropriately combining the examples described in the above embodiments. The series of processes described above can be executed by hardware or by software. In other words, the functional configuration shown in Figure 6 is merely illustrative and not particularly limiting. That is, it is sufficient for the holding system 1 to be equipped with a function that can execute the series of processes described above as a whole, and the type of functional block used to realize this function is not particularly limited to the example in Figure 6. Furthermore, a single functional block may consist of hardware alone, software alone, or a combination of both.

[0124] When a series of processes are executed by software, the programs that make up that software are installed on a computer or other device from a network or storage medium. A computer may be a computer built into dedicated hardware. Alternatively, a computer may be a computer capable of performing various functions by installing various programs, such as a general-purpose personal computer.

[0125] The storage medium for storing programs consists of removable media distributed separately from the main unit, or storage media pre-installed in the main unit. Removable media consists of, for example, magnetic disks, optical disks, magneto-optical disks, or flash memory. Optical disks consist of, for example, CD-ROM (Compact Disk-Read Only Memory), DVD (Digital Versatile Disk), Blu-ray Disc (registered trademark), etc. Magneto-optical disks consist of, for example, MD (Mini-Disk). Flash memory consists of, for example, USB (Universal Serial Bus) memory or SD cards. Furthermore, storage media pre-installed in the main unit consists of, for example, ROM, SSD, HDD, etc., on which programs are stored.

[0126] In this specification, the step of describing a program to be recorded on a recording medium includes not only processes that are performed chronologically in that order, but also processes that are not necessarily performed chronologically, but are executed in parallel or individually. Furthermore, in this specification, the term "system" refers to an overall system composed of multiple devices, means, etc.

[0127] The above embodiments illustrate one example of applying the present invention and do not limit the technical scope of the present invention. That is, the present invention can be modified in various ways, such as by omitting or substituting, without departing from the spirit of the invention, and various embodiments other than those described above are possible. Various embodiments that the present invention can take and their variations are included in the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0128] 1 Holding system, 2 Belt conveyor, 10 Ingredient container, 20 Container supply device, 30 Articulated robot, 31 Gripping mechanism, 311 Connecting member, 312 Air cylinder, 313 Connecting member, 314 Gripping member, 32 Robot arm, 33 Robot base end, 41 Storage state detection sensor, 42 Plating state detection sensor, 51 First weighing scale, 52 Second weighing scale, 60 Base unit, 61 Caster, 62 Plating position, 63 Stand, 70 Control device, 151 Information acquisition unit, 152 Articulated robot control unit, 153 Container supply control unit, 154 Gripping area determination unit, 155 Recording control unit, 171 Parameter storage unit, 172 Learning model storage unit, 173 History database (History DB), 711 CPU, 712 ROM, 713 RAM, 714 Bus, 715 Input unit, 716 Output unit, 717 Storage unit, 718 Communication unit, 719 Drive, 731 Removable media, T1 First transport path, T2 Second transport path

Claims

1. A robot comprising a holding unit for performing object holding, and which performs holding on an object located in a target area, A detection means for detecting parameters related to the height of the object using waves, Control means for controlling the robot, Equipped with, The control means is The insertion depth of the holding part into the object is determined according to the amount of object to be held by the robot, and the shape of the horizontal plane of each of the multiple candidate object regions is determined based on the range of motion of the holding part in the horizontal plane during the holding operation, according to the determined insertion depth. Based on the parameters relating to the height of the object in each of the multiple candidate target areas detected by the detection means, the target area to be actually held is determined from the multiple candidate target areas. A holding system characterized by the following:

2. A robot comprising a holding unit for performing object holding, and which performs holding on an object located in a target area, A detection means for detecting parameters related to the height of the object using waves, Control means for controlling the robot, Equipped with, The control means is Based on a range narrower than the range of motion in the horizontal plane during the holding operation of the holding part, the shape of the horizontal plane of each of the multiple candidate target areas is determined. Based on the parameters relating to the height of the object in each of the multiple candidate target areas detected by the detection means, the target area to be actually held is determined from the multiple candidate target areas. When performing the holding operation, the robot is made to perform the holding operation without changing the range of motion of the holding part in the horizontal plane during the holding operation. A holding system characterized by the following:

3. A robot that holds objects in the target area, A detection means for detecting parameters related to the height of the object using waves, Control means for controlling the robot, Equipped with, The control means is The shape of the target area in the horizontal plane is set, and the shape is positioned in a first direction in the horizontal plane to determine one of a plurality of candidate target areas based on parameters relating to the height of the object, and the shape is positioned in a second direction intersecting the first direction in the horizontal plane to determine one of the other of the plurality of candidate target areas based on parameters relating to the height of the object, Based on the parameters relating to the height of the object in each of the multiple candidate target areas detected by the detection means, the target area to be actually held is determined from the multiple candidate target areas. A holding system characterized by the following:

4. A robot comprising a holding unit for performing object holding, and which performs holding on an object located in a target area, A detection means for detecting parameters related to the height of the object using waves, A computer that controls a system equipped with the following: The insertion depth of the holding part into the object is determined according to the amount of object to be held by the robot, and the shape of the horizontal plane of each of the multiple candidate object regions is determined based on the range of motion of the holding part in the horizontal plane during the holding operation, according to the determined insertion depth. A control function that determines the target area to be actually held from among the multiple target area candidates based on the parameters relating to the height of the object in each of the multiple target area candidates detected by the detection means. A program characterized by achieving this.

5. A robot comprising a holding unit for performing object holding, and which performs holding on an object located in a target area, A detection means for detecting parameters related to the height of the object using waves, A computer that controls a system equipped with the following: Based on a range narrower than the range of motion in the horizontal plane during the holding operation of the holding part, the shape of the horizontal plane of each of the multiple candidate target areas is determined. Based on the parameters relating to the height of the object in each of the multiple candidate target areas detected by the detection means, the target area to be actually held is determined from the multiple candidate target areas. When performing the holding operation, a control function is provided to cause the robot to perform the holding operation without changing the range of motion of the holding part in the horizontal plane during the holding operation. A program characterized by achieving this.

6. A robot that holds objects in the target area, A detection means for detecting parameters related to the height of the object using waves, A computer that controls a system equipped with the following: The shape of the target area in the horizontal plane is set, and the shape is positioned in a first direction in the horizontal plane to determine one of a plurality of candidate target areas based on parameters relating to the height of the object, and the shape is positioned in a second direction intersecting the first direction in the horizontal plane to determine one of the other of the plurality of candidate target areas based on parameters relating to the height of the object, A control function that determines the target area to be actually held from among the multiple target area candidates based on the parameters relating to the height of the object in each of the multiple target area candidates detected by the detection means. A program characterized by achieving this.

Images