System and program

Abstract

To realize a technique that can stably measure an object regardless of physical property of the object.SOLUTION: A system (1, 1A) comprises: a robot (30, 30A) having a collection part (34) for collecting an object (80); and a control apparatus (10, (10A) having an acquisition part (111) that acquires input information including information which represents a collection amount of the object collected by the robot and a target value to be collected, a generation part (114) that generates parameters by use of a generation model (M2) which generates parameters for regulating operations of the robot with reference to the input information, and a control part (115) that controls the robot on the basis of the parameters. The robot performs leveling operation, which drops an amount of the collection amount exceeding the target value from the collection part, as an operation.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a system and a program. [Background technology]

[0002] Conventionally, various technologies related to robots that measure out a desired amount of a target such as powder have been proposed. For example, Patent Document 1 describes an information processing device that includes an acquisition unit that acquires input information including information indicating the amount of a target object collected by a robot having a collection unit that collects the target object and a target value to be collected, and a generation unit that generates parameters using a generation model that generates parameters that define the operation of the robot by referring to the input information, and the operation is an operation for dropping the amount in excess of the target value for the collection amount from the collection unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-176763 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned conventional technology, the target object is dropped by vibrating, moving, or rotating the sampling unit. On the other hand, the physical properties of the target object to be sampled are various, and some of them are difficult to drop the excess accurately because they crumble easily (for example, powders with a small angle of repose). In order to measure such easily crumbling targets, further ingenuity is required in the operation of the robot.

[0005] An object of one aspect of the present invention is to realize a technique that can stably measure an object regardless of the physical properties of the object. [Means for solving the problem]

[0006] In order to solve the above problems, a system according to one embodiment of the present invention comprises a control device having a robot having a collection unit that collects an object, an acquisition unit that acquires input information including information indicating the amount of the object collected by the robot and a target value to be collected, a generation unit that generates parameters using a generative model that generates parameters that define the operation of the robot by referring to the input information, and a robot control unit that controls the robot based on the parameters, and as the operation of the robot, the robot performs a leveling operation to drop the amount of the object collected in excess of the target value from the collection unit.

[0007] The control device provided in the system according to each aspect of the present invention may be realized by a computer. In this case, the control device program that causes the computer to operate as each part (software element) provided in the control device and thereby realizes the control device in a computer, and the computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention. Effect of the Invention

[0008] According to one aspect of the present invention, an object can be stably measured regardless of the physical properties of the object. [Brief description of the drawings]

[0009] [Figure 1] 1 is a configuration diagram showing an example of a system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a functional configuration of a control device provided in the system. [Diagram 3] FIG. 2 is a diagram showing an example of a first operation executed by a robot provided in the system. [Figure 4] FIG. 11 is a diagram showing an example of input and output of a second generation model in the system. [Diagram 5] FIG. 11 is a diagram showing an example of a second operation executed by a robot provided in the system. [Figure 6] 11A to 11C are diagrams illustrating another example of the second operation performed by the robot. [Figure 7] 11A to 11C are diagrams illustrating another example of the second operation performed by the robot. [Figure 8] 11A to 11C are diagrams illustrating another example of the second operation performed by the robot. [Figure 9] 4 is a flowchart showing the flow of a control method executed by the system. [Figure 10] FIG. 2 is a block diagram showing an example of the functional configuration of a learning device provided in the system. [Figure 11] 1 is a flowchart showing the flow of a learning method executed by the system. [Figure 12] FIG. 11 is a configuration diagram showing an example of a system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [Embodiment 1] A system 1 according to a first embodiment of the present invention will be described below.

[0011] <System 1 Configuration> The configuration of system 1 will be described with reference to Figs. 1 and 2. Fig. 1 is a configuration diagram showing the configuration of system 1. Fig. 2 is a block diagram showing the functional configuration of control device 10. As shown in Figs. 1 and 2, system 1 includes control device 10, learning device 20, robot 30, temperature and humidity sensor 40, and supply unit 50. System 1 is a system that causes robot 30 to learn and execute an action for dropping collected object 80.

[0012] The control device 10 and the learning device 20 are communicatively connected to the robot 30 and the temperature and humidity sensor 40, respectively. The control device 10 and the learning device 20 are also communicatively connected to each other. For example, the control device 10, the learning device 20, the robot 30, and the temperature and humidity sensor 40 are connected via a network. In this case, the network is a wired LAN (Local Area Network), a wireless LAN, the Internet, a public line network, a mobile data communication network, or a combination of these.

[0013] In the example shown in FIG. 1, the control device 10 and the learning device 20 are physically different devices, but this does not limit the present embodiment. For example, the control device 10 and the learning device 20 may be physically formed as one integrated computer. In the example shown in FIG. 1, the control device 10 and the robot 30 are physically different devices communicatively connected via a network, but this does not limit the present embodiment. For example, the control device 10 may be built into the robot 30. In the example shown in FIG. 1, the robot 30 and the temperature and humidity sensor 40 are physically connected to each other, but this does not limit the present embodiment. For example, the temperature and humidity sensor 40 may be built into the robot 30.

[0014] 1, the system 1 includes one robot 30 and one temperature and humidity sensor 40, but this does not limit the present embodiment. The system 1 may include multiple robots 30. The system 1 may also include multiple temperature and humidity sensors 40.

[0015] (Overview of the control device 10) The control device 10 is a device that causes the robot 30 to execute a first operation, a second operation, and a third operation. The control device 10 according to this embodiment is also a device that causes the robot 30 to further execute a fourth operation, a fifth operation, and a sixth operation. The detailed configuration of the control device 10 will be described later.

[0016] (1st action) The first operation is an operation in which the robot 30 collects the object 80 from the container 90 storing the object 80. The first operation is defined by a first parameter. The collected object 80 is placed on the collection unit 34. The robot 30 also collects more object 80 than the target value for the amount of object 80 to be collected. For this reason, in the first operation, the robot 30 may collect the object 80 so that it is piled in a cone shape on the collection unit 34, depending on the size of the target value. In the first operation, the robot 30 may also collect only the object 80 to the extent that it fits into the recess of the collection unit 34.

[0017] (2nd action) The second operation is a leveling operation in which the robot 30 drops an excess amount of the target amount of the objects 80 from the collection unit 34. The second operation is defined by a second parameter. The second operation will be described later in detail.

[0018] (3rd action) The third operation is an operation in which the robot 30 discharges the object 80 remaining in the collecting unit 34 after the leveling operation from the collecting unit 34. The third operation according to this embodiment is an operation in which the robot 30 transfers the collected object 80 to a container 91. Specifically, the third operation includes an operation of moving the collecting unit 34 from the collecting position to the transfer position, and an operation of rotating the collecting unit 34. The parameters that define the third operation are determined based on the collecting position determined according to the position of the container 90 and the transfer position determined according to the position of the container 91.

[0019] (4th action) The fourth operation is an operation in which the robot 30 uses the second arm unit 37 to level the surface of the object 80 remaining in the container 90 after collecting the object 80 from the container 90. The leveling operation is an operation to make the surface of the object 80 in the container 90 closer to flat. The leveling operation may be an operation to apply an external force to the container 90. For example, by performing the leveling operation in a state in which the surface of the object 80 in the container 90 is uneven, the surface of the object 80 becomes closer to flat. In this embodiment, the leveling operation is an operation to vibrate the container 90. Specifically, the robot 30 is controlled to have the pair of fingers 37a, 37b grasp the container 90, and the pair of fingers 37a, 37b are vibrated to vibrate the container 90. By performing such a fourth operation by the robot 30, it is possible to prevent the surface of the object 80 in the container 90 from being concave and only an amount of the object 80 less than the target value from being collected when the next measurement is performed.

[0020] (5th action) The fifth operation includes at least one of an operation of opening the lid of the container 90 before the robot 30 collects the object 80 from the container 90 using the second arm unit 37, and an operation of closing the lid of the container 90 after the object 80 is collected. By performing such a fifth operation by the robot 30, the operation of opening and closing the lid of the container 90 can be performed in parallel while the first arm unit 33 is performing the leveling operation or moving the collection unit. Therefore, the efficiency of the series of operations for measuring the object 80 can be improved compared to the case where one arm unit performs all of these operations.

[0021] (6th action) The sixth operation is a recovery operation for returning the amount of the object 80 on the collecting unit 34 to the target value when the amount of the drop measured by the robot 30 exceeds the calculated excess amount (the amount of the object 80 on the collecting unit 34 becomes less than the target value). The recovery operation may be, for example, an operation of redoing the first operation and the second operation. The recovery operation may also be an operation of adding an amount of the object 80 on the collecting unit 34 such that the weight of the object 80 on the collecting unit 34 exceeds the target value, and redoing the second operation. The recovery operation may also be an operation of adding an amount of the object 80 on the collecting unit 34 such that the weight of the object 80 on the collecting unit 34 becomes the target value.

[0022] (Object 80) The object 80 is an object to be collected by the robot 30. The object 80 is, for example, a powder, a granular material, a liquid, or a viscous material. The object 80 is contained in a container 90.

[0023] (container 90) The container 90 has a shape capable of containing the object 80, and has an opening through which the object 80 can be collected. Here, the container 90 is a bottle with an open top. Moreover, the container 90 according to this embodiment has a lid that can be opened and closed. The container 90 is placed on the supply unit 50. The shape of the container 90 is not particularly limited as long as it can contain the object 80. For example, the container 90 may be a cylindrical petri dish with one of two bottom surfaces open, a box with an open top surface, or a plate on which the object 80 can be placed. Moreover, the container 90 does not have to have a lid.

[0024] (Container 91) The container 91 is a container different from the container 90 for storing the collected object 80. The container 91 has a shape capable of storing the object 80, and has an opening through which the object 80 can be transferred. Specific examples of the shape of the container 91 are described in the same manner as the shape of the container 90. However, the shape of the container 91 may or may not be the same as that of the container 90.

[0025] (Overview of learning device 20) The learning device 20 is a device that trains a generative model that generates parameters that define the second action. The learning device 20 acquires the amount of object 80 picked up by the first action. The learning device 20 also acquires the amount of object 80 that has fallen by the second action that is defined by the learning parameters. The learning device 20 also trains a generative model by machine learning using the picked up amount, the dropped amount, and the learning parameters. A detailed configuration of the learning device 20 will be described later.

[0026] (Configuration of Robot 30) The robot 30 executes a first operation, a second operation, a third operation, a fourth operation, and a fifth operation based on the control of the control device 10. Specifically, as shown in Fig. 1, the robot 30 is an articulated robot having a plurality of rotation axes, and has a pedestal unit 31, a base unit 32, a first arm unit 33, a collection unit 34, a measurement unit 35, a camera 36, ​​a second arm unit 37, and a second camera 39. The robot 30 may have three or more arms.

[0027] The base 31 is installed on an installation surface of the robot 30. The installation surface is, for example, a floor, but is not limited to this. The base 31 may be movable on the installation surface based on the control of the control device 10. For example, the base 31 may have wheels that come into contact with the installation surface.

[0028] The base portion 32 is connected to the pedestal portion 31 so as to be capable of pivoting.

[0029] The base end of the first arm unit 33 is rotatably connected to the base unit 32 or the tip end of another arm around a fixed axis. The first arm unit 33 according to this embodiment has a pair of fingers 33a, 33b at the tip. The pair of fingers 33a, 33b move away from each other (open) and move toward each other (close) based on the control of the control device 10. The first arm unit 33 can hold the collection unit 34 by sandwiching the collection unit 34 between the pair of fingers 33a, 33b.

[0030] The collection unit 34 is for collecting the object 80. The collection unit 34 according to this embodiment is a spoon, and has a curved recess at the tip that can hold the object 80. For this reason, the first arm unit 33 according to this embodiment grasps the handle of the spoon with a pair of fingers 33a, 33b. The collection unit 34 is not limited to a spoon. The collection unit 34 may be, for example, a cup, a plate, a spatula, or the like. That is, the recess does not have to be curved, and the collection unit 34 does not have to have a recess. The position and inclination of the collection unit 34 are changed by the robot 30 executing a part or all of the movement of the pedestal unit 31, the rotation of the base unit 32, and the rotation of each arm. The collection unit 34 may be attached to the tip of the first arm unit 33, rather than being grasped by a pair of fingers 33a, 33b.

[0031] The measuring unit 35 measures the weight of the collected object 80 as the collected amount. The measuring unit 35 is a weight sensor that measures the magnitude of the load acting on the first arm unit 33, and is installed on the first arm unit 33. The measuring unit 35 measures the measurement value before collecting the object 80 and the measurement value after collecting the object 80, and calculates the difference between them as the collected amount. The measuring unit 35 also measures the amount of the object 80 that actually falls due to the leveling operation as the dropped amount. The measuring unit 35 measures the measurement value after collecting the object 80 and the measurement value after performing the leveling operation, and calculates the difference between them as the dropped amount.

[0032] The measuring unit 35 may be a scale provided separately from the robot 30. In this case, the container 90 containing the object 80 is placed on the measuring unit 35. Then, the robot 30 measures the weights of the container 90 and the object 80 before collecting the object 80, and the weights of the container 90 and the object 80 after collecting the object, and calculates the difference between them as the collection amount. Then, the measuring unit 35 transmits the measurement result to the control device 10. If the object 80 that has fallen due to the leveling operation is returned to the container, the amount of the object that has fallen can also be calculated.

[0033] The camera 36 according to this embodiment is installed on the first arm unit 33. The camera 36 mainly photographs the tip of the first arm unit 33 (fingers 33a, 33b, and the tools (collection unit 34, etc.) held by the fingers 33a, 33b) to generate image information. The camera 36 can also photograph the tip of the second arm unit 37 (fingers 37a, 37b, and the tools (scraper unit 38, container 90, etc.) held by the fingers 37a, 37b) depending on the orientation of the first arm unit 33 to generate image information. For example, the camera 36 is an RGB camera and is installed on the first arm unit 33. The camera 36 is not limited to an RGB camera, and may be any sensor capable of photographing the surface of the object 80, and may be, for example, a depth camera. The photographing direction by the camera 36 is not particularly limited, and may be from above, from diagonally above, or from the side. When photographing a recess or object 80 from the side, the camera 36 can generate image information that allows the height, relief, etc. of the object 80 to be easily analyzed.

[0034] The base end of the second arm 37 is connected to the base 32 or the tip of another arm so as to be rotatable around a fixed axis. The second arm 37 according to this embodiment has a pair of fingers 37a, 37b at the tip. The pair of fingers 37a, 37b move away from each other (open) and move toward each other (close) based on the control of the control device 10. The second arm 37 can hold the container 90 by pinching the container 90 with the pair of fingers 37a, 37b. The second arm 37 can also hold the leveling portion 38 by changing the object pinched by the pair of fingers 37a, 37b from the container 90 to the leveling portion 38.

[0035] In addition, the second arm unit 37 operates independently of the first arm unit 33. This allows the robot 30 to perform another operation for measuring the target object 80 in parallel while the first arm unit 33 is moving the sampling unit 34. As a result, the robot 30 can efficiently perform a series of operations for measuring the target object 80.

[0036] The leveling part 38 according to this embodiment is a rod-shaped member. The second arm part 37 holds the leveling part 38 so that it extends in the front-rear direction of the robot 30 and so that the lower end is horizontal. The front-rear direction is a direction that is horizontally perpendicular to the left-right direction when the direction in which the first arm part 33 and the second arm part 37 are aligned is the left-right direction. The leveling part 38 changes its position and inclination by the robot 30 executing a part or all of the movement of the pedestal part 31, the rotation of the base part 32, and the rotation of each arm. The leveling part 38 may be attached to the tip part of the second arm part 37 at a position different from the pair of fingers 37a and 37b, instead of being held by the pair of fingers 37a and 37b. In this case, the leveling part 38 may be attached to a position that is directly above the mouth of the container 90 held by the pair of fingers 37a and 37b.

[0037] The second camera 39 according to this embodiment is a camera configured similarly to the camera 36, ​​and is installed on the second arm unit 37. The second camera 39 mainly photographs the tip of the second arm unit 37 (fingers 37a, 37b, and the tools (scraper 38, container 90, etc.) held by the fingers 37a, 37b) to generate image information. The second camera 39 can also photograph the tip of the first arm unit 33 (fingers 33a, 33b, and the tools (collection unit 34, etc.) held by the fingers 33a, 33b) depending on the orientation of the second arm unit 37 to generate image information. The second camera 39 may be a camera different from the camera 36. The photographing direction by the second camera 39 is not particularly limited, and may be from above, from diagonally above, or from the side. When the recess or object 80 is photographed from the side, the second camera 39 can generate image information that allows the height, undulations, etc. of the object 80 to be easily analyzed.

[0038] At least one of the camera 36 and the second camera 39 may be installed in a portion other than the arm portion (for example, the base portion 32, etc.). At least one of the camera 36 and the second camera 39 may be installed in a position away from the robot 30 and at a position where the tip portion of the corresponding arm portion of the first arm portion 33 and the second arm portion 37 can be photographed.

[0039] (Configuration of temperature and humidity sensor 40) The temperature and humidity sensor 40 detects the temperature and humidity around the object 80 and generates environmental information indicating the temperature and humidity. The temperature and humidity sensor 40 is installed, for example, around the object 80. In this way, the temperature and humidity sensor 40 detects the temperature and humidity as the environment around the object 80. Note that the temperature and humidity sensor 40 may be a sensor capable of detecting an environment other than temperature and humidity.

[0040] (Summary of Supply Unit 50) The supply unit 50 supplies a container 90 containing an object 80. The supply unit 50 according to the present embodiment is a tray on which a plurality of containers 90 are placed. The tray is inclined so as to descend as it approaches the robot 30. Therefore, when the robot 30 grabs and moves the container 90 at the front among the plurality of containers 90 placed on the tray, the remaining containers 90 slide closer to the robot 30. The supply unit 50 may be a transport mechanism (such as a belt conveyor or a vehicle equipped with a tray for carrying the container 90) that extends from a location away from the robot 30 to the vicinity of the robot and transports the container 90 placed thereon to the vicinity of the robot 30. The supply unit 50 may also have a function of transporting the container 91 to the next process after the robot 30 has performed the third operation (transferring the object 80).

[0041] <Functional configuration of the control device 10> The functional configuration of the control device 10 will be described with reference to Fig. 2. As shown in Fig. 2, the control device 10 includes a control unit 11 and a storage unit 12. The control unit 11 has an acquisition unit 111, a recognition unit 112, a calculation unit 113, a generation unit 114, and a robot control unit 115. The storage unit 12 stores a first generation model M1 and a second generation model M2. The second generation model M2 is an example of a configuration that realizes a "generation model" as described in the claims.

[0042] (Acquisition part 111) The acquisition unit 111 acquires input information including information indicating the amount of the target object 80 collected by the robot 30 and the target value to be collected. The input information will be described in detail later.

[0043] (Recognition unit 112) The recognition unit 112 recognizes the position and inclination of the sampling unit 34 (the tip of the spoon) based on the image information generated by the camera 36. Furthermore, the recognition unit 112 according to this embodiment recognizes the shape of the recess based on the image information.

[0044] (Calculation unit 113) The calculation unit 113 calculates the curvature of the recognized recess.

[0045] (Generation unit 114) The generating unit 114 generates the second parameter by using the second generating model M2. The second generating model M2 and the second parameter will be described later in detail. The generating unit 114 according to the present embodiment generates the second parameter by taking into account the calculated curvature. Specifically, the volume of the recess is predicted based on the curvature and the width of the recess, and the second parameter is generated according to the predicted volume. This makes it possible to determine whether or not it is necessary to insert the leveling portion 38 into the recess when performing the leveling operation. In addition, the generating unit 114 according to the present embodiment generates the first parameter that defines the operation of the robot 30 to pick up the object 80 by using the first generating model M1. The first generating model M1 and the first parameter will also be described later in detail. The generating unit 114 according to the present embodiment generates the first parameter by further referring to the position and inclination of the recognized tip. In this way, by continuing to monitor the tip of the spoon, the operation of picking up the object 80 can be stably performed regardless of the shape of the spoon or the gripping position.

[0046] (Robot control unit 115) The robot control unit 115 controls the robot 30 based on the parameters. That is, the robot control unit 115 controls the robot 30 to perform a first operation, a second operation defined by the second parameter generated by the generation unit 114, a third operation, a fourth operation, and a fifth operation. In addition, when a separate device for discharging the object 80 from the collection unit 34 is prepared, the robot control unit 115 may not have a function to control the robot 30 to perform the third operation. In addition, when a separate device for leveling the object 80 in the container 90 is prepared, the robot control unit 115 may not have a function to control the robot 30 to perform the fourth operation. In addition, when the container 90 does not have a lid, the robot control unit 115 may not have a function to control the robot 30 to perform the fifth operation.

[0047] (First generation model M1) The first generation model M1 is a model constructed by machine learning so as to generate a first parameter when condition information is input. The first generation model M1 is configured by, for example, one or more neural networks (preferably, deep neural networks). An example of a neural network is, but is not limited to, a CNN (Convolutional Neural Network).

[0048] (Condition information) The condition information is information indicating the conditions for operating the robot 30. Specifically, the condition information indicates the conditions when the robot 30 executes the first operation. For example, the condition information may include some or all of the image information, shape information, environmental information, and physical property information of the target object 80. The image information can be acquired by the camera 36. The shape information can be acquired by a depth camera (not shown). The environmental information can be acquired by the temperature and humidity sensor 40. The physical property information will be described in detail later.

[0049] (First parameter) The first parameter is a parameter that specifies the first operation of the robot 30. In other words, the first operation of the robot 30 is modeled by the first parameter. The first parameter may include, for example, a position at which the collection unit 34 is inserted into the object 80, the depth of insertion, the inclination at the time of insertion, the amount of movement after insertion, the speed of movement after insertion, the inclination at the start of the operation of removing the collection unit from the object 80, and a part or all of the inclination at the end of the operation of removing the collection unit. When the depth of insertion is included, the depth is set so that the resistance force that the collection unit 34 receives from the object 80 does not exceed the force of the robot 30 pressing the collection unit 34. This can prevent the robot 30 from breaking down due to excessive load on the motor, etc., when the robot 30 tries to forcefully push the collection unit in. This can also prevent the collection unit 34 from being deformed or damaged. This also prevents the fingers 33a, 33b (37a, 37b) of the first arm portion 33 (second arm portion 37) of the robot 30 from slipping and moving relative to the collection portion 34 (container 90) grasped by the fingers 33a, 33b (37a, 37b).

[0050] By the control device 10 outputting such a first parameter, in the first operation, the robot 30 first activates the camera 36. As a result, the camera 36 takes an image of the recess (tip of the spoon) of the spoon held at the tip of the first arm unit 33, and generates image information of the recess. Then, the robot 30 recognizes the inclination of the spoon, the position of the recess, and the like, based on the image information of the recess generated by the camera 36. Next, the robot 30 points the camera 36 toward the mouth of the container 90 held by the second arm unit 37, as shown in FIG. 3. As a result, an image of the inside of the container 90 is taken, and image information of the inside of the container 90 is generated. Then, the robot 30 recognizes the inclination of the surface of the object 80, based on the image information of the inside of the container 90 generated by the camera 36.

[0051] Next, the robot 30 inserts the tip of the spoon so that the extension direction of the handle of the spoon is approximately perpendicular to the surface of the object 80. After collecting the object 80, the robot 30 rotates the spoon and the container 90 to make the extension direction of the handle and the surface of the object 80 in the container 90 approximately horizontal. Then, the robot 30 moves the spoon vertically upward. Then, the robot 30 pulls out the tip of the spoon so that the extension direction of the handle is parallel to the surface of the object 80. As a result, the spoon is inserted with the mouth of the container 90 facing up, so that the spoon can be inserted and collected even if many objects 80 are contained in the container 90 (close to the mouth). Also, the tip is not repelled by the object 80, as in the case of inserting the spoon with the mouth of the container 90 facing sideways. The object 80 is placed on the recess of the spoon pulled out of the container 90. For this reason, the camera 36 captures an image of the surface of the object 80 on the spoon and generates image information of the object 80 .

[0052] The robot 30 may be configured to capture an image of at least one of the recess of the spoon and the inside of the container 90 with the second camera 39 when performing the first operation described above. This can eliminate the need to adjust the focus and the camera direction when switching the subject to be captured from the spoon to the container 90. The robot 30 may be configured to capture an image of the surface of the object 80 on the spoon with the second camera 39. This can prevent the object 80 from falling by eliminating the need to operate the arm unit 33 that holds the spoon.

[0053] (Second generation model M2) The second generation model M2 will be described with reference to Fig. 4. Fig. 4 is a schematic diagram illustrating the input and output of the second generation model M2. The second generation model M2 is a model constructed by machine learning so as to output a second parameter when input information is input.

[0054] (Input information) 4, the input information includes information indicating the collection amount and the target value. In addition to these pieces of information, the input information may also include at least a portion of environmental information indicating the environment around the object 80, physical property information indicating the physical properties of the object 80, image information of the object 80, and image information of the collection unit 34. Details of these pieces of information will be described below.

[0055] (Amount collected) The collection amount is the amount of the target object 80 collected by the robot 30 through the first action.

[0056] (Information showing target value) In the present embodiment, the target value itself is applied as the information indicating the target value. The target value is a value that is a target for sampling the object 80 by the first operation.

[0057] (Image information of collection part 34) The image information of the collection unit 34 includes the collection unit 34 as a subject. For example, it is desirable that the image information of the collection unit 34 includes the collection unit 34 in a state in which the target object 80 has not been collected as a subject. For example, the image information of the collection unit 34 may be an image of the collection unit 34 captured by the camera 36 before the first operation is performed. Furthermore, the image information of the collection unit 34 may be stored in advance in the memory of the control device 10.

[0058] (Image information for object 80) The image information of the object 80 includes, as a subject, the object 80 on the collection unit 34 collected by the collection unit 34. The image information of the object 80 may be captured by the camera 36 after the first operation is performed.

[0059] (Environmental information) The environmental information is information that indicates the environment around the target object 80. In this embodiment, the environmental information includes the temperature and humidity acquired by the temperature and humidity sensor 40. The environmental information input to the first generation model M1 is similarly described.

[0060] (Physical properties information) The physical property information is information indicating the physical property of the object 80. When the object 80 is a powder, some or all of the repose angle, collapse angle, and difference angle can be used as an example of the physical property information. Information acquired in advance is used as the repose angle, collapse angle, and difference angle. In addition, the density of the object 80 can be used as another example of the physical property information. However, the physical property information is not limited to the above example. The physical property information input to the first generation model M1 is similarly described. The physical property information may be stored in a memory included in the control device 10, or may be acquired from an external device.

[0061] (2nd parameter) The second parameters are parameters that define the second motion of the robot 30. In other words, the second motion of the robot 30 is modeled by the second parameters. The second parameters include at least a part of, for example, a speed, a time, a number of times, a distance, and an angle.

[0062] The second parameters include parameters d1, v1, and t1 that define the operation of moving the collecting unit 34 relatively to the leveling unit 38 in the left-right direction. Here, the left-right direction is the horizontal direction in which the first arm unit 33 and the second arm unit 37 are aligned. For example, if the direction in which the collecting unit 34 approaches the leveling unit 38 is the left direction, the operation of moving the collecting unit 34 relatively to the left (or right) and then relatively to the right (or left). d1 indicates the distance by which the collecting unit 34 is moved to the left (or right). v1 indicates the speed at which the collecting unit 34 is moved to the left-right direction. t1 indicates the length of time for which the operation of moving the collecting unit 34 to the left-right direction is performed. The second parameters may include parameter n1. n1 indicates the number of times that the relative movement in the left and right direction is repeated when the object 80 on the collection portion 34 is scraped off in multiple steps (when the object 80 is easily crumbled, etc.).

[0063] The second parameters also include parameters d2, v2, and t2 that define the operation of moving the collecting unit 34 relative to the scraping unit 38 in the vertical direction. Here, the vertical direction is a direction perpendicular to the horizontal plane. The relative operation of moving in the vertical direction is the operation of moving the collecting unit 34 relatively upward (or downward). d2 indicates the distance by which the collecting unit 34 is moved upward (or downward). v2 indicates the speed at which the collecting unit 34 is moved in the vertical direction. t2 indicates the length of time for which the operation of moving the collecting unit 34 in the vertical direction is performed. The second parameters may also include parameter n2. n2 indicates the number of times that the relative movement in the vertical direction is repeated when scraping the object 80 on the collecting unit 34 in multiple steps.

[0064] Relative movement includes moving the collecting portion 34 while keeping the leveling portion 38 stationary, moving the leveling portion 38 while keeping the collecting portion 34 stationary, and moving both the collecting portion 34 and the leveling portion 38.

[0065] The second parameters also include parameters r3, v3, t3, and n3 that define the operation of rotating the collection unit 34. Here, the rotation operation is an operation of rotating the collection unit 34 clockwise (or counterclockwise) around a straight line extending in the front-rear direction (a direction horizontally perpendicular to the left-right direction) as the rotation axis. r3 indicates the angle by which the collection unit 34 is rotated clockwise (or counterclockwise). v3 indicates the speed at which the collection unit 34 is rotated. t3 indicates the length of time for which the collection unit 34 is rotated.

[0066] For example, the second generation model M2 outputs second parameters including parameters that define some or all of the following second parameters: (i) an operation of relatively moving the collecting unit 34 in the left-right direction; (ii) an operation of relatively moving the collecting unit 34 in the up-down direction; and (iii) an operation of rotating the collecting unit 34. The second generation model M2 according to this embodiment outputs second parameters that define an operation according to the conditions (such as the amount of the object 80, the crumbling tendency of the object 80, and the environment) when performing the second operation. When the parameters included in the second parameters include parameters that define two or more operations among these (i) to (iii), for example, the second operation may be defined as an operation of sequentially executing those operations. Also, for example, the second operation may be defined as an operation of simultaneously executing at least two of those operations.

[0067] By the control device 10 outputting such second parameters, in the second operation, the robot 30 first operates the second camera 39. As a result, the second camera 39 takes an image of the leveling portion 38 held at the tip of the second arm unit 37 and generates image information of the leveling portion 38. Then, the robot 30 recognizes the position of the leveling portion 38 based on the image information of the leveling portion 38 generated by the second camera 39. Next, as shown in FIG. 5, the robot 30 directs the second camera 39 toward the tip of the spoon held by the first arm unit 33 (object 80). As a result, the second camera 39 takes an image of the object 80 placed on the tip of the spoon and generates image information of the object 80. Then, the robot 30 recognizes the surface shape of the object 80 based on the image information of the collected object 80 generated by the second camera 39.

[0068] Next, the robot 30 moves the collection unit 34 to a predetermined reference position. The reference position is a relative position to the leveling unit 38, a predetermined distance dx in the left-right direction and a predetermined distance dy upward from a starting position described later. Then, the robot 30 positions the lower end of the leveling unit 38 at the starting position. The starting position is a position immediately before the leveling starts, and is a position below the upper end of the object 80 piled on the collection unit and above the collection unit. The lower end of the leveling unit 38 at the starting position may be located inside the object 80 or to the side of the object 80 depending on the size of the excess.

[0069] Then, the robot 30 moves the collection section relative to the leveling section 38 in the horizontal direction, thereby dropping the excess from the collection section. When the lower end of the leveling section 38 at the start position is located inside the target object 80, the leveling operation drops the part of the target object 80 that is above the lower end of the leveling section 38 and is on the side of the leveling section 38 in the relative movement direction of the leveling section 38. On the other hand, when the lower end of the leveling section 38 at the start position is located to the side of the target object 80, the leveling operation drops the part of the target object 80 that is above the lower end of the leveling section 38. As a result, the target object 80 on the collection section after the leveling operation has a relatively stable shape due to the upper part being scraped off, so that the target object 80 is less likely to fall off due to the movement of the collection section.

[0070] The robot 30 may perform such a leveling operation by stopping the second arm unit 37 and fixing the leveling unit 38, and moving the collection unit 34 to level off the excess. In this way, the leveled object 80 falls to a fixed position, so that the leveled object 80 is less likely to scatter. Conversely, the robot 30 may perform such a leveling operation by stopping the first arm unit 33 and fixing the collection unit 34 (object 80), and moving the leveling unit 38 to level off the excess. In this way, the first arm unit 33 does not move, and vibrations associated with it are not generated, so that the object 80 on the collection unit 34 is less likely to collapse. The arm units 33 and 37 may be stopped by stopping control of the motor or by brake lock. The robot 30 may also perform leveling by moving the collection unit 34 and the leveling unit 38, respectively.

[0071] In particular, when the leveling unit 38 is attached to a position directly above the mouth of the container 90 held by the pair of fingers 37a, 37b of the second arm unit 37 as described above, the robot 30 can level off the excess by keeping the first arm unit 33 stationary and fixing the collection unit 34, and operating the second arm unit 37 to move the leveling unit 38 and the container 90. In this way, it is possible to achieve both the effect of making the object 80 on the collection unit 34 less likely to collapse and the effect of making the leveled object 80 less likely to scatter.

[0072] Furthermore, if the object 80 is prone to crumbling or is in an environment in which the object 80 is prone to crumbling, in the second operation, the robot 30 first tilts the surface (the surface on which the recess is formed) on which the object 80 is placed in the collection section with respect to the horizontal plane, as shown in FIG. 6. Then, the robot 30 scrapes off the excess by horizontally moving the scraping section 38 relative to the inclined collection section. This reduces the inclination of the portion of the object 80 to be scraped off, so that even if the object is prone to crumbling, the object 80 can be scraped off without crumbling.

[0073] When the volume of the target amount of object 80 matches the volume of the recess, in the second operation, the robot 30 slides the lower end of the leveling part 38 over the edge of the collection part surrounding the recess, causing the excess to fall from the collection part. As a result, all of the object 80 on the collection part after the leveling operation fits into the recess, making it difficult for the object 80 to spill out due to the movement of the collection part.

[0074] If the volume of the object 80 of the target amount is smaller than the volume of the recess, in the second operation, the robot 30 performs either (1) cutting along the recess, or (2) cutting with the cutting portion 38 tilted.

[0075] (1) In the case of leveling along a recess, the robot 30 relatively moves the leveling part 38 so as to be aligned with the recess of the collection part, as shown in Fig. 7. This makes it possible to measure the target amount of the object 80 even in cases where the target amount of the object 80 cannot be measured unless the leveling part 38 is inserted into the recess (such as when the volume of the object 80 with the target amount is equal to or less than the capacity of the recess, or when the volume of the collected object 80 is equal to or less than the capacity of the recess from the beginning and there is an excess amount).

[0076] (2) In the case of leveling with the leveling part 38 inclined, the robot 30 first tilts the leveling part 38 by using the second arm part 37 to rotate the leveling part 38 about a straight line extending in the relative movement direction of the leveling part 38 with respect to the collection part as a rotation axis. Then, as shown in FIG. 8, the robot 30 scrapes out the excess by moving the leveling part 38 relative to the collection part while inserting the end of the leveling part 38, which is positioned relatively lower due to the inclination, into the recessed part. As a result, even if the target amount of the object 80 cannot be measured unless the leveling part 38 is inserted into the recessed part, and the horizontal width of the leveling part 38 in the horizontal direction perpendicular to the relative movement direction is larger than the width of the recessed part in the same direction, the target amount of the object 80 can be measured.

[0077] Furthermore, by outputting such a second parameter from the control device 10, in the second operation, when the robot 30 moves the collection unit on which the collected object 80 is placed, it moves the container 90 using the second arm unit 37 so that the container 90 remains positioned below the collection unit. As a result, since the container 90 is always positioned below the collection unit during movement, even if the object 80 falls from the collection unit, the fallen object 80 returns to the container 90, and the system does not become dirty.

[0078] Incidentally, when performing the second operation described above, the robot 30 may be configured to photograph at least one of the leveling portion 38 and the object 80 on the spoon with the camera 36. In particular, if the leveling portion 38 is photographed with the camera 36, ​​there is an advantage that the camera 36 can photograph the leveling portion 38 from any direction when the position and orientation of the leveling portion 38 is changed by moving the second arm unit 37 without moving the first arm unit 33 in order to prevent the object 80 from falling off the spoon.

[0079] <Flow of control method S1> The flow of the control method S1 executed by the system 1 configured as above will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the flow of the control method S1 executed by the system 1. As shown in Fig. 9, the control method S1 includes steps S101 to S106.

[0080] (Step S101) In step S101, the generation unit 114 inputs condition information to the first generation model M1, thereby generating first parameters to be output from the first generation model M1.

[0081] (Step S102) In step S102, the robot control unit 115 controls the robot 30 to perform a first operation of picking up the target object 80 stored in the container 90 from the container 90.

[0082] (Step S103) In step S103, the acquisition unit 111 acquires the amount of the object 80 collected by the first operation and the target value to be collected. Here, the collected amount can be calculated as the difference between the weights obtained from the measuring unit 35 before and after the execution of the first operation. For example, the acquisition unit 111 may acquire the target value by reading the target value stored in the memory of the control device 10. Also, for example, the acquisition unit 111 may acquire the target value inputted via an input device. Also, for example, the acquisition unit 111 may acquire the target value from another device via a communication interface. Also, the acquisition unit 111 acquires the environmental information, the image information of the collection unit 34, and the image information of the collected object 80 from the camera 36. Also, the acquisition unit 111 acquires the physical property information by reading it from the memory.

[0083] (Step S104) In step S104, the generation unit 114 inputs input information including the collection amount, the target value, the environmental information, the image information of the collection unit 34, the image information of the target object 80, and the physical property information to the second generation model M2. As a result, the generation unit 114 generates second parameters to be output from the second generation model M2.

[0084] (Step S105) In step S105, the robot control unit 115 controls the robot 30 to execute a second operation of dropping from the collection unit 34 an excess amount of the target object 80 relative to the collection amount target value.

[0085] (Step S106) In step S106, the robot control unit 115 controls the robot 30 to execute a third operation of transferring the collection unit 34.

[0086] <Functional configuration of learning device 20> The functional configuration of the learning device 20 will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the functional configuration of the learning device 20. As shown in Fig. 10, the learning device 20 includes a control unit 21 and a storage unit 22. The control unit 21 includes a learning data acquisition unit 211, a learning unit 212, an update unit 213, and a robot control unit 214. The storage unit 22 stores the second generation model M2.

[0087] (Learning data acquisition unit 211) The learning data acquisition unit 211 acquires a second parameter for learning. The learning data acquisition unit 211 also acquires the amount of the object 80 collected by the robot 30, and the amount of the object 80 that has fallen from the collection unit 34 by the robot 30 performing the second operation defined by the second parameter for learning. The amount of collection is as described above. The learning data acquisition unit 211 also acquires environmental information, image information of the collection unit 34, image information of the collected object 80, and physical property information. The learning data acquisition unit 211 according to this embodiment also acquires a first parameter for learning.

[0088] (Second parameter for learning) The second parameter for learning is a parameter that defines a second operation for learning.

[0089] (Learning section 212) The learning unit 212 trains the second generation model M2 by machine learning so as to output a second parameter for learning when input information including the collected amount and the dropped amount is input. Specifically, the input information includes the collected amount, remaining amount, environmental information, image information of the collection unit 34, image information of the target object 80, and physical property information acquired by the learning data acquisition unit 211.

[0090] (Update section 213) When there is a difference between the drop amount and the excess amount, the update unit 213 updates the second generation model M2. Specifically, the update unit 213 updates the second generation model M2 to generate a second parameter that defines the next leveling operation, the start position of which is adjusted so as to reduce the difference. As a result, the error in the operation of the robot 30 gradually decreases as the operation is repeated.

[0091] (Robot control unit 214) The robot control unit 214 controls the robot 30 to perform a first action and a second action defined by a second parameter for learning.

[0092] <Flow of learning method S2> The flow of the learning method S2 executed by the system 1 configured as above will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the learning method S2 executed by the system 1. As shown in Fig. 11, the learning method S2 includes steps S201 to S207.

[0093] (Step S201) In step S201, the learning data acquisition unit 211 acquires a first parameter for learning. For example, the learning data acquisition unit 211 may acquire the first parameter for learning by reading the first parameter for learning stored in the memory of the learning device 20. Also, for example, the learning data acquisition unit 211 may acquire the first parameter for learning inputted via an input device. Also, for example, the learning data acquisition unit 211 may acquire the first parameter for learning from another device via a communication interface.

[0094] (Step S202) In step S202, the robot control unit 214 controls the robot 30 to perform a first operation of picking up the target object 80 stored in the container 90 from the container 90.

[0095] (Step S203) In step S203, the learning data acquisition unit 211 acquires the amount of the object 80 collected by the first operation. Here, the collected amount can be calculated as the difference between the weights obtained from the measuring unit 35 before and after the first operation is performed. Furthermore, the learning data acquisition unit 211 acquires environmental information, image information of the collection unit 34, and image information of the collected object 80 from the camera 36. Furthermore, the learning data acquisition unit 211 acquires physical property information by reading it from the memory.

[0096] (Step S204) In step S204, the learning data acquisition unit 211 acquires the second parameters for learning. For example, the learning data acquisition unit 211 may acquire the second parameters for learning by reading the second parameters for learning stored in the memory of the learning device 20. Also, for example, the learning data acquisition unit 211 may acquire the second parameters for learning inputted via an input device. Also, for example, the learning data acquisition unit 211 may acquire the second parameters for learning from another device via a communication interface.

[0097] (Step S205) In step S205, the robot control unit 214 controls the robot 30 to execute a second action defined by the second parameter for learning. This causes a part of the collected object 80 to fall from the collection unit .

[0098] (Step S206) In step S206, the learning data acquisition unit 211 acquires the amount of fall of the target object 80 caused by the second motion. Here, the amount of fall can be calculated as the difference in weight obtained from the measurement unit 35 before and after the second motion is performed.

[0099] (Step S207) In step S207, the learning unit 212 inputs input information including the collection amount, remaining amount, environmental information, image information of the collection unit 34, image information of the target object 80, and physical property information obtained in steps S203 and S206, and trains the second generation model M2 by machine learning so as to output the second parameters for learning obtained in step S204.

[0100] The learning device 20 generates the second generation model M2 by repeating the operations of steps S201 to S207 until a predetermined number of times is reached or until the parameters constituting the second generation model M2 converge.

[0101] <Effects of this embodiment> In the system 1 according to the present embodiment, the robot 30 executes a first operation of collecting the target object 80, and then executes a second operation of dropping the excess collected by the first operation from the collection unit 34 when the collected amount exceeds a target value. At this time, the target object 80 is dropped by leveling, and therefore, unlike the conventional case where the target object 80 is dropped by vibration or rotation, it is difficult for a situation to occur in which an amount of the target object 80 greater than expected falls regardless of the physical properties (fluidity) of the target object 80. Therefore, according to the system 1, the target object 80 can be stably measured regardless of the physical properties of the target object 80.

[0102] Moreover, the system 1 according to this embodiment includes a supply unit 50 and a measurement unit 35, and the robot 30 also executes the first and third operations. Therefore, according to the system 1 according to this embodiment, it is possible to automate a series of operations until the target object 80 measured to the target value amount is obtained.

[0103] [Embodiment 2] A second embodiment of the present invention will be described below. For convenience of explanation, the same reference numerals are given to components having the same functions as those described in the first embodiment, and the explanations thereof will not be repeated. Hereinafter, a system 1A according to the second embodiment of the present invention will be described.

[0104] <System 1A Configuration> An overview of the system 1A will be described with reference to Fig. 12. Fig. 12 is a configuration diagram showing an overview of the system 1A. As shown in Fig. 12, the system 1A is different in that it includes a control device 10A and a robot 30A instead of the control device 10 and the robot 30. The control device 10A is connected to the robot 30A so as to be able to communicate with it.

[0105] In the system 1A according to this embodiment, the leveling unit 38 is fixed. The leveling unit 38 according to this embodiment is fixed to a table or a room in which the system 1A is installed. The fixed position of the leveling unit 38 may be a portion other than the first arm unit 33 of the robot 30A, or may be a device other than the robot 30A included in the system 1A.

[0106] <Functional configuration of the control device 10A> The generation unit of the control device 10A generates a first parameter using a first generation model different from the first generation model M1 according to the first embodiment. The generation unit of the control device 10A generates a second parameter using a second generation model different from the second generation model M2 according to the first embodiment. The first parameter according to this embodiment is a parameter that defines a first operation of the robot 30A using the first arm unit 33 (without using the second arm unit 37). The second parameter according to this embodiment is a parameter that defines a second operation of the robot 30A using the first arm unit 33 (without using the second arm unit 37).

[0107] (Configuration of robot 30A) The robot 30A does not have a configuration corresponding to the second arm unit 37 of the robot 30 according to embodiment 1. In other words, the robot 30A is a one-arm type robot.

[0108] Then, the robot 30A scrapes off the excess by moving the collecting unit 34. This makes it possible to limit the location where the target object 80 falls due to the scraping operation to below and around the scraping unit 38. By fixing a plurality of scraping rods in different positions in advance, it is possible to selectively use any of the scraping rods depending on the type of powder, etc.

[0109] The collection unit 34 may be fixed. In this case, the robot 30A holds the container 90 with the first arm unit 33 and collects the object 80 by moving the container 90. The robot 30A may then operate to scrape off the excess by moving the scraping unit 38. In this way, the collection unit on which the collected object 80 is placed does not move, so that it is possible to prevent the object 80 on the collection unit from spilling out during the scraping operation.

[0110] <Effects of this embodiment> According to the system 1A of this embodiment, like the system 1 of the embodiment 1, the object 80 can be stably measured regardless of the physical properties of the object 80. Moreover, the robot 30A of this embodiment has a simpler structure than the robot 30 of the embodiment 1. Therefore, according to the system 1A, the object 80 can be measured at low cost.

[0111] [Variation 1] In each of the above-described embodiments, the "information indicating the target value" included in the input information input by the control device 10 to the second generation model M2 is the target value itself. However, the "information indicating the target value" is not limited to this, and may be information obtained by processing the target value. The information obtained by processing the target value may be information that allows the target value to be calculated, and may be, for example, the difference between the target value and the harvesting amount.

[0112] [Modification 2] In each of the above-described embodiments, an example has been described in which the first parameter defining the first operation of picking the target object 80 is generated using the first generation model M1 constructed by machine learning. Here, the first generation model M1 may be generated by GAN (Generative Adversarial Networks). In this case, in addition to the condition information, a noise parameter is further input to the first generation model M1.

[0113] In each embodiment, the control device 10 may determine the first parameter by other methods without using the first generation model M1. For example, the control device 10 may determine the first parameter by using a rule for obtaining the first parameter according to the condition information. The control device 10 may use the first parameter defined as a constant.

[0114] [Modification 3] In the above-described first embodiment, an example has been described in which the smoothing operation is an operation of vibrating the container 90. However, the smoothing operation is not limited to this, and may be any operation that makes the surface of the object 80 in the container 90 closer to flat. For example, the smoothing operation may be an operation of rotating the container 90.

[0115] [Modification 4] In each of the above-described embodiments, the first parameter and the second parameter may include other parameters in addition to the above-described multiple parameters or in place of at least one of them.

[0116] [Modification 5] Each of the above-described embodiments can be suitably modified to a task of transferring the target object 80 from the container 90 to each of the multiple containers 91. In this modification, the third operation is an operation of transferring the collected target object 80 to each of the multiple containers 91. This provides an effect that an amount of the target object 80 that is closer to the target value can be transferred from the container 90 to each of the multiple containers 91.

[0117] [Modification 6] In each of the above-described embodiments, an example in which only the leveling operation is executed as the second operation has been described. However, the system 1 and 1A may be configured to selectively use the leveling operation, vibration, and rotation operation as the second operation depending on at least one of the physical properties of the target object 80 and the environment.

[0118] [Physical configuration of System 1 and 1A]

[0119] (Physical configuration of control devices 10 and 10A) The control devices 10 and 10A can be configured by a computer having a bus, a processor, a main memory, an auxiliary memory, and a communication interface. The processor, the main memory, the auxiliary memory, and the communication interface are connected to each other via the bus.

[0120] The processor may be, for example, a CPU (Central Processing Unit), a microprocessor, a digital signal processor, a microcontroller, or a combination of these.

[0121] As the main memory, for example, a semiconductor RAM (random access memory) or the like is used.

[0122] The auxiliary memory may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these. The auxiliary memory stores a program for causing the processor to execute the operations of the control devices 10 and 10A described above. The processor loads the program stored in the auxiliary memory onto the main memory and executes each instruction included in the loaded program. In this way, the program causes the computer to function as each of the control blocks 111-115 of the control devices 10 and 10A.

[0123] The communication interface is an interface that connects to a network. The communication interface communicates with each of the learning device 20, the robot 30, and the temperature and humidity sensor 40 via the network. The communication interface included in the control device 10A also communicates with the robot 30 via the network.

[0124] In this example, the processor and the communication interface are examples of hardware elements that realize the control unit 11. The main memory and the auxiliary memory are examples of hardware elements that realize the storage unit 12.

[0125] (Physical configuration of learning device 20) The learning device 20 can be configured by a computer having a bus, a processor, a main memory 202, an auxiliary memory, a communication interface, and an input / output interface. The processor, the main memory, the auxiliary memory, the communication interface, and the input / output interface are connected to each other via the bus. An input device and an output device are connected to the input / output interface.

[0126] The processor may be, for example, a CPU (Central Processing Unit), a microprocessor, a digital signal processor, a microcontroller, a GPU (Graphics Processing Unit), or a combination of these.

[0127] As the main memory, for example, a semiconductor RAM (random access memory) or the like is used.

[0128] The auxiliary memory may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination of these. The auxiliary memory stores a program for causing the processor to execute the above-mentioned operation of the learning device 20. The processor loads the program stored in the auxiliary memory onto the main memory and executes each instruction included in the loaded program.

[0129] The communication interface is an interface that connects to a network and communicates with each of the control devices 10, 10A, the robot 30, and the temperature and humidity sensor 40 via the network.

[0130] The input / output interface may be, for example, a Universal Serial Bus (USB) interface, a short-range communication interface such as infrared or Bluetooth (registered trademark), or a combination of these.

[0131] The input device may be, for example, a keyboard, a mouse, a touch pad, a microphone, or a combination of these, etc. The output device may be, for example, a display, a printer, a speaker, or a combination of these.

[0132] In this example, the processor and the communication interface are examples of hardware elements that realize the control unit 21. The main memory and the auxiliary memory are examples of hardware elements that realize the storage unit 22.

[0133] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0134] 〔summary〕 The system according to aspect 1 of the present invention comprises a control device having a robot having a collection unit for collecting an object, an acquisition unit for acquiring input information including information indicating the amount of the object collected by the robot and a target value to be collected, a generation unit for generating parameters using a generative model which generates parameters that define the operation of the robot by referring to the input information, and a robot control unit for controlling the robot based on the parameters, and the robot is configured to perform a leveling operation as its operation, causing the excess amount of the collection amount over the target value to fall from the collection unit.

[0135] The system according to aspect 2 of the present invention may be configured in the above aspect 1, further comprising a supply unit which supplies a container containing the object, and a measuring unit which measures the weight of the collected object as the collected amount, and the robot further performs an operation of collecting the object from the container, and an operation of discharging the object remaining in the collection section after the leveling operation from the collection section.

[0136] A system according to aspect 3 of the present invention may be configured in the above aspect 1 or 2 such that the robot collects the object in a cone-shaped pile on the collection section, positions the lower end of the leveling section at a starting position below the upper end of the object piled on the collection section and above the collection section, and moves the collection section horizontally relative to the leveling section, causing the excess to fall from the collection section.

[0137] A system according to aspect 4 of the present invention may be configured in any one of aspects 1 to 3 above, wherein the measurement unit measures the amount of the object that actually falls due to the leveling operation as a falling amount, and the control device further includes an update unit that updates the generation model when there is a difference between the falling amount and the excess amount, so as to generate parameters that define the next leveling operation, with the starting position adjusted so as to reduce the difference.

[0138] A system according to aspect 5 of the present invention may be configured in the above aspect 1 or 2 such that the collection section has a recess, and the robot causes the excess to fall from the collection section by sliding the lower end of a scraping section along the edge of the collection section surrounding the recess.

[0139] A system according to aspect 6 of the present invention may be configured in the above aspect 1 or 2, wherein the collection unit has a curved recess, the robot further has a camera that photographs the recess and generates image information of the recess, and the control device further has a recognition unit that recognizes the shape of the recess based on the image information and a calculation unit that calculates the curvature of the recognized recess, and the generation unit generates the parameters by taking the curvature into account.

[0140] A system according to a seventh aspect of the present invention may be configured in the above-mentioned first or second aspect such that the robot relatively moves a leveling portion along a recess of the collecting portion.

[0141] The system according to aspect 8 of the present invention may be configured in any one of aspects 1 to 3 above, such that the robot is capable of tilting the placement surface of the object in the collection section relative to a horizontal plane, and the excess portion is scraped off by moving the scraping section horizontally relative to the inclined collection section.

[0142] The system according to aspect 9 of the present invention may be configured such that, in aspect 1 or 2 above, the measuring unit measures the amount of the object that actually falls due to the leveling operation as the falling amount, and the robot further performs an operation to return the amount of the object above the collection unit to the target value if the falling amount exceeds the excess amount.

[0143] The system according to aspect 10 of the present invention may be configured in the above aspect 1 or 2 such that the robot has a first arm portion that holds the collection portion and a second arm portion that operates independently of the first arm portion.

[0144] A system according to aspect 11 of the present invention may be configured in the above aspect 10 such that, after the robot has collected the object from the container, it further performs an operation of smoothing the surface of the object remaining in the container using the second arm portion.

[0145] A system according to aspect 12 of the present invention may be configured in the above-mentioned aspect 10 such that the second arm unit is capable of holding a container, and when moving the collection unit carrying the collected object, the robot uses the second arm unit to move the container so that it remains positioned below the collection unit.

[0146] A system according to aspect 13 of the present invention may be configured as follows in aspect 1 or 2 above: the collection unit is a spoon, the robot has an arm unit that grasps the handle of the spoon and a camera that photographs the tip of the spoon and generates image information of the tip, the control device further has a recognition unit that recognizes the position and inclination of the tip based on the image information, and the generation unit further refers to the recognized position and inclination of the tip to generate parameters that define the robot's operation of collecting the target object.

[0147] A system according to aspect 14 of the present invention may be configured in the above aspect 10, wherein the robot uses the second arm portion to perform at least one of the following actions: opening the lid of the container before picking up the object from the container, and closing the lid of the container after picking up the object.

[0148] A system according to aspect 15 of the present invention may be configured in the above aspect 1 or 2 such that the collection part is a spoon, and when the robot collects the object contained in the container, it inserts the tip of the spoon so that the extension direction of the spoon handle is perpendicular to the surface of the object, and after collecting the object, it pulls out the tip of the spoon so that the extension direction of the handle is parallel to the surface of the object.

[0149] A system according to aspect 16 of the present invention may be configured in such a way that, in aspect 1 or 2 above, the parameters include a depth to which the collection part is inserted into the object contained in the container, and the depth is set so that the resistance force that the collection part receives from the object does not exceed the force with which the robot pushes the collection part.

[0150] A system according to aspect 17 of the present invention may be configured as in aspect 10 above, wherein the collection section has a recess, the second arm section holds a leveling section, and the robot uses the second arm section to tilt the leveling section by rotating it about a straight line extending in the direction of relative movement of the leveling section with respect to the collection section as a rotation axis, and the end of the leveling section, which is positioned relatively lower due to the tilt, is inserted into the recess while moving the leveling section relative to the collection section, thereby scraping out the excess portion.

[0151] A system according to aspect 18 of the present invention may be configured in the above aspect 10, wherein the second arm unit is capable of holding a container and a leveling unit is attached at a position directly above the mouth of the container held by the second arm unit, and the robot keeps the first arm unit stationary to fix the collection unit, and operates the second arm unit to move the leveling unit and the container to level off the excess.

[0152] A system according to a nineteenth aspect of the present invention may be configured in the above-mentioned first or second aspect such that the scraping portion is fixed and the robot scrapes off the excess portion by moving the collection portion.

[0153] A system according to aspect 20 of the present invention may be configured in the above aspect 1 or 2 such that the collection section is fixed and the robot scrapes off the excess portion by moving a scraping section.

[0154] A program according to aspect 21 of the present invention is a program for causing a computer to function as a control device provided in the system in any of aspects 1 to 20 above, and may be configured to cause a computer to function as the acquisition unit and the generation unit. [Explanation of symbols]

[0155] 1. 1A System 10, 10A Control Device 11 Control section 111 Acquisition Department 112 Recognition part 113 Calculation section 114 Generation part 115 Robot control unit 12 Storage section M1 First generative model M2 Second generative model (generative model) 20 Learning Device 21 Control section 211 Learning Data Acquisition Unit 212 Learning Department 213 Update Department 214 Robot Control Unit 22 Memory section 30, 30A Robot 33 First arm 33a, 33b fingers 34 Collection section 35 Measurement section 36 Camera 37 Second arm section 37a, 37b fingers 38 Sliding section 39 Second Camera 80 Objects 90, 91 container

Claims

1. A robot having a collection unit that collects an object; a control device having an acquisition unit that acquires input information including information indicating an amount of an object collected by the robot and a target value to be collected, a generation unit that generates parameters defining the operation of the robot using a generative model that refers to the input information, and a robot control unit that controls the robot based on the parameters; Equipped with The robot performs a leveling operation to drop the excess of the collection amount relative to the target value from the collection unit as the operation. system.

2. A supply unit that supplies a container containing the object; a measuring unit that measures the weight of the sampled object as the sampled amount; Further comprising: The robot comprises: An operation of collecting the object from the container; An operation of discharging the object remaining in the collection section from the collection section after the leveling operation; Further carry out The system of claim 1 .

3. The robot comprises: The object is collected in a cone-shaped manner on the collection portion, The lower end of the leveling portion is positioned at a start position below the upper end of the object piled on the collection portion and above the collection portion; The collecting part is moved horizontally relative to the leveling part to cause the excess to fall from the collecting part. The system of claim 2.

4. The measuring unit measures an amount of the object that has actually fallen due to the leveling operation as a falling amount, The control device includes: and an update unit that updates the generation model to generate parameters for defining a next leveling operation when there is a difference between the drop amount and the excess amount, the parameters being adjusted in the start position so as to reduce the difference. The system of claim 3.

5. The collection portion has a recess, The robot causes the excess to fall from the collection section by sliding the lower end of the scraping section along the edge of the collection section surrounding the recess.

3. A system according to claim 1 or 2.

6. The collection portion has a curved recess, The robot further includes a camera that captures an image of the recess and generates image information of the recess, The control device includes: a recognition unit that recognizes a shape of the recess based on the image information; A calculation unit that calculates a curvature of the recognized recess; and The generating unit generates the parameters by taking the curvature into account.

3. A system according to claim 1 or 2.

7. The robot relatively moves the leveling portion along the recess of the collection portion.

3. A system according to claim 1 or 2.

8. The robot comprises: The mounting surface of the object in the collection unit can be inclined with respect to a horizontal plane, The excess is scraped off by horizontally moving the scraping portion relative to the inclined collecting portion. The system of claim 3.

9. The measuring unit measures an amount of the object that has actually fallen due to the leveling operation as a falling amount, The robot further performs an operation to return the amount of the object on the collection unit to the target value when the amount of the object that has fallen exceeds the excess amount. The system of claim 2.

10. The robot comprises: A first arm portion that holds the collection portion; a second arm portion that operates independently of the first arm portion; having The system of claim 2.

11. After collecting the object from the container, the robot further performs an operation of smoothing a surface of the object remaining in the container using the second arm unit. The system of claim 10.

12. the second arm portion is capable of holding a container; When moving the collection unit on which the collected object is placed, the robot uses the second arm unit to move the container so that the container is kept positioned below the collection unit. The system of claim 10.

13. The collection part is a spoon, The robot comprises: An arm portion for gripping the handle of the spoon; a camera for capturing an image of the tip of the spoon and generating image information of the tip; having The control device further includes a recognition unit that recognizes a position and an inclination of the tip portion based on the image information, The generation unit further refers to the recognized position and inclination of the tip portion to generate parameters that define an operation of the robot to pick up the object.

3. A system according to claim 1 or 2.

14. The robot performs at least one of an action of opening a lid of the container before collecting the object from the container and an action of closing a lid of the container after collecting the object, using the second arm unit. The system of claim 10.

15. The collection part is a spoon, The robot comprises: When collecting the object contained in the container, insert the tip of the spoon so that the extension direction of the handle of the spoon is perpendicular to the surface of the object; After collecting the object, the tip of the spoon is pulled out so that the extension direction of the handle is parallel to the surface of the object. The system of claim 2.

16. The parameters include a depth to which the collection portion is inserted into the object contained in the container; The depth is set so that the resistance force that the collection unit receives from the object does not exceed the force that the robot presses the collection unit. The system of claim 2.

17. The collection portion has a recess, The second arm portion holds a scraping portion, The robot comprises: The second arm portion is used to tilt the leveling portion by rotating the leveling portion about a straight line extending in a relative movement direction of the leveling portion with respect to the collecting portion as a rotation axis, The end of the leveling portion, which is positioned relatively lower due to the inclination, is inserted into the recess while the leveling portion is moved relative to the collection portion to scrape out the excess. The system of claim 10.

18. The second arm portion includes: It is possible to hold a container, a leveling portion is attached at a position directly above the mouth of the container held by the second arm portion; The robot comprises: The first arm portion is made stationary to fix the collection portion; The second arm portion is operated to move the scraping portion and the container, thereby scraping off the excess. The system of claim 10.

19. The sliding part is fixed, The robot scrapes off the excess by moving the collection unit. The system of claim 1 .

20. The collection portion is fixed, The robot scrapes off the excess portion by moving a scraping unit. The system of claim 1 .

21. A program for causing a computer to function as a control device included in the system according to claim 1, the program causing a computer to function as the acquisition unit and the generation unit.

Images