Control device, trained model generation device, method, and program

Abstract

Provided is a control device for controlling a robot that weighs an object which is a powder, a granular material, or a fluid, the control device acquiring state data representing a state when the robot is weighing the object. The control device generates behavior data including an operation for, when the state data is input, adjusting the amount of the object that the robot has scooped up by inputting the state data to a trained model, which is a pre-generated trained model and which outputs the behavior data. The control device controls the robot such that the operation represented by the behavior data is realized.

Description

Control device, trained model generation device, method, and program

[0001] The present disclosure relates to a control device, a trained model generation device, a method, and a program.

[0002] A learning system that weighs powder such as salt has been known (see, for example, Japanese Patent Application Laid-Open No. 2020-194242, 2020-194242). A spoon is attached to the manipulator of this learning system as an end effector, and the manipulator weighs salt (see, for example, paragraph

[0013] of the document 1).

[0003] Also known is a liquid weighing method that uses a robot arm to pour water from a first container into a second container and weigh the liquid (see, for example, Document 2: JP 2021-164980 A). In this liquid weighing method, a control device that controls the operation of the robot arm acquires an image including the first container taken at a first time point, acquires the angle of the first container at the first time point, acquires the amount of liquid in the second container at the first time point, inputs the acquired image, angle, and amount of liquid into a learning model, acquires an angle at a second time point output by the learning model, and controls the angle of the first container held by the robot arm according to the acquired angle at the second time point (see, for example, claim 1 of Document 2).

[0004] There is also a known machine learning method for teaching a robot to scoop a large amount of powder, granules, or fluid from a container and dispense it into another container in a predetermined amount (see, for example, Document 3: JP 2019-098419 A). This machine learning method consists of a real-world learning process in which reinforcement learning is performed between a multi-axis robot and an object in a real space, and a simulation learning process in which reinforcement learning is performed between a pseudo multi-axis robot that simulates the multi-axis robot and a pseudo object that simulates the object in a simulation space (see, for example, the abstract of Document 3).

[0005] Furthermore, a technique for handling granular media using a robot is known (see, for example, Reference 4: Schenckc, C., Tompson, J., Fox, D., and Levine, S., "Learning Robotic Manipulation of Granular Media," In Proceedings of the First Conference on Robotic Learning (CoRL) (to appear), 2017). Reference 4 discloses a control model for a robot used to scoop up granular media.

[0006] However, when a robot weighs an object such as a powder, granules, or fluid, the robot may scoop up more than the target amount. While a robot typically operates aiming to achieve the target amount, this is difficult to achieve due to various errors, and the robot may end up scooping up more than the target amount. While the technologies disclosed in the above-mentioned documents 1-4 are technologies for weighing powders and the like, they do not take into consideration how the robot should behave if it scoops up more than the target amount.

[0007] The present disclosure has been made in consideration of the above points, and aims to adjust the amount of an object when a robot scoops up more than the target amount of an object, which is a powder, granules, or fluid, when weighing the object.

[0008] In order to achieve the above-mentioned object, the control device of the present disclosure is a control device that controls a robot that weighs an object that is a powder, granules, or fluid, and includes: an acquisition unit that acquires state data that represents the state of the robot when weighing the object; a generation unit that generates the behavioral data according to the state data acquired by the acquisition unit by inputting the state data acquired by the acquisition unit into a trained model that is generated in advance and that outputs behavioral data that includes an action of adjusting the amount of the object scooped up by the robot when the state data is input; and a control unit that controls the robot so that the action represented by the behavioral data generated by the generation unit is realized.

[0009] Furthermore, the control method disclosed herein is a control method for controlling a robot that weighs an object that is a powder, granules, or fluid, in which state data representing the state of the robot when weighing the object is acquired, and the acquired state data is input to a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input, thereby generating behavioral data according to the acquired state data, and controlling the robot so that the action represented by the generated behavioral data is realized, in a control method in which a computer executes processing.

[0010] In addition, the control program disclosed herein is a control program for controlling a robot that weighs an object that is a powder, granules, or fluid, and is a control program for causing a computer to execute processing to acquire status data representing the state of the robot when weighing the object, input the acquired status data to a trained model that is generated in advance and that outputs behavioral data including an action to adjust the amount of the object scooped up by the robot when the status data is input, thereby generating the behavioral data according to the acquired status data, and controlling the robot so that the action represented by the generated behavioral data is realized.

[0011] In addition, the trained model generation device of the present disclosure is a trained model generation device that includes: a training acquisition unit that acquires training data obtained by computer simulating the actions of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid; and a training unit that generates a trained model based on the training data acquired by the training acquisition unit, and when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, outputs behavioral data including actions of the robot adjusting the amount of the object scooped up by the robot.

[0012] In addition, the trained model generation method disclosed herein is a trained model generation method in which a computer executes a process to acquire training data obtained by computer simulation of the actions of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid, and generate a trained model based on the acquired training data, in which, when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, behavioral data including actions of the robot adjusting the amount of the object scooped up by the robot is output.

[0013] In addition, the trained model generation program of the present disclosure is a trained model generation program for causing a computer to execute a process that acquires training data obtained by computer simulation of the actions of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid, and generates a trained model based on the acquired training data, in which, when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, behavioral data including actions of the robot adjusting the amount of the object scooped up by the robot is output.

[0014] According to the control device, trained model generation device, method, and program disclosed herein, when a robot weighs an object that is a powder, granules, or fluid, it is possible to adjust the amount of the object in a situation where the robot scoops up more than the target amount.

[0015] FIG. 1 is a diagram for explaining an overview of the present embodiment. FIG. 2 is a diagram for explaining an overview of the present embodiment. FIG. 3 is a diagram for explaining the operation of a spoon of the present embodiment. FIG. 4 is a diagram for explaining randomization of physical parameters in the present embodiment. FIG. 5 is a block diagram showing the hardware configuration of a trained model generation device according to the present embodiment. FIG. 6 is a block diagram showing a schematic configuration of a trained model generation device of the present embodiment. FIG. 7 is a diagram for explaining a trained model of the present embodiment. FIG. 8 is a block diagram showing a schematic configuration of a control system of the present embodiment. FIG. 9 is a block diagram showing the hardware configuration of a control device according to the present embodiment. FIG. 10 is a flowchart showing the flow of trained model generation processing in the present embodiment. FIG. 11 is a flowchart showing the flow of control processing in the present embodiment.

[0016] An example of an embodiment of the present disclosure will be described below with reference to the drawings. In this embodiment, a control system equipped with a control device according to the present disclosure will be described as an example. Note that the same reference numerals are used in the drawings to designate identical or equivalent components and parts. Furthermore, the dimensions and proportions of the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0017] 1 and 2 are diagrams for explaining an overview of this embodiment. As shown in FIG. 1, a robot 24 of this embodiment weighs powder filled in a container Ct. In this embodiment, as shown in FIG. 2, a trained model used when weighing the powder is generated using training data obtained by executing a computer simulation. Specifically, as shown in FIG. 2, in the learning phase Tr, a policy is generated by executing reinforcement learning based on virtual state data Sv and virtual behavior data Av obtained by executing a simulator. Note that this policy is realized by a known machine learning model.

[0018] Then, in the operation phase Op, the robot performs an operation to weigh out the powder using this policy. Specifically, when current state data S is input to the policy, the policy outputs new action data. After the robot performs an operation according to the action data, it weighs out the powder W using the electronic balance. New state data S is generated by the robot's operation, and this state data is input to the policy, and the next action data is output from the policy. By executing this cycle in the operation phase Op, the robot weighs out the powder.

[0019] When a robot weighs a powder object, it may scoop up more than the target amount. In this embodiment, such a case is assumed, and the robot adjusts the amount of the object by dropping the object it has scooped up. If the amount of the object scooped up is less than the target amount, the robot may perform the scooping action again. This will be explained in detail below.

[0020] <Problem Formulation> In this embodiment, reinforcement learning is used to generate a trained model that reflects the above-mentioned measures. In this embodiment, the problem is defined by a Markov decision process (S, A, μ, P, r, γ). S is a set of states obtained from the environment, and A is a set of selectable actions. P ａ ｓｓ’ represents the probability that action a is selected under the circumstances in which state s is observed, and that the state transitions to state s' at the next time step. ａ ｓｓ’ represents the reward when action a is selected in a situation where state s is observed and a transition to state s' is made at the next time step. γ∈[0,1) is the discount rate. Policy π(a|s) is the probability that action a is selected in a situation where state s is given. The objective of reinforcement learning is to find the optimal policy π that maximizes the total reward expressed by the following equation (1). ＊ The goal is to find the time t.

[0021] (1)

[0022] In addition, r in the above formula (1) ｓｔ is expressed by the following equation (2).

[0023] (2)

[0024] In the weighing of powder in this embodiment, the state s is the current powder mass w ｃｕｒｒｅｎｔ and the target mass of powder w ｇｏａｌ and the spoon's tilt angle θ ｓｐｏｏｎ and a vector [w ｃｕｒｒｅｎｔ , θ ｓｐｏｏｎ , w ｇｏａｌ In this embodiment, in order to weigh a powder of an arbitrary target mass, the state s is represented by the target mass w of the powder. ｇｏａｌ The action a is incorporated into the action a of tilting the spoon. ｉｎｃｌｉｎｅ and the action of shaking the spoon a ｓｈａｋｅ A vector [a ｉｎｃｌｉｎｅ , a ｓｈａｋｅ ]. In addition, action a ｉｎｃｌｉｎｅ corresponds to the relative pitch angle of the spoon.

[0025] Figure 3 shows the action of tilting a spoon. ｉｎｃｌｉｎｅ and the action of shaking the spoon a ｓｈａｋｅ As shown in FIG. 3, the action of shaking a spoon ｓｈａｋｅ is an action of moving a spoon back and forth, and the action of shaking a spoon is ｓｈａｋｅ is an action of moving the spoon back and forth, and the action of tilting the spoon a ｉｎｃｌｉｎｅ is the action of changing the spoon's position. ｓｈａｋｅ can be expressed by the movement distance and movement acceleration of the spoon. ｓｈａｋｅ is expressed by the proportionality coefficient between the spoon's moving distance and its moving acceleration.

[0026] Therefore, the state s and the action a in this embodiment are defined as follows.

[0027]

[0028] The reward r is calculated by the following equation (3): ｇｏａｌ and the current powder mass w ｃｕｒｒｅｎｔ is defined as the difference between

[0029] (3)

[0030] (Generation of Trained Model by Computer Simulation) In this embodiment, a machine learning model is used to generate a policy π ＊ Specifically, in this embodiment, a neural network model having a long short-term memory (LSTM) structure, which is an example of a machine learning model, is used to realize the policy π ＊ (a|s). The neural network model with an LSTM structure can take into account the time-series state of the powder.

[0031] As described above, in this embodiment, a neural network model having an LSTM structure is trained using training data obtained by performing a computer simulation, thereby generating a trained model to be used when weighing powder.

[0032] When performing the computer simulation, physical parameters in the virtual space of the simulation are randomly changed. Specifically, the simulation is performed by randomly changing the coefficient of friction between particles that make up the powder, the number of particles, the radius of the particles, the mass of the particles, the coefficient of friction between the spoon and the particles, a parameter representing the strength of the spoon swing, gravity, and the target mass of the powder. By randomly changing the physical parameters in the virtual space, various environments are virtually realized in the simulation, and a trained model that can respond to various situations is generated.

[0033] 4 is a diagram illustrating changing physical parameters in a virtual space. As shown in FIG. 4 , in a learning phase Tr in which a trained model is generated by executing a simulation, various physical parameters in the virtual space are randomly changed (Rd in FIG. 4 ), and the operation of moving a spoon in the virtual space to adjust the amount of powder on the spoon is reflected in the trained model. Then, as shown in FIG. 4 , in an operation phase Op, the trained model is used to perform an operation of adjusting the amount of powder on the spoon by moving the spoon in real space.

[0034] The coefficient of friction between particles that make up the powder is also a parameter that represents the difficulty of the powder flowing. The coefficient of friction between the spoon and particles is also a parameter that affects the amount of powder remaining on the spoon. The parameter that represents the strength of the spoon swing is randomly changed to reflect the difference in the strength of the spoon swing by the robot between the simulator and reality in the trained model. The gravity is randomly changed to reproduce the phenomenon of powder flying. In order to more closely reproduce the phenomenon of powder flying, the gravity in the virtual space may be set to a value that is significantly smaller than the gravitational acceleration in reality. The target mass is randomly changed to weigh the powder at an arbitrary target mass.

[0035] (Trained model generation device 10) Fig. 5 is a block diagram showing the hardware configuration of the trained model generation device 10 according to this embodiment. As shown in Fig. 5, the trained model generation device 10 has a CPU (Central Processing Unit) 42, a memory 44, a storage device 46, an input / output I / F (Interface) 48, a storage medium reading device 50, and a communication I / F 52. Each component is connected to each other via a bus 54 so as to be able to communicate with each other.

[0036] The storage device 46 stores a trained model generation program for executing each process described below. The CPU 42 is a central processing unit that executes various programs and controls each component. That is, the CPU 42 reads the program from the storage device 46 and executes the program using the memory 44 as a work area. The CPU 42 controls each component and performs various arithmetic processes in accordance with the program stored in the storage device 46.

[0037] The memory 44 is configured with a RAM (Random Access Memory) and serves as a working area to temporarily store programs and data. The storage device 46 is configured with a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and stores various programs including the operating system and various data.

[0038] The input / output I / F 48 is an interface for inputting data from the outside and outputting data to the outside. Furthermore, input devices for various inputs, such as a keyboard or a mouse, and output devices for various information outputs, such as a display or a printer, may be connected. A touch panel display may be used as the output device, allowing it to function as an input device.

[0039] The storage medium reader 50 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

[0040] The communication I / F 52 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0041] Next, the functional configuration of the trained model generation device 10 will be described. As shown in Figure 6, the trained model generation device 10 functionally includes a simulation unit 12, a learning acquisition unit 14, and a learning unit 16. A trained model storage unit 18 is also provided in a predetermined storage area of ​​the trained model generation device 10. Each functional configuration is realized by the CPU 42 reading each program stored in the storage device 46, expanding it into the memory 44, and executing it.

[0042] The trained model storage unit 18 stores a trained model generated by the process described below. As described above, the trained model of this embodiment is a neural network model having an LSTM structure. FIG. 7 is a diagram for explaining the trained model of this embodiment. As shown in FIG. 7, the trained model of this embodiment is a neural network model having an LSTM structure. ＊ This is a model that realizes (a|s), and when state data s is input, action data a is output.

[0043] The simulation unit 12 executes a computer simulation of the behavior of a virtual robot in a virtual space when weighing a virtual object, which is a virtual powder. A virtual spoon is provided on the virtual robot, and the virtual robot weighs the virtual powder using the virtual spoon. When executing the computer simulation, the simulation unit 12 randomly changes physical parameters in the virtual space.

[0044] The learning acquisition unit 14 acquires learning data. The learning data is data obtained by computer simulation of the behavior of the virtual robot when weighing a virtual object, which is virtual powder. The learning data is composed of data such as the position and orientation of a specific part of the virtual robot (e.g., the part where the spoon is attached), the position and orientation of the virtual spoon, the amount of the virtual object present on the virtual spoon, and a reward r when a certain action a is selected in a certain state s.

[0045] The learning unit 16 generates a trained model based on the training data acquired by the training acquisition unit 14. When state data s is input when the robot weighs powder, the trained model outputs behavior data a including an action of adjusting the amount of the object scooped up by the robot.

[0046] Specifically, the learning unit 16 performs deep reinforcement learning on a neural network model having an LSTM structure so as to maximize the reward function shown in the above formula (1). For example, the deep reinforcement learning may be performed to train the current mass of powder to approach a target amount. The learning unit 16 then stores the resulting trained model in the trained model storage unit 18. Note that the reward function is not limited to the above formula (1) and may be of another type. For example, to prevent the user from dropping too much powder from the spoon, causing the amount of powder on the spoon to deviate from the target amount, a term that imposes a large penalty on such behavior may be added to the above formula (1). In this case, the trained model is more likely to take actions that bring the amount of powder scooped up closer to the target amount and prevent excessive powder dropping, which is more favorable for weighing the object.

[0047] This results in a trained model being obtained in which, when state data s is input when the robot is weighing powder, behavioral data a is output, including the robot's actions to adjust the amount of object it scoops up, and this trained model can be used to control a real robot.

[0048] (Control System 20) FIG. 8 is a block diagram showing a schematic configuration of the control system 20 of this embodiment. As shown in FIG. 8, the control system 20 includes a sensor group 22, a robot 24, a spoon 26 installed on the robot 24, and a control device 30. The control device 30 according to this embodiment controls the operation of the robot 24 using the trained model generated by the trained model generation device 10. The spoon 26 may be any object capable of obtaining an object whose amount is to be adjusted, such as a medicine spoon or a cup-shaped object that can be held by the robot, or may have a structure with a recess that allows the object to fit between the robot's fingers.

[0049] The sensor group 22 sequentially detects the state of the robot 24, the state of the spoon 26, the state of the object on the spoon 26, and the state of the object in the container. The sensor group 22 includes, for example, an electronic balance that measures the mass of the object on the spoon or in the container, a sensor that detects the posture and position of a specific location on the robot 24, and a sensor that detects the posture and position of the spoon 26. Note that the measurement target is not limited to the mass of the object. For example, the state of the object may be determined by estimating a quantity (e.g., the volume of the object) from a photographed image of the object using known image processing technology. Furthermore, the amount of the object dropped from the spoon may be estimated by estimating the sound emitted when the object is dropped using known audio processing technology. Furthermore, the state of the object to be detected is not limited to the state of the object on the spoon 26 or the state of the object in the container, but may also be the amount of the object dropped from the spoon 26.

[0050] The robot 24 operates in response to control commands output from a control device 30, which will be described later. The spoon 26 is attached to the robot 24, for example, as shown in Fig. 1, and scoops up an object in a container in response to the operation of the robot 24.

[0051] 9 is a block diagram showing the hardware configuration of the control device 30 according to this embodiment. As shown in FIG. 9, the control device 30 includes a CPU (Central Processing Unit) 62, a memory 64, a storage device 66, an input / output I / F (Interface) 68, a storage medium reader 70, and a communication I / F 72. Each component is connected to each other via a bus 74 so as to be able to communicate with each other.

[0052] The storage device 66 stores control programs for executing the various processes described below. The CPU 62 is a central processing unit that executes various programs and controls the various components. That is, the CPU 62 reads the programs from the storage device 66 and executes them using the memory 64 as a work area. The CPU 62 controls the various components and performs various arithmetic operations in accordance with the programs stored in the storage device 66.

[0053] The memory 64 is configured with a RAM (Random Access Memory) and serves as a working area to temporarily store programs and data. The storage device 66 is configured with a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including the operating system and various data.

[0054] The input / output I / F 68 is an interface for inputting data from the outside and outputting data to the outside. Furthermore, input devices for various inputs, such as a keyboard or a mouse, and output devices for various information outputs, such as a display or a printer, may be connected. A touch panel display may be used as the output device, allowing it to function as an input device.

[0055] The storage medium reader 70 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

[0056] The communication I / F 72 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0057] Next, the functional configuration of the control device 30 will be described. As shown in Fig. 8, the control device 30 functionally includes an acquisition unit 34, a generation unit 36, and a control unit 38. A trained model storage unit 32 is also provided in a predetermined storage area of ​​the control device 30. Each functional configuration is realized by the CPU 62 reading each program stored in the storage device 66, expanding it in the memory 64, and executing it.

[0058] The trained model storage unit 32 stores the trained model generated by the trained model generation device 10.

[0059] The acquisition unit 34 acquires the state of the robot 24, the state of the spoon 26, the state of the object on the spoon 26, and the state of the object in the container, all of which are detected by the sensor group 22. Next, the acquisition unit 34 generates state data s that represent the state of the robot 24 when weighing the object. Specifically, the acquisition unit 34 calculates the mass w of the object currently present on the spoon 26 based on the various data acquired by the sensor group 22. ｃｕｒｒｅｎｔ and the current inclination θ of the spoon 26 ｓｐｏｏｎ Then, the acquisition unit 34 calculates the current state data s=[w ｃｕｒｒｅｎｔ , θ ｓｐｏｏｎ , w ｇｏａｌ ] is set. The target mass of the object is w ｇｏａｌ is preset by the user.

[0060] The generation unit 36 ​​inputs the state data s acquired by the acquisition unit 34 into the trained model stored in the trained model storage unit 32, and generates behavior data a=[a ｉｎｃｌｉｎｅ , a ｓｈａｋｅ] is generated. This behavior data a includes an action to adjust the amount of the object scooped up by the robot 24, and corresponds to an action that the robot 24 should take at the next time. Note that the action to adjust the amount of the object scooped up by the robot 24 may include at least one of an action to tilt the spoon 26 and an action to shake the spoon 26. Therefore, the action a to tilt the spoon 26 ｉｎｃｌｉｎｅ and the action of shaking the spoon 26 a ｓｈａｋｅ Either one of the actions may be performed, but only one of the actions may be performed. ｉｎｃｌｉｎｅ and the action of shaking the spoon 26 a ｓｈａｋｅ Both may be performed.

[0061] The control unit 38 calculates the behavior data a=[a ｉｎｃｌｉｎｅ , a ｓｈａｋｅ Specifically, the control unit 38 controls the robot 24 so as to realize the action represented by the action a of tilting the spoon generated by the generation unit 36. ｉｎｃｌｉｎｅ The control unit 38 outputs a control command to the robot 24 so that the spoon-waving action a generated by the generation unit 36 ​​is realized. ｓｈａｋｅ A control command is output to the robot 24 so that the above is realized. As a result, an action is performed to adjust the amount of the object present on the spoon 26. Specifically, this is an action in which the robot 24 drops the object scooped up with the spoon 26. The action of dropping the object is an action in which the posture of a specific part of the robot 24 and the position of a specific part of the robot 24 are changed. Note that the method by which the action of the robot 24 is controlled using the generated behavioral data before (preparation stage) and after (processing of the powder after weighing) the robot 24 operates may be based on well-known teaching techniques.

[0062] Next, the operation of the trained model generation device 10 according to this embodiment will be described.

[0063] When the trained model generation device 10 receives a predetermined instruction signal, the CPU 42 of the trained model generation device 10 reads the trained model generation program from the storage device 46, expands it into the memory 44, and executes it. As a result, the CPU 42 functions as each functional component of the trained model generation device 10, and the trained model generation process shown in FIG. 10 is executed.

[0064] In step S100, the simulation unit 12 executes a computer simulation of the behavior of a virtual robot in a virtual space when weighing a virtual object, which is a virtual powder. A virtual spoon is provided on the virtual robot, and the virtual robot weighs the virtual powder using the virtual spoon. When executing the computer simulation, the simulation unit 12 randomly changes physical parameters in the virtual space.

[0065] In step S102, while the simulation unit 12 is performing a simulation, the learning acquisition unit 14 acquires learning data including the position and posture of a specific part of the virtual robot (for example, the part where the spoon is attached), the position and posture of the virtual spoon, the amount of virtual objects present on the virtual spoon, and the reward r when a certain action a is selected in a certain state s.

[0066] In step S104, the learning unit 16 generates a trained model based on the training data acquired in step S102.

[0067] In step S106, the learning unit 16 stores the trained model generated in step S104 in the trained model storage unit 18.

[0068] Next, the operation of the control system 20 according to this embodiment will be described.

[0069] When the trained model generated by the trained model generation device 10 is input to the control device 30, the trained model is stored in the trained model storage unit 32 of the control device 30. Then, when the control system 20 receives a predetermined instruction signal, the CPU 62 of the control device 30 reads the control program from the storage device 66, expands it into the memory 64, and executes it. As a result, the CPU 62 functions as each functional component of the control device 30, and the control process shown in FIG. 11 is executed.

[0070] In step S200, the acquisition unit 34 acquires current state data s=[w ｃｕｒｒｅｎｔ , θ ｓｐｏｏｎ , w ｇｏａｌ ]to obtain.

[0071] In step S202, the generation unit 36 ​​inputs the state data s acquired in step S200 into the trained model stored in the trained model storage unit 32, thereby generating behavior data a=[a ｉｎｃｌｉｎｅ , a ｓｈａｋｅ ]to generate.

[0072] In step S204, the behavior data a=[a ｉｎｃｌｉｎｅ , a ｓｈａｋｅ ] is controlled to realize the action represented by the symbol .

[0073] The control process shown in FIG. 11 is repeated, and control signals are repeatedly output to the robot 24, thereby appropriately weighing the object.

[0074] As described above, the control device according to this embodiment is a control device that controls a robot that weighs a powder object. The control device acquires state data that represents the state of the robot when weighing the object, and inputs the state data to a trained model that has been generated in advance. The trained model is a model that outputs behavioral data that includes an action to adjust the amount of the object scooped up by the robot when the state data is input. The control device uses this trained model to generate behavioral data according to the state data. The control device then controls the robot so that the action represented by the generated behavioral data is realized. This makes it possible to adjust the amount of the object when the robot scoops up more than the target amount of the object when weighing the powder object.

[0075] Specifically, a spoon is attached to the robot, and the status data includes the spoon's posture and the target amount of the object, so that even in a situation where more object than the target amount is scooped up, the amount of object can be adjusted by tilting or moving the spoon.

[0076] The action of adjusting the amount of the object is the action of dropping the object that the robot has scooped up. The action of dropping the object is the action of changing the posture and position of a specific part of the robot. This makes it possible to finely adjust the amount of the object and perform weighing with high accuracy.

[0077] Furthermore, the trained model generation device according to this embodiment acquires training data obtained by computer simulation of the behavior of a virtual robot when weighing a virtual object, which is a virtual powder. Based on the training data, the trained model generation device generates a trained model that outputs behavioral data including behaviors of the robot adjusting the amount of the object scooped up when state data representing the state when weighing the powder object is input. This makes it possible to obtain a trained model for adjusting the amount of the object when the robot scoops up more than the target amount when weighing the powder object.

[0078] Furthermore, the trained model generation device according to this embodiment generates a trained model based on data obtained by computer simulating the behavior of a virtual robot when weighing a virtual object, which is a virtual powder. This makes it possible to generate a trained model without operating a real robot. When collecting training data using a real robot, it is expected that an incident such as scattering powder may occur. In contrast, by collecting training data using computer simulation as in this embodiment, it is possible to prevent the occurrence of an incident such as scattering powder.

[0079] Furthermore, the trained model generation device according to this embodiment executes simulations by randomly varying physical parameters in a virtual space, thereby virtually realizing various environments in the simulation and generating trained models that can adapt to various situations.

[0080] Next, an example will be described. In this example, the above-described method was applied to weighing four powders. The following table shows the results of this example. The following table shows the weighing results when the above-described method was applied to four powders: flour, rice flour, salt, and coal. In this example, 5 mg, 10 mg, and 15 mg of powder were weighed using a robot. The results below consist of the mean absolute error and standard deviation. For example, for the weighing result of 5 mg of flour (0.0±0.4), 0.0 represents the mean absolute error and 0.4 represents the standard deviation. In this example, five policies were trained for one powder type, and five experiments were conducted for each policy. Therefore, 25 experimental results were obtained for each powder type, and the results are shown in the following table. As can be seen from the table, accurate weighing was achieved.

[0081]

[0082] The following table explains the case where physical parameters in the virtual space are randomly changed in the simulation when generating a trained model. The following table lists the physical parameters that are randomly changed.

[0083]

[0084] "Powder friction coefficient" represents the coefficient of friction between particles that make up the powder. "Powder particle number" represents the number of particles. "Powder particle radius" represents the radius of the particle. "Powder particle mass" represents the mass of the particle. "Spoon friction coefficient" represents the coefficient of friction between the spoon and the particle. "Shake speed weight" is a parameter that represents the strength of the spoon shake. "Gravity" represents gravity. "Goal powder amount" represents the target mass of the powder.

[0085] The following table shows the weighing results when physical parameters in a virtual space are randomly changed in a simulation to generate a trained model. The results are also composed of mean absolute error and standard deviation. In the table, "Ours" refers to the weighing results obtained using the method proposed in this embodiment. Furthermore, "Incline" refers to the action of tilting the spoon, and "Shake" refers to the action of shaking the spoon. Note that "MLP" and "P-controller" in the second row of the table refer to existing methods. "MLP" refers to a multilayer neural network model, and "P-controller" refers to P-control. The weighing results shown in the fourth row of the table are those obtained when the listed physical parameters are fixed without random change. Furthermore, the weighing results shown in the fifth row of the table are those obtained by randomly changing the top n physical parameters from the weighing results shown in the fourth row. As can be seen from the table, randomly changing the physical parameters results in more accurate weighing. Here, the first row is counted as Ours (Incline & Shake), the second row as MLP and P-Controller, the third row as Ours (Incline only) and Ours (Shake only), and so on.

[0086]

[0087] Although the embodiments and examples have been described above, it goes without saying that the present disclosure can be embodied in various forms without departing from the spirit and scope of the present disclosure.

[0088] For example, in the above embodiment, the object is a powder, but the present invention is not limited to this. For example, the above embodiment and examples can be applied to objects such as particles or fluids.

[0089] The configuration of the state data s and the behavior data a in the above embodiment can be changed as appropriate. For example, the target mass can be configured without being incorporated into the state data s. For example, a trained model can be generated for each target mass.

[0090] In addition, the group of sensors used to acquire the status data s may include, for example, a camera or a microphone, and the status data s (e.g., the mass of the powder) may be obtained from an image of the amount of powder piled onto the spoon or the sound of the powder dropping from the spoon.

[0091] In the above embodiment, the trained model is implemented by a neural network model having an LSTM structure, but the trained model is not limited to this. Any machine learning model may be used.

[0092] Furthermore, various processors other than the CPU may execute the processes executed by loading software (programs) in the above embodiments. Examples of processors in this case include programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors having circuit configurations designed specifically to execute specific processes. Each process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0093] In the above embodiment, the programs are pre-stored (installed) in a storage device, but this is not limiting. The programs may be provided in a form stored in a storage medium such as a CD-ROM, DVD-ROM, Blu-ray Disc, or USB memory. The programs may also be downloaded from an external device via a network.

[0094] (Supplementary Notes) The following supplementary notes are provided regarding aspects of the present disclosure.

[0095] (Supplementary Note 1) A control device for controlling a robot that weighs an object that is a powder, granules, or fluid, comprising: an acquisition unit that acquires state data representing a state of the robot when weighing the object; a generation unit that generates the behavior data according to the state data acquired by the acquisition unit by inputting the state data acquired by the acquisition unit into a trained model that is generated in advance and that outputs behavior data including an action of adjusting the amount of the object scooped up by the robot when the state data is input; and a control unit that controls the robot so that the action represented by the behavior data generated by the generation unit is realized.

[0096] (Supplementary Note 2) The control device according to Supplementary Note 1, wherein the trained model is a model trained based on data obtained by computer simulating the behavior of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid.

[0097] (Supplementary Note 3) The control device according to Supplementary Note 2, wherein the simulation is performed by randomly changing physical parameters in a virtual space.

[0098] (Supplementary Note 4) The control device according to any one of Supplementary Notes 1 to 3, wherein a spoon is provided on the robot, and the state data is data including an attitude of the spoon and a target amount of the object.

[0099] (Supplementary Note 5) The control device according to any one of Supplementary Notes 1 to 4, wherein the action of adjusting the amount of the objects is an action of dropping the objects scooped up by the robot.

[0100] (Supplementary Note 6) The control device according to Supplementary Note 5, wherein the action of dropping the object is an action of changing the posture of a specific part of the robot and the position of the specific part of the robot.

[0101] (Supplementary Note 7) A trained model generation device including: a training acquisition unit that acquires training data obtained by computer simulation of the behavior of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid; and a training unit that generates a trained model based on the training data acquired by the training acquisition unit, and that, when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, outputs behavioral data including the behavior of the robot adjusting the amount of the object scooped up by the robot.

[0102] (Supplementary Note 8) A control method for controlling a robot that weighs an object that is a powder, granules, or fluid, comprising the steps of: acquiring state data representing the state of the robot when weighing the object; inputting the acquired state data into a trained model that is generated in advance and that outputs behavioral data including an action to adjust the amount of the object scooped up by the robot when the state data is input; generating the behavioral data according to the acquired state data; and controlling the robot so that the action represented by the generated behavioral data is realized.

[0103] (Supplementary Note 9) A trained model generation method in which a computer executes the following process: acquiring training data obtained by computer simulating the behavior of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid; and generating a trained model based on the acquired training data, in which, when state data representing the state of the robot when weighing the object that is a powder, granules, or fluid is input, behavioral data including behavior of the robot adjusting the amount of the object scooped up by the robot is output.

[0104] (Supplementary Note 10) A control program for controlling a robot that weighs an object that is a powder, granules, or fluid, the control program causing a computer to execute processes including: acquiring state data representing a state of the robot when weighing the object; inputting the acquired state data into a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the state data is input; generating the behavioral data according to the acquired state data; and controlling the robot so that the action represented by the generated behavioral data is realized.

[0105] (Supplementary Note 11) A trained model generation program for causing a computer to execute a process of acquiring training data obtained by computer simulation of the behavior of a virtual robot when weighing a virtual object that is a virtual powder, granules, or fluid, and generating a trained model based on the acquired training data, which outputs behavioral data including behavior of the robot adjusting the amount of the object scooped up by the robot when state data representing the state of the robot when weighing the object that is a virtual powder, granules, or fluid is input.

[0106] The disclosure of Japanese Patent Application No. 2023-124132, filed on July 31, 2023, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A control device for controlling a robot that weighs an object, which is a powder, granule, or fluid, comprising: an acquisition unit that acquires status data representing a state of the robot when weighing the object; a generation unit that generates behavioral data corresponding to the status data acquired by the acquisition unit by inputting the status data acquired by the acquisition unit into a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the status data is input; and a control unit that controls the robot so that the action represented by the behavioral data generated by the generation unit is realized.

2. The control device according to claim 1, wherein the trained model is a model trained based on data obtained by computer simulating the operation of a virtual robot when weighing a virtual object which is a virtual powder, granules, or fluid.

3. The control device according to claim 2, wherein the computer simulation is a simulation performed by randomly varying physical parameters in a virtual space.

4. The control device according to any one of claims 1 to 3, wherein a spoon is provided on the robot, and the state data includes a posture of the spoon and a target amount of the object.

5. The control device according to any one of claims 1 to 3, wherein the action of adjusting the amount of the objects is an action of dropping the objects scooped up by the robot.

6. The control device according to claim 5, wherein the action of dropping the object is an action of changing the attitude of a specific part of the robot and the position of the specific part of the robot.

7. A trained model generation device including: a learning acquisition unit that acquires learning data obtained by computer simulating the actions of a virtual robot when weighing a virtual object which is a virtual powder, granules, or fluid; and a learning unit that generates a trained model based on the learning data acquired by the learning acquisition unit, and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when state data representing the state of the robot when weighing the object which is a virtual powder, granules, or fluid is input.

8. A control method for controlling a robot that weighs an object which is a powder, granules, or fluid, comprising the steps of: acquiring status data representing the state of the robot when weighing the object; inputting the acquired status data into a trained model which is generated in advance and which outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the status data is input; generating the behavioral data according to the acquired status data; and controlling the robot so as to realize the action represented by the generated behavioral data.

9. A trained model generation method in which a computer executes a process of acquiring training data obtained by computer simulating the actions of a virtual robot when weighing a virtual object which is a virtual powder, granules, or fluid, and generating a trained model based on the acquired training data, in which, when state data representing the state of the robot when weighing the object which is a powder, granules, or fluid is input, behavioral data including an action of adjusting the amount of the object scooped up by the robot is output.

10. A control program for controlling a robot that weighs an object, which is a powder, granule, or fluid, comprising the steps of: acquiring status data representing the state of the robot when weighing the object; inputting the acquired status data into a trained model that is generated in advance and that outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when the status data is input; generating behavioral data corresponding to the acquired status data; and controlling the robot so as to realize the action represented by the generated behavioral data.

11. A trained model generation program for causing a computer to execute a process of acquiring training data obtained by computer simulating the actions of a virtual robot when weighing a virtual object which is a virtual powder, granules, or fluid, and generating a trained model based on the acquired training data, which outputs behavioral data including an action of adjusting the amount of the object scooped up by the robot when state data representing the state of the robot when weighing the object which is a virtual powder, granules, or fluid is input.

Images