Learned model generation device, controller, method for generating learned model, and learned model generation program

Abstract

To easily generate operation data of a robot from operation data of a person.SOLUTION: A learned model generation device 10 acquires learning data including operation data of a person for learning. The device 10 generates a learned generator which outputs operation datal of a control target robot when operation data showing an operation of a person as a target is input by causing a hostile generation network model including a generator and a discriminator to do a machine learning on the basis of learning data.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a trained model generation device, a control device, a trained model generation method, and a trained model generation program. [Background technology]

[0002] Conventionally, there is known a technique for teaching a dual-arm robot having two arms how to move (for example, see Non-Patent Document 1). In this technique, the dual-arm robot learns a predetermined movement by trial and error.

[0003] Also, when teaching a robot having multiple arms a motion, a technique is known in which a different instructor teaches a motion to each of the multiple arms (see, for example, Non-Patent Document 2). In this technique, an instructor remotely controls the robot to teach it a motion. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Rohan Chitnis, Shubham Tulsiani, Saurabh Gupta, Abhinav Gupta, "Intrinsic Motivation for Encouraging Synergistic Behavior", ICLR 2020. [Non-Patent Document 2] Albert Tung, Josiah Wong, Ajay Mandlekar, Roberto Martin, Yuke Zhu, Li Fei-Fei, Silvio Savarese, "Learning Multi-Arm Manipulation Through Collaborative Teleoperation", ICRA, 2021. Summary of the Invention [Problem to be solved by the invention]

[0005] Incidentally, when a human teaches a robot an action, it is necessary to take into consideration physical constraints on the robot's action. For example, if the range of motion of a robot is different from that of a human, the robot may not be able to execute an action that a human can easily perform. In addition, when a robot has multiple movable parts, such as a dual-arm robot, it is necessary to coordinate the operation of the multiple movable parts. When a human teaches a robot an action, it is difficult to teach the robot an action while coordinating the operation of such multiple movable parts.

[0006] This makes it difficult for humans to teach robots how to move.

[0007] The present disclosure has been made in consideration of the above points, and aims to easily generate robot motion data from human motion data. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the trained model generation device of the present disclosure is a trained model generation device that includes a training acquisition unit that acquires training data including human motion data for training, and a learning unit that generates a trained generator that outputs motion data of a robot to be controlled when motion data representing the motion of a target human is input, by machine learning a generative adversarial network model including a generator and a discriminator based on the training data acquired by the training acquisition unit.

[0009] In addition, the trained model generation method disclosed herein is a trained model generation method in which a computer executes a process to acquire training data including human motion data for training, and machine-learn a generative adversarial network model including a generator and a discriminator based on the acquired training data, thereby generating a trained generator that outputs motion data of a robot to be controlled when motion data representing the motion of a target human is input.

[0010] In addition, the trained model generation program of the present disclosure is a trained model generation program for causing a computer to execute a process of acquiring training data including human motion data for training, and performing machine learning on a generative adversarial network model including a generator and a discriminator based on the acquired training data, thereby generating a trained generator that outputs motion data of a robot to be controlled when motion data representing the motion of a target human is input. Effect of the Invention

[0011] According to the trained model generation device, control device, trained model generation method, and trained model generation program disclosed herein, robot motion data can be easily generated from human motion data. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining an overview of the present embodiment. [Diagram 2] FIG. 1 is a diagram for explaining an overview of the present embodiment. [Diagram 3] FIG. 1 is a diagram for explaining an overview of a framework according to an embodiment of the present invention. [Figure 4] FIG. 2 is a diagram for explaining a generative adversarial network model in the present embodiment. [Diagram 5] FIG. 1 is a block diagram showing a schematic configuration of a trained model generation system according to an embodiment of the present invention. [Figure 6]FIG. 2 is a block diagram showing a hardware configuration of the trained model generation device according to the present embodiment. [Figure 7] FIG. 1 is a block diagram showing a schematic configuration of a control system according to an embodiment of the present invention. [Figure 8] 2 is a block diagram showing a hardware configuration of a control device according to the embodiment. FIG. [Figure 9] 1 is a flowchart showing the flow of a trained model generation process in this embodiment. [Figure 10] 1 is a flowchart showing the flow of a trained model generation process in this embodiment. [Figure 11] 4 is a flowchart showing the flow of a control process in the present embodiment. [Figure 12] FIG. 1 is a diagram for explaining the present embodiment. [Figure 13] FIG. 13 is a diagram showing the results of this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] An example of an embodiment of the present disclosure will be described below with reference to the drawings. In this embodiment, a control system equipped with a control device according to the present disclosure will be described as an example. Note that the same or equivalent components and parts in each drawing are given the same reference numerals. Also, the dimensions and ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0014] <Overview of the embodiment> 1 and 2 are diagrams for explaining an overview of this embodiment. As shown in Fig. 1, in this embodiment, the motion of a human H is taught to a dual-arm robot having two arms R1 and R2. The example shown in Fig. 1 is an example of the case where the dual-arm robot is taught a motion to achieve a task of moving an object b in the direction of an arrow C.

[0015] Specifically, as shown in T1 of Fig. 1, a demonstration is performed in which the first arm R1 of the dual-arm robot and the arm of the human H are made to move in a coordinated manner, and thereby the positions of the two-dimensional barcodes Q1 and Q2 are imaged by a camera (not shown). Also, as shown in T2 of Fig. 1, a demonstration is performed in which the second arm R2 of the dual-arm robot and the arm of the human H are made to move in a coordinated manner, and thereby the positions of the two-dimensional barcodes Q1 and Q2 are imaged by a camera (not shown).

[0016] In the demonstration, the motion data of the arms of the human H and the object b is generated based on the positions of the two-dimensional barcodes Q1 and Q2. In addition, the motion data of the first arm R1 and the second arm R2 of the dual-arm robot in the demonstration is obtained from a control device (not shown) that controls the dual-arm robot.

[0017] In this embodiment, based on the motion data thus obtained, motion data to be taught to the first arm R1 and the second arm R2 of the dual-arm robot is generated.

[0018] Then, in the execution phase, the first arm R1 and the second arm R2 of the dual-arm robot execute a task of moving the object b in the direction of the arrow C, as shown in E of FIG.

[0019] In this regard, for example, when teaching a dual-arm robot a movement, rather than teaching the movement to each arm one at a time as described above, it is also possible to consider a method of teaching the movement to both arms, the first arm R1 and the second arm R2, at once.

[0020] For example, a method can be considered in which a human executes a task of moving object b in the direction of arrow C using both arms, and motion data at that time is obtained. However, in this case, the human may perform a motion that is impossible for a dual-arm robot to execute. For example, when a human executes a task of moving object b in the direction of arrow C, the human may move object b at a speed that exceeds the upper limit of the moving speed of the arms of the dual-arm robot. Furthermore, when attempting to execute a task that is more complicated than the above-mentioned task, the human may execute a motion that does not take into account the range of motion of the arms of the dual-arm robot.

[0021] For this reason, even if motion data of a human using both arms is obtained, it is difficult to directly teach that motion data to a dual-arm robot.

[0022] Also, for example, a method is conceivable in which a human remotely controls the first arm R1 and the second arm R2 of a dual-arm robot to execute a task of moving an object b in the direction of arrow C, and obtain motion data at that time. In this case, however, it is necessary to coordinate the remote control of the first arm R1 and the remote control of the second arm R2.

[0023] If the first arm R1 is remotely controlled by a different person from the second arm R2, the more complex the task, the more difficult it becomes to coordinate and obtain appropriate motion data. Also, if the first arm R1 is remotely controlled by a different person from the second arm R2, the equipment required to achieve the remote control will need to be complex, which will result in a huge cost for preparing the equipment.

[0024] For this reason, it is difficult for a human being to obtain motion data by remotely operating the first arm R1 and the second arm R2 of a dual-arm robot.

[0025] Therefore, in this embodiment, as described above, motion data is obtained by performing a demonstration in which a human and a dual-arm robot perform cooperative operations, and motion data to be taught to the dual-arm robot is generated based on that motion data.

[0026] Specifically, in this embodiment, a known generative adversarial network model is used to generate motion data of a dual-arm robot from motion data of a human. In the following, a scene in which the first arm R1 and a human H move, such as T1 in FIG. 1, is denoted as "Robot1-Human". In addition, a scene in which the second arm R2 and a human move, such as T2 in FIG. 1, is denoted as "Human-Robot2". In addition, a scene in which a human H and the arms of a dual-arm robot simply move is denoted as "Human-Robot". In addition, a scene in which the first arm R1 and the second arm R2 of a dual-arm robot move is denoted as "Robot1-Robot2" or "Robot-Robot".

[0027] In the following, the method proposed in this embodiment is also referred to as "learning from demonstrations by human and robotic arms (LfD-HR)."

[0028] 3 is a diagram for explaining an outline of the framework of the present embodiment. The framework of the present embodiment will be explained below with reference to FIG.

[0029] (A.Problem formulation) In this embodiment, the problem is defined by a Markov decision process (S, A, P, ρ0), where state s∈S, action a∈A, transition function P(s'∈S|s,a), and ρ0 is the initial state. In this embodiment, two domains are defined.

[0030] Domain X is a domain that belongs to the demonstration of tasks by humans and robots, and domain Y is a domain that belongs to task operations by a dual-arm robot.

[0031] The motion data belonging to domain X includes the arm state x of human H. H and the state x of the first arm R1 and the second arm R2 of the dual-arm robot. Ri (i=1,2) and state x of object b b and the action a of the first arm R1 and the second arm R2 of the dual-arm robot. Ri The following are included:

[0032] The motion data belonging to domain Y includes the state y of the first arm R1 and the second arm R2 of the dual-arm robot. Ri and state y of object b b and the action a of the first arm R1 and the second arm R2 of the dual-arm robot. Ri The following are included:

[0033] In this embodiment, a state x=(x H ,x Ri ,x b ) and state y=(y R1 ,y R2 ,y b ) and action (a R1 ,a R2 In this embodiment, in the Markov decision process in domain X and domain Y, M x =(S x ,A x ,P x ,ρ 0x ) and M y =(S y ,A y ,P y ,ρ 0y In this embodiment, a policy π:S is defined for the control model of the dual-arm robot, as will be described later. y →A y Learn:

[0034] B. Data Collection First, in this embodiment, random data is collected as shown in Fig. 3. Specifically, as shown in Fig. 3, random data {x R1 ,x H ,x b ,a R1} |D| are collected, where |D| represents the number of data sets.

[0035] In addition, as shown in FIG. 3, the random data {x H ,x R2 ,x b ,a R2} |D| is collected.

[0036] In addition, as shown in FIG. 3, the random data {y R1 ,y R2 ,y b ,a R1 ,a R2} |D| is collected.

[0037] Next, demonstration data is collected as shown in Fig. 3. The demonstration data is data obtained when the human H and the arm (the first arm R1 or the second arm R2) of the dual-arm robot are coordinated.

[0038] Specifically, as shown in FIG. 3, the demonstration data {x R1 ,x H ,x b ,a R1} |D| is collected.

[0039] In addition, as shown in FIG. 3, the demonstration data {x H ,x R2 ,x b ,a R2} |D| is collected.

[0040] (C. Forward Dynamics Model and Inverse Dynamics Model) In this embodiment, a forward dynamics model F and an inverse dynamics model F for the arm of a dual-arm robot are -1 is defined as follows.

[0041] The forward dynamics model F is defined by the following equation (1A).

[0042]

number

[0043] In addition, the inverse dynamics model F -1 is defined by the following formula (1B):

[0044]

number

[0045] In the above equations, t represents time. Forward dynamics model F and inverse dynamics model F -1 is achieved by utilizing known dynamics models or machine learning models.

[0046] Forward Dynamics Model F and Inverse Dynamics Model F -1 The parameters are set in advance so that the following expressions (1) and (2) are satisfied.

[0047]

number

[0048] In the above equations, ||·||2 represents the L2 norm, and E represents the expected value.

[0049] In this embodiment, the loss function L in the above formula (1) fwd A forward dynamics model F is generated in advance so that the loss function L in the above formula (2) is minimized. inv The inverse dynamics model F such that -1 are generated in advance. The forward dynamics model F and the inverse dynamics model F -1 is used in the domain translation framework described below.

[0050] (D. Domain Translation Framework) 3, the domain translation framework of this embodiment is a framework that realizes conversion from a domain X of cooperative motion between a human and a dual-arm robot to a domain Y of motion of the dual-arm robot. A specific description will be given below.

[0051] (1) Domain translation In this embodiment, adversarial learning is used to generate a state transition function G that performs mapping from a state x of action data to a state y of action data. Specifically, a generator in the adversarial generative network model is generated, and the generator is set as the state transition function. By adversarial learning, when the state x of action data is input, the generator G generates a pseudo state y^ of action data according to the following formula. Note that the generator G generates a state y^ of action data that deceives the discriminator Dy in the adversarial generative network model.

[0052]

number

[0053] On the other hand, the discriminator Dy in the generative adversarial network model attempts to distinguish whether the input motion data is a state y^ of motion data generated by the generator G or a state y of real motion data.

[0054] Fig. 4 is a diagram for explaining the generative adversarial network model in this embodiment. As shown in Fig. 4, when a state x of motion data is input to a generator G, the generator G outputs a pseudo state y^ of motion data corresponding to the state x of the motion data. This state y^ of motion data is data simulating the state of a dual-arm robot. A discriminator Dy judges whether or not this motion data y^ is motion data representing the actual state of the dual-arm robot.

[0055] As shown in Fig. 4, in adversarial learning, the generator G is trained to output a state y^ of the motion data that deceives the discriminator Dy. Also, in adversarial learning, the discriminator Dy is trained to be able to determine that the state y^ of the motion data output by the generator G is fake.

[0056] Specifically, in the adversarial learning of this embodiment, a generator G and a discriminator Dy in the generative adversarial network model are generated so that the following formula (3) is satisfied: In the following formula, p(x) represents the probability distribution of x appearing, and p(y) represents the probability distribution of y appearing.

[0057]

number

[0058] For this reason, in this embodiment, the loss function L for the generative adversarial network model in the above formula (3) is adv A generator G is generated so that (G, Dy) is minimized, and the loss function L advA discriminator Dy is generated so that (G, Dy) is maximized. In this embodiment, in order to prevent the generator G from overfitting the demonstration data of domain X and the random data of domain Y, the random data of domain X is also used as training data.

[0059] (2) Dynamics consistency In this embodiment, the generative adversarial network model is trained while taking into consideration the consistency of the dynamics of the dual-arm robot. If a generator G is generated without any constraints, the state y^ of the motion data converted by the generator G is a state transition P y There may be a lack of consistency.

[0060] Therefore, in this embodiment, the state y t ^ and state y at time t+1 t+1 ^ and state transition P y Generate a generator G such that the consistency of

[0061] Specifically, the inverse dynamics model F -1 Then, according to the following formula, the action a taken by the dual-arm robot at time t is t ~ is calculated.

[0062]

number

[0063] In addition, the dynamics model F calculates the state y at time t+1 according to the following equation: t+1 ~ is calculated.

[0064]

number

[0065] Dynamics Model F and Inverse Dynamics Model F-1 The state y at time t+1 calculated by t+1 ~ is the state y output from generator G at time t+1. t+1 Must match ^.

[0066] Therefore, in this embodiment, the loss function L dyn (G) is set as the loss function L for the consistency of dynamics. dyn (G) is expressed by the following equation (4).

[0067]

number

[0068] In the above formula, ||||1 represents the L1 norm. In this embodiment, the loss function L dyn A generator G is generated such that (G) is minimized.

[0069] (3) Partial Identity Mapping Even if we take into account domain translation and consistency of dynamics, the generator G can only provide real data, i.e., state x Ri A state far from Ri For example, the generator G may output a state y Ri may be output.

[0070] Therefore, in this embodiment, the loss function L id (G) Set the loss function L for partial identity mapping. id (G) is expressed by the following equation (5).

[0071]

number

[0072] In this embodiment, the loss function L for partial identity mapping is id A generator G is generated such that (G) is minimized.

[0073] (4) Integrated loss function In this embodiment, the above-mentioned loss functions are integrated to obtain the following loss function L full The loss function L in the following equation (6) is set. full The generator G and discriminator Dy of the generative adversarial network model are trained so that λ in the following equation (6) is minimized. adv ,λ dyn ,λ id is a weight, which is preset.

[0074]

number

[0075] As mentioned above, the forward dynamics model F and the inverse dynamics model F -1 are trained in advance based on random data. In addition, the forward dynamics model F and the inverse dynamics model F -1 may be updated when training the generative adversarial network model. For example, the parameters of the forward dynamics model F and the inverse dynamics model F -1 The parameters of the generative adversarial network model may be updated together when the generative adversarial network model is trained.

[0076] (Trained model generation system 1) Fig. 5 is a block diagram showing a schematic configuration of the trained model generation system 1 of this embodiment. As shown in Fig. 5, the trained model generation system 1 includes a camera 2A, a dual-arm robot 4A, and a trained model generation device 10. The trained model generation device 10 according to this embodiment generates a trained model for generating motion data of the dual-arm robot from motion data of a human.

[0077] The camera 2A sequentially captures images of the first arm R1 and the second arm R2 of the dual-arm robot 4A, which is the control target, and the human H performing an action. For example, the camera 2A sequentially captures images of the demonstrations T1 and T2 being performed as shown in FIG. 1. The camera 2A also sequentially captures images of the first arm R1 and the second arm R2 of the dual-arm robot 4A performing random actions. The camera 2A also sequentially captures images of the human performing random actions. The camera 2A then outputs the obtained image data to the trained model generation device 10.

[0078] The dual-arm robot 4A is a robot as shown in FIG. 1, and is equipped with a first arm R1 and a second arm R2.

[0079] Fig. 6 is a block diagram showing a hardware configuration of the trained model generation device 10 according to this embodiment. As shown in Fig. 6, the trained model generation device 10 has a CPU (Central Processing Unit) 42, a memory 44, a storage device 46, an input / output I / F (Interface) 48, a storage medium reading device 50, and a communication I / F 52. Each component is connected to each other via a bus 54 so as to be able to communicate with each other.

[0080] The storage device 46 stores a trained model generation program for executing each process described below. The CPU 42 is a central processing unit, and executes various programs and controls each configuration. That is, the CPU 42 reads the program from the storage device 46 and executes the program using the memory 44 as a working area. The CPU 42 controls each of the above configurations and performs various calculation processes according to the program stored in the storage device 46.

[0081] The memory 44 is made up of a RAM (Random Access Memory) and temporarily stores programs and data as a working area. The storage device 46 is made up of a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including an operating system, and various data.

[0082] The input / output I / F 48 is an interface for inputting data from the camera 2A and the dual-arm robot 4A and outputting data to the camera 2A and the dual-arm robot 4A. In addition, an input device for performing various inputs, such as a keyboard or a mouse, and an output device for outputting various information, such as a display or a printer, may be connected. A touch panel display may be used as the output device to function as the input device.

[0083] The storage medium reader 50 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

[0084] The communication I / F 52 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0085] Next, the functional configuration of the trained model generation device 10 will be described. As shown in Fig. 5, the trained model generation device 10 functionally includes a learning acquisition unit 12 and a learning unit 16. In addition, a data storage unit 14, a trained model storage unit 18, and a control model storage unit 19 are provided in a predetermined storage area of ​​the trained model generation device 10. Each functional configuration is realized by the CPU 42 reading each program stored in the storage device 46, expanding it in the memory 44, and executing it.

[0086] Image data captured by the camera 2A is stored in the data storage unit 14. In addition, the data storage unit 14 stores control data used when the double-arm robot 4A operates.

[0087] The trained model storage unit 18 stores a trained generative adversarial network model generated by the process described below.

[0088] The control model storage unit 19 stores a control model for controlling the dual-arm robot 4A.

[0089] The learning acquisition unit 12 acquires learning data. Specifically, the learning data is calculated to be used when training a generative adversarial network model, which will be described later, by processing image data and control data stored in the data storage unit 14. The learning data in this embodiment includes motion data of a human H for learning and motion data of a dual-arm robot 4A for learning.

[0090] Specifically, the learning data in this embodiment includes demonstration data representing cooperative movement between the first arm R1 and the arm of human H, demonstration data representing cooperative movement between the second arm R2 and the arm of human H, random data representing random movement of the dual-arm robot 4A, and random data representing random movement of human H.

[0091] The demonstration data showing the cooperative motion between the first arm R1 and the arm of the human H is the demonstration data of "Robot1-Human" shown in FIG. R1 ,x H ,x b ,a R1} |D| The demonstration data showing the cooperative motion between the second arm R2 and the arm of the human H is the demonstration data of "Human-Robot2" shown in FIG. H ,x R2 ,x b ,aR2} |D| It is.

[0092] The random data representing the random motion of the dual-arm robot 4A and the random data representing the random motion of the human H are the random data {x R1 ,x H ,x b ,a R1} |D| And the random data of "Human-Robot2" {x H ,x R2 ,x b ,a R2} |D| And the random data of "Robot1-Robot2" {y R1 ,y R2 ,y b ,a R1 ,a R2} |D| And so.

[0093] The learning acquisition unit 12 identifies the positions and moving speeds of the two-dimensional barcodes Q1, Q2 shown in Fig. 1 by analyzing the image data and control data stored in the data storage unit 14. The learning acquisition unit 12 then acquires the above-mentioned demonstration data and random data by combining the positions and moving speeds of the two-dimensional barcodes Q1, Q2 with the control data of the dual-arm robot 4A.

[0094] The learning unit 16 performs machine learning on a generative adversarial network model including a generator G and a discriminator Dy based on the learning data acquired by the learning acquisition unit 12, thereby generating a trained generator G that outputs motion data y of the dual-arm robot 4A when motion data x representing the motion of a target human H is input.

[0095] Specifically, the learning unit 16 calculates the integrated loss function L full The generative adversarial network model is machine-learned so that the integrated loss function Lfull is the loss function L for the generative adversarial network model. adv and the loss function L for the consistency of dynamics dyn and the loss function L for partial identity mapping id The minimization of each loss function will be explained below.

[0096] (Loss function L for generative adversarial network model adv ) The learning unit 16 essentially calculates the loss function L for the generative adversarial network model shown in the above formula (3). adv While training the generator G so that adv The discriminator Dy is trained so that

[0097] (The loss function L for the consistency of dynamics dyn ) The learning unit 16 essentially calculates the loss function L dyn Train the generator G so that

[0098] As described above, in this embodiment, the state y t and the action a of the dual-arm robot 4A at time t t When this is input, the state y of the dual-arm robot 4A at time t+1 is t+1 In this embodiment, a forward dynamics model F is prepared in advance, which outputs the state y t and the state y of the dual-arm robot 4A at time t+1 t+1 When the input is "a", the action a that the dual-arm robot 4A would have taken at time t is t The inverse dynamics model F -1 The above is prepared in advance.

[0099] For this reason, when the learning unit 16 trains the adversarial generative network model by machine learning, the inverse dynamics model F-1 In contrast, the state y of the dual-arm robot 4A at time t output from the generator G is t ^ and state y at time t+1 t+1 By inputting ^, the action a that the dual-arm robot 4A is estimated to have taken at time t is t ~ Calculate.

[0100] In addition, when the learning unit 16 performs machine learning on the generative adversarial network model, the state y of the dual-arm robot 4A output from the generator G is applied to the dynamics model F. t ^ and the inverse dynamics model F -1 Action a calculated by t ~ By inputting the above, the state y of the dual-arm robot 4A at time t+1 is t+1 ~ Calculate.

[0101] Then, the learning unit 16 calculates the state y of the dual-arm robot 4A at time t output from the generator G. t+1 ^ and the calculated state y of the dual-arm robot 4A at time t+1 t+1 ~ The generator G is trained so that the difference between

[0102] (The loss function L for partial identity mapping id ) The learning unit 16 essentially calculates the loss function L for the partial identity mapping shown in the above formula (5). id Train the generator G so that

[0103] Specifically, the learning unit 16 generates a trained generator G by training the generator G so that the difference between the operation data x of the learning dual-arm robot 4A and the state y^ of the dual-arm robot 4A output from the generator G is small.

[0104] Then, the learning unit 16 stores the trained generative adversarial network model including the trained generator G and the trained discriminator Dy in the trained model storage unit 18.

[0105] Next, the learning unit 16 generates robot motion data for the dual-arm robot 4A by inputting target motion data representing the motion of a target human to the learned generator G. Here, the target human motion corresponds to the motion to be taught to the dual-arm robot 4A.

[0106] The learning unit 16 learns a control model for controlling the dual-arm robot 4A based on the generated operation data of the dual-arm robot 4A, thereby generating a learned control model that outputs action a of the operation data when state y of the operation data is input.

[0107] For example, the learning unit 16 generates a trained control model using known imitation learning. This makes it possible to obtain a trained control model reflecting a measure for generating an action a from a state y. Note that a known function or machine learning model can be adopted as the control model.

[0108] Then, the learning unit 16 stores the learned control model in the control model storage unit 19.

[0109] (Control System 20) Fig. 7 is a block diagram showing a schematic configuration of a control system 20 according to this embodiment. As shown in Fig. 7, the control system 20 includes a camera 2B, a dual-arm robot 4B, and a control device 30. The control device 30 according to this embodiment uses a trained control model generated by the trained model generation device 10 to control the operation of the dual-arm robot 4B.

[0110] The camera 2B has the same configuration as the camera 2A described above, and sequentially captures images of the first arm R1 and the second arm R2 of the dual-arm robot 4B, which is the control target, while they are in motion. The camera 2B then outputs the obtained image data to the control device 30.

[0111] The dual-arm robot 4B has a similar configuration to the dual-arm robot 4A described above, and is a robot as shown in FIG.

[0112] Fig. 8 is a block diagram showing a hardware configuration of the control device 30 according to this embodiment. As shown in Fig. 8, the control device 30 has a CPU (Central Processing Unit) 62, a memory 64, a storage device 66, an input / output I / F (Interface) 68, a storage medium reading device 70, and a communication I / F 72. Each component is connected to each other via a bus 74 so as to be able to communicate with each other.

[0113] The storage device 66 stores control programs for executing each process described below. The CPU 62 is a central processing unit, and executes various programs and controls each component. That is, the CPU 62 reads the programs from the storage device 66 and executes the programs using the memory 64 as a working area. The CPU 62 controls each component and performs various arithmetic processes according to the programs stored in the storage device 66.

[0114] The memory 64 is made up of a RAM (Random Access Memory) and temporarily stores programs and data as a working area. The storage device 66 is made up of a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including the operating system, and various data.

[0115] The input / output I / F 68 is an interface for inputting data from the camera 2B and the dual-arm robot 4B and outputting data to the camera 2B and the dual-arm robot 4B. In addition, an input device for performing various inputs, such as a keyboard or a mouse, and an output device for outputting various information, such as a display or a printer, may be connected. A touch panel display may be used as the output device to function as the input device.

[0116] The storage medium reader 70 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

[0117] The communication I / F 72 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0118] Next, the functional configuration of the control device 30 will be described. As shown in Fig. 7, the control device 30 functionally includes an acquisition unit 34, a generation unit 36, and a control unit 38. A control model storage unit 32 is provided in a predetermined storage area of ​​the control device 30. Each functional configuration is realized by the CPU 62 reading out each program stored in the storage device 66, expanding it in the memory 64, and executing it.

[0119] The control model storage unit 32 stores the control trained model generated by the trained model generation device 10.

[0120] The acquisition unit 34 acquires the state of the dual-arm robot 4B and the state of the object b. Specifically, the acquisition unit 34 acquires the state (y R1 ,y R2 ,y b) to calculate

[0121] The generation unit 36 ​​generates a state (y R1 ,y R2 ,y b ) to determine the state (y R1 ,y R2 ,y b The behavior of the dual-arm robot 4B according to R1 ,a R2 )

[0122] The control unit 38 determines whether the action (a R1 ,a R2 Specifically, the control unit 38 controls the dual-arm robot 4B so as to take the action (a R1 ,a R2 ) is output to the first arm R1 and the second arm R2 of the dual-arm robot 4B.

[0123] Next, the operation of the trained model generation system 1 according to this embodiment will be described.

[0124] First, data on the movements of the human H and the dual-arm robot 4A is collected and input to the trained model generation device 10. The data on the movements of the human H and the dual-arm robot 4A is stored in the data storage unit 14. Then, when the trained model generation system 1 receives a predetermined instruction signal, the CPU 42 of the trained model generation device 10 reads out a trained model generation program from the storage device 46, expands it in the memory 44, and executes it. As a result, the CPU 42 functions as each functional component of the trained model generation device 10, and the trained model generation process shown in Figs. 9 and 10 is executed.

[0125] In step S100, the learning data acquisition unit 12 acquires learning data from the data stored in the data storage unit .

[0126] In step S102, the learning unit 16 performs machine learning on a generative adversarial network model including a generator G and a discriminator Dy based on the learning data acquired in step S100, thereby generating a trained generator G that outputs motion data y of the dual-arm robot 4A when motion data x representing the motion of a target human H is input.

[0127] In step S104, a trained generative adversarial network model including the trained generator G and the trained discriminator Dy is stored in the trained model storage unit 18.

[0128] Next, when the trained model generation system 1 receives a predetermined instruction signal, the trained model generation device 10 executes a trained model generation process shown in FIG.

[0129] In step S200, the learning unit 16 acquires target motion data representing a motion of a target human H. For example, the learning unit 16 acquires, as the target motion data, motion data to be taught to the double-arm robot 4A from among the motion data of the human H included in the learning data. Note that data different from the learning data may be used as the target motion data.

[0130] In step S202, the learning unit 16 reads out the trained generator G from the trained model storage unit 18.

[0131] In step S204, the learning unit 16 generates motion data for the double-arm robot 4A by inputting the target motion data acquired in step S200 to the learned generator G read out in step S202.

[0132] In step S206, the learning unit 16 learns a control model for controlling the dual-arm robot 4A based on the operation data of the dual-arm robot 4A obtained in step S204, thereby generating a learned control model that outputs action a of the operation data when state y of the operation data is input.

[0133] In step S208, the learning unit 16 stores the learned control model generated in step S206 in the control model storage unit 19.

[0134] Next, the operation of the control system 20 according to this embodiment will be described.

[0135] When the trained model for control generated by the trained model generation system 1 is input to the control device 30, the trained model for control is stored in the control model storage unit 19. Then, when the control system 20 receives a predetermined instruction signal, the CPU 62 of the control device 30 reads out a control program from the storage device 66, expands it in the memory 64, and executes it. As a result, the CPU 62 functions as each functional component of the control device 30, and the control process shown in FIG. 11 is executed.

[0136] In step S300, the acquisition unit 34 acquires the state (y R1 ,y R2 ,y b ) to get the

[0137] In step S302, the generation unit 36 ​​reads out the learned control model stored in the control model storage unit 32.

[0138] In step S304, the generation unit 36 ​​applies the state (y R1 ,y R2 ,y b ) to determine the state (y R1 ,yR2 ,y b The behavior of the dual-arm robot 4B according to R1 ,a R2 )

[0139] In step S306, the control unit 38 determines whether the action (a R1 ,a R2 The dual-arm robot 4B is controlled so as to pick up the object.

[0140] The control process shown in FIG. 11 is repeated, and a control signal is repeatedly output to the double-arm robot 4A, whereby a task for the object b is executed.

[0141] As described above, the trained model generation device according to the present embodiment trains a generative adversarial network model including a generator and a discriminator by machine learning based on training data including training human motion data, thereby generating a trained generator that outputs motion data of a robot to be controlled when motion data representing a target human motion is input. This makes it possible to obtain a trained generator that can easily generate robot motion data from human motion data.

[0142] In addition, the trained model generation device according to this embodiment uses a loss function L dyn Loss function L including full By generating a trained generator so that is minimized, it is possible to obtain a trained generator that can generate motion data that takes into account the consistency of the dynamics of the dual-arm robot 4A. This prevents the generation of motion data that ignores the dynamics of the dual-arm robot 4A.

[0143] In addition, the trained model generation device according to this embodiment uses a loss function L id Loss function L including fullBy generating a trained generator so that the is minimized, it is possible to obtain a trained generator capable of generating motion data that matches the real motion of the dual-arm robot 4A. This prevents the generation of motion data that ignores the real motion of the dual-arm robot 4A.

[0144] Loss function L for dynamics consistency dyn and the loss function L for partial identity mapping id By generating a trained generator taking this into consideration, for example, generation of motion data that ignores the range of motion of the dual-arm robot 4A is suppressed.

[0145] In addition, by including random data representing the random movements of the dual-arm robot 4A in the learning data, a learned generator can be generated taking into account factors such as the range of motion of the robot to be controlled. EXAMPLES

[0146] Next, an example will be described. In this example, a simulation is performed to verify the effectiveness of the proposed LfD-HR.

[0147] FIG. 12 is a diagram for explaining this embodiment. As shown in FIG. 12, three simulations are performed in this embodiment. "Demonstration" shown in FIG. 12 corresponds to demonstrations T1 and T2 in FIG. 1 described above. Also, "Execution" shown in FIG. 12 corresponds to execution phase E in FIG. 2 described above. Dual-arm block push in FIG. 12(a) is intended for cooperative operation in which Robot1 and Robot2 push out a block. Also, dual-arm peg insertion in FIG. 12(b) is intended for cooperative operation in which a peg held by Robot2 is inserted into a hole held by Robot1.

[0148] Fig. 13 shows the results of this embodiment. As shown in Fig. 13, according to the proposed LfD-HR, Robot1 and Robot2 can cooperate to accomplish the target task.

[0149] In the above embodiment, the robot to be controlled is a dual-arm robot, but the present invention is not limited to this and may be any robot. For example, a robot having one arm may be the target. The robot to be controlled may also be, for example, a robot having multiple fingers. In this case, operation data relating to the movements of the multiple fingers is generated.

[0150] In addition, various processors other than the CPU may execute each process executed by reading the software (program) in the above embodiment. Examples of the processor in this case include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. Each process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, a combination of a CPU and an FPGA, etc.). More specifically, the hardware structure of these various processors is an electric circuit that combines circuit elements such as semiconductor elements.

[0151] In the above embodiment, the programs are stored (installed) in advance in a storage device, but the present invention is not limited to this. The programs may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, a Blu-ray disc, or a USB memory. The programs may be downloaded from an external device via a network.

[0152] (Additional Note) The following additional notes are given regarding aspects of the present disclosure.

[0153] (Appendix 1) A learning acquisition unit that acquires learning data including learning human motion data; a learning unit that performs machine learning on a generative adversarial network model including a generator and a discriminator based on the learning data acquired by the learning acquisition unit, thereby generating a trained generator that outputs motion data of a robot to be controlled when motion data representing a target human motion is input; A trained model generating device comprising:

[0154] (Appendix 2) The operation data includes a state y and an action a of the robot to be controlled, The state y of the robot to be controlled at time t t and the action a of the controlled robot at time t t When the input is made, the state y of the robot to be controlled at time t+1 is t+1 A forward dynamics model F that outputs The state y of the robot to be controlled at time t t and the state y of the controlled robot at time t+1 t+1 When the input is "a", the action a taken by the robot to be controlled at time t is t The inverse dynamics model F -1 and are prepared in advance, The learning unit is The inverse dynamics model F -1 The state y of the robot to be controlled at time t output from the generator is t ^ and state y at time t+1 t+1 By inputting ^, the action a that the controlled robot would have taken at time t is obtained. t ~ Calculate For the forward dynamics model F, the state y t ^ and the above action at ~ By inputting the above, the state y of the controlled robot at time t+1 is t+1 ~ Calculate The state y of the robot to be controlled at time t output from the generator t+1 ^ and the calculated state y of the controlled robot at time t+1 t+1 ~ generating the trained generator by training the generator so that the difference between 2. A trained model generation device according to claim 1.

[0155] (Appendix 3) The learning data further includes motion data x of a robot to be controlled for learning, The learning unit generates the trained generator by training the generator so that a difference between the motion data x of the controlled robot for learning and the state y^ of the controlled robot output from the generator becomes small. 3. The trained model generation device according to claim 1 or 2.

[0156] (Appendix 4) The robot to be controlled is a robot equipped with at least one arm. The trained model generating device according to any one of Supplementary Note 1 to Supplementary Note 3.

[0157] (Appendix 5) the robot to be controlled is a dual-arm robot having a first arm and a second arm, The learning data is demonstration data representing a coordinated motion between the first arm and the human arm; and demonstration data representing a cooperative motion between the second arm and the human arm. 5. The trained model generation device according to claim 4.

[0158] (Appendix 6) The learning data is Random data representing a random movement of the robot to be controlled; and random data representing random movements of the human. The trained model generating device according to any one of Supplementary Note 1 to Supplementary Note 3.

[0159] (Appendix 7) The learning unit is generating motion data of the robot to be controlled by inputting target motion data representing a motion of a target human to the learned generator; a control trained model for controlling the control target robot based on the generated operation data of the control target robot, the control trained model being for outputting an action of the operation data when a state of the operation data is input; The trained model generating device according to any one of Supplementary Note 1 to Supplementary Note 3.

[0160] (Appendix 8) An acquisition unit for acquiring a state of a robot to be controlled; A generation unit that generates a behavior of the control target robot according to the state by inputting the state acquired by the acquisition unit into the control trained model generated by the trained model generation device described in Supplementary Note 7; A control unit that controls the target robot to take the action generated by the generation unit; A control device comprising:

[0161] (Appendix 9) acquiring training data including training human motion data; A trained generator is generated that outputs motion data of a robot to be controlled when motion data representing a motion of a target human is inputted, by performing machine learning on a generative adversarial network model including a generator and a discriminator based on the acquired training data. A method for generating trained models in which processing is performed by a computer.

[0162] (Appendix 10) acquiring training data including training human motion data; A trained generator is generated that outputs motion data of a robot to be controlled when motion data representing a motion of a target human is inputted, by performing machine learning on a generative adversarial network model including a generator and a discriminator based on the acquired training data. A trained model generation program for allowing a computer to execute processing. [Explanation of symbols]

[0163] 4A, 4B Dual-arm robot 10 Trained model generator 12 Learning acquisition section 14 Data storage unit 16 Learning Department 18 Trained model memory 19,32 Control model memory section 20. Control System 30 Control device 34 Acquisition Department 36 Generation part 38 Control Unit

Claims

1. A learning acquisition unit that acquires learning data including learning human motion data; a learning unit that performs machine learning on a generative adversarial network model including a generator and a discriminator based on the learning data acquired by the learning acquisition unit, thereby generating a trained generator that outputs motion data of a robot to be controlled when motion data representing a target human motion is input; A trained model generating device comprising:

2. The operation data includes a state y and an action a of the robot to be controlled, The state y of the robot to be controlled at time t ｔ and the action a of the controlled robot at time t ｔ When the input is made, the state y of the robot to be controlled at time t+1 is ｔ＋１ A forward dynamics model F that outputs The state y of the robot to be controlled at time t ｔ and the state y of the controlled robot at time t+1 ｔ＋１ When the input is "a", the action a taken by the robot to be controlled at time t is ｔ The inverse dynamics model F is －１ and are prepared in advance, The learning unit is The inverse dynamics model F －１ The state y of the robot to be controlled at time t output from the generator is ｔ ^ and state y at time t+1 ｔ＋１ By inputting ^, the action a that the controlled robot would have taken at time t is obtained. ｔ ～ Calculate For the forward dynamics model F, the state y ｔ ^ and the action a ｔ ～ By inputting the above, the state y of the robot to be controlled at time t+1 is obtained. ｔ＋１ ～ Calculate The state y of the robot to be controlled at time t output from the generator ｔ＋１ ^ and the calculated state y of the controlled robot at time t+1 ｔ＋１ ～ generating the trained generator by training the generator so that the difference between The trained model generating device according to claim 1 .

3. The learning data further includes motion data x of a robot to be controlled for learning, The learning unit generates the trained generator by training the generator so that a difference between the motion data x of the controlled robot for learning and the state y^ of the controlled robot output from the generator becomes small. The trained model generating device according to claim 1 or 2.

4. The robot to be controlled is a robot equipped with at least one arm. The trained model generating device according to claim 1 or 2.

5. the robot to be controlled is a dual-arm robot including a first arm and a second arm, The learning data is demonstration data representing a coordinated motion between the first arm and the human arm; and demonstration data representing a cooperative motion between the second arm and the human arm. The trained model generating device according to claim 4.

6. The learning data is Random data representing a random movement of the robot to be controlled; and random data representing random movements of the human. The trained model generating device according to claim 1 or 2.

7. The learning unit is generating motion data of the robot to be controlled by inputting target motion data representing a motion of a target human to the learned generator; a control trained model for controlling the control target robot based on the generated operation data of the control target robot, the control trained model being for outputting an action of the operation data when a state of the operation data is input; The trained model generating device according to claim 1 or 2.

8. An acquisition unit for acquiring a state of a robot to be controlled; A generation unit that generates a behavior of the target robot according to a state by inputting the state acquired by the acquisition unit into the control trained model generated by the trained model generation device according to claim 7; A control unit that controls the target robot to take the action generated by the generation unit; A control device comprising:

9. acquiring training data including training human motion data; A trained generator is generated that outputs motion data of a robot to be controlled when motion data representing a motion of a target human is inputted, by performing machine learning on a generative adversarial network model including a generator and a discriminator based on the acquired training data. A method for generating trained models in which processing is performed by a computer.

10. acquiring training data including training human motion data; A trained generator is generated that outputs motion data of a robot to be controlled when motion data representing a motion of a target human is inputted, by performing machine learning on a generative adversarial network model including a generator and a discriminator based on the acquired training data. A trained model generation program for allowing a computer to execute processing.

Images