Information processing device, method, and program

Abstract

A first inference unit (331) uses a first trained model (321) to infer command information at a first unit time. A second inference unit (332) uses a second trained model (322) to infer command information at a second unit time that is shorter than the first unit time from information that corresonds to command information derived by means of the first trained model (321). An operation control unit (333) uses the command information inferred by the second inference unit (332) to operate a control target device (10B).

Description

[Technical field]

[0001] The present invention relates to an information processing device, a method, and a program. [Background technology]

[0002] There are known technologies for controlling robots. For example, Patent Document 1 describes a robot that is remotely controlled by a master-slave system. In addition, a technology has been proposed in which the remote operation of a robot using a master-slave system is machine-learned to generate a trained model, and the robot is operated using the trained model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2015 / 041046 publication Summary of the Invention [Problem to be solved by the invention]

[0004] When operating a robot using a trained model generated through machine learning, the robot may not operate properly if the operating conditions (movement speed, etc.) or operating environment (physical characteristics such as the shape and position of the object being operated, flexibility, etc., obstacle positions, etc.) change. In order to respond to these changes in operating conditions and operating environment and stabilize the robot's operation predictions, a huge amount of training data is required, and there is also the problem that the calculation time becomes enormous.

[0005] An aspect of the present invention aims to realize a control target device, such as a robot, with stable motion prediction when controlling the motion of the control target device. [Means for solving the problem]

[0006] An information processing device according to aspect 1 of the present invention includes a first estimation unit that estimates command information for every first unit time using a first learned model that derives command information to be supplied to the control target device from information representing a state of the control target device output by the control target device operating in accordance with command information, a second estimation unit that estimates the command information for every second unit time shorter than the first unit time using a second learned model that derives the command information to be supplied to the control target device from the information representing the state output by the control target device and the command information derived by inputting the information representing the state into the first learned model, and an operation control unit that operates the control target device using the command information estimated by the second estimation unit.

[0007] According to the above configuration, the information processing device operates the controlled device using a first trained model that derives command information from information representing a state, and a second trained model that derives command information from information corresponding to the output of the first trained model. By using these multiple trained models hierarchically in the operation control of the controlled device, it is possible to stabilize the operation prediction of the controlled device without requiring a large amount of teacher data. In addition, although a two-layer trained model is described here, it is also possible to increase the number of layers to use an n-layer model, or to select whether or not to use each hierarchical model depending on the operation purpose of the controlled device.

[0008] In an information processing device according to aspect 2 of the present invention, in aspect 1, the command information is information issued from a device that outputs the command information, and the information input to the second trained model may include command information derived by the first trained model, and information representing the command information and the state.

[0009] According to the above configuration, the input data of the second trained model is a set of the output of the first trained model estimated in the first unit time and the state output by the controlled device output in the second unit time. Since the first unit time is set longer than the second unit time, the second trained model can predict the operation of the controlled device as the output of the first trained model fluctuates less, and the prediction can be stabilized.

[0010] In the information processing device according to aspect 3 of the present invention, in aspect 1 or 2, the information input to the second trained model may be data obtained by reducing correlation (converting to reduce correlation) between a pair of command information derived by the first trained model and information representing the state. For example, the correlation can be reduced by generating data of the sum and difference between the two pieces of data.

[0011] According to the above configuration, the correlation between the input data of the second trained model is reduced. This makes it possible to stabilize the operation prediction of the controlled device. The lower the correlation between the data, the more stable the operation prediction of the controlled device.

[0012] In the information processing device of aspect 4 of the present invention, in aspects 1 to 3, the device that outputs the command information and the device to be controlled may each be equipped with a robot arm, and the command information and the information representing the state may each include information indicating the operation of the robot arm.

[0013] According to the above configuration, the information processing device controls the operation of the controlled device using a hierarchical trained model that derives command information including information indicating the operation of the robot arm. This makes it possible to stabilize the operation prediction of the controlled device that includes a robot arm.

[0014] In an information processing device according to aspect 5 of the present invention, in aspects 1 to 4, the device that outputs the command information is a master device that operates in response to a user operation, the controlled device is a slave device that operates in accordance with the command information output from the master device, and the first trained model and the second trained model may be trained using the command information and information representing the state collected in a master-slave system including the master device and the slave device.

[0015] According to the above configuration, the information processing device operates the controlled device using a plurality of hierarchical trained models generated using command information and information representing states collected in the master-slave system, thereby making it possible to stabilize the prediction of the operation of the controlled device.

[0016] An information processing device according to aspect 6 of the present invention includes a first generation unit that generates a first trained model by using a set of information representing a state of a controlled device output by the controlled device operating in accordance with command information and the command information supplied to the controlled device, and a second generation unit that generates a second trained model by using a set of input information including information representing a relationship between the information representing the state output by the controlled device and the command information derived by inputting the information representing the state into the first trained model, and the command information supplied to the controlled device.

[0017] According to the above configuration, the information processing device generates a first trained model that derives command information from information representing a state, and a second trained model that derives command information from information corresponding to the output of the first trained model. These trained models having a plurality of estimated unit times are used hierarchically in the operation control of the controlled device, so that the first generation unit and the second generation unit can be operated independently. In other words, since the first trained model and the second trained model can be obtained independently, learning and prediction of the operation of the controlled device can be stabilized without requiring a huge amount of teacher data.

[0018] The information processing device of aspect 7 of the present invention, in aspect 6, may further include a first estimation unit that estimates the command information by inputting information representing the state output by the controlled device to the first learned model every first unit time, a second estimation unit that estimates the command information by inputting input data including information representing a relationship between the information representing the state output by the controlled device and the command information derived by inputting the information representing the state into the first learned model every second unit time shorter than the first unit time, and an operation control unit that operates the controlled device using the command information estimated by the second estimation unit.

[0019] According to the above configuration, the information processing device operates the controlled device using a first trained model that derives command information from information representing a state, and a second trained model that derives command information from information corresponding to the output of the first trained model. These multiple trained models are used hierarchically in the operation control of the controlled device, so that the operation prediction of the controlled device can be stabilized without requiring a huge amount of teacher data.

[0020] An information processing device according to an eighth aspect of the present invention is an information processing device according to the sixth or seventh aspect, wherein the device that outputs the command information is a master device that operates in response to a user operation, the controlled device is a slave device that operates in accordance with the command information output from the master device, the first generation unit generates the first trained model using the command information and information representing the state collected in a master-slave system including the master device and the slave device, and the second generation unit generates the second trained model using the command information and information representing the state collected in the master-slave system.

[0021] According to the above configuration, the information processing device generates multiple hierarchical trained models using command information and state information collected in the master-slave system. Since the master device operates in response to user operation, it is possible to collect, as the command information, information that is controlled while a human feels the control delay of the slave device and dynamic interaction with the environment through the master device. By using these multiple trained models in the operation control of the controlled device, it is possible to stabilize the operation prediction of the controlled device.

[0022] An information processing device according to a 9th aspect of the present invention, in any of the first to eighth aspects, may be a learned model that derives the command information to be supplied to the control target device after the control target device outputs information representing the state from input data including information representing the state and information including an operating speed that specifies the operation of the control target device.

[0023] According to the above configuration, the information processing device estimates command information using a first trained model trained using teacher data including information including an operating speed that defines the operation of the controlled device. This makes it possible to stabilize the operation prediction of the controlled device, whose operating speed is variable.

[0024] In the information processing device according to aspect 10 of the present invention, in accordance with aspect 9, the information including the operation speed may include information indicating a frequency indicating an operation cycle of the control target device or information indicating an operation execution time.

[0025] According to the above configuration, the information processing device estimates command information using the first trained model trained using teacher data including a frequency indicating an operation period of the controlled device. This makes it possible to stabilize operation prediction of the controlled device that performs a periodic operation.

[0026] An information processing device according to aspect 11 of the present invention, in aspect 9 or 10, may be a learned model that derives, from the input data, the command information to be supplied to the control target device after a predetermined unit time has elapsed since the control target device output information representing the state.

[0027] According to the above configuration, the information processing device estimates the command information using the first trained model that derives the command information to be supplied to the control target device after a unit time has elapsed since the control target device output information representing the state. This makes it possible to stabilize the operation prediction of the control target device, whose operation speed is variable.

[0028] In an information processing device according to aspect 12 of the present invention, in aspect 6 or 7, the first learned model is a learned model that derives, from input data including information representing the state and information including an operating speed that specifies an operation to be instructed to the control target device, the command information to be supplied to the control target device as output data after the control target device outputs the information representing the state, and the first generation unit may generate the first learned model using the command information, the information representing the state, and the information including the operating speed collected by operating the control target device while changing the operating speed of the control target device.

[0029] According to the above configuration, the first trained model is generated using teacher data collected by operating the control target device while changing the operation speed of the control target device. The information processing device controls the operation of the control target device using this trained model, thereby making it possible to stabilize the operation prediction of the control target device, which has a variable operation speed.

[0030] An information processing device according to aspect 13 of the present invention, in aspects 1 to 12, may be a learned model that is trained using as input information information representing a second state derived by a third trained model that derives, from the command information, information representing a second state that the control target device outputs after the command information is supplied to the control target device.

[0031] According to the above configuration, information representing a state derived by the third trained model is trained as input information for the first trained model to generate the first trained model. The information processing device can stabilize the operation prediction of the controlled device by controlling the operation of the controlled device using the first trained model.

[0032] A method according to a fourteenth aspect of the present invention is a method in which an information processing device executes the following steps: a first step of estimating command information for each first unit time using a first trained model that derives the command information to be supplied to the control target device from information representing the state of the control target device output by the control target device that operates according to command information; a second step of estimating the command information for each second unit time shorter than the first unit time using a second trained model that derives the command information to be supplied to the control target device from information representing the relationship between the information representing the state output by the control target device and the command information derived by inputting the information representing the state into the first trained model; and a third step of operating the control target device using the command information estimated in the second step. Here, up to the third step is described, but the number of steps may be n steps, which is a finite integer.

[0033] According to the above configuration, the information processing device operates the controlled device using a first trained model that derives command information from information representing a state, and a second trained model that derives command information from information corresponding to the output of the first trained model. These multiple trained models are used hierarchically in the operation control of the controlled device, so that the operation prediction of the controlled device can be stabilized without requiring a huge amount of teacher data.

[0034] In an information processing device according to aspect 15 of the present invention, in aspects 1 to 5, the second estimation unit estimates the command information using n (n is a natural number greater than or equal to 2) second trained models, the n second trained models are used hierarchically, and information representing a relationship according to the command information derived from the second trained models in each layer may become input information for a second trained model in a lower layer of the second trained model.

[0035] According to the above configuration, multiple trained models are used hierarchically, thereby making it possible to stabilize prediction of the operation of the controlled device.

[0036] An information processing device according to aspect 16 of the present invention, in aspects 1 to 5, may further include a change control unit that changes one or both of the first unit time and the second unit time depending on the hardness or softness of the object, wherein the controlled device is a device that performs work on an object.

[0037] According to the above configuration, the estimation period can be changed depending on the hardness or softness of the object, thereby stabilizing the prediction of the operation of the device to be controlled.

[0038] An information processing device according to aspect 17 of the present invention, in aspects 1 to 12, may be such that the first trained model is a trained model that derives information representing a second state of the control target device after a predetermined time has elapsed and the command information from information representing the state, and is a trained model trained using the information representing the second state derived by the first trained model as input information.

[0039] According to the above configuration, learning is performed using information representing a state derived by the first trained model as input information for the first trained model, and the first trained model is generated. The information processing device can stabilize the operation prediction of the control target device by controlling the operation of the control target device using this first trained model.

[0040] An information processing device according to aspect 18 of the present invention, in aspects 1 to 12, may be a trained model that derives information representing a second state of the control target device after a predetermined time has elapsed and the command information from the command information and information representing the state, and the first trained model is trained using the command information and the information representing the second state derived by the first trained model as input information.

[0041] According to the above configuration, learning is performed using information representing a state derived by the first trained model as input information for the first trained model, and the first trained model is generated. The information processing device can stabilize the operation prediction of the control target device by controlling the operation of the control target device using this first trained model.

[0042] In an information processing device according to a nineteenth aspect of the present invention, in the fifteenth aspect, the n second trained models may each have a different operating frequency.

[0043] According to the above configuration, by using a plurality of trained models in a hierarchical manner, each of a plurality of operations performed by the controlled device can be trained at an operating frequency suitable for that operation.

[0044] The information processing device according to aspect 20 of the present invention is the same as that according to aspect 13 , 15, 17, and 18 In the method, the input information may be a value that is quantitatively expressed.

[0045] According to the above configuration, since the input information is a value expressed quantitatively, the information processing device can correct the command speed according to the actual operation speed. In other words, since the information processing device can learn autonomously, it is possible to easily train a trained model.

[0046] In an information processing device according to aspect 21 of the present invention, in aspects 1 to 5, the controlled device grasps a grasp object or contacts a contact object, and the first learned model and the second learned model store an action of grasping the grasp object or contacting the contact object, and the action of grasping the grasp object or contacting the contact object may be learned in either the first learned model or the second learned model depending on the hardness of the grasp object or the contact object.

[0047] According to the above configuration, even for the same motion, the trained model to be trained differs depending on the hardness of the object, so that the trained model can be trained at a more suitable sampling time. Therefore, according to the above configuration, the time required for training can be reduced, and the memory capacity required for storing the trained model can be reduced.

[0048] In an information processing device of aspect 22 of the present invention, in aspect 23, the first trained model learns the behavior when the object to be grasped is a soft object, and the second trained model learns the behavior when the object to be grasped is a hard object, and the first trained model and the second trained model may have different operating frequencies.

[0049] According to the above configuration, when the object is a soft object, a low operating frequency is preferable, and therefore the object is trained into a trained model with a low operating frequency, whereas when the object is a hard object, a high operating frequency is preferable, and therefore the object is trained into a trained model with a high operating frequency. Therefore, according to the above configuration, the trained model can be trained with a more suitable sampling time. Therefore, according to the above configuration, the time required for training can be reduced, and the memory capacity required for storing the trained model can be reduced.

[0050] A program according to a twenty-third aspect of the present invention is a program for causing a computer to function as the information processing device of any one of the first to thirteenth and fifteenth to twenty-second aspects, and is a program for causing a computer to function as each of the above-mentioned units.

[0051] In an information processing device according to aspect 24 of the present invention, in aspects 1 to 5 and 7, the first estimation unit may estimate the command information at multiple time points, and the second estimation unit may estimate the command information for each of the estimated command information at the multiple time points. According to the above configuration, the upper layer learning model and the lower layer learning model can be trained separately and independently. Also, according to the above configuration, the state can be estimated more accurately with less training.

[0052] The scope of the present invention also includes the program according to aspect 19 and a computer-readable recording medium having the program recorded thereon.

[0053] The information processing device according to each aspect of the present invention may be realized by a computer. In this case, the control program of the information processing device that realizes the information processing device on a computer by causing the computer to operate as each part (software element) of the information processing device, and the computer-readable recording medium on which the control program is recorded, also fall within the scope of the present invention. Effect of the Invention

[0054] According to one aspect of the present invention, it is possible to stabilize operation prediction when controlling the operation of a controlled device using a learned model. [Brief description of the drawings]

[0055] [Figure 1] FIG. 1 is a diagram illustrating an outline of a configuration of a system according to a first embodiment of the present invention. [Diagram 2] FIG. 1 is a diagram illustrating an example of a first trained model. [Diagram 3] FIG. 13 is a diagram illustrating an example of a second trained model. [Figure 4] 11 is a flowchart illustrating a flow of processing performed by an information processing device. [Diagram 5] FIG. 1 is a diagram illustrating a system for collecting teacher data. [Figure 6] FIG. 10 is a diagram illustrating an outline of a system configuration according to a second embodiment of the present invention. [Figure 7] 11 is a flowchart illustrating a flow of processing performed by an information processing device. [Figure 8] FIG. 1 is a diagram illustrating a hierarchical trained model. [Figure 9] FIG. 11 is a diagram illustrating an outline of a system configuration according to a third embodiment of the present invention. [Figure 10] FIG. 11 is a diagram illustrating a schematic configuration of a system according to a fourth embodiment of the present invention. [Figure 11] 1 is a graph showing changes over time in state values ​​of a robot arm controlled by a conventional method. [Figure 12] 1 is a graph showing changes over time in state values ​​of a robot arm controlled by a conventional method. [Figure 13] 11 is a graph showing changes over time in the state value of a robot arm. [Figure 14] FIG. 11 is a diagram illustrating a schematic configuration of a system according to a fifth embodiment of the present invention. [Figure 15] FIG. 13 is a diagram illustrating a schematic configuration of a system according to a sixth embodiment of the present invention. [Figure 16]FIG. 13 is a diagram illustrating an example of the relationship between a first learned model and a third learned model. [Figure 17] FIG. 13 is a diagram illustrating an example of the relationship between a first learned model and a third learned model. [Figure 18] FIG. 13 is a diagram illustrating a schematic configuration of a system according to a seventh embodiment of the present invention. [Figure 19] FIG. 13 is a diagram illustrating a schematic configuration of a system according to an eighth embodiment of the present invention. [Figure 20] FIG. 13 is a diagram illustrating a schematic configuration of a system according to a ninth embodiment of the present invention. [Figure 21] FIG. 1 is a diagram illustrating an example of the contents of a first trained model. [Figure 22] FIG. 1 is a diagram illustrating an example of the contents of a first trained model. [Figure 23] FIG. 1 is a diagram illustrating an example of the contents of a first trained model. [Figure 24] FIG. 13 is a diagram illustrating an outline of a system configuration according to a modified example of the ninth embodiment of the present invention. [Diagram 25] FIG. 13 is a diagram illustrating a schematic configuration of a system according to a tenth embodiment of the present invention. [Figure 26] FIG. 23 is a diagram illustrating the functional configuration of a system according to a tenth embodiment of the present invention. [Figure 27] FIG. 23 is a schematic diagram for explaining in more detail a hierarchical action generator according to a tenth embodiment of the present invention. [Figure 28] FIG. 23 is a diagram illustrating an example in which a slave device according to a tenth embodiment of the present invention writes characters with a writing implement. [Figure 29] FIG. 23 is a diagram showing an example of a character written on a sheet of paper by a slave device according to a tenth embodiment of the present invention. [Diagram 30] FIG. 23 is a diagram for explaining ideal values ​​and response values ​​when a system according to an eleventh embodiment of the present invention is used. [Diagram 31] FIG. 1 is a diagram illustrating state prediction according to a conventional technique. [Diagram 32] 10A and 10B are diagrams for explaining the problem of deviation accumulation according to the present invention. [Diagram 33]FIG. 23 is a diagram illustrating a prediction of a state according to the sixth embodiment of the present invention. [Diagram 34] FIG. 13 is a diagram illustrating a state prediction according to the ninth embodiment of the present invention. [Diagram 35] FIG. 12 is a schematic diagram for explaining in more detail the hierarchical trained model according to the twelfth embodiment of the present invention. [Diagram 36] FIG. 23 is a diagram illustrating a state prediction according to the thirteenth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0056] [Embodiment 1] Hereinafter, one embodiment of the present invention will be described.

[0057] 1 is a diagram illustrating a schematic configuration of a system 1 including an information processing device 30 according to a first embodiment of the present invention. The system 1 is a system that generates a trained model used to operate a controlled device. The system 1 includes a slave device 10, a master device 20, and an information processing device 30.

[0058] The slave unit 10 and the master unit 20 are, for example, robots equipped with a robot arm. The slave unit 10 is a device that operates according to command information MASTER(t) supplied at time t. The slave unit 10 is an example of a controlled device according to the present invention. The slave unit 10 outputs status information SLAVE(t) that represents the status of the slave unit 10 at time t. The status information SLAVE(t) includes, for example, information indicating the operation of the robot arm included in the slave unit 10 at time t. The information indicating the operation of the robot arm includes, for example, information indicating the position of the robot arm, posture such as joint angles, angular velocity of the joints, torque, and an image indicating the relationship between the surrounding situation and the robot arm. Note that the information included in the status information SLAVE(t) is not limited to these, and other information may be included.

[0059] The master device 20 is a device that operates in response to a user operation. The master device 20 is an example of a device that outputs command information. The master device 20 supplies command information MASTER(t) to the slave device 10. The command information MASTER(t) is information emitted from a device that outputs command information. In this embodiment, the command information MASTER(t) is information that represents a state of the master device 20 that has changed in response to a user operation on the master device 20. The command information MASTER(t) includes, for example, information that represents an operation of a robot arm provided in the master device 20 at time t. The information that represents the operation of the robot arm includes, for example, information that represents an image that represents the position of the robot arm, posture such as the angle of the joints, angular velocity of the joints, torque, and a relationship between the surrounding situation and the robot arm. Note that the information included in the command information MASTER(t) is not limited to these, and other information may be included.

[0060] The actions performed by the slave device 10 and the master device 20 include almost all actions performed by humans, such as drawing letters or images using a writing instrument, cleaning using a cleaning tool, cooking, grasping an object, painting, polishing, cutting, drilling, or assembling, as well as actions performed in cooperation between humans and robots, such as transporting or moving heavy objects.

[0061] The information processing device 30 is a device that generates a trained model for operating a controlled device, using command information MASTER(t) output from the master device 20 and state information SLAVE(t) output from the slave device 10. The information processing device 30 is, for example, a server device or a personal computer.

[0062] The information processing device 30 includes a learning phase execution unit 31 and a storage unit 32. The learning phase execution unit 31 includes a first generation unit 311 and a second generation unit 312. The storage unit 32 stores a first trained model 321, a second trained model 322, and a teacher dataset 323.

[0063] The first generation unit 311 uses a pair of input information and output information to generate a first trained model 321 that has been trained to learn the correlation between the input information and the output information. In this embodiment, the input information input to the first trained model 321 is state information SLAVE(t) output by the slave device 10 at time t. The output information is command information MASTER(t+T1) supplied to the slave device 10 at time (t+T1) after a predetermined unit time T1 has elapsed from time t. Note that the unit time T1 may be zero or a negative value.

[0064] The first trained model 321 may be any trained model capable of generating command information MASTER(t+T1) from state information SLAVE(t). The first trained model 321 may be realized, for example, by a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Long Short-Term Memory (LSTM), a Deep Neural Network (DNN), or a combination thereof.

[0065] FIG. 2 is a diagram showing a schematic example of the first trained model 321. As described above, state information SLAVE(t) is input as input data to the first trained model 321. The first trained model 321 is composed of, for example, a convolutional layer, a pooling layer, and a combination layer. In the convolutional layer, the input data is convoluted by filtering. The convolutional data is pooled in the pooling layer. This improves the model's ability to recognize changes in the position of features in the data. The pooling data is processed in the combination layer, and is converted into the output data of the first trained model 321, that is, the command information MASTER(t+T1) supplied to the slave device 10 at time (t+T1), and output. The unit time T1 is, for example, a sampling period of the LSTM, and is, for example, 400 msec.

[0066] When the master device 20 and the slave device 10 are equipped with a robot arm, the state information SLAVE(t) includes, for example, nine parameters of the joint angles (θs1(t), θs2(t), θs3(t)), the joint angular velocities (θs1'(t), θs2'(t), θs3'(t)), and the torques (τs1(t), τs2(t), τs3(t)) of the robot arm at time t. The command information MASTER(t) also includes nine parameters of the joint angles (θm1(t), θm2(t), θm3(t)), the joint angular velocities (θm1'(t), θm2'(t), θm3'(t)), and the torques (τm1(t), τm2(t), τm3(t)) of the robot arm at time t.

[0067] The second generation unit 312 uses a pair of input information and output information to generate a second trained model 322 that has learned the correlation between the input information and the output information. The input information of the second trained model 322 may be the state information SLAVE(t) output by the slave device 10 and the command information MASTER(t+T1) derived by inputting the state information SLAVE(t) to the first trained model 321, or data with these information reduced in correlation. For example, as shown in FIG. 3, the input information may be in a form including information DATA(t, t+T1) that represents the relationship between the state information SLAVE(t) output by the slave device 10 and the command information MASTER(t+T1) derived by inputting the state information SLAVE(t) to the first trained model 321. The output information of the second trained model 322 is the command information MASTER(t+T2) supplied to the slave device 10 after the second unit time T2 has elapsed. That is, the output information of the second trained model 322 is command information MASTER(t+T2) supplied to the slave device 10 after a second unit time T2, which is shorter than the first unit time T1, has elapsed since the slave device 10 output the state information SLAVE(t), which is the input information of the first trained model 321. In this embodiment, the unit time T2 may be shorter than the unit time T1.

[0068] The second trained model 322 may be any trained model capable of generating command information MASTER(t) from state information SLAVE(t). The second trained model 322 may be realized by, for example, CNN, RNN, LSTM, DNN, or a combination thereof. The unit time T2 is, for example, a sampling period of LSTM, and is, for example, 20 msec. The unit time T2 may be zero.

[0069] Information DATA(t, t+T1) is information that represents the relationship between status information SLAVE(t) and command information MASTER(t+T1). In this embodiment, information DATA(t, t+T1) is data that has a low correlation with command information MASTER(t+T1) output by the first trained model 321. Data with a low correlation is data that has no correlation with command information MASTER(t+T1) or data that has a low correlation. Information DATA(t, t+T1) is, for example, information derived from status information SLAVE(t) and command information MASTER(t+T1).

[0070] The information DATA(t, t+T1) is, for example, information that represents the difference or ratio between the status information SLAVE(t) and the command information MASTER(t+T1). As an example, the information DATA(t, t+T1) is expressed by the following equation (1). By using the difference or ratio between the status information SLAVE(t) and the command information MASTER(t+T1), the correlation between the input and output of the second trained model 322 is reduced.

[0071] DATA(t,t+T1)=SLAVE(t)-MASTER(t+T1)…(1) Fig. 3 is a diagram illustrating an example of the second trained model 322. In the example of Fig. 3, the input information input to the second trained model 322 includes command information MASTER(t+T1) and information DATA(t, t+T1). That is, in the example of Fig. 3, command information (t+T1) and information DATA(t, t+T1) are input to the second trained model 322 as input data.

[0072] The second trained model 322 is composed of, for example, a convolutional layer, a pooling layer, and a combination layer. In the convolutional layer, the input data is convoluted by filtering. The convolutional data is pooled in the pooling layer. This improves the model's ability to recognize changes in the position of features in the data. The pooling data is processed in the combination layer, and is converted into the output data of the second trained model 322, that is, the command information MASTER(t+T2) supplied to the slave device 10 at time (t+T2), and output.

[0073] 4 is a flowchart showing the flow of the trained model generation process performed by the information processing device 30. Note that some steps may be executed in parallel or in a different order. In step S100, the first generation unit 311 generates the first trained model 321 by using training data included in the training data set 323. As described above, the training data used in training the first trained model 321 is a set of state information SLAVE(t) and command information (t+T1).

[0074] In step S101, the second generation unit 312 generates the second trained model 322 by using the training data included in the training data set 323. The training data used in training the second trained model 322 is, as described above, a set of command information (t+T1) and information DATA(t, t+T1), and command information (t+T2).

[0075] The first trained model 321 and the second trained model 322 generated by this embodiment are used hierarchically when operating the controlled device. More specifically, information according to the output of the first trained model 321 is used as input information for the second trained model 322, and command information (t+T2), which is the output of the second trained model 322, is used to control the operation of the controlled device. The operation control of the controlled device will be described in the second embodiment described later.

[0076] Next, a specific example of the configuration of this embodiment will be described with reference to the drawings.

[0077] (Example) Fig. 5 is a diagram showing a schematic diagram of a system for collecting teacher data according to the present embodiment. In Fig. 5, the system 2 is a master-slave system including a master device 20 and a slave device 10, in which the slave device 10 operates according to the operation of the master device 20. A first trained model 321 and a second trained model 322 are trained using command information MASTER(t) and state information SLAVE(t) collected in the system 2 as teacher data.

[0078] The system 2 is a system for collecting command information MASTER(t) and status information SLAVE(t) in the operation of erasing a line using an eraser. The master device 20 includes a base unit 211, a rotating unit 212 placed on the base unit 211, a first arm unit 213 connected to the rotating unit 212, and a second arm unit 214 connected to the first arm unit 213. The rotating unit 212 is rotatable around the Z axis with respect to the base unit 211. The first arm unit 213 has a first end connected to the rotating unit 212 and is swingable around the Y axis with the first end as a fulcrum. As a result, the second end of the first arm unit 213 is swingable in the Z axis direction. The second arm unit 214 has a first end connected to the first arm unit 213 and is swingable around the Y axis with the first end as a fulcrum. As a result, the second end of the second arm unit 214 is able to swing in the X-axis direction, which allows the operator to grasp the second end of the second arm unit 214 and move it in the X-, Y-, and Z-axis directions.

[0079] The slave device 10 operates according to command information MASTER(t) output from the master device 20. The slave device 10 includes a base unit 111, a rotating unit 112 placed on the base unit 111, a first arm unit 113 connected to the rotating unit 112, and a second arm unit 114 connected to the first arm unit 113. The rotating unit 112 is rotatable around the Z axis with respect to the base unit 111. The first arm unit 113 has a first end connected to the rotating unit 112 and is swingable around the Y axis with the first end as a fulcrum. As a result, the second end of the first arm unit 113 is swingable in the Z axis direction. The second arm unit 114 has a first end connected to the first arm unit 113 and is swingable around the Y axis with the first end as a fulcrum. As a result, the second end of the second arm unit 114 is able to swing in the X-axis direction. An eraser E is attached to the second end of the second arm unit 114. The eraser E moves in response to the operator's movement operation on the second end of the second arm unit 214.

[0080] The slave device 10 performs control to change the state of its own device so that the difference between the state represented by the command information MASTER(t) supplied from the master device 20 (e.g., the angle, angular velocity, and torque of the joints of the robot arm) and the state of its own device becomes zero.

[0081] A sheet of paper P on which a line has been drawn is placed under the second arm unit 114, and the height of the sheet of paper P in the Z-axis direction is adjustable. For example, with the height of the sheet of paper P adjusted to 20 mm, the operator first grasps the second end of the second arm unit 214 and moves it down in the Z-axis direction. When the eraser E attached to the second end of the second arm unit 114 comes into contact with the sheet of paper P, the operator moves the second end of the second arm unit 214 in the X-axis and Y-axis directions while the eraser E is still in contact with the sheet of paper P. In this way, the operator erases the line drawn on the sheet of paper P with the eraser E.

[0082] The information processing device 30 collects the time-series command information MASTER(t) and state information SLAVE(t) in the above series of operations, and stores them in the storage unit 32. The information processing device 30 executes the generation process of the first trained model 321 and the second trained model 322 described above, using the collected command information MASTER(t) and state information SLAVE(t).

[0083] The above-described operation of erasing lines with the eraser E may be performed in the same manner when the height of the paper surface P is adjusted to 50 mm and 80 mm.

[0084] [Modifications] In the above embodiment, a configuration has been described in which the master device 20 supplies command information MASTER(t) to the slave device 10. The device that supplies command information MASTER(t) to the slave device 10 is not limited to the master device 20, and may be another device.

[0085] In the above embodiment, a case has been described in which a two-layer trained model including a trained model in an upper layer (first trained model 321) and a trained model in a lower layer (second trained model 322) is generated. The number of layers of the trained model is not limited to two, and may be three or more. Even when the number of layers is three or more, information according to the output of the trained model in the upper layer is used as the input of the trained model in the lower layer, as in the above embodiment.

[0086] That is, the second estimation unit 332 estimates the command information using n (n is a natural number greater than or equal to 2) second trained models 322, and the n second trained models 322 are used hierarchically, with information DATA corresponding to the command information MASTER derived from the second trained models 322 in each layer becoming input information for the second trained models 322 in the layer below the second trained models 322.

[0087] In this case, it is preferable that the sampling period (estimation period) of the trained model in the upper layer is longer than the sampling period (estimation period) of the trained model in the lower layer.

[0088] For example, a three-layer trained model including a first trained model 321 and two second trained models 322a, 322b may be used. In this case, the input of the first trained model 321 is state information SLAVE(t), and the output is command information MASTER(t+T1). The output of the first trained model 321, command information MASTER(t+T1) and information DATA(t, t+T1), become the input of the second trained model 322a. The output of the second trained model 322a is command information MASTER(t+T2a).

[0089] In addition, the command information MASTER(t+T2a) and information DATA(t, t+T2a), which are the outputs of the second trained model 322a, are input to the second trained model 322b, which is in a lower layer than the second trained model 322a. The output of the second trained model 322b is command information MASTER(t+T2b). The command information MASTER(t+T2b), which is the output of the second trained model 322b in the lowest layer, is used to control the operation of the slave device 10.

[0090] In the above embodiment, the system 1 including one slave device 10 and one master device 20 has been exemplified, but the number of slave devices 10 and the number of master devices 20 included in the system 1 may be two or more. In this case, the information processing device 30 collects status information SLAVE(t) and command information MASTER(t) from the multiple slave devices 10 and the multiple master devices 20, and generates a trained model using the collected information as teacher data.

[0091] In the above embodiment, the slave device 10 and the information processing device 30 are separate devices. However, the slave device 10 and the information processing device 30 may be integrated into one device.

[0092] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0093] FIG. 6 is a diagram illustrating a schematic configuration of a system 1B including an information processing device 30B according to a second embodiment of the present invention. The system 1B is a system that operates a control target device 10B using a learned model of multiple layers. The system 1B includes a control target device 10B and an information processing device 30B. The control target device 10B is a device that operates according to command information MASTER(t). The control target device 10B is, for example, a device equipped with a robot arm, for example, the slave device 10 according to the first embodiment. The information processing device 30B operates the control target device 10B by supplying command information MASTER(t) to the control target device 10B.

[0094] The information processing device 30B includes an estimation phase execution unit 33 and a storage unit 32. The estimation phase execution unit 33 includes a first estimation unit 331, a second estimation unit 332, and an operation control unit 333. The first estimation unit 331 estimates command information MASTER(t+T1) from state information SLAVE(t) for each first unit time τ1 using the first learned model 321. The second estimation unit 332 estimates command information MASTER(t+T2) from input data including information DATA(t, t+T1) for each second unit time τ2 using the second learned model 322. The operation control unit 333 operates the control target device 10B using the command information MASTER(t+T2) estimated by the second estimation unit 332. The second unit time τ2 is shorter than the first unit time τ1. The first unit time τ1 is, for example, 400 ms. The second unit time τ2 is, for example, 20 ms.

[0095] 7 is a flowchart showing the flow of the operation control process of the control target device 10B performed by the information processing device 30B. Note that some steps may be executed in parallel or in a different order.

[0096] In step S200, the first estimation unit 331 acquires state information SLAVE(t) from the control target device 10B. In step S201 (an example of the first step), the first estimation unit 331 estimates command information MASTER(t+T1) for each unit time τ1 using the acquired state information SLAVE(t) and the first learned model 321. That is, the first estimation unit 331 estimates command information MASTER(t+T1) derived by inputting the state information SLAVE(t) to the first learned model 321.

[0097] In step S202, the first estimation unit 331 calculates input information for the second trained model 322 using the state information SLAVE(t) input to the first trained model 321 and the command information MASTER(t+T1) estimated in step S200. For example, this is a set of command information MASTER(t+T1) and information DATA(t, t+T1). The information DATA(t, t+T1) is calculated, for example, by the above-mentioned formula (1).

[0098] In step S203 (an example of the second step), the second estimation unit 332 estimates command information MASTER(t, t+T2) for each unit time τ2 from information DATA(t, t+T1) representing the relationship between state information SLAVE(t) and command information MASTER(t+T1) using the second trained model 322. More specifically, for example, the second estimation unit 332 estimates command information MASTER(t+T2) derived by inputting command information MASTER(t+T1) and information DATA(t, t+T1) as input data to the second trained model 322. In this embodiment, since the unit time τ2 is shorter than the unit time τ1, the frequency at which input data is input to the second trained model 322 is higher than the frequency at which command information MASTER(t+T1) is output from the first trained model 321. Therefore, one piece of command information MASTER(t+T1) output from the first trained model 321 may be input to the second trained model 322 multiple times.

[0099] That is, in this embodiment, multiple trained models (first trained model 321 and second trained model 322) are hierarchically organized, and information corresponding to the output of a higher-level trained model is used as input to a lower-level trained model.

[0100] FIG. 8 is a diagram illustrating a hierarchical trained model. In the example of FIG. 8, a first trained model 321 and a second trained model 322 are used hierarchically. The first trained model 321 is a trained model in an upper layer, for example, an LSTM with a sampling period of 400 msec. The second trained model 322 is a trained model in a lower layer, for example, an LSTM with a sampling period of 20 msec. That is, in this embodiment, the sampling period of the trained model in the upper layer is longer than the sampling period of the trained model in the lower layer.

[0101] In the example of Figure 8, information DATA(t, t+T1) corresponding to output command information MASTER(t+T1) of the first trained model 321 in the upper layer is used as input to the second trained model 322 in the lower layer.

[0102] In Figure 8, state information SLAVE(t), which is input information of the first trained model 321, includes, for example, nine parameters of the joint angles (θs1(t), θs2(t), θs3(t)) of the robot arm at time t, the angular velocities of the joints (θs1'(t), θs2'(t), θs3'(t)), and the torques (τs1(t), τs2(t), τs3(t)). In addition, the command information MASTER(t+T1), which is the output information of the first trained model 321, includes, for example, nine parameters, namely, the joint angle of the robot arm at time t (θm1(t+T1), θm2(t+T1), θm3(t+T1)), the joint angular velocity (θm1'(t+T1), θm2'(t+T1), θm3'(t+T1)), and the torque (τm1(t+T1), τm2(t+T1), τm3(t+T1)).

[0103] On the other hand, in Figure 8, the input information of the second trained model 322 includes, for example, command information MASTER(t+T1) including nine parameters, and information DATA(t, t+T1) including nine parameters, totaling 18 parameters.

[0104] Returning to the description of FIG. 7, in step S204 (an example of the third step), the operation control unit 333 operates the control target device 10B at time (t+T2) using the command information MASTER(t+T2) estimated in step S202. As described above, the command information MASTER(t+T2) includes, for example, the angle of the joint, the angular velocity of the joint, and the torque of the robot arm at time (t+T2). The control target device 10B operates the robot arm according to the command information MASTER(t+T2) supplied from the information processing device 30B at time (t+T2). For example, the control target device 10B performs control to change the state of the own device so that the difference between the state represented by the command information MASTER(t) supplied from the information processing device 30B (for example, the angle, angular velocity, and torque of the joint of the robot arm) and the state of the own device becomes zero.

[0105] In step S205, the operation control unit 333 judges whether to end the operation control. This judgment may be made, for example, by judging whether or not an operation to end the operation control has been performed by the operator. If the operation control is to be ended (step S205; YES), the operation control unit 333 ends the processing. On the other hand, if the operation control is not to be ended (step S205; NO), the operation control unit 333 returns to the processing of step S200, and continues the estimation processing of the command information MASTER(t+T2) using the state information SLAVE(t) and the operation control processing of the control target device 10B.

[0106] However, if we try to estimate the command information MASTER(t) to be supplied to the controlled device 10B using a single trained model, we need to deepen the hierarchy of the trained model in order to stabilize the operation prediction. This requires a huge amount of training data and requires a huge amount of calculation time.

[0107] In contrast, in this embodiment, the information processing device 30B operates the control target device 10B using a first trained model 321 that derives command information MASTER(t+T1) from state information SLAVE(t) and a second trained model 322 that derives command information (t+T2) from information corresponding to the output of the first trained model. That is, in this embodiment, a plurality of trained models are hierarchically arranged, and the output of a trained model (first trained model 321) in a higher layer is input to a trained model (second trained model 322) in a lower layer. By using these multiple trained models hierarchically in the operation control of the control target device 10B, it is possible to stabilize the operation prediction of the control target device 10B without requiring a huge amount of teacher data.

[0108] In addition, when detecting the state of a robot using multiple sensors with different sensing cycles, it may be difficult to merge the detection values ​​of the multiple sensors as inputs to a single trained model. In contrast, according to this embodiment, since the trained models are hierarchical, the trained models of each layer can be designed independently. Therefore, even when multiple sensors with different sensing cycles are used, trained models corresponding to each sensor can be designed individually, making it easy to generate trained models.

[0109] [Modifications] In the above embodiment, the case has been described where the control target device 10B that is the control target is the slave device 10. However, the control target device 10B may be another device different from the slave device 10.

[0110] Furthermore, in the above embodiment, the configuration has been described in which the control target device 10B and the information processing device 30B are separate entities, but the control target device 10B and the information processing device 30B may be configured as an integrated device.

[0111] [Embodiment 3] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0112] 9 is a diagram illustrating a schematic configuration of a system 1C including an information processing device 30C according to a third embodiment of the present invention. The system 1C includes a slave device 10, a master device 20, and an information processing device 30C. The information processing device 30C generates a trained model for operating the slave device 10, and operates the slave device 10 using the trained model.

[0113] The information processing device 30C includes a learning phase executing unit 31, a storage unit 32, and an estimation phase executing unit 33. Each of these components has been described in the above-mentioned first or second embodiment, and therefore description thereof will be omitted.

[0114] According to this embodiment, the information processing device 30C generates a first trained model 321 that derives command information MASTER(t+T1) from state information SLAVE(t) and a second trained model 322 that derives command information (t+T2) from information DATA(t, t+T1) corresponding to the output of the first trained model, and operates the slave device 10 using the generated first trained model 321 and second trained model 322. That is, in this embodiment, a plurality of trained models are hierarchically arranged, and the output of the trained model (first trained model 321) in the upper layer is input to the trained model (second trained model 322) in the lower layer. These multiple trained models are used hierarchically in the operation control of the slave device 10, so that the operation prediction of the slave device 10 can be stabilized without requiring a huge amount of teacher data.

[0115] [Embodiment 4] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0116] 10 is a diagram illustrating a schematic configuration of a system 1D including an information processing device 30D according to a fourth embodiment of the present invention. The system 1D includes a slave device 10, a master device 20, an information processing device 30D, and an input device 40. The information processing device 30D executes a learning phase in which a trained model for operating the slave device 10 is generated, and also executes an estimation phase in which the slave device 10 is operated using the trained model.

[0117] The input device 40 inputs information including an operation speed (hereinafter referred to as "speed information") that defines the operation of the slave device 10 in the learning phase to the information processing device 30D. In this embodiment, the operation speed of the slave device 10 is variable. The input device 40 is, for example, a keyboard, a mouse, or a touch panel operated by an operator. Furthermore, the input device 40 may be a device that transmits the speed information to the information processing device 30D via a communication network such as the Internet.

[0118] In this embodiment, the input data input to the first trained model 321D includes speed information v(t) at time t in addition to state information SLAVE(t) at time t. That is, the first trained model 321D is a trained model that derives command information (t+T1) from input data including state information SLAVE(t) and speed information v(t).

[0119] For example, when the operation of the slave device 10 is a periodic operation, the speed information v(t) may be a frequency that represents the operation period of the slave device 10. The periodic operation may be, for example, an operation in which the robot arm of the slave device 10 repeatedly draws a line using a writing implement, or a periodic cleaning operation performed using a cleaning tool. Note that the speed information v(t) is not limited to the frequency of the operation period, and may be other information. The speed information may be, for example, the execution time of one operation, which is information that is highly correlated with the frequency, a target value of the slave device 10, an object to be operated, or conversation information with a human being.

[0120] In the teacher data collection phase, the operator of the master device 20 operates the master device 20 while changing the operation speed of the master device 20. As a result, the slave device 10 operates while changing its own operation speed in response to the command information MASTER(t) supplied from the master device 20. That is, in this operation example, the command information MASTER(t), status information SLAVE(t), and speed information v(t) are collected by operating the slave device 10 while changing the operation speed of the slave device 10.

[0121] In the learning phase, the first generation unit 311D generates the first trained model 321D using the collected command information MASTER(t), state information SLAVE(t), and speed information v(t). The teacher data used to generate the first trained model 321D is a set of the state information SLAVE(t) and speed information v(t) and the command information MASTER(t+T1).

[0122] In the estimation phase, the first estimation unit 331 estimates command information MASTER(t+T1) derived by inputting the state information SLAVE(t) and the speed information v(t) as input data to the first trained model 321D. Similarly to the above-mentioned third embodiment, the second estimation unit 332 estimates command information (t+T2) derived by inputting the command information MASTER(t+T1) and information DATA(t, t+T1) corresponding to the command information MASTER(t+T1) as input data to the second trained model 322. The operation control unit 333 supplies the estimated command information (t+T2) to the slave unit 10 at time (t+T2) to operate the slave unit 10.

[0123] Incidentally, in order to make the movement speed of the slave device 10 different from the movement speed of the slave device 10 when the teacher data was collected, it is possible to use the teacher data by shortening or expanding it in the time axis direction. However, when the teacher data is compressed or expanded in the time axis direction, the movement of the slave device 10 differs from the movement when the teacher data was collected, making it difficult to reproduce the movement. This is because the movement of the robot arm of the slave device 10 is affected by inertial force and friction.

[0124] Fig. 11 and Fig. 12 are diagrams illustrating graphs showing temporal changes in state values ​​of a robot arm controlled by a conventional method. Fig. 11 is a diagram illustrating a graph showing teacher data and actual state values ​​(state information SLAVE(t) output from the slave device 10) in a case where teacher data is shortened in the time axis direction and the slave device 10 is operated in a conventional information processing device. Fig. 12 is a diagram illustrating a graph showing teacher data and actual state values ​​(state information SLAVE(t) output from the slave device 10) in a case where teacher data is expanded in the time axis direction and the slave device 10 is operated in a conventional information processing device.

[0125] 11 and 12, the horizontal axis indicates time, and the vertical axis indicates the angle of the robot arm in a predetermined axial direction. The dashed line indicates the response to the teacher data, and the solid line indicates the response during execution. In the example of FIG. 11, it can be seen that when the motion speed of the slave device 10 is made faster than the motion speed when the teacher data was collected, the amplitude of the periodic motion of the slave device 10 becomes smaller. Also, in the example of FIG. 12, it can be seen that when the motion speed of the slave device 10 is made slower than the motion speed when the teacher data was collected, the amplitude of the periodic motion of the slave device 10 becomes larger.

[0126] FIG. 13 is a diagram showing the change over time in the state value of the robot arm when the slave device 10 is operated by the information processing device 30D according to this embodiment. In the diagram, the horizontal axis indicates time, and the vertical axis indicates the angle of the joint in a predetermined axial direction of the robot arm of the slave device 10. In the diagram, a graph f11 is a graph showing the relationship between time and angular velocity represented by the teacher data stored in the storage unit 32. A graph f12 is a graph showing the actual state of the slave device 10 (angle of the robot arm) when the teacher data is supplied to the slave device 10. It is clearly shown that there is a difference (error) between the graph f13 predicting the operation of the master device 20 and the graph f12 predicting the operation of the slave device 10 from the graph f11 showing the original teacher data.

[0127] Graph f13 is a graph showing the change over time in the state value when the slave device 10 is operated by the information processing device 30D according to this embodiment. The amplitude of the waveform of graph f12 is smaller than that of graph f11, whereas the amplitude of the waveform of graph f13 is approximately the same as that of graph f11. According to the information processing device 30D according to this embodiment, it is possible to reduce the difference between the operation of the slave device 10 indicated by the teacher data and the actual operation. That is, according to this embodiment, it is possible to reproduce the operation of the slave device 10 indicated by the teacher data at different operation speeds.

[0128] As described above, according to this embodiment, the information processing device 30D estimates the command information MASTER(t+T1) using the first trained model 321D trained using teacher data including speed information v(t) indicating the motion speed of the slave device 10. This makes it possible to stabilize the motion prediction of the slave device 10, whose motion speed is variable.

[0129] [Modifications] In the above-described fourth embodiment, a case has been described in which the speed information of the slave device 10 is input to the information processing device 30D by the input device 40. The speed information may be information obtained by the slave device 10 measuring the operating speed of the own device. In this case, the slave device 10 outputs status information SLAVE(t) of the own device and also outputs speed information v(t) representing the operating speed of the own device. The information processing device 30D trains the first trained model 321D using the status information SLAVE(t) and speed information v(t) output from the slave device 10 as teacher data.

[0130] According to this embodiment, the slave device 10 can autonomously generate new teaching data, so that the amount of teaching data taught by humans can be significantly reduced.

[0131] [Embodiment 5] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0132] 14 is a diagram illustrating a schematic configuration of a system 1E according to embodiment 5. The system 1E includes a slave device 10, a master device 20, an information processing device 30E, and an input device 40. The information processing device 30E differs from the information processing device 30D according to embodiment 4 described above in that the information processing device 30E does not include the second generation unit 312, the second trained model 322, and the second estimation unit 332.

[0133] In the above-described embodiment 4, the information processing device 30D controls the operation of the slave device 10 using multiple hierarchical trained models (the first trained model 321D and the second trained model 322) in the same manner as in the above-described embodiment 3. In contrast, in the present embodiment, the information processing device 30E controls the operation of the slave device 10 using one first trained model 321D, not multiple hierarchical trained models. That is, in the present embodiment, command information (t+T1) which is the output of the first trained model 321D is used to control the operation of the slave device 10, instead of command information (t+T2) which is the output of the second trained model 322.

[0134] The operation control unit 333 operates the slave device 10 by supplying the command information MASTER(t+T1) estimated by the first estimation unit 331 to the slave device 10 at time (t+T1).

[0135] The functions implemented by the information processing device 30E according to the above embodiment may be shared and implemented by a plurality of devices. For example, the information processing device 30E may be realized by a system including a first device including the learning phase executing unit 31, a second device including the estimation phase executing unit 33, and a third device including the storage unit 32.

[0136] The information processing device 30E according to this embodiment can be identified as a device including: an estimation unit that estimates command information using a learned model that derives the command information to be supplied to the control target device after the control target device outputs state information from input data including information representing the state of the control target device output by the control target device operating in accordance with command information and information including an operating speed that specifies the operation to be instructed to the control target device; and an operation control unit that operates the control target device using the command information estimated by the estimation unit.

[0137] Furthermore, the information processing device 30E according to this modified example can be identified as a device including a generation unit that generates a learned model that learns the correlation between the input information and the command information, using a set of input information including information output by a controlled device operating according to command information and information including an operating speed that specifies an operation to be instructed to the controlled device, and the command information that is supplied to the controlled device after the controlled device outputs the information representing the state.

[0138] [Embodiment 6] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0139] 15 is a diagram illustrating a schematic configuration of a system 1F according to a sixth embodiment of the present invention. The system 1F includes a slave device 10, a master device 20, and an information processing device 30F. The information processing device 30F differs from the information processing device 30C of the third embodiment in that the information processing device 30F includes a first generation unit 311F instead of the first generation unit 311, and in that the information processing device 30F includes a third trained model 324.

[0140] The third trained model 324 is a trained model that uses command information MASTER(t) as input data and derives state information SLAVE(t+T3). The state information SLAVE(t+T3) derived by the third trained model 324 is used as input information for the first trained model 321F. The third trained model 324 can be realized by, for example, CNN, RNN, LSTM, DNN, or a combination thereof.

[0141] FIG. 16 is a diagram illustrating a relationship between the first trained model 321F and the third trained model 324. In the diagram, when command information MASTER(t) is input as input data to the third trained model 324, the third trained model 324 derives state information SLAVE(t+T3). The first generation unit 311F trains the first trained model 321F using a pair of state information SLAVE(t+T3) derived by the third trained model 324 and command information MASTER(t+T1+T3) as teacher data. The state information SLAVE(t+T3) derived by the third trained model 324 is an example of information representing a second state according to this embodiment (hereinafter referred to as "second state information").

[0142] In the learning phase, the first generation unit 311F generates a third trained model 324 by training the teacher data (a set of command information MASTER(t) and state information SLAVE(t+T3)) stored in the storage unit 32F. The first generation unit 311F also generates a first trained model 321F by training the teacher data (a set of state information SLAVE(t+T3) and command information MASTER(t+T1+T3)) stored in the storage unit 32F. At this time, the state information SLAVE(t+T3) used in generating the first trained model 321F is not the actual state value (state information SLAVE(t+T3)) output from the slave device 10, but the state information SLAVE(t+T3) derived by the third trained model 324.

[0143] It is also possible to link the first trained model 321F and the third trained model 324 and train both models simultaneously. In that case, the following five methods A to E are possible.

[0144] A. The state information SLAVE(t) is made to output the state information SLAVE(t+T1+T3). In other words, the state information SLAVE(t) → the first trained model 321F → the third trained model 324 → the state information SLAVE(t+T1+T3).

[0145] B. The state information SLAVE(t+T1+T3) is output from the command information MASTER(t). That is, the command information MASTER(t) → the third trained model 324 → the first trained model 321F → the state information SLAVE(t+T1+T3).

[0146] C. The above A and B are repeated multiple times (n times). For example, command information MASTER(t) → third trained model 324 → first trained model 321F → ... → third trained model 324 → first trained model 321F → command information MASTER(t+nT1+nT3) or status information SLAVE(t) → first trained model 321F → third trained model 324 → ... → first trained model 321F → third trained model 324 → status information SLAVE(t+nT1+nT3).

[0147] D. Repeat A above n times, and omit the step of predicting the status information SLAVE from the final command information MASTER. In other words, start from the status information SLAVE(t) and predict the command information MASTER(t+nT1+(n-1)T3).

[0148] E. Repeat step B above n times, and omit the step of predicting command information MASTER from the final state information SLAVE. In other words, starting from command information MASTER(t), predict state information SLAVE(t+nT3+(n-1)T1).

[0149] FIG. 17 is a diagram illustrating the relationship between the first trained model 321F and the third trained model 324 when T1=T3. In the diagram, an arrow d11 indicates the relationship between the input and output of the first trained model 321F. An arrow d12 indicates the relationship between the input and output of the third trained model 324. In the example of FIG. 17, the first trained model 321F derives command information MASTER(t+T1) from state information SLAVE(t). Furthermore, the third trained model 324 derives state information (t+2T1) from command information MASTER(t+T1). In the absence of the third trained model 324, the first trained model 321F is an open-loop model that predicts the next step. There is no relationship between the response of the first trained model 321F that predicts command information from state information (t) and the response of the first trained model 321F that predicts command information from state information (t+2T1). On the other hand, when the third trained model 324 is involved, the output of the first trained model 321F, which predicts command information from state information (t), propagates to state information (t+2T1), resulting in a closed-loop model in which the response of the first trained model 321F at state information (t) affects the response of the first trained model 321F at time (t+T2).

[0150] That is, in this embodiment, in the learning phase of the first trained model 321F, the output of the third trained model 324 is used as the input of the first trained model 321F.

[0151] In the estimation phase, the third trained model 324 is not used, and the first trained model 321F (and the second trained model 322) is used to estimate the command information. The estimation process performed by the first estimation unit 331 and the second estimation unit 332 is similar to the process described in the above-mentioned second embodiment, and detailed description thereof will be omitted here.

[0152] According to this embodiment, the information processing device 30F performs learning using the state information SLAVE(t+T3) derived by the third trained model 324 as input data for the first trained model 321F, thereby generating the first trained model 321F. This process corresponds to modeling the learning as a closed loop in the learning phase. On the other hand, in the estimation phase, the information processing device 30F forms a closed loop to control the operation of the slave device 10 using the first trained model 321F. Therefore, in the learning phase, learning can be performed as the same closed loop model as in the estimation phase, so that the operation prediction of the controlled device can be stabilized.

[0153] [Embodiment 7] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0154] 18 is a diagram illustrating a schematic configuration of a system 1G according to embodiment 7. The system 1G includes a slave device 10, a master device 20, and an information processing device 30G. The information processing device 30G differs from the information processing device 30F according to embodiment 6 in that the information processing device 30G does not include the second generation unit 312, the second trained model 322, and the second estimation unit 332.

[0155] In the above-described embodiment 6, the information processing device 30F controls the operation of the slave device 10 using multiple hierarchical learned models (the first learned model 321F and the second learned model 322) in the same manner as in the above-described embodiment 3. In contrast, in the present embodiment, the information processing device 30G controls the operation of the slave device 10 using the first learned model 321F and the third learned model 324, rather than multiple hierarchical learned models. That is, in the present embodiment, the command information MASTER(t+T1), which is the output of the first learned model 321F, is used to control the operation of the slave device 10, rather than the command information MASTER(t+T2), which is the output of the second learned model 322.

[0156] The operation control unit 333 operates the slave device 10 by supplying the command information MASTER(t+T1) estimated by the first estimation unit 331 to the slave device 10 at time (t+T1).

[0157] The functions implemented by the information processing device 30G according to the above embodiment may be shared and implemented by a plurality of devices. For example, the information processing device 30G may be realized by a system including a first device including a learning phase executing unit 31, a second device including an estimation phase executing unit 33, and a third device including a storage unit 32.

[0158] The information processing device 30G according to this embodiment includes an estimation unit that estimates command information using a first learned model that derives command information from information representing a state of a controlled device output by a controlled device operating in accordance with command information, and that is supplied to the controlled device after the controlled device outputs information representing the state, and an operation control unit that operates the controlled device using the command information estimated by the estimation unit, and the first learned model can be identified as a learned model that has been trained to learn, as information representing the state, information representing a second state derived by a third learned model that derives, from the command information, information representing a second state output by the controlled device after the command information is supplied to the controlled device.

[0159] In addition, the information processing device 30G according to this modification includes a first generation unit that generates a first trained model in which a correlation between information representing the first state and command information is learned using a set of information representing the first state of a control target device that operates according to command information and the command information supplied to the control target device after the control target device outputs the information representing the first state, and a third generation unit that generates a third trained model in which a correlation between the command information and information representing the second state is learned using a set of information representing the second state of the control target device that is output by the control target device after the command information is supplied to the control target device, and the first generation unit uses the second state information derived in the third trained model as the state information for training the first trained model. The first generation unit 311F according to this embodiment is an example of the third generation unit.

[0160] [Embodiment 8] Other embodiments of the present invention will be described below. For convenience of explanation, the same reference numerals are given to members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0161] 19 is a diagram illustrating a schematic configuration of a system 1H including an information processing device 30H according to an eighth embodiment of the present invention. The system 1H includes a slave device 10, a master device 20, an information processing device 30H, and a hardness / softness sensor 50. The information processing device 30H executes a learning phase in which a trained model for operating the slave device 10 is generated, and also executes an estimation phase in which the slave device 10 is operated using the trained model.

[0162] The estimation phase execution unit 33H of the information processing device 30H includes a change control unit 334 in addition to a first estimation unit 331, a second estimation unit 332, and an operation control unit 333. The change control unit 334 changes one or both of the unit times τ1 and τ2 depending on the degree of hardness of an object that is the target of an operation performed by the slave device 10. Examples of the object include an object that is the target of a gripping operation, foodstuffs that are the target of a cooking operation, a writing implement and writing paper that are the target of a drawing operation, and an object that is the target of a polishing operation.

[0163] In addition, for control and operation of interactions with an environment having a low operating frequency, such as soft objects, the first generation unit with a long unit time can mainly estimate the movements, and for control and operation of interactions with an environment having a high operating frequency, such as hard objects, the first generation unit with a short unit time can mainly estimate the movements.

[0164] The hardness sensor 50 is provided in the slave device 10, detects the degree of hardness of an object, and outputs a detected signal to the information processing device 30H.

[0165] The change control unit 334 changes one or both of the unit time τ1 and the unit time τ2 according to the degree of hardness of the object detected by the hardness sensor 50.

[0166] For example, the change control unit 334 may perform control such that the harder the object is, the shorter the unit times τ1 and τ2 are set, and the softer the object is, the longer the unit times τ1 and τ2 are set.

[0167] [Modifications] The functions implemented by the information processing device 30H according to the above embodiment may be shared and implemented by a plurality of devices. For example, the information processing device 30J may be realized by a system including a first device including the learning phase executing unit 31, a second device including the estimation phase executing unit 33H, and a third device including the storage unit 32.

[0168] [Embodiment 9] Another embodiment of the present invention will now be described. For convenience of explanation, the same reference numerals will be used to designate members having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0169] 20 is a diagram illustrating a schematic configuration of a system 1J including an information processing device 30J according to a ninth embodiment of the present invention. The system 1J includes a slave device 10, a master device 20, and an information processing device 30J. The information processing device 30J differs from the seventh embodiment in that the information processing device 30J includes a first learned model 321J, a first generation unit 311J, and a first estimation unit 331J instead of the first learned model 321F, the third learned model 324, the first generation unit 311F, and the first estimation unit 331.

[0170] 21 is a diagram illustrating an example of the contents of the first trained model 321J. The first trained model 321J is a trained model that receives state information SLAVE(k) as an input and derives state information SLAVE(k+n) and command information SLAVE(k+m).

[0171] In the learning phase, the first generation unit 311J trains the first trained model 321J using the state information SLAVE(k) and a set of the state information SLAVE(k+n) and command information MASTER(k+m) as teacher data.

[0172] In addition, the first generation unit 311J trains the first trained model 321J using state information SLAVE(k+n) which is the output of the first trained model 321J as teacher data. That is, the state information SLAVE(k+n) derived by the first trained model 321J is used as input information of the first trained model 321J.

[0173] Fig. 22 is a diagram for explaining the first trained model 321J. In the figure, the arrow d21 indicates the relationship between the input and output of the first trained model 321J. In the example of Fig. 22, as described above, the state information SLAVE(t+n) and the command information MASTER(k+m) are derived from the state information SLAVE(k).

[0174] In the estimation phase, the first estimation unit 331J estimates state information SLAVE(k+n) and command information MASTER(k+m) from state information SLAVE(k) every unit time τ1 using the first trained model 321J.

[0175] The operation control unit 333 uses the command information MASTER(k+m) estimated by the first estimation unit 331 to operate the control target device (slave device 10).

[0176] In this embodiment, a feature common to the slave device 10 and the master device 20 can be extracted by simultaneously executing the motion prediction of the slave device 10 and the motion prediction of the master device 20. Also, in this embodiment, it is sufficient to train one first trained model 321J, and training is easier than training a plurality of trained models.

[0177] The information processing device 30J according to this embodiment includes an estimation unit that estimates command information using a first learned model that derives command information to be supplied to a control target device from information representing a state of the control target device output by the control target device operating in accordance with command information, and an operation control unit that operates the control target device using the command information estimated by the estimation unit, and the first learned model can be identified as a learned model that derives information representing a second state of the control target device after a predetermined time has elapsed and the command information from information representing the state, and is learned using the information representing the second state derived by the first learned model as input information.

[0178] [Modifications] [Variation 1] The first trained model 321J is not limited to the configuration described in the above-mentioned embodiment 9, and may have another configuration.

[0179] 23 is a diagram illustrating the contents of the first trained model 321K according to this modified example. The first trained model 321K is a trained model that receives state information SLAVE(k) and command information MASTER(k) as input and derives state information SLAVE(k+n) and command information SLAVE(k+m).

[0180] In the learning phase, the first generation unit 311J trains the first trained model 321K using a pair of status information SLAVE(k) and command information SLAVE(k) and a pair of status information SLAVE(k+n) and command information MASTER(k+m) as teacher data.

[0181] In addition, the first generation unit 311J trains the first trained model 321J using the state information SLAVE(k+n) and command information MASTER(k+m) that are outputs of the first trained model 321J as teacher data. That is, the state information SLAVE(k+n) and command information MASTER(k+m) derived by the first trained model 321J are used as input information for the first trained model 321J.

[0182] In the estimation phase, the first estimation unit 331J uses the first learned model 321J to estimate state information SLAVE(k+n) and command information MASTER(k+m) from state information SLAVE(k) and command information SLAVE(k) every unit time τ1.

[0183] The operation control unit 333 operates the controlled device (slave device 10) using the command information MASTER(k+m) estimated by the first estimation unit 331. In addition, the prediction error of MASTER(k+m) can be reduced by feeding back information relating to the difference between the estimated SLAVE(k+n) and the actually measured SLAVE(k+n), such as a difference or ratio, to the predicted value of MASTER(k+m).

[0184] In the information processing device 30J of this modified example, the first learned model can be identified as a learned model that derives information representing a second state of the control target device after a predetermined time has elapsed and the command information from the command information and information representing the state, and is a learned model that has been trained using the command information and the information representing the second state derived by the first learned model as input information.

[0185] [Modification 2] The functions implemented by the information processing device 30J according to the above embodiment may be shared and implemented by a plurality of devices. For example, the information processing device 30J may be realized by a system including a first device including a learning phase executing unit 31J, a second device including an estimation phase executing unit 33J, and a third device including a storage unit 32J.

[0186] [Modification 3] In the system 1J according to the ninth embodiment, the second generation unit 312, the second trained model 322, and the second estimation unit 332 according to the sixth embodiment described above may be included in the system.

[0187] 24 is a diagram illustrating a schematic configuration of a system 1H according to this modification. An information processing device 30H of the system 1H differs from the information processing device 30J according to the above-described ninth embodiment in that the information processing device 30H includes a second generation unit 312, a second trained model 322, and a second estimation unit 332.

[0188] In this modification, the information processing device 30H uses multiple hierarchical trained models (the first trained model 321J and the second trained model 322) to control the operation of the slave device 10, similarly to the above-described embodiment 3. That is, in this modification, command information (t+T2) which is the output of the second trained model 322 is used to control the operation of the slave device 10, instead of command information MASTER(k+m) which is the output of the first trained model 321J.

[0189] In this modification, the command information MASTER(k+m) estimated by the first estimation unit 331 and DATA(k+m) calculated based on the command information MASTER(k+m) are used as inputs to the second trained model 322, and the command information MASTER(k+T2) is derived by the second trained model 322. The operation control unit 333 controls the operation of the slave device 10 using the command information MASTER(k+T2) derived by the second trained model 322.

[0190] [Embodiment 10] FIG. 25 is a diagram illustrating a schematic configuration of a system 1L including an information processing device 30L according to the tenth embodiment of the present invention. The system 1L includes a slave device 10, a master device 20, and an information processing device 30L. The information processing device 30L is a modified example of the information processing device 30C described with reference to FIG. 3. The information processing device 30L will be described with reference to FIG. 25. The configuration described in the above-mentioned third embodiment may be denoted by the same reference numerals and description thereof may be omitted.

[0191] The first generation unit 311 generates the first trained model 321 based on the state information SLAVE(t) of the slave device 10 at time t, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1. Note that the first generation unit 311 may generate the first trained model 321 based on a teacher dataset 323.

[0192] The second generation unit 312 generates the second trained model 322 based on the state information SLAVE(t) of the slave device 10 at time t, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1. Note that the first generation unit 311 may generate the first trained model 321 based on the teacher dataset 323.

[0193] The first estimation unit 331 estimates command information MASTER(t+T1) based on the learned first trained model 321 and state information SLAVE(t) of the slave device 10 at time t.

[0194] The second estimation unit 332 estimates command information MASTER(t+T1) based on the learned second trained model 322 and state information SLAVE(t) of the slave device 10 at time t.

[0195] FIG. 26 is a diagram for explaining the functional configuration of a system 1L including an information processing device 30L according to the tenth embodiment of the present invention. The functional configuration of the information processing device 30L will be explained with reference to the same figure. The information processing device 30L includes a motion generator 610-1, a motion generator 610-2, and a motion generator 610-3. In the following explanation, when the motion generators 610-1, 610-2, and 610-3 are not distinguished from each other, they may be simply referred to as motion generators 610. Also, the motion generator 610-1 may be described as a higher layer relative to the motion generator 610-3, and the motion generator 610-3 may be described as a lower layer relative to the motion generator 610-1.

[0196] The action generator 610-1 includes a first generating unit 311 and a first estimating unit 331. The action generator 610-1 generates a first trained model 321 based on the state information SLAVE(t) of the slave device 10, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1. When there is no need to distinguish between the state information SLAVE(t) of the slave device 10 input to the action generator 610-1, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1, they may be referred to as an input signal IN1. The behavior generator 610-1 estimates command information MASTER(t+T1) based on the learned first trained model 321 and state information SLAVE(t) of the slave device 10 at time t, and outputs it to the slave device 20. The behavior generator 610-1 also generates an interrupt INT1 to the lower layer behavior generator 610-2 based on the learned first trained model 321 and state information SLAVE(t) of the slave device 10 at time t. The signal output by the behavior generator 610-1 may be referred to as an output signal OUT1.

[0197] The action generator 610-2 includes a second generator 312-1 and a second estimator 332-2. The action generator 610-2 generates a second trained model 322-1 based on the state information SLAVE(t) of the slave device 10, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1. When the state information SLAVE(t) of the slave device 10 input to the action generator 610-2, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1 are not distinguished from each other, they may be referred to as an input signal IN2. The behavior generator 610-2 estimates command information MASTER(t+T1) based on the trained second trained model 322-1 and state information SLAVE(t) of the slave device 10 at time t, and outputs it to the slave device 20. The behavior generator 610-2 also generates an interrupt INT2 to the lower layer behavior generator 610-3 based on the trained second trained model 322 and state information SLAVE(t) of the slave device 10 at time t. The signal output by the behavior generator 610-2 may be referred to as an output signal OUT2.

[0198] The action generator 610-3 includes a second generator 312-2 and a second estimator 332-2. The action generator 610-3 generates a second trained model 322-2 based on the state information SLAVE(t) of the slave device 10, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1. When the state information SLAVE(t) of the slave device 10 input to the action generator 610-3, the command information MASTER(t) of the master device 20 at time t, and the command information MASTER(t+1) of the master device 20 at time t+1 are not distinguished from each other, they may be referred to as an input signal IN3. The action generator 610-3 estimates command information MASTER(t+T1) based on the learned second trained model 322-2 and state information SLAVE(t) of the slave device 10 at time t, and outputs the command information MASTER(t+T1) to the slave device 20. The signal output by the action generator 610-3 may be referred to as an output signal OUT3.

[0199] 27 is a schematic diagram for explaining the hierarchical action generator 610 according to the tenth embodiment of the present invention in more detail. With reference to the drawing, the hierarchical action generator 610 according to the tenth embodiment will be explained in more detail.

[0200] The motion generator 610-1 performs, for example, self-location estimation. For example, self-location estimation may be estimating the position of a writing implement held by the slave device 20 in a case where the slave device 20 writes a character with the writing implement. Since the motion generator 610-1 is the motion generator 610 in the highest layer, it performs estimation of motions that only require a certain degree of accuracy due to a slower frequency compared to the motion generators 610 in the lower layers.

[0201] The motion generator 610-2 performs, for example, image recognition. For example, in the case where the slave device 20 writes characters with a writing implement, the image recognition may be image recognition of characters written on a paper and the paper on which the characters are to be written. The motion generator 610-2 is located between the motion generator 610 of the higher layer and the motion generator 610 of the lower layer, and therefore performs estimation of motions that require a frequency faster than the motion generator 610 of the higher layer and a frequency slower than the motion generator 610 of the lower layer.

[0202] The motion generator 610-3 performs, for example, real-time control. For example, in the case where the slave device 20 writes a character using a writing implement, the real-time control may be estimation of a motion that requires real-time control, such as the force with which the writing implement is gripped and the speed at which the writing implement is moved. As the motion generator 610-3 is the lowest layer motion generator, it performs estimation of motions that require higher accuracy at a faster frequency than the motion generators 610 in higher layers.

[0203] That is, the operating frequency of the motion generator 610-1 is slower than that of the motion generator 610-2, which is slower than that of the motion generator 610-3. Also, the motion generator 610-1 controls more complex motions than the motion generator 610-2, which in turn controls more complex motions than the motion generator 610-3. That is, according to this embodiment, the motion generator 610 in the upper layer controls slower and more complex motions than the motion generator 610 in the lower layer, and the motion generator 610 in the lower layer controls faster and simpler motions than the motion generator 610 in the upper layer.

[0204] According to this embodiment, by generating an interrupt from the motion generator 610 in the upper layer to the motion generator 610 in the lower layer, each motion generator 610 can be individually trained according to each motion. For example, when controlling a slow and complex motion such as self-location estimation, the motion generator 610 trains at a low frequency. Also, when controlling a fast and simple motion such as real-time control, the motion generator 610 trains at a high frequency. Here, when learning a slow and complex motion, if learning at a high frequency, a huge amount of memory capacity may be consumed, and when learning a fast and simple motion, if learning at a low frequency, an error may become large due to insufficient sampling frequency. In this embodiment, by hierarchizing the trained model according to the motion, each motion can be trained at a frequency suitable for the motion, so that no unnecessary memory capacity is required.

[0205] 28 is a diagram showing a schematic example of a case where the slave device according to the tenth embodiment of the present invention writes characters with a writing implement. As a specific example for explaining the effect of this embodiment, an example of a case where the system 2 causes the slave device 10 to write characters with a writing implement W will be described. The slave device 10 in FIG. 28 differs from the slave device 10 shown in FIG. 5 in that it includes a writing implement W instead of an eraser E. The slave device 10 writes characters on a sheet of paper P with the writing implement W.

[0206] Figure 29 is a diagram showing an example of a character written on a sheet of paper by a slave unit according to embodiment 10 of the present invention. The slave unit 10 in this embodiment may write, for example, the character C as shown in Figure 29(A), or the character string CS as shown in Figure 29(B). In the example shown in the figure, the direction in which the character string CS is arranged may be referred to as the x-axis direction, and the direction perpendicular to the x-axis direction may be referred to as the y-direction.

[0207] In an example shown in FIG. 29(A), the slave device 10 writes the alphabet "A" as the character C. In an example shown in FIG. 29(B), the slave device 10 writes the alphabet "ABC" as the character string CS. When writing the character string "ABC", it is necessary to write the character C and then move the writing implement W to a position where the next character C is to be written. In an example shown in FIG. 29(B), the slave device 10 writes the alphabet "A" in a range L1 on the x-axis, then moves the writing implement W by a distance L2 in the x-axis direction, and writes the next alphabet "B". In addition, the slave device 10 writes the alphabet "B" in a range L3, then moves the writing implement W by a distance L4 in the x-axis direction, and writes the next alphabet "C".

[0208] Here, when writing characters in ranges L1, L2, or L3, a higher operating frequency is preferable. However, when moving distance L2 or L4 in the x-axis direction, a lower operating frequency is acceptable, and therefore a lower operating frequency is preferable when memory consumption and time required for learning are taken into consideration. In other words, when writing a character string CS using the slave device 10, there are a mixture of cases where a higher operating frequency is preferable and a case where a lower operating frequency is preferable. According to this embodiment, by hierarchizing the motion generator 610, it is possible to cause each motion to learn at a suitable operating frequency.

[0209] [Embodiment 11] FIG. 30 is a diagram for explaining ideal values ​​and response values ​​when a system according to an eleventh embodiment of the present invention is used. The eleventh embodiment is a modified example of the fourth embodiment. With reference to the same figure, the system according to the eleventh embodiment will be described in more detail. In this embodiment, the slave device 10 learns the first learned model 321D by comparing speed information (i.e., command speed) at which the own device is scheduled to operate as a result of the operation with the measured operating speed (i.e., operating speed) of the own device. An example shown in FIG. 30 shows the correspondence relationship between the command speed and the operating speed.

[0210] FIG. 30 shows ideal values ​​and response values. The ideal value refers to the case where the command speed and the motion speed are the same. For example, when the command speed is speed v12, the ideal value of the motion speed is speed v22. However, in this example, when the command speed is speed v12, the actual motion speed is speed v21. In this case, the first learned model 321D learns that the command speed, which has been learned to be speed v12, is speed v11, which is the ideal value of the command speed corresponding to the actual motion speed v21.

[0211] In this embodiment, the input information and output information of the first trained model 321 are speeds. That is, the input information and output information of the first trained model 321 are values ​​that can be quantitatively expressed. Therefore, the first trained model 321 can learn autonomously from the relationship between the input information and the output information. Furthermore, by configuring as in this embodiment, the information processing device 30D can easily train the first trained model 321.

[0212] [Prediction of state according to embodiment 6 and embodiment 9] The state prediction described with reference to FIG. 17 in the sixth embodiment and the state prediction described with reference to FIG. 22 in the ninth embodiment will be described in more detail with reference to FIGS. 31 to 34.

[0213] FIG. 31 is a diagram for explaining prediction of a state according to the conventional technology. With reference to the figure, the state of a controlled device and prediction of a future state in the state will be explained. In the explanation of the figure, the horizontal axis is shown as time t. In the conventional technology, since a slave device and a master device are not provided, the input and the output have the same value, and the controlled device can predict the future state of the device itself. That is, according to the conventional technology, the state of time t=k+1 is predicted in the state of time t=k, and in the state of time t=k+1, the state of time t=k+2 can be predicted using the predicted result of the device itself as the initial value. That is, according to the conventional technology, since the future state is predicted according to the state of the device itself, there is no accumulation of deviation even when predicting the distant future.

[0214] Fig. 32 is a diagram for explaining the problem of deviation accumulation according to the present invention. With reference to the diagram, the state of the controlled device and prediction of the future state in that state when the slave device 10 and the master device 20 are used will be explained. Fig. 32(A) is a diagram of an example during learning, and Fig. 32(B) is a diagram of an example during execution.

[0215] First, future prediction during learning will be described with reference to Fig. 32(A). During learning, the learned model predicts the future state of the master device 20 (e.g., the state of the master device 20 at time t = k+1) based on a state of the slave device 10 (e.g., the state of the slave device 10 at time t = k). Furthermore, the learned model predicts the further future state of the master device 20 (e.g., the state of the master device 20 at time t = k + 2) based on the state of the slave device 10 at time t = k + 1. That is, the state predicted by the learned model at time t = k and the state predicted at time t = k + 1 are independent of each other, so an open loop is formed.

[0216] Next, future prediction at run time will be described with reference to FIG. 32(B). At run time, the learned model predicts the future state of the master device 20 based on the state of the slave device 10 and the response from the master device 20. For example, at time t=k+1, the learned model predicts the state of the master device 20 at time t=k+2 based on the state of the slave device 10 at time t=k+1 and the response from the master device 20. That is, a closed loop is configured to predict the future state at time t=k+1 according to the result of the learned model operating based on the state predicted at time t=k. Therefore, according to the model shown in FIG. 32, there is a problem in that the model is different at the time of learning and at the time of run time. According to the model shown in FIG. 32, although it is an open loop at the time of learning, a closed loop is configured at the time of run time, so that a deviation that is not taken into account at the time of learning occurs at the time of run time, and when predicting the distant future, the deviation accumulates, resulting in a problem that the future state cannot be predicted accurately. In addition, if the deviation is large, there is a risk of control becoming unstable.

[0217] Fig. 33 is a diagram for explaining state prediction according to the sixth embodiment of the present invention. The state prediction according to the sixth embodiment of the present invention is intended to solve the problem described with reference to Fig. 32. With reference to Fig. 33, the state of the controlled device and prediction of the future state in that state in the sixth embodiment of the present invention will be described. Fig. 33(A) is a diagram of an example during learning, and Fig. 33(B) is a diagram of an example during execution.

[0218] First, future prediction during learning will be described with reference to FIG. 33(A). During learning, the learned model predicts the state of the master device 20 at time t=k+1 based on the state of the slave device 10 at time t=k (hereinafter, the prediction of the state of the master device 20 based on the state of the slave device 10 will be referred to as "human behavior prediction"). Also, the learned model predicts the state of the slave device 10 at time t=k+2 based on the state of the master device 20 at time t=k+1 (hereinafter, the prediction of the state of the slave device 10 based on the state of the master device 20 will be referred to as "environment response prediction"). That is, in this embodiment, the learned model predicts the state predicted at time t=k and a response based on the prediction.

[0219] Next, future prediction during execution will be described with reference to FIG. 33(B). During execution, the learned model predicts the future state of the master device 20 based on the state of the slave device 10 and the response from the master device 20. For example, at time t=k+1, the learned model predicts the state of the master device 20 at time t=k+2 based on the state of the slave device 10 at time t=k+1 and the response from the master device 20. That is, a closed loop is configured to predict the future state at time t=k+1 according to the result of the learned model operating based on the state predicted at time t=k. However, in this embodiment, the response from the master device 20 at time t=k+1 has already been predicted by "environment response prediction", so there is no accumulation of deviation. Therefore, according to the model shown in FIG. 31, the future state can be predicted more accurately.

[0220] Here, according to the model shown in Fig. 33, the "environment response prediction" learned during learning is not used during execution. Therefore, according to the model shown in Fig. 33, since the "environment response prediction" is not used during execution, there is a problem that the data collected by consuming time and memory capacity during learning cannot be utilized during execution.

[0221] Fig. 34 is a diagram for explaining state prediction according to the ninth embodiment of the present invention. The state prediction according to the sixth embodiment of the present invention is intended to solve the new problems that have arisen from the method described with reference to Fig. 33. With reference to Fig. 34, the state of the controlled device and prediction of the future state in that state according to the ninth embodiment of the present invention will be described. Fig. 34(A) is a diagram of an example during learning, and Fig. 34(B) is a diagram of an example during execution.

[0222] First, future prediction during learning will be described with reference to FIG. 34(A). During learning, the learned model predicts the state of the master device 20 at time t=k+1 and the state of the slave device 10 at time t=k+2 based on the state of the slave device 10 at time t=k. That is, in this embodiment, the learned model predicts the state of the master device 20 after a predetermined time has elapsed and the state of the slave device 10 at a time further ahead than the predetermined time based on the state of the slave device 10. This embodiment differs from the example in FIG. 33 in that "environment response prediction" is performed based on the state of the slave device 10.

[0223] Next, future prediction during execution will be described with reference to FIG. 34(B). During execution, the learned model predicts the future state of the slave device 10 and the future state of the master device 20 based on the current state of the slave device 10, the response from the master device 20, and the state predicted from the past state of the slave device 10. For example, at time t=k+1, the learned model predicts the state of the master device 20 and the state of the slave device 10 at time t=k+2 based on the state of the slave device 10 at time t=k+1, the response from the master device 20, and the state of the slave device 10 predicted at time t=k. That is, according to this embodiment, control can be performed based on the actual response and information predicted in the past, so that the future state can be predicted more accurately.

[0224] [Embodiment 12] Fig. 35 is a schematic diagram for explaining in more detail the hierarchical trained model according to the twelfth embodiment of the present invention. The twelfth embodiment is a modified example of the eighth embodiment. With reference to the same figure, the system according to the twelfth embodiment will be explained. In the eighth embodiment, an example has been explained in which the system 1H is provided with the hardness sensor 50, and one or both of the unit times τ1 and τ2 are changed according to the degree of hardness of the object to be grasped. However, the twelfth embodiment differs from the eighth embodiment in that the system 1H does not include the hardness sensor 50.

[0225] In the twelfth embodiment, the information processing device 30H includes a plurality of trained models, and is trained into different trained models depending on the hardness of an object to be gripped. Specifically, the information processing device 30H is configured with a plurality of motion generators 611-n (n is an integer equal to or greater than 1). In the example shown in FIG. 35, the information processing device 30H is configured with a motion generator 611-1 and a motion generator 611-2. Both the motion generator 611-1 and the motion generator 611-2 perform gripping motions. The motion generator 611-1 has a lower motion frequency than the motion generator 611-2, and the motion generator 611-2 has a lower motion frequency than the motion generator 611-1.

[0226] Here, in the motion of gripping an object, the sampling time suitable for learning varies depending on the hardness of the object. For example, the sampling time when gripping a soft object may be longer than the sampling time when gripping a hard object. In this embodiment, in the slave unit 10, the motion of gripping a soft object is learned by the motion generator 611-1 with a low motion frequency, and the motion of gripping a hard object is learned by the motion generator 611-2 with a high motion frequency. That is, in this embodiment, the slave unit 10 learns different learned models depending on the hardness of the object.

[0227] The configuration in the 12th embodiment has the same configuration as the hierarchical structure described in the 10th embodiment, but the two are different in that the trained model trained according to the movement of the own device is different in the 12th embodiment, and the trained model trained according to the object to be grasped is different in the 12th embodiment. A configuration in which the configurations of the 10th embodiment and the 12th embodiment are combined may be used.

[0228] By configuring as in embodiment 12, the trained model can be trained at a more suitable sampling time. According to embodiment 12, the trained model can be trained at a more suitable sampling time, so that the time required for training can be reduced and the memory capacity required for storing the trained model can be reduced.

[0229] [Embodiment 13] 36 is a diagram for explaining state prediction according to the 13th embodiment of the present invention. The 13th embodiment differs from the above-mentioned embodiments in that the first estimation unit 331 in the upper layer estimates command information at a plurality of different time points, and the second estimation unit 332 estimates command information for each of the command information at the estimated plurality of time points. The plurality of different time points may be past time points, may be the present, or may be future time points. In the figure, an example will be described in which the period of the upper layer is five times that of the lower layer. The first estimation unit 331 of the upper layer predicts the state at three time points: time k+5, time k+10, and time k+15.

[0230] 36(A) to 36(C) are diagrams for explaining the estimation by the second estimating unit 332, which is a lower layer, during the period from time k to time k+5. In Fig. 36(A), the second estimation unit 332 predicts the state at time k+1. In Fig. 36(B), the second estimation unit 332 predicts the state at time k+2. In Fig. 36(C), the second estimation unit 332 predicts the state at time k+3. The second estimating unit 332 can eliminate interference between the upper layer and the lower layer by not updating the input from the first estimating unit 331 which is an upper layer.

[0231] The first estimation unit 331 in the upper layer estimates command information at multiple different time points, and the second estimation unit 332 estimates command information for each of the command information estimated at the multiple time points, so that information related to time changes in command information can be transmitted to the lower layer. Therefore, even if the output of the upper layer is only position information, for example, the lower layer can obtain information related to speed equivalently. Therefore, even if the upper layer outputs command information using a sensor that is difficult to generate speed information, such as image information or GPS, it becomes possible to transmit appropriate command information to the hierarchy.

[0232] Furthermore, among the pairs of input information and output information used by the first generation unit 311 when generating the first learned model 321, the output information is not limited to the example of command information MASTER(t+T1) supplied to the slave device 10, but may also be a model that outputs information that is highly correlated with the command information MASTER(t+T1), such as status information SLAVE(t+T1). In particular, predicting the state information SLAVE(t+T1) is effective because when predicting the command information MASTER(t+T1), it is not possible to verify whether the prediction result was correct because only slaves exist during autonomous operation, and no master exists. On the other hand, when predicting the state information SLAVE(t+T1), since slaves exist even during autonomous operation, it is possible to evaluate the prediction error by comparing it with the response value one step later. This makes it easier to reduce prediction errors and stabilize predictions.

[0233] Furthermore, the first trained model 321 is not limited to an example of an arbitrary trained model capable of generating command information MASTER(t+T1) from status information SLAVE(t), but may be a model that outputs information that is highly correlated with command information MASTER(t+T1), such as status information SLAVE(t+T1). The command information MASTER(t+T1) output by the first trained model 321 in the above description can be replaced with information that is highly correlated with command information MASTER(t+T1). In particular, predicting the state information SLAVE(t+T1) is effective because when predicting the command information MASTER(t+T1), it is not possible to verify whether the prediction result was correct because only slaves exist during autonomous operation, and no master exists. On the other hand, when predicting the state information SLAVE(t+T1), since slaves exist even during autonomous operation, it is possible to evaluate the prediction error by comparing it with the response value one step later. This makes it easier to reduce prediction errors and stabilize predictions.

[0234] Furthermore, the n second trained models 322 used by the second estimation unit 332 for estimation are not limited to a case where n is a natural number of 2 or more, and may be one, i.e., n is a natural number of 1 or more. In this embodiment, the first estimation unit 331 and the second estimation unit 332 use different trained models. Therefore, in the learning stage, the first trained model 321 used by the first estimation unit 331 and the second trained model 322 used by the second estimation unit 332 can be trained separately and independently, which can significantly reduce design costs. Also, even when it becomes necessary to learn a new operation, the amount of learning can be reduced by newly training the second trained model 322 and reusing the first trained model 321. Similarly, the amount of learning can be reduced by newly training the first trained model 321 and reusing the second trained model 322.

[0235] In addition, in the estimation phase execution unit 33H of the information processing device 30H, the change control unit 334 may change one or both of the unit times τ1 and τ2 according to the degree of hardness of a contact object that is simply in contact, in addition to an object that is the subject of a gripping operation, an ingredient that is the subject of a cooking operation, a writing implement and writing paper that are the subject of a drawing operation, and an object that is the subject of a polishing operation. In other words, the operation of the controlled device (master device and slave device) is not limited to a gripping operation, but also includes an operation of simply contacting.

[0236] [Software implementation example] The control blocks (particularly the learning phase execution unit 31 and the estimation phase execution unit 33) of the information processing devices 30, 30B to 30L may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0237] In the latter case, the information processing device 30, 30B to 30L includes a computer that executes instructions of a program, which is software that realizes each function. The computer includes, for example, one or more processors, and a computer-readable recording medium that stores the program. In the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present invention. The processor may be, for example, a CPU (Central Processing Unit). The recording medium may be a "non-transient tangible medium", such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like. The computer may further include a RAM (Random Access Memory) that expands the program. The program may be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) that can transmit the program. Note that one aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0238] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0239] 1, 1B, 1C, 1D, 1E, 1F, 1G, 1H, 1I, 1J, 1K, 1L, 2 systems 10 Slave Device 10B Controlled Device 20 Master Device 30, 30B, 30C, 30D, 30E, 30F, 30G, 30H, 30I, 30K, 30K, 30L Information processing device 31 Learning Phase Execution Unit 32 Storage section 33 Estimation Phase Execution Unit 40 Input Devices 111, 211 Base 112, 212 Rotating part 113, 213 First arm 114, 214 Second arm 311, 311D, 311F 1st generation section 312 Second generation part 321, 321D, 321F First trained model 322 Second trained model 323 Training Dataset 324 Third trained model 331 1st estimation part 332 Second estimation part 333 Motion Control Unit 610, 610-1, 610-2, 610-3 motion generator

Claims

1. a first estimation unit that estimates command information for each first unit time using a first learned model that derives the command information to be supplied to a control target device from information output by the control target device that operates in accordance with command information and that represents a state of the control target device; a second estimation unit that estimates the command information for each second unit time shorter than the first unit time by using a second learned model that derives the command information to be supplied to the control target device from information representing the state output by the control target device and the command information derived by inputting the information representing the state into the first learned model; an operation control unit that operates the control target device using the command information estimated by the second estimation unit; An information processing device comprising:

2. the command information is information issued by a device that outputs the command information, The information input to the second trained model includes command information derived by the first trained model and information representing the state. The information processing device according to claim 1 .

3. The input of the second trained model is data obtained by converting a set of information representing the state output by the controlled device and information of the output of the first trained model so as to reduce correlation.

3. The information processing device according to claim 1 or 2.

4. the device that outputs the command information and the device to be controlled each include a robot arm; The command information and the information representing the state each include information indicating an operation of the robot arm.

4. The information processing device according to claim 1.

5. the device that outputs the command information is a master device that operates in response to a user operation, the controlled device is a slave device that operates in accordance with the command information output from the master device, The first learned model and the second learned model are learned using the command information and the information representing the state collected in a master-slave system including the master device and the slave device. The information processing device according to any one of claims 1 to 4.

6. a first generation unit that generates a first trained model by using a set of information representing a state of a control target device output by the control target device operating according to command information and the command information supplied to the control target device, the first trained model being trained to learn a correlation between the information representing the state and the command information; a second generation unit that generates a second learned model by learning a correlation between the input information and the command information using a set of input information including information representing a relationship between information representing the state output by the controlled device and the command information derived by inputting the information representing the state into the first learned model, and the command information supplied to the controlled device; and An information processing device comprising:

7. a first estimation unit that estimates the command information by inputting information representing the state output by the controlled device to the first trained model every first unit time; a second estimation unit that estimates the command information by inputting input data, including information representing a relationship between information representing the state output by the control target device and the command information derived by inputting the information representing the state into the first trained model, into the second trained model for each second unit time shorter than the first unit time; an operation control unit that operates the control target device using the command information estimated by the second estimation unit; The information processing device according to claim 6 , further comprising:

8. the device that outputs the command information is a master device that operates in response to a user operation, the controlled device is a slave device that operates in accordance with the command information output from the master device, The first generation unit generates the first trained model by using the command information and the information representing the state collected in a master-slave system including the master device and the slave device; The second generation unit generates the second trained model by using the command information and the information representing the state collected in the master-slave system.

8. The information processing device according to claim 6 or 7.

9. The first trained model is a trained model that derives the command information to be supplied to the control target device after the control target device outputs information representing the state from input data including information representing the state and information including an operating speed that specifies the operation of the control target device. The information processing device according to any one of claims 1 to 8.

10. The information including the operation speed includes information indicating a frequency or an operation execution time representing an operation cycle of the control target device. The information processing device according to claim 9.

11. The first trained model is a trained model that derives, from the input data, the command information to be supplied to the control target device after a predetermined unit time has elapsed since the control target device output information representing the state. The information processing device according to claim 9 or 10.

12. The first trained model is a trained model that derives, from input data including information representing the state and information including an operation speed that specifies an operation to be instructed to the control target device, the command information to be supplied to the control target device after the control target device outputs information representing the state, as output data; The first generation unit generates the first learned model using the command information, the information representing the state, and information including the operation speed, which are collected by operating the control target device while changing the operation speed of the control target device.

8. The information processing device according to claim 6 or 7.

13. The first trained model is a trained model trained using, as input information, information representing a second state derived by a third trained model that derives, from the command information, information representing a second state output by the control target device after the command information is supplied to the control target device. The information processing device according to any one of claims 1 to 12.

14. a first step of estimating command information for each first unit time using a first trained model that derives the command information to be supplied to a control target device from information output by the control target device that operates according to command information and that represents a state of the control target device; a second step of estimating the command information for each second unit time shorter than the first unit time using a second learned model that derives the command information to be supplied to the control target device from information representing a relationship between information representing the state output by the control target device and the command information derived by inputting the information representing the state into the first learned model; a third step of operating the controlled device using the command information estimated in the second step; A method executed by an information processing device.

15. The second estimation unit estimates the command information using n (n is a natural number equal to or greater than 2) second trained models, The n second trained models are used hierarchically, and information representing a relationship according to command information derived from the second trained model of each layer becomes input information of a second trained model below the second trained model. The information processing device according to any one of claims 1 to 5.

16. the control target device is a device that performs an operation on an object, a change control unit that changes one or both of the first unit time and the second unit time in accordance with the degree of hardness of the object; The information processing device according to any one of claims 1 to 5, further comprising:

17. The first trained model is a trained model that derives information representing a second state of the control target device after a predetermined time has elapsed and the command information from information representing the state, and is a trained model that has been trained using the information representing the second state derived by the first trained model as input information. The information processing device according to any one of claims 1 to 12.

18. The first trained model is a trained model that derives information representing a second state of the control target device after a predetermined time has elapsed and the command information from the command information and information representing the state, and is a trained model that has been trained using the command information and the information representing the second state derived by the first trained model as input information. The information processing device according to any one of claims 1 to 12.

19. The n second trained models each have a different operating frequency. The information processing device according to claim 15.

20. The input information is a quantitative value.

19. The information processing device according to claim 13, 15, 17, or 18.

21. The control target device grasps a grasp target or contacts a contact target, The first trained model and the second trained model store an action of gripping the gripping object or touching the contact object, The action of gripping the gripping object or touching the touching object is learned by either the first trained model or the second trained model depending on the degree of hardness of the gripping object or the touching object. The information processing device according to any one of claims 1 to 5.

22. The first trained model learns a behavior when the object to be grasped is a soft object, The second trained model learns a behavior when the object to be grasped is a hard object, The first trained model and the second trained model have different operating frequencies. The information processing device according to claim 21.

23. A program for causing a computer to function as the information processing device according to any one of claims 1 to 13 and 15 to 22, the program causing the computer to function as each of the units.

24. The first estimation unit estimates the command information at a plurality of time points, The second estimation unit estimates the command information for each of the command information at a plurality of estimated time points. The information processing device according to any one of claims 1 to 5 and 7.

Images