Information processing device, information processing method, and program

Abstract

An information processing device (100) according to the present disclosure is provided with: an acquisition unit (121) that acquires a dynamics model, which approximates the dynamics of a control target and outputs a state of the control target in response to a control input, and an uncertainty model, which represents the uncertainty of the dynamics model; a generation unit (122) that generates control input samples for the control target using the uncertainty model; and a calculation unit (123) that calculates optimal control inputs from the samples using the dynamics model.

Description

Information processing device, information processing method, and program

[0001] The present disclosure relates to an information processing device, an information processing method, and a program.

[0002] When performing optimal control of a control object such as a robot, a method has been proposed that uses an approximate dynamics model of the control object obtained by machine learning. For example, Patent Literature 1 discloses a technique for performing sampling-based optimal control using a dynamics model approximated by a neural network.

[0003] JP 2018-124982 A

[0004] However, in Patent Document 1, the trained dynamics model may have an error with respect to the actual dynamics model, and therefore, when an optimal control input is calculated using such a dynamics model, there is a risk that an error will be large during execution. As a result, there is a problem that there is a risk that the control accuracy using the dynamics model of the controlled object will be reduced.

[0005] Therefore, an object of the present disclosure is to solve the above-mentioned problem that there is a risk of a decrease in control accuracy when control is performed using a dynamics model of the controlled object.

[0006] An information processing device according to one aspect of the present disclosure includes: an acquisition unit that acquires a dynamics model approximating the dynamics of a controlled object that outputs a state of the controlled object according to a control input, and an uncertainty model representing the uncertainty of the dynamics model, a generation unit that generates samples of a control input for the controlled object using the uncertainty model, and a calculation unit that calculates an optimal control input from the samples using the dynamics model.An information processing method according to one aspect of the present disclosure includes: acquiring a dynamics model approximating the dynamics of a controlled object that outputs a state of the controlled object according to a control input, and the uncertainty model representing the uncertainty of the dynamics model, generating samples of a control input for the controlled object using the uncertainty model, and calculating an optimal control input from the samples using the dynamics model. Furthermore, a program according to one aspect of the present disclosure has a configuration for causing a computer to execute the following processes: acquiring a dynamics model that approximates the dynamics of a controlled object and outputs a state of the controlled object according to a control input; and an uncertainty model that represents the uncertainty of the dynamics model; generating samples of a control input for the controlled object using the uncertainty model; and calculating an optimal control input from the samples using the dynamics model.

[0007] With the above-described configuration, the present disclosure can improve control accuracy when performing control using a dynamics model of a controlled object.

[0008] FIG. 1 is a block diagram showing the overall configuration of a control system according to the present disclosure. FIG. 2 is a block diagram showing the hardware configuration of a learning device according to the present disclosure. FIG. 3 is a block diagram showing the hardware configuration of a control device according to the present disclosure. FIG. 4 is a block diagram showing the configuration of a learning device according to the present disclosure. FIG. 5 is a block diagram showing the configuration of a control device according to the present disclosure. FIG. 6 is a diagram showing the processing state of a control device according to the present disclosure. FIG. 7 is a flowchart showing the processing operation of a learning device according to the present disclosure. FIG. 8 is a flowchart showing the processing operation of a control device according to the present disclosure. FIG. 9 is a block diagram showing the hardware configuration of an information processing device according to the present disclosure. FIG. 10 is a block diagram showing the configuration of an information processing device according to the present disclosure.

[0009] First Embodiment A first embodiment of the present disclosure will be described with reference to the drawings. Note that the drawings may be relevant to any embodiment.

[0010] [Controlled Object] The control system according to the present disclosure can be used, for example, when a robot arm, which is the controlled object, is used to move an object to be manipulated, for example, by pushing it to a desired position. In this case, the controlled object is the robot arm and the object. The control input to the controlled robot arm can be, for example, the position of the robot arm's hand, and the state of the controlled object can be, for example, the position and velocity of the robot arm's hand. Note that the state of the controlled object may also be the position and velocity of an object manipulated by the robot arm's hand. Note that, in this embodiment, the controlled object by the control system is described as a robot arm, but the controlled object is not limited to being a robot arm and may be anything.

[0011] [System Configuration] Fig. 1 is a diagram showing an example of the configuration of a control system according to this embodiment. As shown in Fig. 1, a control system 5 includes a learning device 1, a storage device 2, a control device 3, and a control target 4. The learning device 1 performs data communication with the storage device 2 via a communication network or by wireless or wired direct communication. Furthermore, the control device 3 performs data communication with the storage device 2 and the control target 4 via a communication network or by wireless or wired direct communication.

[0012] The controlled object 4 performs an operation related to a control target based on a control input provided from the control device 3. The controlled object 4 also supplies a state signal representing the state of the controlled object 4 to the control device 3. Here, the controlled object 4 is, for example, a robot that is the target of optimal control, such as a robot arm that autonomously behaves as described above. The state signal representing the state of the controlled object is, for example, an output signal from various sensors that detect the position and orientation of the robot. In this embodiment, the state signal is particularly the position of the hand of the robot arm that is the controlled object 4. However, the controlled object 4 may also include an object manipulated by the robot arm described above. In this case, the state signal may be a detection signal representing the position, velocity, or the like of the object manipulated by the controlled object 4. For example, the state signal may be a detection signal representing the position of an object detected from an image of the object captured by a camera attached to the robot arm.

[0013] The control device 3 receives the current state of the control target 4 as an input, calculates a control input based on a control target, and outputs the calculated control input to the control target 4. Here, the control input is calculated by a sampling-based optimal control method using an approximate dynamics model and an uncertainty model stored in the storage device 2. Here, the sampling-based optimal control method may be, for example, Model Predictive Path Integral control or a control method based on the Cross-Entropy Method.

[0014] The learning device 1 learns an approximate dynamics model of the control target 4 from previously provided learning data, for example, by machine learning using a neural network. The learning device 1 also learns an uncertainty model of the learned approximate dynamics model. The learning device 1 then registers the learned dynamics model and uncertainty model in the storage device 2.

[0015] The above-mentioned approximate dynamics model is a machine learning model that receives as input the state of the controlled object at the current time and a control input, and outputs the state of the controlled object at the next time. Furthermore, the learning data is data on the state of the controlled object, the control input, and the subsequent state of the controlled object, collected using the controlled object or a simulator of the controlled object, etc., in order to use it for learning the above-mentioned approximate dynamics model.

[0016] The uncertainty model described above is learned by machine learning of the error of the approximate dynamics model, the probabilistic uncertainty of the dynamics of the controlled object, etc. The uncertainty model may be learned by the same model as the approximate dynamics, for example, using a Bayesian neural network model, an ensemble model, etc.

[0017] The storage device 2 stores the dynamics model and uncertainty model learned by the learning device 1. The storage device 2 may be an external storage device such as a hard disk connected to or built into the learning device 1 or the control device 3, a storage medium such as a flash memory, or a server device that communicates data with the learning device 1 and the control device 3. The storage device 2 may also be composed of multiple storage devices, and may have the above-mentioned storage units distributed among them.

[0018] 1 is an example, and various modifications may be made to the configuration. For example, the control device 3 and the controlled object 4 may be integrated. In another example, at least two of the learning device 1, the storage device 2, and the control device 3 may be integrated.

[0019] [Hardware Configuration] Next, an example of the hardware configuration of the above-mentioned learning device 1 and control device 3 will be described. Fig. 2 is a diagram showing an example of the hardware configuration of the learning device 1. The learning device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 10.

[0020] The processor 11 executes a program stored in the memory 12 to function as a controller (arithmetic unit) that performs overall control of the learning device 1. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0021] Memory 12 is composed of various types of volatile and non-volatile memory, such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. Memory 12 also stores programs for executing the processes performed by learning device 1. Note that some of the information stored in memory 12 may be stored in one or more external storage devices (e.g., storage device 2) that can communicate with learning device 1, or may be stored in a storage medium that is detachable from learning device 1.

[0022] The interface 13 is an interface for electrically connecting the learning device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to other devices, or hardware interfaces for connecting to other devices via cables or the like. For example, the interface 13 may interface with input devices that accept user input (external input), such as a touch panel, buttons, keyboard, or voice input device, display devices such as a display or projector, or sound output devices such as speakers.

[0023] The hardware configuration of the learning device 1 is not limited to the configuration shown in Figure 2. For example, the learning device 1 may incorporate at least one of a display device, an input device, and a sound output device. The learning device 1 may also include a storage device 2.

[0024] 3 is a diagram illustrating an example of the hardware configuration of the control device 3. The control device 3 includes, as hardware, a processor 31, a memory 32, and an interface 33. The processor 31, the memory 32, and the interface 33 are connected via a data bus 30.

[0025] The processor 31 executes a program stored in the memory 32 to function as a controller (arithmetic unit) that performs overall control of the control device 3. The processor 31 is, for example, a processor such as a CPU, a GPU, or a TPU. The processor 31 may be composed of multiple processors.

[0026] The memory 32 is configured by various types of volatile and non-volatile memory, such as RAM, ROM, and flash memory. The memory 32 also stores programs for executing processes performed by the control device 3. Note that part of the information stored in the memory 32 may be stored in one or more external storage devices (e.g., the storage device 2) that can communicate with the control device 3, or may be stored in a storage medium that is detachable from the control device 3.

[0027] The interface 33 is an interface for electrically connecting the control device 3 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0028] The hardware configuration of the control device 3 is not limited to the configuration shown in Fig. 3. For example, the control device 3 may incorporate at least one of a display device, an input device, and a sound output device. Furthermore, the control device 3 may be configured to include the storage device 2.

[0029] [Configuration of Learning Device] Next, a detailed configuration of the above-mentioned learning device 1 will be described. Fig. 4 is a diagram showing an example of the configuration of the learning device 1 that learns an approximate dynamics model and an uncertainty model. The learning device 1 has the function of receiving input of learning data and learning an approximate dynamics model and an uncertainty model. To realize this function, the learning device 1 includes a dynamics model learning unit 14 and an uncertainty model learning unit 15. The functions of the dynamics model learning unit 14 and the uncertainty model learning unit 15 are realized when the equipped processor 11 executes a program stored in the memory 12.

[0030] In this embodiment, when the state of the controlled object 4 at time t is represented as x_t and the control input given to the controlled object at time t as u_t, the learning data is given by a set of pairs {x_t, u_t, x_(t+1)} of the state x_t and control input u_t at the current time t, and the state x_(t+1) at the next time t+1. This data may be given by a sequence of states and control inputs x_{0:T}, u_({0:T-1}). Here, when written as x_({0:T}), it represents the sequence of states x from time 0 to time T.

[0031] The dynamics model learning unit 14 receives the above-mentioned learning data as input, and learns an approximate dynamics model F as shown in the following equation (1).

[0032] Various models can be used to learn the approximate dynamics model F. For example, a neural network may be used as the model, but the present invention is not limited to this.

[0033] The uncertainty model learning unit 15 receives training data as input and learns a model that represents the uncertainty of the dynamics model. In this embodiment, the probability distribution p(x_t, u_t) of training data {x_t, u_t} learned by a generative model, or something similar, is used as the uncertainty model. The reason why the distribution of the training data is used as the uncertainty model of the trained dynamics model is that it is expected that the accuracy of the trained dynamics model will be high in areas where the training data density is high.

[0034] Here, various generative models can be used when learning the probability distribution of training data, such as, but not limited to, a flow-based model and an energy-based model.

[0035] As described above, the dynamics model and uncertainty model learned by the learning device 1 are stored in the storage device 2. However, the approximate dynamics model and uncertainty model do not necessarily have to be learned by the learning device 1, but may be stored in advance in the storage device 2. Furthermore, the approximate dynamics model and uncertainty model do not necessarily have to be generated by learning, but may be generated by a method other than learning. For example, the dynamics model and uncertainty model may be generated from design data, motion analysis data, simulation data, etc. of the robot, which is the control object 4, and may be generated by any method.

[0036] [Configuration of Control Device] Next, a detailed configuration of the above-mentioned control device 3 will be described. FIG. 5 is a diagram showing an example of the configuration of the control device 3 that calculates the control input. The control device 3 calculates the control input by a sampling-based method using the current state provided by the controlled object 4 and information on the approximate dynamics model and uncertainty model read from the storage device 2. In this case, in the above-mentioned example of a control system that pushes an object using a robot arm, the control input is the hand position of the robot arm, and the control input sequence is the time series of the hand position of the robot arm, i.e., the hand trajectory. Furthermore, the state of the controlled object is the position and velocity of the hand of the robot arm, i.e., the position and velocity of the object manipulated by the robot arm.

[0037] As shown in FIG. 5 , the control device 3 includes an initial control input sequence generator 34, a state / control input sequence sample generator 35, an evaluation cost calculator 38, and a control input sequence updater 39. The state / control input sequence sample generator 35 includes a control input sample generator 36 and a state transition calculator 37. As will be described later, the control device 3 updates the control input sequence u_(0:T_h−1) and calculates an optimal control input sequence by repeatedly performing the processes of the units 35 to 38. The functions of the initial control input sequence generator 34, the state / control input sequence sample generator 35, the control input sample generator 36, the state transition calculator 37, the evaluation cost calculator 38, and the control input sequence updater 39 are realized by the processor 31 executing a program stored in the memory 32.

[0038] The initial control input sequence generator 34 outputs a time series of control inputs as an initial solution for performing optimization calculations of the control input sequence. This may be given as a zero vector, for example, or a control input policy that has been separately learned or designed in advance may be used.

[0039] The state and control input sequence sample generator 35 calculates different control input sequence samples and state sequence samples calculated from the respective control input sequence samples, using the state of the controlled object, the control input sequence of the current optimization step, and the uncertainty model and approximate dynamics model learned by the learning device 1. The samples are generated by repeatedly executing the processes by the control input sample generator 36 and the state transition calculator 37 for the length of the time series.

[0040] The control input sample generation unit 36 ​​generates samples using a state at a certain time, a nominal input, and an uncertainty model. Here, the state at a certain time is either the initial state x_0 provided by the control object 4, or the state at the previous time provided by the state transition calculation unit 37, or a state calculated from the input sample. The nominal input means either the initial solution of the control input sequence provided by the initial control input sequence generation unit 34, or the control input sequence updated by optimization calculation provided by the control input sequence update unit 39.

[0041] 6, the control input sample generation unit 36 ​​can generate samples by regarding a normal distribution with the nominal input as the mean as the likelihood distribution and a distribution of the control input conditioned on the state at the current time obtained from the uncertainty model as the prior distribution, and then performing an appropriate number of samples from the posterior distribution obtained from these. For example, the posterior distribution can be obtained by combining or integrating the likelihood distribution and the prior distribution.

[0042] More specifically, for example, suppose the state at the current time is given by x_t, the nominal input at the current time is given by u_t, and the uncertainty model is given by the probability distribution p(x_t,u_t) of the training data. In this case, when the normal distribution shown in the following equation 2, in which the mean is the nominal input u_t and the variance is a pre-given parameter Σ, is used as the likelihood distribution, and the probability distribution p(u_t|x_t) of the control input conditioned on the state at the current time obtained from the training data distribution p(x_t,u_t) is used as the prior distribution, sampling from the posterior distribution can be performed, for example, by the Markov chain Monte Carlo method.

[0043] As another sample generation method, when generating N samples, for example, as shown in Figure 7, M (>N) samples can be obtained from a normal distribution with the nominal input as the mean, N samples with the smallest uncertainty obtained from the uncertainty model can be selected, and the remaining samples can be discarded.

[0044] The state transition calculation unit 37 calculates the next-time state sample x_(t+1)^k from the input sample u_t^k at a certain time generated by the control input sample generation unit 36 ​​and the state sample x_t^k at the corresponding time, using the approximate dynamics learned by the learning device 1. Here, when N input samples are generated, it is assumed that k∈N. The input samples and the next-time state sample are held until sampling is completed. If further generation of the next-time sample is required, the calculated next-time state is output to the control input sample generation unit 36. When generation of the time-series samples is completed, the held input and state are output to the evaluation cost calculation unit 38 as a sample sequence shown in Equation 3.

[0045] The evaluation cost calculation unit 38 calculates an evaluation cost S^k based on the control target from each state sequence and control input sequence sample, and outputs the calculated evaluation cost to the control input sequence update unit 39 together with the control input sequence sample.

[0046] The control input sequence updater 39 calculates an updated nominal input sequence based on the nominal input sequence, the control input sequence samples, and the evaluation costs corresponding to each sample. Here, the nominal input sequence is either the sequence provided by the initial control input sequence generator 34 or the nominal input sequence previously updated by the control input sequence updater 39. The control input sequence updater 39 determines whether optimization has ended, and upon completion, outputs a control input or a portion of the control input to the control object 4. If optimization is to continue, the control input sequence updater 39 outputs an updated nominal input sequence to the control input sample generator 36. The determination of the end of optimization can be made, for example, by determining convergence of the evaluation cost, a threshold related to the number of updates of the nominal input sequence, or both.

[0047] More specifically, the nominal input sequence is updated as follows: When the minimum cost of each sample is expressed by Equation 4 and the difference between each sample and the nominal input sequence is expressed by Equation 5, the updated nominal input sequence u_(0:T_h-1)^new is given by Equation 6.

[0048] [Operation] Next, an example of the operation of the above-described control system will be described, in particular the operation of the learning device 1 and the control device 3. Fig. 8 is a flowchart showing the operation of the learning device 1 in this embodiment.

[0049] (Step S101) The learning device 1 receives input of learning data. (Step S102) The dynamics model learning unit 14 and the uncertainty model learning unit 15 learn the approximate dynamics model and the uncertainty model, respectively. The learning of each model may be performed by parallel processing or in any order. (Step S103) The learning device 1 outputs the learning results, the approximate dynamics model and the uncertainty model, to the storage device 2.

[0050] The approximate dynamics model and the uncertainty model do not necessarily have to be learned by the learning device 1, but may be stored in advance in the storage device 2. Furthermore, the approximate dynamics model and the uncertainty model do not necessarily have to be generated by learning, but may be generated by a method other than learning.

[0051] Next, a description will be given of an example of the operation of the control device 3. Fig. 9 is a flowchart showing the operation of the control device 3 in the first embodiment.

[0052] (Step S301) First, the control device 3 receives a current state input from the controlled object 4. (Step S302) The initial control input sequence generator 34 generates an initial solution for the control input sequence. (Step S303) The control device 3 starts a loop L31 that performs sampling processing of the state sequence and the control input sequence. In the loop L31, sampling calculations in the time series direction are performed by repeating the loop, and therefore the number of loop iterations is represented by time t.

[0053] (Step S304) The control input sample generation unit 36 ​​calculates a control input sample u_t^k from the state sample x_t^k of the current step t and the nominal input u_t^. In the case of the initial step, N samples are calculated based on the current state x_t received from the control device 3. The samples are generated by sampling using the Markov chain Monte Carlo method or the like from the posterior distribution when the normal distribution obtained from the nominal input is used as the prior distribution and the conditional distribution of the input obtained from the training data distribution learned by the learning device 1 is used as the likelihood distribution.

[0054] (Step S305) The state transition calculation unit 37 calculates the state sample x_(t+1)^k for the next step t+1 from the state sample x_t^k for the current step t and the control input sample u_t^k. This calculation is performed using the approximate dynamics model learned by the learning device 1. (Step S306) The control device 3 performs termination processing of the loop L31. Specifically, the control device 3 refers to the number of iterations t and determines whether the sampling calculation has been completed up to the termination time. If it is determined that the sampling processing has not been completed, the control device 3 continues the sampling processing for the next time. In this case, the processing returns to step S304. On the other hand, if it is determined that the sampling processing has been completed, the control device 3 terminates the loop L31. In this case, the processing proceeds to step S307.

[0055] (Step S307) The evaluation cost calculation unit 38 calculates the evaluation cost for each of the generated N sample sequences. (Step S308) The control input update unit 39 updates the nominal input sequence based on each sample and its evaluation cost. (Step S309) The control device 3 determines whether optimization of the control input sequence has been completed. (Step S310) If optimization is to be continued (step S310: YES), the process proceeds to step S303. On the other hand, if optimization is to be completed (step S310: NO), the process proceeds to step S311. (Step S311) The control device 3 outputs the control input sequence resulting from the optimization calculation, or a portion of it, to the control object 4.

[0056] As described above, by learning an approximate dynamics model and its uncertainty model in the learning device 1, sampling can be performed taking into account the uncertainty of the dynamics model when performing a sampling-based optimization method. As a result, optimal control can be performed using a more likely region of the learned dynamics model, thereby improving the accuracy of control execution. Furthermore, by treating the distribution of the training data as an uncertainty model, it becomes possible to consider the posterior distribution as described above, and sampling from the distribution can be performed.

[0057] Second Embodiment Next, a second embodiment of the present disclosure will be described. In this embodiment, the uncertainty model is different from that in the first embodiment. In this embodiment, when the learning device 1 learns the uncertainty model, the learning device 1 may learn the uncertainty model so that the uncertainty model is obtained as an amount corresponding to the variance of the output of the dynamics model. In this case, the uncertainty model indicates that the greater the variance of the output of the dynamics model, the greater the uncertainty. Specifically, the uncertainty model can be obtained by learning a dynamics model using, for example, an ensemble model or a Bayesian neural network. Note that the configurations of the control system 5 and the control device 3 in the second embodiment are the same as those in the first embodiment.

[0058] In the second embodiment, when it is considered to generate, for example, N samples, the control input sampling in the control input sample generation unit 36 ​​can be generated by a method such as obtaining M (>N) samples from a normal distribution with the nominal input as the mean, selecting N samples with the smallest uncertainties obtained from the uncertainty model, and discarding the remaining samples, as shown in FIG. 7 .

[0059] Third Embodiment Next, a third embodiment of the present disclosure will be described with reference to the drawings. This embodiment shows an outline of the configuration of the control device 3 described in the above-mentioned embodiment. Note that the drawings may be relevant to any of the embodiments.

[0060] The information processing device 100 in this embodiment is configured as a general information processing device, and is equipped with the following hardware configuration, for example: CPU (Central Processing Unit) 101 (arithmetic device); ROM (Read Only Memory) 102 (storage device); RAM (Random Access Memory) 103 (storage device); programs 104 loaded into RAM 103; a storage device 105 that stores the programs 104; a drive device 106 that reads and writes data from and to a storage medium 110 external to the information processing device; a communication interface 107 that connects to a communication network 111 external to the information processing device; an input / output interface 108 that inputs and outputs data; and a bus 109 that connects the various components.

[0061] 10 shows an example of the hardware configuration of the information processing device 100, and the hardware configuration of the information processing device is not limited to the above-described case. For example, the information processing device may be configured with only a part of the above-described configuration, such as excluding the drive device 106. Furthermore, the information processing device may use a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point Number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof, instead of the above-described CPU.

[0062] The information processing device 100 can be equipped with an acquisition unit 121, a generation unit 122, and a calculation unit 123 shown in FIG. 11 by having the CPU 101 acquire and execute the program group 104. The program group 104 may be stored in advance in the storage device 105 or the ROM 102, for example, and loaded into the RAM 103 by the CPU 101 as needed. The program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read and supply the program to the CPU 101. However, the acquisition unit 121, the generation unit 122, and the calculation unit 123 described above may be constructed using dedicated electronic circuits for realizing such means.

[0063] The acquisition unit 121 acquires a dynamics model that approximates the dynamics of a controlled object that has been trained to output a state of the controlled object according to a control input, and an uncertainty model that represents the uncertainty of the dynamics model. The generation unit 122 generates samples of a control input for the controlled object using the uncertainty model. The calculation unit 123 calculates an optimal control input from the samples using the dynamics model.

[0064] With the above-described configuration, the present disclosure can generate control input samples that take into account the uncertainty of a dynamics model, and can calculate an optimal control input from the samples using the dynamics model. As a result, optimal control can be performed using a more likely region of the dynamics model, thereby improving the accuracy of control execution.

[0065] In addition, at least one of the functions of the above-mentioned acquisition unit 121, generation unit 122, and calculation unit 123 may be executed by an information processing device installed and connected anywhere on the network, that is, they may be executed by so-called cloud computing.

[0066] The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-RWs, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program can also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electric wires and optical fibers, or via wireless communication paths.

[0067] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each of the above-described embodiments can be combined with other embodiments as appropriate.

[0068] <Supplementary Notes> Some or all of the above embodiments can also be described as in the following supplementary notes. Below, an outline of the configurations of an information processing device, an information processing method, and a program according to the present disclosure will be described. However, the present disclosure is not limited to the following configurations. (Supplementary Note 1) An information processing device comprising: an acquisition unit that acquires a dynamics model that approximates the dynamics of a controlled object and outputs a state of the controlled object according to a control input, and an uncertainty model that represents the uncertainty of the dynamics model; a generation unit that generates samples of a control input for the controlled object using the uncertainty model; and a calculation unit that calculates an optimal control input from the samples using the dynamics model. (Supplementary Note 2) The information processing device according to Supplementary Note 1, wherein the acquisition unit acquires the uncertainty model based on learning data used to learn the dynamics model. (Supplementary Note 3) The information processing device according to Supplementary Note 2, wherein the acquisition unit acquires the uncertainty model based on a probability distribution of the learning data. (Supplementary Note 4) The information processing device according to Supplementary Note 3, wherein the generation unit generates the samples using a distribution based on a preset control input and a probability distribution of the training data. (Supplementary Note 5) The information processing device according to Supplementary Note 3, wherein the generation unit generates the samples using a normal distribution with a preset control input as an average and a probability distribution of the training data. (Supplementary Note 6) The information processing device according to Supplementary Note 5, wherein the generation unit generates candidate samples of the control input from a normal distribution with a preset control input as an average, and generates the samples from the candidate samples using the probability distribution of the training data. (Supplementary Note 7) The information processing device according to Supplementary Note 6, wherein the generation unit generates the samples with smaller uncertainty of the dynamics model from the candidate samples using the probability distribution of the training data. (Supplementary Note 8) The information processing device according to Supplementary Note 1, wherein the acquisition unit acquires the uncertainty model based on a variance of the output of the dynamics model.(Supplementary Note 9) An information processing method comprising: acquiring a dynamics model approximating the dynamics of a controlled object that outputs a state of the controlled object according to a control input; and an uncertainty model representing the uncertainty of the dynamics model; generating samples of a control input for the controlled object using the uncertainty model; and calculating an optimal control input from the samples using the dynamics model. (Supplementary Note 10) An information processing method according to Supplementary Note 9, wherein the uncertainty model is acquired based on learning data used to learn the dynamics model. (Supplementary Note 11) An information processing method according to Supplementary Note 10, wherein the uncertainty model is acquired based on a probability distribution of the learning data. (Supplementary Note 12) An information processing method according to Supplementary Note 11, wherein the samples are generated using a distribution based on a predetermined control input and a probability distribution of the learning data. (Supplementary Note 13) The information processing method according to Supplementary Note 12, wherein the samples are generated using a normal distribution with a preset control input as an average and a probability distribution of the training data. (Supplementary Note 14) The information processing method according to Supplementary Note 12, wherein candidate samples of the control input are generated from a normal distribution with a preset control input as an average, and the samples are generated from the candidate samples using the probability distribution of the training data. (Supplementary Note 15) The information processing method according to Supplementary Note 14, wherein the samples with smaller uncertainty in the dynamics model are generated from the candidate samples using the probability distribution of the training data. (Supplementary Note 16) The information processing method according to Supplementary Note 9, wherein the uncertainty model is obtained based on the variance of the output of the dynamics model.(Supplementary Note 17) A computer-readable storage medium storing a program that causes a computer to execute the following processes: acquiring a dynamics model that approximates the dynamics of a controlled object, which outputs a state of the controlled object according to a control input, and an uncertainty model that represents the uncertainty of the dynamics model; generating samples of a control input for the controlled object using the uncertainty model; and calculating an optimal control input from the samples using the dynamics model.

[0069] REFERENCE SIGNS LIST 1 Learning device 11 Processor 12 Memory 13 Interface 14 Dynamics model learning unit 15 Uncertainty model learning unit 2 Storage device 3 Control device 31 Processor 32 Memory 33 Interface 34 Initial control input sequence generation unit 35 State / control input sequence sample generation unit 36 ​​Control input sample generation unit 37 State transition calculation unit 38 Evaluation cost calculation unit 39 Control input sequence update unit 4 Control target 5 Control system 100 Information processing device 101 CPU 102 ROM 103 RAM 104 Program group 105 Storage device 106 Drive device 107 Communication interface 108 Input / output interface 109 Bus 110 Storage medium 111 Communication network 121 Acquisition unit 122 Generation unit 123 Calculation unit

Claims

1. An information processing device comprising: an acquisition unit that acquires a dynamics model that approximates the dynamics of a controlled object and outputs the state of the controlled object according to a control input; and an uncertainty model that represents the uncertainty of the dynamics model; a generation unit that generates samples of control inputs for the controlled object using the uncertainty model; and a calculation unit that calculates an optimal control input from the samples using the dynamics model.

2. An information processing device according to claim 1, wherein the acquisition unit acquires the uncertainty model based on learning data used in learning the dynamics model.

3. An information processing device according to claim 2, wherein the acquisition unit acquires the uncertainty model based on a probability distribution of the training data.

4. An information processing device according to claim 3, wherein the generation unit generates the samples using a distribution based on a preset control input and a probability distribution of the training data.

5. An information processing device according to claim 3, wherein the generation unit generates the samples using a normal distribution with a preset control input as the mean and a probability distribution of the training data.

6. An information processing device according to claim 5, wherein the generation unit generates candidate samples of the control input from a normal distribution with a preset control input as the mean, and generates the samples from the candidate samples using the probability distribution of the learning data.

7. An information processing device according to claim 6, wherein the generation unit generates, from the candidate samples, the samples for which the uncertainty of the dynamics model is smaller, using a probability distribution of the training data.

8. An information processing device according to claim 1, wherein the acquisition unit acquires the uncertainty model based on a variance of the output of the dynamics model.

9. An information processing method comprising: acquiring a dynamics model that approximates the dynamics of a controlled object that outputs a state of the controlled object according to a control input; and an uncertainty model that represents the uncertainty of the dynamics model; generating samples of the control input for the controlled object using the uncertainty model; and calculating an optimal control input from the samples using the dynamics model.

10. An information processing method according to claim 9, wherein the uncertainty model is obtained based on training data used to train the dynamics model.

11. An information processing method according to claim 10, further comprising the step of: acquiring the uncertainty model based on a probability distribution of the training data.

12. An information processing method according to claim 11, wherein the samples are generated using a distribution based on a preset control input and a probability distribution of the training data.

13. An information processing method according to claim 12, wherein the samples are generated using a normal distribution with a preset control input as the mean and a probability distribution of the training data.

14. An information processing method according to claim 12, comprising generating candidate samples of the control input from a normal distribution with a preset control input as the mean, and generating the sample from the candidate samples using the probability distribution of the training data.

15. An information processing method according to claim 14, further comprising generating, from the candidate samples, samples for which the uncertainty of the dynamics model is smaller, using a probability distribution of the training data.

16. An information processing method according to claim 9, further comprising: obtaining the uncertainty model based on a variance of the output of the dynamics model.

17. A computer-readable storage medium storing a program that causes a computer to execute the following processes: acquiring a dynamics model that approximates the dynamics of a controlled object and outputs the state of the controlled object according to a control input; and an uncertainty model that represents the uncertainty of the dynamics model; generating samples of control inputs for the controlled object using the uncertainty model; and calculating an optimal control input from the samples using the dynamics model.

Images