Information processing apparatus, information processing method, and program

Abstract

To suitably control a control object even when observation noise is included in observation data.SOLUTION: An information processing apparatus includes an acquisition unit that acquires a series of observation values in each step from a certain step t to a step t+N after N steps (N is an integer equal to or greater than 2) from the certain step t, which are observation values related to a control target controlled by repetition of steps, and a generation unit that generates a control model used to control the control target by a learning process including a first learning process that refers to observation values in each of two consecutive steps among the series of observation values and a second learning process that refers to the observation values.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There are known techniques for controlling control targets such as robots. For example, Patent Literature 1 discloses a learning device that performs self-organized updating of a dynamics memory network composed of multiple nodes based on observed time-series data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-299366 Summary of the Invention [Problem to be solved by the invention]

[0004] Generally, in a technique for controlling a control target based on observation data, the observation data may contain observation noise. Therefore, even if the technique of Patent Document 1 is used, it is not easy to suitably control the control target.

[0005] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of its purpose is to provide a technology that can suitably control a control target even when observation noise is included in the observation data. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention includes an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more), and a generation means for generating a control model to be used to control the control object by a learning process including a first learning process that references the observation values ​​at each of two consecutive steps among the series of observation values, and a second learning process that references the series of observation values.

[0007] An information processing device according to one aspect of the present invention includes an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N, which is N steps after the certain step t (N is an integer of 2 or more), and a control means for controlling the control object using a control model generated by a learning process including a first learning process that references the observation values ​​at each of two consecutive steps among the series of observation values, and a second learning process that references the series of observation values.

[0008] An information processing method according to one aspect of the present invention includes obtaining a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or greater), and generating a control model to be used to control the control object by a learning process including a first learning process that references the observation values ​​at each of two consecutive steps among the series of observation values, and a second learning process that references the series of observation values.

[0009] An information processing method according to one aspect of the present invention includes obtaining a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N that is N steps after the certain step t (N is an integer of 2 or more), and controlling the control object using a control model generated by a learning process that includes a first learning process that references the observation values ​​at each of two consecutive steps among the series of observation values, and a second learning process that references the series of observation values.

[0010] A program according to one aspect of the present invention is a program that causes a computer to function as an information processing device, and the program causes the computer to execute an acquisition process that acquires a series of observation values ​​related to a control object that is controlled by repeating steps, from a certain step t to a step t+N that is N steps after the certain step t (N is an integer of 2 or more), and a generation process that generates a control model to be used to control the control object by a learning process that includes a first learning process that references observation values ​​at each of two consecutive steps out of the series of observation values, and a second learning process that references the series of observation values.

[0011] A program according to one aspect of the present invention is a program that causes a computer to function as an information processing device, and causes the computer to execute an acquisition process that acquires a series of observation values ​​related to a control object that is controlled by repeating steps, from a certain step t to a step t+N that is N steps after the certain step t (N is an integer of 2 or more), and a control process that controls the control object using a control model generated by a learning process that includes a first learning process that references observation values ​​at each of two consecutive steps among the series of observation values, and a second learning process that references the series of observation values. [Effects of the Invention]

[0012] According to one aspect of the present invention, even when observation noise is included in the observation data, it is possible to suitably control the control target. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 3] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 4] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 5] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 6] FIG. 2 is a diagram for explaining processing by an information processing device according to the present disclosure. [Figure 7] FIG. 2 is a block diagram illustrating a configuration example of a generation unit included in the information processing device according to the present disclosure. [Figure 8] FIG. 1 is a flow diagram showing a processing flow in an information processing device according to the present disclosure. [Figure 9] FIG. 2 is a diagram for explaining processing in an information processing device according to the present disclosure. [Figure 10] FIG. 2 is a diagram for explaining processing in an information processing device according to the present disclosure. [Figure 11] FIG. 2 is a diagram for explaining processing in an information processing device according to the present disclosure. [Figure 12] FIG. 2 is a diagram for explaining processing in an information processing device according to the present disclosure. [Figure 13] FIG. 2 is a diagram for explaining processing in an information processing device according to the present disclosure. [Figure 14] FIG. 2 is a diagram for explaining processing in an information processing device according to the present disclosure. [Figure 15] FIG. 1 is a block diagram illustrating a hardware configuration of an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0015] [First embodiment] A first exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referred to in describing this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise.

[0016] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 may also be called a learning device, a control model generation device, a diffusion coefficient estimation device, or the like. As shown in Fig. 1, the information processing device 1 includes an acquisition unit 11 and a generation unit 12.

[0017] (Acquisition part 11) The acquisition unit 11 acquires a series of observation values ​​(observation data) related to a control object, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more). Here, the control object is a control object that is controlled by repeating steps.

[0018] Here, t is an index for distinguishing between multiple steps, and in this exemplary embodiment, it is an index assigned to the steps executed sequentially in ascending order. Step t may also be referred to as the t-th step or the step associated with the t-th time. For example, when t=3 and N=5, the acquisition unit 11 acquires the observed values ​​for each step from step 3, which is the third step, to step 8, which is the eighth step.

[0019] Furthermore, specific examples of observed values ​​related to the control object do not limit this exemplary embodiment, but examples include the position of the control object, the speed of the control object, the orientation of the control object, the angular velocity of the control object, and the angle of a joint of the control object. Furthermore, the observed values ​​can be obtained, for example, by a camera that captures an image of the control object or a sensor attached to the control object. However, these examples do not limit this exemplary embodiment.

[0020] (Generation part 12) The generation unit 12 generates a control model used to control the control target by: a first learning process that refers to the observed values ​​at each of two successive steps among the series of observed values; a second learning process using the series of observed values; The generated image is generated by a learning process including:

[0021] For example, in the above example, when t=3 and N=5, the generation unit 12 calculates a first learning process that refers to the observation values ​​in two consecutive steps, step 3 and step 4, among the observation values ​​in each step from step 3 to step 8 acquired by the acquisition unit 11; A second learning process that refers to the observed values ​​at each step from step 3 to step 8 acquired by the acquisition unit 11. The control model is generated by a learning process including:

[0022] As an example, the generator 12 may determine a diffusion coefficient to be used in the control model by referring to the result of the first learning process and the result of the second learning process. Here, the generator 12 may be configured to determine the diffusion coefficient by calculating the difference between the result of the first learning process and the result of the second learning process. However, these examples do not limit the present exemplary embodiment.

[0023] Furthermore, the control model generated by the generation unit 12 may be used in online control (online learning) of the control object. Here, online control, for example, refers to control that sequentially repeats the acquisition of observation values, the updating (learning) of a control model by referring to the observation values, and the control of the control object using the updated control model. As an example, the acquisition unit 11 may be configured to sequentially acquire observation values ​​at each step, and the control model generated by the generation unit 12 by referring to the observation values ​​of the control object acquired up to a certain step t' may be used to control the control object in steps after the certain step t' (or in steps after the certain step t'). However, this example does not limit the present exemplary embodiment.

[0024] (Effects of information processing device 1) As described above, in the information processing device 1, Obtaining a series of observations of a control object that is controlled by repeating steps, from a certain step t to step t+N, which is N steps after the certain step t (N is an integer of 2 or more), A control model used to control the control target, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; Generated by a learning process including The following configuration is adopted.

[0025] As described above, according to the above configuration, the control model is generated by a learning process including the first learning process and the second learning process. Here, according to the knowledge gained by the inventors, by using the first learning process and the second learning process, even if observation noise is included in the observation values ​​(observation data) related to the control object, it is possible to generate a control model in which the influence of the observation noise is reduced. Therefore, according to the above configuration, it is possible to provide a technology that can suitably control the control object even if the observation data includes observation noise.

[0026] (Flow of information processing method S1) Next, the flow of the information processing method S1 according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1. The information processing method S1 may also be called a learning method, a control model generation method, a diffusion coefficient estimation method, or the like. As shown in Fig. 2, the information processing method S1 includes a step (process) S11 of acquiring observed values ​​(observation data) and a step (process) S12 of generating a control model.

[0027] (Step S11) In step S11, the acquisition unit 11 acquires a series of observation values ​​(observation data) relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more). The specific processing by the acquisition unit 11 has been described above, and therefore will not be described here.

[0028] (Step S12) Next, in step S12, The generation unit 12 generates a control model used to control the control target by: a first learning process that refers to the observed values ​​at each of two successive steps among the series of observed values; a second learning process using the series of observed values; The specific processing by the generation unit 12 has been described above, and therefore will not be described here.

[0029] (Effect of information processing method S1) As described above, in the information processing method S1, Obtaining a series of observations of a control object that is controlled by repeating steps, from a certain step t to step t+N, which is N steps after the certain step t (N is an integer of 2 or more), A control model used to control the control target, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; Generated by a learning process including According to the above configuration, the same effects as those of the information processing device 1 are achieved.

[0030] (Configuration of information processing device 2) Next, the configuration of the information processing device 2 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2. The information processing device 2 may also be called a control device. As shown in Fig. 3, the information processing device 2 includes an acquisition unit 21 and a control unit 22.

[0031] (Control unit 21) The acquisition unit 21 acquires a series of observation values ​​(observation data) related to a control object, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more). Here, the control object is a control object that is controlled by repeating steps.

[0032] Here, t is an index for distinguishing between multiple steps, and in this exemplary embodiment, it is an index assigned to the steps executed sequentially in ascending order. Step t may also be referred to as the t-th step or the step associated with the t-th time. For example, when t=3 and N=5, the acquisition unit 21 acquires the observed values ​​for each step from step 3, which is the third step, to step 8, which is the eighth step.

[0033] Furthermore, specific examples of observed values ​​related to the control object do not limit this exemplary embodiment, but examples include the position of the control object, the speed of the control object, the orientation of the control object, the angular velocity of the control object, and the angle of a joint of the control object. Furthermore, the observed values ​​can be obtained, for example, by a camera that captures an image of the control object or a sensor attached to the control object. However, these examples do not limit this exemplary embodiment.

[0034] (control unit 22) The control unit 22 a first learning process that refers to the observed values ​​at each of two successive steps among the series of observed values; a second learning process using the series of observed values; The control object is controlled using a control model generated by a learning process including the above.

[0035] For example, in the above example, when t=3 and N=5, the control model is as follows: a first learning process in which, among the observed values ​​in each step from step 3 to step 8 acquired by the acquisition unit 21, the observed values ​​in two consecutive steps, step 3 and step 4, are referenced; A second learning process that refers to the observed values ​​at each step from step 3 to step 8 acquired by the acquisition unit 21. The control model is generated by a learning process including:

[0036] As an example, the control model may include a diffusion coefficient determined by referring to the result of the first learning process and the result of the second learning process. Here, the diffusion coefficient may be determined by calculating the difference between the result of the first learning process and the result of the second learning process. However, these examples do not limit the present exemplary embodiment.

[0037] Furthermore, the control unit 22 may use the control model in online control (online learning) of the control object. Here, online control, for example, refers to control that sequentially repeats the acquisition of observation values, the updating (learning) of a control model by referring to the observation values, and the control of the control object using the updated control model. For example, the acquisition unit 21 may sequentially acquire observation values ​​at each step, and the control unit 22 may use the control model generated by referring to the observation values ​​of the control object acquired up to a certain step t', for the control of the control object in a step after the certain step t' (or a step after the certain step t'). However, this example does not limit the present exemplary embodiment.

[0038] (Effects of information processing device 2) As described above, in the information processing device 2, Obtaining a series of observations of a control object that is controlled by repeating steps, from a certain step t to step t+N, which is N steps after the certain step t (N is an integer of 2 or more), a first learning process that refers to the observed values ​​at each of two successive steps among the series of observed values; a second learning process using the series of observed values; The control object is controlled using a control model generated by a learning process including The following configuration is adopted.

[0039] As described above, according to the above configuration, a control object is controlled using the control model generated by a learning process including the first learning process and the second learning process. According to the inventor's findings, by using the first learning process and the second learning process, even if observation noise is included in the observation values ​​(observation data) related to the control object, a control model in which the influence of the observation noise is reduced can be generated. Therefore, according to the above configuration, even if observation noise is included in the observation data, the control object can be suitably controlled.

[0040] (Flow of information processing method S2) Next, the flow of the information processing method S2 according to this exemplary embodiment will be described with reference to Fig. 4. Fig. 4 is a flow diagram showing the flow of the information processing method S2. The information processing method S2 may also be called a control method. As shown in Fig. 4, the information processing method S2 includes a step (process) S21 of acquiring an observation value (observation data) and a step (process) S22 of controlling a control target.

[0041] (Step S21) In step S21, the acquisition unit 21 acquires a series of observation values ​​(observation data) relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more). The specific processing by the acquisition unit 21 has been described above, and therefore will not be described here.

[0042] (Step S22) Subsequently, in step S22, the control unit 22 a first learning process that refers to the observed values ​​at each of two successive steps among the series of observed values; a second learning process using the series of observed values; The control unit 22 controls the controlled object using a control model generated by a learning process including the following: The specific process performed by the control unit 22 has been described above, and therefore a description thereof will be omitted here.

[0043] Note that steps S21 and S22 may constitute online control of the control object. As an example, in step S21, the acquisition unit 21 may acquire observed values ​​at a certain step t', and in step S22, the control unit 22 may use the control model generated by referring to the observed values ​​of the control object acquired up to step t' for controlling the control object in steps after step t' (or steps after step t'). After step S22 is completed, the process may return to step S21, acquire observed values ​​at the next step, step t'+1, and in step S22, the control unit 22 may use the control model generated by referring to the observed values ​​of the control object acquired up to step t'+1 for controlling the control object in steps after step t'+1 (or steps after step t'+1).

[0044] (Effect of information processing method S2) As described above, in the information processing method S2, Obtaining a series of observations of a control object that is controlled by repeating steps, from a certain step t to step t+N, which is N steps after the certain step t (N is an integer of 2 or more), a first learning process that refers to the observed values ​​at each of two successive steps among the series of observed values; a second learning process using the series of observed values; The control object is controlled using a control model generated by a learning process including The above configuration provides the same effects as the information processing device 2.

[0045] Second Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0046] (Configuration of information processing system 100A) The configuration of an information processing system 100A according to this exemplary embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the information processing system 100A. As shown in Fig. 5, the information processing system 100A includes an information processing device 1A, and a robot 50 and an observation device 60 connected to the information processing device 1A via a network N. Here, the specific configuration of the network N does not limit this exemplary embodiment, but as an example, a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks can be used.

[0047] (Robot 50) The robot 50 is an example of an object to be controlled by the information processing device 1A. The robot 50 is a device that operates based on a control input supplied by the information processing device 1A, and is, for example, a two-wheeled mobile robot. However, the object to be controlled is not limited to a two-wheeled mobile robot, and may be other robots, etc. Furthermore, the robot 50 may have one or more arms and one or more joints.

[0048] Specific examples of the control input generated by the information processing device 1A and supplied to the robot 50 do not limit the present exemplary embodiment, but as an example, Velocity v of robot 50, acceleration dv / dt of robot 50 Angular velocity ω of the robot 50, angular acceleration dω / dt of the robot 50 The acceleration of the robot 50 is expressed as a "dotted v", and the angular acceleration of the robot 50 is expressed as a "dotted ω".

[0049] (Observation equipment 60) The observation device 60 is a device that observes the robot 50 and the environment around the robot 50. As an example, the observation device 60 includes: A camera that captures images of the robot 50 and the environment around the robot 50; one or more sensors attached to the robot 50; A receiving unit that acquires sensing data from one or more sensors attached to the robot 50 The observation device 60 is equipped with the above-mentioned cameras and sensors. Position (x,y) of robot 50 ·Robot 50 direction θ The angle of one or more joints of the robot 50 The observation device 60 observes the robot 50 and supplies the observation values ​​(observation data) obtained by the observation to the information processing device 1A. The observation by the observation device 60 is performed sequentially, for example. In step t, the observation values ​​regarding the robot 50 obtained by the observation device 60 are referred to as s t , or s obs tIt is sometimes written as:

[0050] (Configuration of information processing device 1A) Next, the configuration of the information processing device 1A according to this exemplary embodiment will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the information processing device 1A. As shown in Fig. 5, the information processing device 1A includes a control unit 10A, a storage unit 20A, a communication unit 30, and an input / output unit 40.

[0051] (Communication unit 30) The communication unit 30 communicates with devices external to the information processing device 1A via a network. For example, the communication unit 30 transmits control inputs supplied from the control unit 10A to the robot 50, and supplies observation values ​​related to the robot 50 received from the observation device 60 to the control unit 10A.

[0052] (Input / output section 40) The input / output unit 40 is configured to include at least one of input / output devices such as a keyboard, a mouse, a display, a printer, and a touch panel. Alternatively, the input / output unit 40 may be configured to have input / output devices such as a keyboard, a mouse, a display, a printer, and a touch panel connected to it. In this configuration, the input / output unit 40 accepts various types of information input to the information processing device 1A from the connected input devices. Furthermore, the input / output unit 40 outputs various types of information to the connected output devices under the control of the control unit 10A. An example of the input / output unit 40 is an interface such as a USB (Universal Serial Bus).

[0053] (Storage unit 20A) The storage unit 20A stores various data referenced by the control unit 10A and various data generated by the control unit 10A. Observations OBS {s1,s2,...} Control input values ​​CI {u1,u2,...} 1 step parameter P1 N-step parameter PN Diffusion coefficient parameter DC ·Variance-covariance matrix VC is stored.

[0054] (Observed value OBS) The observation values ​​OBS are sequential observation values ​​of the robot 50 obtained by the observation device 60. For example, the observation values ​​OBS may include an observation value s1 in the first step, step 1, an observation value s2 in the second step, step 2, an observation value s3 in the t-th step, step t, and so on. t etc., a series of observations {s1,s2,...,s t ,···} It is sometimes written as.

[0055] (Control input value CI) The control input value CI is a value of a sequential control input derived by the robot control unit 22, which will be described later. The control input value CI is transmitted to the robot 50 via the communication unit 30 as described above, and the robot 50 performs an operation based on the control input value CI. For example, the control input value CI may be a control input value u1 in step 1, which is the first step; a control input value u2 in step 2, which is the second step; a control input value u3 in step t, which is the t-th step; t etc., a series of control input values {u1,u2,...,u t ,···} The process of deriving the control input value CI will be described later.

[0056] (1 step parameter P1) The one-step parameter P1 is an observation value of the robot 50, and is a series of observation values ​​{s t ,···,s t+N}, the parameter P1 is obtained by the first learning process by referring to the observed values ​​in each of two successive steps.t and the observed value s t+1 The one-step parameters P1 include, for example, the following parameters obtained by the first learning process with reference to the above. Observation s t and the observed value s t+1 The diffusion coefficient (also denoted as ψ1) obtained by the first learning process with reference to Observation s t and the observed value s t+1 The variance-covariance matrix (also denoted as Σ1) obtained by the first learning process with reference to Includes:

[0057] (N step parameter PN) The N-step parameter PN is an observation value of the robot 50, and is a series of observation values ​​{s t ,···,s t+N}. For example, the N-step parameter PN is a parameter obtained by a second learning process using a series of observations {s t ,···,s t+N}. More specifically, the N-step parameters PN include, for example, A set of observations {s t ,···,s t+N}, the diffusion coefficient (ψ N Also written as A set of observations {s t ,···,s t+N}, the variance-covariance matrix (Σ N Also written as Includes:

[0058] (Diffusion coefficient parameter DC) 5 indicates a diffusion coefficient parameter derived (generated, updated) by a later-described generating unit 12 with reference to a one-step parameter P1 and an N-step parameter PN. The process of deriving the diffusion coefficient parameter DC by the generating unit 12 will be described later.

[0059] (variance-covariance matrix VC) 5 indicates a variance-covariance matrix derived (generated, updated) by the generating unit 12, which will be described later, with reference to the one-step parameter P1 and the N-step parameter PN. The process of deriving the variance-covariance matrix VC by the generating unit 12 will be described later.

[0060] (Technical Overview of Processing by Information Processing Device 1A) Before describing each part of the control unit 10A of the information processing device 1A in detail, a technical overview of the processing by the information processing device 1A will be described. This technical overview includes the knowledge obtained by the inventors regarding the invention described in this specification.

[0061] (Robot motion modeling) In the present exemplary embodiment, the motion of robot 50 is described by the following stochastic differential equation:

number

[0062] In the above formula 1, f(st ,u t ) and σ(s t ,u t ) argument s t is a parameter that represents the state of the robot 50. t For example, the observed value s t However, this is not limited to this. As will be described later, the observed value s t , the true state s t and the observation noise n t When expressed as a sum of the above argument s t is the true state s t On the other hand, f(s t ,u t ) and σ(s t ,u t ) argument u t is the control input u t represents.

[0063] (observation noise) The second term in the above formula 1, σ(s t ,u t ) is obtained (generated, updated) by learning with reference to observed data, ideally, as shown in the upper part of Figure 6, the control input u t The first term f(s) of the above formula 1 is determined according to t ,u t )dt and the state change ds obtained by observation t Based on the difference with σ(s t ,u t ) can be estimated.

[0064] However, in reality, observation data contains observation noise (in other words, observation error), and the influence of this observation noise causes σ(s t ,u t ) is difficult to estimate correctly. t ,u t) is often larger than the true value due to the influence of observation noise, which can result in a problem that it is difficult to appropriately control the robot 50. The information processing device 1A according to this exemplary embodiment provides a method for reducing the influence of such observation noise and appropriately controlling the robot 50.

[0065] (Definition of robot motion and diffusion coefficient estimation model) Below, the motion of the robot 50 will be defined anew, and a diffusion coefficient estimation model will be defined. First, the motion of the robot 50 will be defined anew by the following stochastic differential equation.

number

[0066] As an example, when the robot 50 is a two-wheeled mobile robot, the movement of the robot 50 is expressed as follows:

number

[0067] When the robot 50 is a two-wheeled mobile robot, the expression of the motion of the robot 50 is not limited to the above example. As an example, the diffusion coefficient can be decomposed into components of motion. For example, when the diffusion coefficient is decomposed into translation and rotation, the motion of the robot 50 can be expressed as

number

[0068] In the above expression of the motion of the robot 50, the diffusion coefficient D ψ is modeled as follows:

number

number

[0069] On the other hand, the bold φ(s,t) on the right-hand side of Eq. T represents the transpose of the basis vector. Here, the basis vector includes, for example, the basis corresponding to each component [x, y, θ] included in the observation value and each component [v, ω, v with a dot, ω with a dot] included in the control input,

number

[0070] (Elements that make up observation information) Next, we will explain the elements that make up the observation information (observation values, observation data) obtained by the observation device 60. The observation values ​​at steps t and t+1 are respectively denoted by s obs t and s obs t+1 , the difference between these observations is expressed as follows:

number

number

[0071] where the control noise e u As an example,

number

number

[0072] (Removal of the effects of observation noise based on the inventor's knowledge) As explained above, when learning is performed in an environment with observation noise, the model obtained as a result of the learning may be affected by noise e p This will include the impact of

number

number

number

[0073] The inventors have found that by using the diffusion coefficient given by one dt and the diffusion coefficient given by Ndt, which is N dt, it is possible to obtain a diffusion coefficient from which the influence of noise has been removed. More specifically,

number

number

number

[0074] In the above discussion, the control noise e associated with the translational motion of the robot 50 u,p , and the observation noise e associated with the translational motion of the robot 50 obs,p However, this is not intended to limit the present exemplary embodiment, and the control noise e associated with the rotational movement of the robot 50 is u,θ , and the observation noise e associated with the rotational motion of the robot 50 obs,θ The same applies to the other movements of the robot 50.

[0075] (Control unit 10A) Returning to FIG. 5, the description of each unit included in the control unit 10A will continue. As shown in FIG. 5, the control unit 10A includes an acquisition unit 11, a generation unit 12, and a robot control unit 22. Here, the acquisition unit 11 has the functions of the acquisition unit 11 described in exemplary embodiment 1 and the functions of the acquisition unit 21. For this reason, the acquisition unit 11 may also be referred to as the acquisition unit 11 (21). The generation unit 12 has the functions of the generation unit 12 described in exemplary embodiment 1. The robot control unit 22 has the functions of the control unit 22 described in exemplary embodiment 1. In the following, descriptions of matters that overlap with those described in exemplary embodiment 1 may be omitted.

[0076] (Acquisition part 11(21)) The acquisition unit 11 (21) acquires a series of observation values ​​(observation data) relating to the controlled object (robot 50) from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more). obs t ,···,s obs t+N ) to get the

[0077] Here, specific examples of observed values ​​related to the control object (robot 50) do not limit this exemplary embodiment, but include, for example, the position (x, y) of the control object, the velocity of the control object, the orientation (θ) of the control object, the angular velocity of the control object, and the angle of the joints of the control object. Also, for example, the observation values ​​are acquired by an observation device 60. However, these examples do not limit this exemplary embodiment.

[0078] (Generation part 12) The generation unit 12 generates a control model used to control the controlled object (robot 50) by The series of observations (s obs t ,···,s obs t+N ), the observations at each of two successive steps (e.g., s obs t ,s obs t+1 ) a first learning process referring to The series of observations (s obs t ,···,s obs t+N ) and the second learning process The generated image is generated by a learning process including:

[0079] As shown in FIG. 5, the generation unit 12 according to this exemplary embodiment includes, for example, a first learning unit 121, a second learning unit 122, a diffusion coefficient parameter calculation unit 123, and a variance-covariance matrix calculation unit 124.

[0080] The first learning unit 121 executes the first learning process described above. The second learning unit 122 executes the second learning process described above. The diffusion coefficient parameter calculation unit 123 calculates the diffusion coefficient ψ by referring to the results of the first learning process and the second learning process. The variance-covariance matrix calculation unit 124 calculates the variance-covariance matrix Σ by referring to the results of the first learning process and the second learning process.

[0081] Fig. 7 is a block diagram showing an example of the configuration of the generation unit 12 according to this exemplary embodiment. As shown in Fig. 7, the generation unit (estimation unit) 12 includes a one-step learning unit (first learning unit) 121, an N-step learning unit (second learning unit) 122, a parameter calculation unit (diffusion coefficient parameter calculation unit) 123, and a variance-covariance matrix calculation unit 124. Note that the arrows in the figure simply indicate the direction of data flow and do not exclude bidirectionality.

[0082] As shown in FIG. 7, the generation unit 12 includes: ·Control input values ​​{u1,u2,...} observations {s1, s2,...} is input and supplied to the 1-step learning device 121 and the N-step learning device 122. Here, the observed value is, as described above, {s obs 1,s obs 2,···}.

[0083] (One-step learning unit (first learning unit) 121) As shown in FIG. 7, the one-step learning device 121 includes a one-step observation diffusion coefficient calculation unit 121a and a parameter update amount calculation unit 121c.

[0084] (One-step observation diffusion coefficient calculation unit 121a) The one-step observation diffusion coefficient calculation unit 121a calculates the state change amount in one step for the robot 50. ds t = s obs t+1 -s obs t and the expected movement f(s t ,u t )dt (the first term of Equation 2 above), the one-step observed diffusion coefficient D obs,t,i As an example, the one-step observation diffusion coefficient calculation unit 121a calculates the error e t of,

number

number

number

number

[0085] (Parameter update amount calculation unit 121c) The parameter update amount calculation unit 121c calculates the one-step observed diffusion coefficient D obs,t,i As an example, the parameter update amount calculation unit 121c calculates the diffusion coefficient parameter ψ (or ψ with a hat) from the one-step observed diffusion coefficient D obs,t,i from

number

[0086] First, the diffusion coefficient Dψ is modeled as follows (corresponding to Equation 5 above):

number

number

number

[0087] The expected value E and variance V of the error ε are

number

number

number

number

number

number

[0088] The parameter update amount calculation unit 121c calculates the one-step observed diffusion coefficient D obs,t,i The diffusion coefficient parameter ^ψ (with a hat) derived from the equation and the basis φ (or basis vector Φ) are used to calculate the one-step estimated diffusion coefficient ~D ψ (s,u,dt) (D with tilde) ψ (s, u, dt), or D(s, u, dt) with a tilde. This one-step estimated diffusion coefficient corresponds to the second term on the left side of Equation 17 above.

[0089] The parameter update amount calculation unit 121c calculates the one-step observed diffusion coefficient D obs,t,i The diffusion coefficient parameter ψ (or ψ with a hat) derived from is stored in the storage unit 20A as a one-step parameter P1. The diffusion coefficient parameter derived by the parameter update amount calculation unit 121c may also be expressed as ψ1.

[0090] Furthermore, the parameter update amount calculation unit 121c calculates the one-step observed diffusion coefficient D obs,t,i See the variance-covariance matrix (or simply variance)

number

[0091] (N-step learning device (second learning unit) 122) On the other hand, as shown in FIG. 7, the N-step learning device 122 includes an N-step observation diffusion coefficient calculation unit 122a, an N-step control input calculation unit 122b, and a parameter update amount calculation unit 122c.

[0092] (N-step observation diffusion coefficient calculation unit 122a) The N-step observation diffusion coefficient calculation unit 122a calculates the state change amount of the robot 50 over N steps. ds t = s obs t+N -s obs t and the expected movement amount determined deterministically by the structure and dynamics of the robot 50 (the sum of the first term of the above-mentioned formula 2 over N steps).

number

number

number

number

number

[0093] (N-step control input calculation unit 122b) The N-step control input calculation unit 122b calculates the average of the control input values ​​from the series of control input values ​​{u1, u2, . . .}.

number

number

[0094] (Parameter update amount calculation unit 122c) The parameter update amount calculation unit 122c calculates the N-step observed diffusion coefficient D obs,t,i As an example, the parameter update amount calculation unit 122c calculates the diffusion coefficient parameter ^ψ (or ψ with a hat) by using the N-step observation diffusion coefficient D obs,t,i from

number

number

[0095] First, the diffusion coefficient Dψ is modeled as follows (corresponding to Equation 5 above):

number

number

number

[0096] The expected value E and variance V of the error ε are

number

number

number

number

number

number

number

number

number

[0097] Note that the autocorrelation matrix Ω ρ Calculating the inverse matrix of requires a high calculation cost. Therefore, the parameter update amount calculation unit 122c may be configured to derive an approximate solution as described below.

[0098] First, Omega ρ Consider an approximate solution by taking advantage of the fact that the influence of is local. Assuming an appropriate vector r,

number

number

number

number

number

number

number

[0099] The parameter update amount calculation unit 122c calculates the N-step observed diffusion coefficient D obs,t,i The N-step estimated diffusion coefficient ~D is calculated using the diffusion coefficient parameter ^ψ (ψ with a hat) derived from ψ (s,u,Ndt) (D with tilde) ψ (s, u, Ndt), also denoted as D(s, u, Ndt) with a tilde, is obtained. The N-step estimated diffusion coefficient corresponds to the first term on the left side of Equation 17 above.

[0100] The parameter update amount calculation unit 122c calculates the N-step observed diffusion coefficient D obs,t,i The diffusion coefficient parameter ψ (or ψ with a hat) derived from is stored in the storage unit 20A as an N-step parameter PN. N It is sometimes written as:

[0101] Furthermore, the parameter update amount calculation unit 122c calculates the N-step observation diffusion coefficient D obs,t,i See the variance-covariance matrix (or simply variance)

number

[0102] (Parameter calculation unit 123) The parameter calculation unit 123 derives a diffusion coefficient parameter from which observation noise has been removed (reduced) by referring to the one-step parameter P1 derived by the one-step learning device 121 and the N-step parameter PN derived by the N-step learning device 122. As an example, the parameter calculation unit 123 The one-step estimated diffusion coefficient ∼D(s,u,dt) (D(s,u,dt) with a tilde) derived by the one-step learning unit 121, Using the N-step estimated diffusion coefficients ∼D(s, u, Ndt) (D(s, u, Ndt) with a tilde) derived by the N-step learning device 122,

number

[0103] Alternatively, the parameter calculation unit 123 may be expressed as executing a process to cancel the influence of observation noise by utilizing the difference in the contribution of observation noise between the parameters (diffusion coefficient parameters) learned using data from one step and the parameters (diffusion coefficient parameters) learned using data from N steps. The derived diffusion coefficient D(s, u) is stored in the storage unit 20A as a diffusion coefficient parameter DC and is also supplied to the robot control unit 22, which will be described later.

[0104] (Variance-covariance matrix calculation unit 124) The variance-covariance matrix calculation unit 124 calculates the variance-covariance matrix of the estimation model (the variance-covariance matrix of the diffusion coefficient D(s,u)). Here, the variance of the estimation model (the variance of the diffusion coefficient D(s,u)) is expressed as follows:

number

number

[0105] In addition, in the third line of Equation 63, an approximation is performed when the magnitude of the observation noise is not excessively large. More specifically, the following approximation is performed:

number

number

[0106] The above processing by the variance-covariance matrix calculation unit 124 is as follows: The one-step estimated diffusion coefficient ∼D(s,u,dt) (D(s,u,dt) with a tilde) derived by the one-step learning unit 121, The N-step estimated diffusion coefficient ∼D(s,u,Ndt) (D(s,u,Ndt) with a tilde) derived by the N-step learner 122, It can also be expressed as deriving the variance-covariance matrix used in controlling the controlled object (robot 50) by referring to the above.

[0107] (Robot control unit 22) Returning to Fig. 5, the robot control unit 22 will be described. As shown in Fig. 5, the robot control unit 22 includes an evaluation function calculation unit 221 and a control input determination unit 222. As an example, the robot control unit 22 performs risk-sensitive stochastic control with reference to the evaluation function, as will be described below.

[0108] (Evaluation function calculation unit 221) The evaluation function calculation unit 221 calculates the risk-sensitive evaluation function J by referring to the diffusion coefficient (or diffusion coefficient parameter) D(s, u) calculated by the parameter calculation unit 123 and the variance-covariance matrix calculated by the variance-covariance matrix calculation unit 124.

number

[0109] (Control input determination unit 222) The control input determination unit 222 refers to the risk-sensitive evaluation function J and controls the robot 50 so as to minimize the evaluation function. More specifically, it determines a control input to the robot 50 that minimizes the evaluation function. This allows the information processing device 1A to perform control such as moving the robot 50 slowly when the robot 50 is near an obstacle and moving quickly when the robot 50 is not near an obstacle.

[0110] (Processing flow by information processing device 1A) Next, the flow of processing by the information processing device 1A will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing an example of the flow of processing by the information processing device 1A. In the example shown in Fig. 8, the information processing device 1A shows the flow of online control (online learning) of the robot 50. Here, online control refers to control that sequentially repeats the acquisition of observation values, updating (learning) of a control model that references the observation values, and control of the control target using the updated control model.

[0111] (Process S11 (S21)) In step S11 (S21), the acquisition unit 11 (21) acquires an observation value regarding the robot 50. As an example, the acquisition unit 11 (21) acquires an observation value s t (s obs t (also written as ".

[0112] (Process S12) Subsequently, in step S12, the generation unit 12 generates a control model by referring to a series of observation values ​​including the observation values ​​acquired in step S11 (S21). As an example, the generation unit 12 generates a control model (a diffusion coefficient estimation model D) to be used for controlling the robot 50. ψ )of, A series of observations (s obs t-N ,···,s obs t ), the observations at each of two successive steps (e.g., s obs t-1 ,s obs t ) and the first learning process (deriving D(s,u,dt) with a tilde) A series of observations (s obs t-N ,···,s obs t ) and the second learning process (deriving D(s,u,Ndt) with a tilde) The diffusion coefficient estimation model D is generated by a learning process including ψ The details of the derivation (generation, update) process of (in other words, the derivation, generation, and update process of the diffusion coefficient parameter ψ) are as described above.

[0113] However, as in this example, when the information processing device 1A performs online control, the one-step learning device (first learning unit) 121 of the generation unit 12 uses the following as one-step parameters:

number

number

number

number

number

[0114] Then, the parameter calculation unit 123 calculates the one-step parameter P1(ψ1) derived by the one-step learning device 121 and the N-step parameter PN(ψ N ) to derive diffusion coefficient parameters from which observation noise has been removed (reduced). Furthermore, the variance-covariance matrix calculation unit 124 calculates the variance-covariance matrix of the diffusion coefficients D(s,u).

[0115] (Step S22) Subsequently, in step S22, the evaluation function calculation unit 221 of the robot control unit 22 refers to the diffusion coefficient (or diffusion coefficient parameter) D(s, u) calculated by the parameter calculation unit 123 and the variance-covariance matrix calculated by the variance-covariance matrix calculation unit 124, and calculates the risk-sensitive evaluation function J

number

[0116] Then, the control input determination unit 222 of the robot control unit 22 refers to the risk-sensitive evaluation function J and controls the robot 50 so as to minimize the evaluation function. More specifically, it determines a control input to the robot 50 that minimizes the evaluation function. The value of the control input is supplied to the robot 50 and is also referred to in step S12 in the next step.

[0117] (Step S23) Subsequently, in step S23, the observation device 60 observes the robot 50 and the environment around the robot 50, and supplies the observation values ​​(observation data) obtained by the observation to the information processing device 1A. The observation values ​​are used in step S11 (S21) relating to the next step as, for example, observation values ​​s t+1 (s obs t+1 (also written as ".

[0118] In this way, in the information processing device 1A that performs the processing according to the above example, The acquisition unit 11 (21) acquires the observed value (s obs t ) are obtained sequentially, The robot control unit 22 uses the control model generated by the generation means by referring to the observation values ​​of the control object acquired up to a certain step t', for controlling the control object in steps after the certain step t' (or in steps after the certain step t').

[0119] (Effects of information processing device 1A) As described above, in the information processing device 1A, A series of observations (e.g., s) at each step from a certain step t to step t+N (N is an integer equal to or greater than 2) N steps after the certain step t, which is a control object controlled by repeating steps. obs t ,···,s obs t+N ) and A control model used to control the control target, Among the series of observations, the observations at each of two successive steps (e.g., s obs t ,s obs t+1 ) a first learning process (e.g., deriving the diffusion coefficient ψ1) with reference to The series of observations (s obs t,···,s obs t+N ) (e.g., the diffusion coefficient ψ N (Derivation of) and Generated by a learning process including The following configuration is adopted.

[0120] As described above, according to the above configuration, the control model is generated by a learning process including the first learning process and the second learning process. As described above, by using the first learning process and the second learning process, even if observation noise is included in the observation values ​​(observation data) related to the control object, a control model in which the influence of the observation noise is reduced can be generated. Therefore, according to the above configuration, it is possible to provide a technology that can suitably control the control object even if the observation data includes observation noise.

[0121] <Examples and Additional Notes Regarding Information Processing Device 1A> Next, an embodiment of the information processing device 1A and additional notes regarding the information processing device 1A will be described with reference to FIGS. 9 to 14. FIG. 9 shows the N-step learning device (second learning unit) 122. obs t ,···,s obs t+N ) when N, which specifies the number of steps in the diffusion coefficient estimation model, is changed (standard deviation of the diffusion coefficient D(s,u)) σ N The upper part of FIG. 9 shows the observed value s of the state of the robot 50. obs Standard deviation σ obs and the standard deviation σ of the control input u to the robot 50 ctrl The ratio of (σ obs / σ ctrl ) is about 0.15. On the other hand, the bottom part of FIG. obs / σ ctrl 9 is a graph showing the case where N=2 or N=3 is preferable as long as the observation noise is not too large.

[0122] The N-step learning device (second learning unit) 122 calculates the above σ when the value of N is changed. N Alternatively, the N-step learning device (second learning unit) 122 may be configured to determine a change in the value of the ratio (σ obs / σ ctrl ) and N, and obs / σ ctrl ) may be configured to set an appropriate N depending on the value of

[0123] Next, with reference to FIGS. 10 and 11, the diffusion coefficient estimation model (diffusion coefficient D p The error reduction effect in the above case will be explained. The problem setting that is the premise of Figs. 10 and 11 is as follows. A two-wheeled mobile robot was used as the robot 50. T sets of control inputs (v, ω) for the robot 50 are generated. In other words, T steps of control inputs are generated. The robot's motion was simulated M times using a series of control inputs, and a diffusion coefficient estimation model was learned for each simulation. In other words, M different models were learned due to the influence of random numbers. The value of M was set to M=1000. As a diffusion coefficient model

number

number

number

[0124] The upper part of FIG. 10 shows the diffusion coefficient (D p 10 is a graph showing the relationship between the mean square error (Mean Square Error) of the 1-step learning unit (first learning unit) 121 and the number of data sets T. As shown in the upper part of FIG. 10, the error is significantly reduced by referring to the learning results of the 1-step learning unit (first learning unit) 121 and the learning results of the N-step learning unit (second learning unit) 122.

[0125] On the other hand, the bottom of Figure 10 shows the diffusion coefficient parameters (ψ1, ψ v ,ψ ω 10 is a graph showing the relationship between each value of ψ1 and the number of data sets T. As shown in the lower part of FIG. 10, the value of the diffusion coefficient parameter (ψ1: constant term of the diffusion coefficient) is reduced by referring to the learning result of the 1-step learning device (first learning unit) 121 and the learning result of the N-step learning device (second learning unit) 122. This indicates that the diffusion coefficient parameter from which the influence of observation noise has been removed can be correctly estimated by referring to the learning result of the 1-step learning device (first learning unit) 121 and the learning result of the N-step learning device (second learning unit) 122.

[0126] FIG. 11 shows the diffusion coefficient estimation model (diffusion coefficient D θ11 is a graph showing the error reduction in the case of the sparsely packed ...

[0127] The upper part of FIG. 11 shows the diffusion coefficient (D θ 11 is a graph showing the mean square error (Mean Square Error) of the 1-step learning unit (first learning unit) 121 and the change with respect to the number of data sets T. As shown in the upper part of FIG. 11, the error is significantly reduced by referring to the learning results of the 1-step learning unit (first learning unit) 121 and the learning results of the N-step learning unit (second learning unit) 122.

[0128] On the other hand, the bottom part of Fig. 11 shows the diffusion coefficient parameters (ψ1, ψ v ,ψ ω 11 is a graph showing the relationship between each value of ψ1 and the number of data sets T. As shown in the lower part of FIG. 11, the value of the diffusion coefficient parameter (ψ1: constant term of the diffusion coefficient) is reduced by referring to the learning result of the 1-step learning device (first learning unit) 121 and the learning result of the N-step learning device (second learning unit) 122. This indicates that the diffusion coefficient parameter from which the influence of observation noise has been removed can be correctly estimated by referring to the learning result of the 1-step learning device (first learning unit) 121 and the learning result of the N-step learning device (second learning unit) 122.

[0129] In Figure 12, unlike Figures 10 and 11, the observation noise is

number

[0130] The upper part of Fig. 12 is a graph showing the relationship between the mean square error of the diffusion coefficient derived by the parameter calculation unit 123 and the number of data sets T. As shown on the left side of Fig. 12, the error is significantly reduced by referring to the learning results of the 1-step learning unit (first learning unit) 121 and the learning results of the N-step learning unit (second learning unit) 122.

[0131] On the other hand, the bottom part of Fig. 12 shows the diffusion coefficient parameters (ψ1, ψ v ,ψ ω 12 is a graph showing the relationship between each value of the diffusion coefficient parameters (ψ1, ψ v ,ψ ω ) are significantly reduced. This indicates that by referring to the learning results of the 1-step learning device (first learning unit) 121 and the N-step learning device (second learning unit) 122, the diffusion coefficient can be correctly estimated even when the observation noise depends on the velocity v and the angular velocity ω.

[0132] In FIG. 13, the variance of the estimation model (variance of the diffusion coefficient) calculated by the variance-covariance matrix calculation unit 124 is The variance obtained numerically by the first line of Equation 63 using T=10000 steps and M=1000 models ("std calculated by M sampled models" shown in Figure 13) and The variance obtained approximately by the second line of Equation 63 ("proposed std" in Figure 13) The variance obtained approximately by the third line of Equation 63 (shown in Figure 13)

number

[0133] The upper graph of Figure 13 (σ obs / σ ctrl As can be seen from the graph when σ is about 0.15), when the observation noise is relatively small, the approximation can be performed appropriately using either the approximation formula in the second or third line of Equation 63. However, the lower part of Figure 13 (σ obs / σ ctrl As can be seen from the graph in which σ is about 0.75, when the observation noise is relatively large, it is preferable to use the approximation formula in the second line of Equation 63 rather than the approximation formula in the third line.

[0134] In FIG. 14, in the N-step learning device (N-step estimator, second learning unit) 122, the autocorrelation matrix Ω ρ The calculation cost ("Approximation (Proposed)" in Figure 14) when applying the approximation method (Equation 59, Equation 60) to the autocorrelation matrix Ω ρ The calculation cost was compared with that of the case where the approximation method was not applied (Equation 53) (“Accurate” in FIG. 14). As can be seen from Figure 14, the autocorrelation matrix Ω ρ By applying the approximation method (Equation 59, Equation 60) to the above equation, the calculation cost can be significantly reduced.

[0135] [Software implementation example] Some or all of the functions of the information processing devices 1, 2, 1A (hereinafter also referred to as "each of the above devices") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0136] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 15. Figure 15 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0137] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0138] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0139] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0140] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0141] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0142] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0143] (Appendix A1) an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; A generating means for generating the image by a learning process including An information processing device comprising:

[0144] (Appendix A2) The generating means a result of the first learning process; and The result of the second learning process; The diffusion coefficients used in the control model are determined by referring to 10. The information processing device according to claim 1,

[0145] (Appendix A3) The generating means The diffusion coefficient is determined by calculating a difference between the result of the first learning process and the result of the second learning process. 10. The information processing device according to claim 9, wherein the information processing device is a

[0146] (Appendix A4) the diffusion coefficient is expressed by a predetermined basis and a diffusion coefficient parameter that is a coefficient of the basis, The generating means determines the diffusion coefficient parameter by referring to the result of the first learning process and the result of the second learning process. 10. The information processing device according to claim 9, wherein the information processing device is a device for processing information.

[0147] (Appendix A5) The generating means a result of the first learning process; and The result of the second learning process; and deriving a variance-covariance matrix to be used in controlling the controlled object by referring to the 10. The information processing device according to claim 9, wherein the information processing device is a device for processing a plurality of data.

[0148] (Appendix A6) the acquiring means successively acquires the observed values ​​at each step; a control means that uses the control model generated by the generation means with reference to observed values ​​related to the control object acquired up to a certain step t', for controlling the control object in steps after the certain step t'; 10. The information processing device according to claim 9, further comprising:

[0149] (Appendix A7) an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; a control means for controlling the controlled object using a control model generated by a learning process including the An information processing device comprising:

[0150] (Appendix A8) the acquiring means successively acquires the observed values ​​at each step; The control means uses the control model generated by referring to the observation values ​​related to the control object acquired up to a certain step t' for controlling the control object in steps after the certain step t'. 10. The information processing device according to claim 7,

[0151] [Appendix B] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0152] (Appendix B1) an acquisition process in which at least one processor acquires a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process performed by the at least one processor, the first learning process referring to observation values ​​at each of two successive steps among the series of observation values; the at least one processor performs a second learning process with reference to the series of observed values; a generation process in which the at least one processor generates the image by a learning process including: An information processing method comprising:

[0153] (Appendix B2) In the generation process, the at least one processor a result of the first learning process; and The result of the second learning process; The diffusion coefficients used in the control model are determined by referring to 1. The information processing method described in Appendix B1.

[0154] (Appendix B3) In the generation process, the at least one processor The diffusion coefficient is determined by calculating a difference between the result of the first learning process and the result of the second learning process. 1. The information processing method described in Appendix B2.

[0155] (Appendix B4) the diffusion coefficient is expressed by a predetermined basis and a diffusion coefficient parameter that is a coefficient of the basis, In the generation process, the at least one processor determines the diffusion coefficient parameter by referring to a result of the first learning process and a result of the second learning process. 1. An information processing method according to Appendix B2 or B3.

[0156] (Appendix B5) In the generation process, the at least one processor a result of the first learning process; and The result of the second learning process; and deriving a variance-covariance matrix to be used in controlling the controlled object by referring to the 1. An information processing method according to any one of Appendices B2 to B4.

[0157] (Appendix B6) In the acquisition process, the at least one processor sequentially acquires the observed values ​​at each step; a control process in which the at least one processor uses the control model generated by the generation process with reference to observation values ​​related to the control object acquired up to a certain step t', for control of the control object in steps after the certain step t'; An information processing method according to any one of appendices B1 to B5, comprising:

[0158] (Appendix B7) an acquisition process in which the at least one processor acquires a series of observation values ​​relating to a control target that is controlled by repeating steps, from a certain step t to a step t+N that is N steps after the certain step t (N is an integer of 2 or more); a first learning process performed by the at least one processor, the first learning process referring to observation values ​​at each of two successive steps among the series of observation values; the at least one processor performs a second learning process with reference to the series of observed values; a control process in which the at least one processor controls the control target using a control model generated by a learning process including the steps of: An information processing method comprising:

[0159] (Appendix B8) In the acquisition process, the at least one processor sequentially acquires the observed values ​​at each step; In the control process, the at least one processor uses the control model generated by referring to the observation values ​​related to the controlled object acquired up to a certain step t' for controlling the controlled object in steps after the certain step t'. An information processing method as described in Appendix B7.

[0160] [Appendix C] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0161] (Appendix C1) A program that causes a computer to function as an information processing device, The computer an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; A generating means for generating the image by a learning process including An information processing program that functions as a

[0162] (Appendix C2) The generating means a result of the first learning process; and The result of the second learning process; The diffusion coefficients used in the control model are determined by referring to An information processing program as described in Appendix C1.

[0163] (Appendix C3) The generating means The diffusion coefficient is determined by calculating a difference between the result of the first learning process and the result of the second learning process. An information processing program as described in Appendix C2.

[0164] (Appendix C4) the diffusion coefficient is expressed by a predetermined basis and a diffusion coefficient parameter that is a coefficient of the basis, The generating means determines the diffusion coefficient parameter by referring to the result of the first learning process and the result of the second learning process. An information processing program according to Appendix C2 or C3.

[0165] (Appendix C5) The generating means a result of the first learning process; and The result of the second learning process; and deriving a variance-covariance matrix to be used in controlling the controlled object by referring to the An information processing program according to any one of appendices C2 to C4.

[0166] (Appendix C6) the acquiring means successively acquires the observed values ​​at each step; The computer a control means for using the control model generated by the generation means with reference to observed values ​​relating to the control object acquired up to a certain step t', for controlling the control object in steps after the certain step t'; The information processing program according to any one of appendices C1 to C5,

[0167] (Appendix C7) The computer an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; a control means for controlling the controlled object using a control model generated by a learning process including the An information processing program that functions as a

[0168] (Appendix C8) the acquiring means successively acquires the observed values ​​at each step; The control means uses the control model generated by referring to the observation values ​​related to the control object acquired up to a certain step t' for controlling the control object in steps after the certain step t'. An information processing program as described in Appendix C7.

[0169] [Appendix D] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0170] (Appendix D1) at least one processor, an acquisition process for acquiring a series of observation values ​​for a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; A generation process that generates a signal by a learning process including An information processing device that executes the above.

[0171] The information processing device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes.

[0172] (Appendix D2) In the generation process, the at least one processor a result of the first learning process; and The result of the second learning process; The diffusion coefficients used in the control model are determined by referring to 10. The information processing device according to claim 9, wherein the information processing device is an information processing device according to claim 1.

[0173] (Appendix D3) In the generation process, the at least one processor The diffusion coefficient is determined by calculating a difference between the result of the first learning process and the result of the second learning process. 10. The information processing device according to claim 9, wherein the information processing device is an information processing device according to claim 1, wherein

[0174] (Appendix D4) the diffusion coefficient is expressed by a predetermined basis and a diffusion coefficient parameter that is a coefficient of the basis, In the generation process, the at least one processor determines the diffusion coefficient parameter by referring to a result of the first learning process and a result of the second learning process. An information processing device according to appendix D2 or D3.

[0175] (Appendix D5) In the generation process, the at least one processor a result of the first learning process; and The result of the second learning process; and deriving a variance-covariance matrix to be used in controlling the controlled object by referring to the An information processing device according to any one of appendices D2 to D4.

[0176] (Appendix D6) In the acquisition process, the at least one processor sequentially acquires the observed values ​​at each step; The at least one processor: A control process in which the control model generated by the generation process with reference to observed values ​​relating to the control object acquired up to a certain step t' is used to control the control object in steps after the certain step t'. The information processing device according to any one of appendices D1 to D5,

[0177] (Appendix D7) The at least one processor: an acquisition process for acquiring a series of observation values ​​for a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; a control process for controlling the controlled object using a control model generated by a learning process that executes the above-mentioned An information processing device that executes the above.

[0178] (Appendix D8) In the acquisition process, the at least one processor sequentially acquires the observed values ​​at each step; In the control process, the at least one processor uses the control model generated by referring to the observation values ​​related to the controlled object acquired up to a certain step t' for controlling the controlled object in steps after the certain step t'. 10. The information processing device according to claim 7,

[0179] [Appendix E] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0180] (Appendix E1) A program that causes a computer to function as an information processing device, The computer, an acquisition process for acquiring a series of observation values ​​for a control object controlled by repeating steps, from a certain step t to a step t+N N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; A generation process that generates a signal by a learning process including A non-transitory recording medium on which an information processing program for executing the above is recorded. [Explanation of symbols]

[0181] 1, 2, 1A ···Information processing device 11,21 ···Acquisition part 12,22 Control unit 121... First Learning Section 122...Second Learning Section 123 Diffusion coefficient parameter calculation section 124 ...Variance-covariance matrix calculation section

Claims

1. an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N, which is N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; A generating means for generating the image by a learning process including An information processing device comprising:

2. The generating means a result of the first learning process; and The result of the second learning process; The diffusion coefficients used in the control model are determined by referring to The information processing device according to claim 1 .

3. The generating means The diffusion coefficient is determined by calculating a difference between the result of the first learning process and the result of the second learning process. The information processing device according to claim 2 .

4. the acquiring means successively acquires the observed values ​​at each step; a control means that uses the control model generated by the generation means with reference to observed values ​​related to the control object acquired up to a certain step t', for controlling the control object in steps after the certain step t'; The information processing device according to claim 1 , further comprising:

5. an acquisition means for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N, which is N steps after the certain step t (N is an integer of 2 or more); a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; a control means for controlling the controlled object using a control model generated by a learning process including the An information processing device comprising:

6. the acquiring means successively acquires the observed values ​​at each step; The control means uses the control model generated by referring to the observation values ​​related to the controlled object acquired up to a certain step t' for controlling the controlled object in steps after the certain step t'. The information processing device according to claim 5 .

7. Obtaining a series of observation values ​​for a control target that is controlled by repeating steps, from a certain step t to a step t+N that is N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; and generating the data by a learning process including An information processing method comprising:

8. Obtaining a series of observation values ​​for a control target that is controlled by repeating steps, from a certain step t to a step t+N that is N steps after the certain step t (N is an integer of 2 or more); a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; Controlling the controlled object using a control model generated by a learning process including the following: An information processing method comprising:

9. A program that causes a computer to function as an information processing device, The program causes the computer to: an acquisition process for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N, which is N steps after the certain step t (N is an integer of 2 or more); a control model used to control the controlled object, a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; A generation process that generates a signal by a learning process including A program that executes the following.

10. A program that causes a computer to function as an information processing device, The program causes the computer to: an acquisition process for acquiring a series of observation values ​​relating to a control object controlled by repeating steps, from a certain step t to a step t+N, which is N steps after the certain step t (N is an integer of 2 or more); a first learning process that refers to the observed values ​​in each of two consecutive steps among the series of observed values; a second learning process that refers to the series of observed values; a control process for controlling the controlled object using a control model generated by a learning process including the A program that executes the following.

Images