Information processing device, information processing method, and information processing program

Abstract

This information processing device performs machine learning, and comprises: a conversion unit that uses a first model for converting observation into a latent variable to convert observation information ot indicating an observation result in step t into a latent variable zt; a selection unit that selects, as an input to a second model for predicting a future latent variable from the latent variable, one of (i) a first latent variable zt-n obtained through conversion by the conversion unit in step t-n and (ii) a second latent variable zt-n obtained by inputting a latent variable zt-2n to the second model; a prediction unit that predicts a latent variable zt by inputting the latent variable zt-n selected by the selection unit into the second model; and an update unit that updates model parameters of at least one of the first model and the second model by using the latent variable zt obtained through conversion by the conversion unit and the latent variable zt predicted by the prediction unit, to thereby enable support of decision making using a trained model.

Description

Information processing device, information processing method, and information processing program

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program.

[0002] There are known techniques for predicting the future state of an environment based on observational information observed in the environment. One example of such a technique is described in Non-Patent Document 1. Non-Patent Document 1 describes a method called PlaNet (Deep Planning Network) that combines a world model and reinforcement learning. PlaNet uses a method called latent overshooting that incorporates multi-step future predictions into learning. In latent overshooting, learning is performed using all results of multi-step future predictions.

[0003] Danijar Hafner, et al., "Learning Latent Dynamics for Planning from Pixels", 2019

[0004] In the technology described in Non-Patent Document 1, all results of multi-step future predictions are incorporated into learning, so there is a problem that the greater the number of steps in future predictions, the greater the amount of calculations becomes.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and one exemplary purpose thereof is to provide a technology that can reduce the amount of calculation required for learning a learning model for multi-step future prediction.

[0006] An information processing device according to an exemplary aspect of the present disclosure is configured to: ｔ and a first model that converts observations into latent variables, and ｔ the latent variable z ｔ and (i) a first latent variable z converted by the conversion means in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎand (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection means for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction means for predicting the latent variable z ｔ and the latent variable z predicted by the prediction means ｔ and an updating unit that updates model parameters of at least one of the first model and the second model using the model parameters.

[0007] An information processing method according to an exemplary aspect of the present disclosure includes: at least one processor generating observation information o indicating an observation result in step t; ｔ and the at least one processor uses a first model that converts observations into latent variables to obtain the observation information o ｔ the latent variable z ｔ and the at least one processor (i) converts the first latent variable z converted in the conversion process in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection process for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction process for predicting the latent variable z ｔ and the latent variable z predicted in the prediction process. ｔ and an update process for updating model parameters of at least one of the first model and the second model using the model parameters.

[0008] An information processing program according to an exemplary aspect of the present disclosure is a program that causes a computer to function as an information processing device, the program causing the computer to receive observation information o indicating an observation result in step t. ｔ and a first model that converts observations into latent variables, and ｔ the latent variable z ｔ and (i) a first latent variable z converted by the conversion means in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection means for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction means for predicting the latent variable z ｔ and the latent variable z predicted by the prediction means ｔ and an updating unit that updates model parameters of at least one of the first model and the second model using the model parameters.

[0009] According to one exemplary aspect of the present disclosure, an exemplary effect is provided in which a technology can be provided that can reduce the amount of calculation required for learning a learning model for multi-step future prediction.

[0010] 1 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 2 is a flow diagram showing a flow of an information processing method according to the present disclosure. FIG. 3 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 4 is a block diagram showing an example of a functional configuration and processing flow of an information processing device according to the present disclosure. FIG. 5 is a diagram showing an example of a transition of a latent variable obtained by one-step prediction and a transition of a latent variable obtained by multi-step prediction. FIG. 6 is a diagram showing an example of a transition of a latent variable predicted by a prediction unit in each step. FIG. 7 is a diagram showing an example of a transition of a latent variable predicted by a prediction unit in each step. FIG. 8 is a diagram showing an example of a processing flow performed by a prediction unit according to the present disclosure. FIG. 9 is a block diagram showing an example of a functional configuration and processing flow of an information processing device according to the present disclosure. FIG. 10 is a diagram for explaining an example of a learning process of an information processing device according to the present disclosure. FIG. 11 is a block diagram showing a configuration of an information processing device according to the present disclosure. FIG. 12 is a diagram showing an overview of a medical robot using an information processing device according to the present disclosure. FIG. 13 is a block diagram showing a configuration of a computer functioning as an information processing device according to the present disclosure.

[0011] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0012] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0013] (Configuration of information processing device) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes an acquisition unit 11, a conversion unit 12, a selection unit 13, a prediction unit 14, and an update unit 15.

[0014] The acquisition unit 11 acquires observation information o indicating the observation result at step t. ｔ The conversion unit 12 converts the observation information o acquired by the acquisition unit 11 into a latent variable using a first model that converts the observation into a latent variable. ｔ the latent variable z ｔ The selection unit 13 selects (i) the first latent variable z converted by the conversion unit 12 in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding the latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ The prediction unit 14 selects either the latent variable z ｔ－ｎ By inputting into the second model, the latent variable z ｔ The update unit 15 predicts the latent variable z ｔ and the latent variable z predicted by the prediction unit 14 ｔ and update model parameters of at least one of the first model and the second model using the above.

[0015] (Effects of the Information Processing Device) As described above, in the information processing device 1, the observation information o ｔ and a first model that converts observations into latent variables, and the observation information o ｔ the latent variable z ｔ and (i) the first latent variable z converted by the conversion unit 12 in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding the latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection unit 13 for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ and a prediction unit 14 predicting the latent variable z ｔ and the latent variable z predicted by the prediction unit 14 ｔ and an update unit 15 that updates model parameters of at least one of the first model and the second model using the above. Therefore, the information processing device 1 has an effect of reducing the amount of calculation required for learning a learning model for multi-step future prediction.

[0016] (Flow of Information Processing Method) The flow of information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of information processing method S1. As shown in Fig. 2, information processing method S1 includes an acquisition process S11, a conversion process S12, a selection process S13, a prediction process S14, and an update process S15.

[0017] In the acquisition process S11, at least one processor acquires observation information o indicating the observation result at step t. ｔ In the conversion process S12, the at least one processor converts the observation information o acquired in the acquisition process S11 into a latent variable using a first model that converts the observation into a latent variable. ｔ the latent variable z ｔIn the selection process S13, the at least one processor (i) selects the first latent variable z converted in the conversion process in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding the latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ In the prediction process S14, the at least one processor selects either the latent variable z ｔ－ｎ By inputting into the second model, the latent variable z ｔ In the update process S15, the at least one processor predicts the latent variable z ｔ and the latent variable z predicted in the prediction process S14. ｔ and update model parameters of at least one of the first model and the second model using the above.

[0018] (Effects of Information Processing Method) As described above, in the information processing method S1, at least one processor generates observation information o ｔ and the at least one processor uses a first model that converts observations into latent variables to obtain the observation information o ｔ the latent variable z ｔ and (i) in step t-n (n is a natural number equal to or greater than 1), the at least one processor converts the first latent variable z ｔ－ｎ and (ii) adding the latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection process for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction process for predicting the latent variable zｔ and the latent variable z predicted in the above prediction process. ｔ and updating model parameters of at least one of the first model and the second model using the first model and the second model. Therefore, the information processing method S1 has an effect of reducing the amount of calculation required for learning a learning model for multi-step future prediction.

[0019] Second Exemplary Embodiment A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be denoted by the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technical means shown in each drawing referenced to describe this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0020] <Overview of Information Processing Device> An information processing device 1A according to the present disclosure is a device that trains a learning model for future prediction. As an example of the learning model trained by the information processing device 1A, a latent variable z ｔ The future latent variable z from step t+n (n is a natural number greater than or equal to 1) ｔ＋ｎ or observation information o indicating the observation result at step t. ｔ and the past latent variable z at step t-n ｔ－ｎ From the latent variable z ｔ These models are inference models that infer, for example, the behavior a of the controlled object. ｔThe control object is also called an agent. Examples of the control object include, but are not limited to, a robot arm, a character operating in a video game, a drone, or a moving object such as an autonomous vehicle. The information processing device 1A according to the present disclosure is applicable to various fields such as automatic play of a video game, robot arm control, and physical simulation.

[0021] Observation information ｔ is action a ｔ The observation information o is data showing the observation results in the environment where the control target is placed. ｔ As an example, the image data of the tth frame (time t) of the environment is shown. ｔ is a set of video data {o ｔ} ｔ However, this is not intended to limit the present disclosure. ｔ By way of example, and not a limitation of the present disclosure, τ may be one or more variables that define or represent an action by a controlled object (agent) at frame t (time t).

[0022] Latent variable z ｔ is the data that represents the environment, and the observed information o ｔ The latent variables are also called latent representations, features, or states. ｔ are one or more variables representing the environment and the object, by way of example, but this is not intended to limit the present disclosure. ｔ By way of example, the latent variable z may include factors that cannot be directly observed, but this is not intended to limit the present disclosure. ｔ is represented by a vector or a distribution function, for example.

[0023] <Configuration of information processing device> The configuration of an information processing device 1A according to the present disclosure will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A includes a control unit 10A, a storage unit 20A, a communication unit 30A, an input unit 40A, and an output unit 50A.

[0024] (Communication Unit) The communication unit 30A communicates with devices external to the information processing device 1A via a communication line. While the specific configuration of the communication line does not limit the present exemplary embodiment, examples of the communication line include a wireless local area network (LAN), a wired LAN, a wide area network (WAN), a public line network, a mobile data communication network, or a combination thereof. The communication unit 30A transmits data supplied from the control unit 10A to other devices, and supplies data received from other devices to the control unit 10A.

[0025] (Input Unit) The input unit 40A is a component for receiving input to the information processing device 1A, and includes, for example, input devices such as a keyboard, a mouse, a touch panel, etc. The input unit 40A may be configured to receive data from the input devices via an interface such as a USB (Universal Serial Bus).

[0026] (Output Unit) The output unit 50A is a component for performing output from the information processing device 1A, and includes, for example, an output device such as a display, a printer, a touch panel, etc. The output unit 50A may be configured to include, for example, an interface such as a USB, and to output data to the output device via the interface.

[0027] (Storage Unit) The storage unit 20A stores various types of information referenced by the control unit 10A. Examples of such information include a latent variable Z and a model parameter θ. The latent variable Z is data referenced by the control unit 10A, which will be described later. The latent variable Z is a latent variable z converted by the conversion unit 12A, which will be described later. １ , z ２ , ..., and the latent variable z predicted by the prediction unit 14A described later １ , z ２ The model parameter θ is a model parameter of at least one of a transition prediction model MA and an inference model MB, which will be described later.

[0028] (Control Unit) The control unit 10A includes an acquisition unit 11A, a conversion unit 12A, a prediction unit 14A, and an update unit 15A. The acquisition unit 11A and the conversion unit 12A are examples of an acquisition means and a conversion means, respectively, according to the present disclosure. The prediction unit 14A is an example of a selection means and a prediction means according to the present disclosure. The update unit 15A is an example of an update means and an output means according to the present disclosure.

[0029] 4 is a block diagram showing an example of the functional configuration and processing flow of the information processing device 1A. Note that the unidirectional arrows in Fig. 4 simply indicate the direction of the flow of a certain signal (data) or the direction of the flow of processing, and do not exclude bidirectionality.

[0030] (Acquisition unit) The acquisition unit 11A acquires the observation information o ｔ The acquisition unit 11A acquires, for example, the observation information o input via the input unit 40A. ｔ The acquisition unit 11A also acquires the observation information o from a storage destination (which may be in a storage device of the information processing device 1A or a storage device outside the information processing device 1A) designated by the user of the information processing device 1A. ｔ The acquisition unit 11A may also acquire the observation information o from another device via the communication unit 30A. ｔ By receiving the observation information o ｔ may be obtained.

[0031] (Conversion Unit / Inference Model) The conversion unit 12A converts the observation information o ｔ and the latent variable z ｔ－ｋ (k is a natural number greater than or equal to 1) and the latent variable z ｔ The inference model MB infers the following from the observed information o ｔ and the latent variable z ｔ－ｋ From this, the latent variable z ｔ The inference model MB is, for example, a neural network. The input of the inference model MB is the observation information o ｔ and the latent variable z at step t−k ｔ－ｋ The output of the inference model MB is the latent variable z ｔHowever, the input and output of the inference model MB are not limited to the above example. For example, the input of the inference model MB is the latent variable z ｔ－ｋ The input of the inference model MB may not include the action a ｔ－ｋ may include:

[0032] (Prediction Unit / Transition Prediction Model) The prediction unit 14A predicts future latent variables using a transition prediction model MA. The transition prediction model MA is a model for predicting future latent variables from latent variables. The transition prediction model MA is, for example, a world model. Specific configurations of the transition prediction model MA (and the inference model MB) include, but are not limited to, a variational autoencoder (VAE), a generative adversarial network (GAN), a recurrent neural network (RNN), a convolutional neural network (CNN), a graph neural network (GNN), a transformer, and combinations thereof.

[0033] The input of the transition prediction model MA is the latent variable z ｔ The output of the transition prediction model MA is the latent variable z after n steps. ｔ＋ｎ The prediction unit 14A uses the transition prediction model MA to predict the latent variable z ｔ＋ｎ The process of predicting the value of the saturation level will be described in detail later.

[0034] (Update Unit) The update unit 15A updates the latent variable z ｔ and the latent variable z predicted by the prediction unit 14A. ｔ The model parameter θ of at least one of the inference model MA and the transition prediction model MB is updated using the latent variable z ｔ , and the latent variable z predicted by the prediction unit 14A ｔ are the first latent variables z ｔ , and the second latent variable z ｔ As an example, the update unit 15A updates the latent variable z ｔand the latent variable z predicted by the prediction unit 14A. ｔ The model parameter θ is updated so that the difference between

[0035] Furthermore, when updating the inference model MA, the update unit 15A updates the latent variable set {z ｔ}, the image o at time t reconstructed from ｔ And the actual image ｔ The model parameters are updated to reduce the reconstruction error.

[0036] The update unit 15A outputs the updated model parameter θ. As an example, the update unit 15A outputs the model parameter θ by writing it to a storage destination (which may be in a storage device of the information processing device 1A or a storage device external to the information processing device 1A) designated by the user of the information processing device 1A. Furthermore, the update unit 15A may output the model parameter θ to the output unit 50A, or may output the model parameter θ by transmitting it to another device via the communication unit 30A.

[0037] (Details of Prediction Process) The prediction process performed by the prediction unit 14A will be described. The prediction unit 14A selects either (i) one-step prediction or (ii) multi-step prediction to predict the transition of the latent variable, and predicts the transition of the latent variable z ｔ Here, one-step prediction is a step in which the latent variable z ｔ－ｎ In other words, in the case of one-step prediction, the prediction unit 14A selects the observation information o ｔ－ｎ The latent variable z is the output obtained by inputting the above into the inference model MB. ｔーｎ By inputting the transition prediction model MA, the latent variable z ｔ Predict.

[0038] On the other hand, in multi-step prediction, the latent variable z predicted by the prediction unit 14A at step t-n is used as an input to the transition prediction model MA. ｔ－ｎ In other words, in the case of multi-step prediction, the prediction unit 14A selects the latent variable z ｔ－２ｎThe latent variable z is the output obtained by inputting the transition prediction model MA. ｔーｎ By inputting the transition prediction model MA, the latent variable z ｔ Predict.

[0039] In this way, the prediction unit 14A receives, as an input to the transition prediction model MA, (i) the latent variable z ｔ－ｎ (ii) The transition prediction model MA includes the latent variable z ｔ－２ｎ The latent variable z obtained by inputting ｔ－ｎ Select either one of the following.

[0040] Figure 5 shows the latent variable z obtained by one-step prediction. ｔ and the latent variable z obtained by multi-step prediction. ｔ In the example of FIG. 5, n=k=1. In FIG. 5, the latent variable z ｉ｜ｊ is the observation information up to step j. ｊ 5 indicates that the latent variable z is inferred (or predicted) at step i (i≧j) using the transition prediction model MA. ｉ｜ｊ On the other hand, the dotted arrow indicates the process of predicting the observed information o ｉ and the latent variable z ｉ－１｜ｊ－１ From this, the latent variable z ｉ｜ｊ For example, the latent variable z １｜１ is the observation information o at step t = 1 １ is a latent variable obtained by inputting into the inference model MB. ２｜２ is the observation information o at step t = 2 ２ is a latent variable obtained by inputting into the inference model MB. On the other hand, the latent variable z ２｜１ is the latent variable z １｜１ are latent variables obtained by inputting these into the transition prediction model MA.

[0041] In FIG. 5, when one-step prediction is selected at step t=3, the prediction unit 14A calculates the latent variable z ２｜２ By inputting the transition prediction model MA, the latent variable z ３｜２On the other hand, in the example of FIG. 5, when multi-step prediction is selected at step t=3, the prediction unit 14A predicts the latent variable z ２｜１ By inputting the transition prediction model MA, the latent variable z ３｜１ Predict.

[0042] FIG. 6 shows the latent variable z predicted by the prediction unit 14A at each step. ｉ｜ｊ 6 is a diagram showing an example of the transition of the observation information o. In the example of FIG. 6, n=k=1 as in FIG. 5. ｉ is omitted from the illustration.

[0043] When one-step prediction is selected at step t=3, as shown in FIG. 6, the prediction unit 14A predicts the latent variable z ２｜２ By inputting this into the transition prediction model MA, the latent variable z ３｜２ On the other hand, when multi-step prediction is selected at step t=3, the prediction unit 14A predicts the latent variable z ２｜１ By inputting this into the transition prediction model MA, the latent variable z ３｜１ Predict.

[0044] Furthermore, when one-step prediction is selected at step t=3 and when one-step prediction is selected at step t=4, the prediction unit 14A selects the latent variable z ３｜３ By inputting this into the transition prediction model MA, the latent variable z ４｜３ Predict.

[0045] On the other hand, when one-step prediction is selected at step t=3 and multi-step prediction is selected at step t=4, the prediction unit 14A selects the latent variable z ３｜２ By inputting this into the transition prediction model MA, the latent variable z ４｜２ Predict.

[0046] Furthermore, when multi-step prediction is selected at step t=3 and one-step prediction is selected at step t=4, the prediction unit 14A selects the latent variable z３｜３ By inputting this into the transition prediction model MA, the latent variable z ４｜３ Predict.

[0047] On the other hand, when multi-step prediction is selected at step t=3 and when multi-step prediction is selected at step t=4, the prediction unit 14A selects the latent variable z ３｜１ By inputting this into the transition prediction model MA, the latent variable z ４｜１ Predict.

[0048] FIG. 7 shows the latent variable z predicted by the prediction unit 14A at each step. ｉ｜ｊ In the example of FIG. 7, the prediction unit 14A selects multi-step prediction at step t=3, the prediction unit 14A selects one-step prediction at step t=4, and the prediction unit 14A selects multi-step prediction at steps t=5 to t=7. ｉ｜ｊ In this example, at step t=6, the update unit 15A updates the latent variable z ６｜６ and z predicted by the prediction unit 14A. ６｜３ and are used to update the model parameter θ.

[0049] (Specific Example of Selection Method) Next, a specific example of a selection method in which the prediction unit 14 selects either one-step prediction or multi-step prediction will be described. As an example, the prediction unit 14A selects a latent variable z ｔ－ｎ and the latent variable z, which is the output of the transition prediction model MA. ｔ－ｎ More specifically, as an example, the prediction unit 14 selects one-step prediction with a probability p (0≦p≦1) and selects multi-step prediction with a probability 1−p.

[0050] In this case, the prediction unit 14A may change the probability p as the learning of the model progresses. In other words, the prediction unit 14A predicts the latent variable z ｔ－ｎ and the latent variable z, which is the output of the transition prediction model MA. ｔ－ｎ The probability of selecting either of the above may be changed as the learning progresses.

[0051] The prediction unit 14A also calculates the latent variable z ｔ－ｎ The probability p of selecting the latent variable z obtained by the multi-step prediction may be decreased as the learning progresses. In this case, the probability that the prediction unit 14A performs multi-step prediction increases as the learning progresses. ｔ＋ｎ The probability that a multi-step prediction will be used for learning increases. When learning is not yet advanced, the accuracy of multi-step predictions is expected to be low, but as learning progresses, the accuracy of future predictions is expected to improve. Therefore, by increasing the probability of selecting multi-step predictions as learning progresses, the accuracy of multi-step predictions of the generated model can be improved.

[0052] As another example, the prediction unit 14A may select a prediction method for each step by selecting one of a plurality of transition paths prepared in advance. For example, consider a case where five transition paths, [nnnnn], [nnnny], [nnnyy], and [nnyyy], are prepared in advance as transition paths for learning with the number of steps T=5. Here, "n" and "y" in each transition path indicate whether one-step prediction or multi-step prediction is selected for each step. "n" indicates that one-step prediction is selected for that step, and "y" indicates that multi-step prediction is selected for that step. In this case, the prediction unit 14A selects one of the plurality of transition paths and selects a prediction method for each step according to the selected transition path.

[0053] In this case, the prediction unit 14A may randomly select one of the multiple transition paths, or may change the selected transition path as the learning progresses. Alternatively, the prediction unit 14A may select the multiple transition paths in order.

[0054] Furthermore, when randomly selecting a transition path, the prediction unit 14A may change the probability of selection for each transition path. For example, the prediction unit 14A may change the probability p ｔMore specifically, for example, in the learning of the number of steps T=2, the prediction unit 14A may select the probability p １ = 1 / 3, probability p ２ = 1 / 3, probability p ３ = 1 / 3, where the probability p １ is the probability of selecting [nn], and probability p ２ is the probability of selecting [ny], and probability p ３ is the probability of selecting [yy].

[0055] As another example, the prediction unit 14A may select either one-step prediction or multi-step prediction according to a predetermined rule. In other words, the prediction unit 14A may select the latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ The prediction unit 14A may select one of the two prediction methods in accordance with a predetermined rule. More specifically, as an example, the prediction unit 14A may alternately select one-step prediction and multi-step prediction. As another example, the prediction unit 14A may select a prediction method for each step in accordance with a predetermined transition path.

[0056] <Flow of Processing Executed by Prediction Unit> FIG. 8 is a diagram showing an example of the flow of processing executed by the prediction unit 14A. In step S21, the prediction unit 14A determines whether to perform multi-step prediction. If multi-step prediction is to be performed (YES in step S21), the prediction unit 14A proceeds to the processing in step S22, and calculates the latent variable z ｔ－ｎ On the other hand, if one-step prediction is to be performed (NO in step S21), the prediction unit 14A proceeds to the process of step S23, and selects the latent variable z ｔ－ｎ Select .

[0057] In step S24, the prediction unit 14A predicts the selected latent variable z ｔ－ｎ By inputting the transition prediction model MA, the latent variable z ｔ In step S25, the prediction unit 14A predicts the predicted latent variable z ｔ to the update unit 15A.

[0058] <Effects of Information Processing Device> As described above, in the information processing device 1A, the prediction unit 14A selects either one-step prediction or multi-step prediction at each step, and predicts the latent variable z ｔ By selecting the prediction method in this way, the latent variable z ｔ become one.

[0059] In the PlaNet technology described in Non-Patent Document 1, learning is performed using prediction results of a predetermined number of multi-steps, so the amount of calculations in learning depends on the total number of steps T and the number of multi-steps S. That is, in the technology described in Non-Patent Document 1, the greater the total number of steps T, the greater the amount of calculations, and also the greater the number of multi-steps S. In contrast, in the information processing device 1A, the amount of calculations related to learning does not depend on the number of consecutive prediction steps. Therefore, with the information processing device 1A, the amount of calculations related to learning a learning model for multi-step future prediction can be reduced compared to when learning is performed using the results of multiple multi-step predictions.

[0060] Furthermore, in the PlaNet technology described in the above-mentioned Non-Patent Document 1, it is necessary to determine the maximum number of prediction steps in advance, whereas in the information processing device 1A, it is not necessary to determine the maximum number of prediction steps in advance, and the number of consecutive prediction steps during learning can be made variable.

[0061] Furthermore, in the technology described in Non-Patent Document 1, multiple prediction results are obtained in one step, and loss calculations are performed using weighting coefficients that indicate which prediction results are to be emphasized in one step, which results in a large number of hyperparameters. In contrast, with the information processing device 1A, the only hyperparameter required is the probability p.

[0062] Furthermore, the information processing device 1A employs a configuration in which the update unit 15A outputs updated model parameters, making it possible to provide a learning model used for predicting future states.

[0063] In the information processing device 1A, the input of the inference model MB is the observation information o ｔ and the latent variable z at step t-k (k is a natural number). ｔ－ｋ The output of the inference model MB is the latent variable z ｔ Therefore, according to the information processing device 1A, the latent variable z is included in the input of the inference model MB in the learning phase. ｔ－ｋ By including the above, the inference accuracy of at least one of the inference model MB and the transition prediction model MA can be improved.

[0064] In the information processing device 1A, the prediction unit 14A predicts the latent variable z ｔ－ｎ and the latent variable z predicted by the prediction unit 14A. ｔ－ｎ Therefore, the information processing device 1A has an advantage that it is possible to make a model learn a variety of state transitions that are selected probabilistically.

[0065] In the information processing device 1A, the prediction unit 14A predicts the latent variable z ｔ－ｎ and the latent variable z predicted by the prediction unit 14A. ｔ－ｎ The information processing device 1A has a configuration in which the probability of selecting either of the two is changed as the learning progresses. Therefore, the information processing device 1A has an effect of making the model learn a state transition that is appropriate for the progress of the learning in the learning phase.

[0066] In the information processing device 1A, the prediction unit 14A predicts the latent variable z ｔ－ｎ The probability p of selecting a multi-step prediction is reduced as learning progresses. By increasing the probability of selecting a multi-step prediction as learning progresses, it is possible to improve the accuracy of multi-step predictions in the generated future prediction model.

[0067] In the information processing device 1A, the update unit 15A updates the latent variable z ｔ－ｎ and the latent variable z predicted by the prediction unit 14A. ｔ－ｎTherefore, according to the information processing device 1A, it is possible to obtain an effect that a user of the information processing device 1A can set the above rule, and thereby have the model learn state transitions that are in line with the user's intention.

[0068] <Another Configuration Example 1 of Information Processing Device> Figure 9 is a block diagram showing another example of the functional configuration and processing flow of an information processing device 1B according to the present disclosure. Note that the unidirectional arrows in Figure 9 simply indicate the direction of flow of a certain signal (data) or the direction of flow of a process, and do not exclude bidirectionality. In Figure 9, the control unit 10B of the information processing device 1B includes a behavior acquisition unit 13B, a conversion unit 12B, and a prediction unit 14B in addition to the acquisition unit 11A and update unit 15A of the information processing device 1A.

[0069] The behavior acquisition unit 13B acquires behavior a ｔ The behavior acquisition unit 13B acquires the behavior a ｔ may or may not be determined by referring to the latent variables predicted by the prediction unit 14A (latent variables of past steps).

[0070] The conversion unit 12B converts the observation information o ｔ and Action A ｔ and the latent variable z ｔ－ｋ From this, the latent variable z ｔ In this example, the input of the inference model MB2 is the observed information o ｔ and the latent variable z at step t−k ｔ－ｋ In addition to this, action a ｔ The output of the inference model MB2 is the latent variable z ｔ Includes:

[0071] The prediction unit 14B predicts future latent variables using the transition prediction model MA2. In this example, the input of the transition prediction model MA2 is the latent variable z ｔ－ｎ and Action A ｔ The output of the transition prediction model MA2 is the latent variable z ｔ＋ｎ Includes:

[0072] As described above, in the information processing device 1B, the input of the transition prediction model MA2 is the latent variable z ｔ－ｎ and Action A ｔ Therefore, according to the information processing device 1B, in the learning phase, the action a is input to the transition prediction model MA2. ｔ By including this, the inference accuracy of at least one of the inference model MB2 and the transition prediction model MA2 can be improved.

[0073] (Operation example) Fig. 10 is a diagram for explaining an example of the learning process of the information processing device 1B. Fig. 10 illustrates the flow of the learning process of a model having an RNN structure. In the example of Fig. 10, n = k = 1. In addition, in the example of Fig. 10, the observed information o ｔ are the frames that make up the video. The inference model MB2 is ｔ and Action A ｔ－１ and the latent variable z ｔ In addition, the transition prediction model MA2 outputs the latent variable z ｔ－１ and Action A ｔ－１ and the latent variable z ｔ The update unit 15A also outputs the latent variable z ｔ and the latent variable z, which is the output of the inference model MB2. ｔ and update the model parameters using

[0074] h ｔ is the latent variable z up to step t １ , z ２ , …, z ｔ The prediction unit 14B calculates the latent variable z ｔ and the latent variable z ｔ and select the selected latent variable z ｔ The hidden variable h of the RNN of the transition prediction model MA2 ｔThe update unit 15A performs the same process as in step t in each step after step t+1, which is the output of the transition prediction model MA2, and this process is repeated to learn the model.

[0075] 11 is a block diagram showing the configuration of an information processing device 1C according to the present disclosure. A control unit 10C of the information processing device 1C includes an acquisition unit 11A, a conversion unit 12A, a prediction unit 14A, and an update unit 15A provided in the information processing device 1A, as well as a second acquisition unit 16C, a calculation unit 17C, and an output unit 18C. The second acquisition unit 16C, the calculation unit 17C, and the output unit 18C are examples of a second acquisition means, a calculation means, and a second output means, respectively, according to the present disclosure.

[0076] The second acquisition unit 16C acquires the observation information o ｔ and Action A ｔ The second acquisition unit 16C acquires, for example, the observation information o input via the input unit 40A. ｔ and Action A ｔ The second acquiring unit 16C may acquire the observation information o from a storage destination (which may be in a storage device of the information processing device 1A or a storage device outside the information processing device 1A) designated by the user of the information processing device 1A. ｔ and Action A ｔ The second acquiring unit 16C may also acquire the observation information o from another device via the communication unit 30A. ｔ and Action A ｔ By receiving the observation information o ｔ and Action A ｔ and may be obtained.

[0077] The calculation unit 17C calculates the observation information o acquired by the second acquisition unit 16C. ｔ and the transition prediction model MA, the behavior a acquired by the second acquisition unit 16C is ｔ More specifically, the calculation unit 17C calculates the success probability by, for example, the following process. First, the calculation unit 17C calculates the success probability of the observation information o ｔ and the past latent variable z ｔ－ｎ and are input into the inference model MB to obtain the latent variable z ｔand the inferred latent variable z ｔ By inputting this into the transition prediction model MA, the future latent variable z ｔ＋ｎ Furthermore, the calculation unit 17C predicts the predicted future latent variable z ｔ＋ｎ The second acquisition unit 16C acquires the behavior a ｔ The calculation unit 17C outputs the calculated success probability to the output unit 18C. As an example, the calculation unit 17C calculates the success probability using the evaluation model LM3.

[0078] The evaluation model LM3 is obtained by the calculation unit 17C performing the action a ｔ The evaluation model LM3 is a trained model constructed by machine learning using training data, and is used to calculate the success probability of the evaluation model LM3. The machine learning method for the evaluation model LM3 is not limited, and for example, a decision tree-based, linear regression, or neural network method may be used, or two or more of these methods may be used.

[0079] The success probability calculated by the calculation unit 17C is used to create a plan for operating a control target such as a robot arm. For example, the information processing device 1C determines an action plan to instruct the control target by performing an action optimization process using the calculated success probability. However, the use of the success probability calculated by the information processing device 1C is not limited to the above-mentioned example.

[0080] The output unit 18C generates control information for the control target based on the success probability calculated by the calculation unit 17C, and outputs the generated control information. The output unit 18C may output the generated control information via the output unit 50A, or may transmit the control information via the communication unit 30A. Furthermore, the output unit 18C may output the control information by writing the control information to a storage destination designated by the user of the information processing device 1C (which may be in a storage device of the information processing device 1B or a storage device external to the information processing device 1C).

[0081] As described above, in the information processing device 1C, the observation information o ｔ and Action A ｔand a second acquisition unit 16C that acquires the observation information o ｔ and the transition prediction model MA learned by the update unit 15A, the behavior a acquired by the second acquisition unit 16C means ｔ and an output unit 18C that generates control information for the control object based on the success probability calculated by the calculation unit 17C and outputs the generated control information. Therefore, according to the information processing device 1C, it is possible to generate control information for the control object using the model learned by the update unit 15A.

[0082] <Application Example> An application example using the information processing device 1A will be described with reference to Fig. 12. Fig. 12 is a diagram showing an overview of a medical robot rb using the information processing device 1A. Note that this application example may be applied to the configuration of the first exemplary embodiment.

[0083] The medical robot rb shown in Fig. 12 uses a machine-learned world model to grasp the environment in the operating room, the patient's physical condition, and the instruments and procedures required for the surgery. The world model is a model trained by machine learning to input sensor information indicating changes in images and the environment and output information required for the surgery. As an example, the world model includes the transition prediction model MA and inference model MB described above.

[0084] For example, the medical robot rb acquires images from a camera that photographs the operating room and acquires sensor information from a sensor that detects changes in the operating room environment. Next, the medical robot rb inputs the acquired images and sensor information into a world model to acquire information necessary for surgery. The medical robot rb grasps the internal condition of the patient in real time based on the output of the world model and optimizes the selection of instruments and procedures necessary for surgery.

[0085] In this way, the medical robot rb using the information processing device 1A and the world model can predict the movements of doctors, nurses, etc. Therefore, the medical robot rb can cooperate with doctors, nurses, etc. and autonomously execute optimized operations.

[0086] Furthermore, the medical robot rb using the information processing device 1A and the world model can accurately recognize the patient's condition, allowing the medical robot rb to perform surgery with minimal invasiveness.

[0087] [Example of implementation by software] Some or all of the functions of the information processing devices 1, 1A, 1B, and 1C (hereinafter also referred to as "each of the above devices") may be implemented by hardware such as an integrated circuit (IC chip), or by software.

[0088] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 13. Figure 13 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0089] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to function as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0090] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0091] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0092] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0093] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0094] (Appendix A1) Observation information o indicating the observation result at step t ｔ and a first model for converting observations into latent variables, and ｔ the latent variable z ｔ and (i) a first latent variable z converted by the conversion means in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎand a selection means for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction means for predicting the latent variable z ｔ and the latent variable z predicted by the prediction means ｔ and an update unit that updates model parameters of at least one of the first model and the second model using the first model and the second model.

[0095] (Supplementary Note A2) The information processing device according to Supplementary Note A1, further comprising: an output unit configured to output the model parameters updated by the update unit.

[0096] (Appendix A3) The input of the first model is the observation information o ｔ and the latent variable z at step t-k (k is a natural number). ｔ－ｋ and the output of the first model is the latent variable z ｔ The information processing device according to Appendix A1 or A2, comprising:

[0097] (Appendix A4) The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ and

[0098] (Appendix A5) The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ and a probability of selecting either of the first and second values ​​is changed as learning progresses.

[0099] (Appendix A6) The selection means selects the first latent variable z ｔ－ｎ The information processing device according to Appendix A5, wherein the probability of selecting the first parameter is reduced as learning progresses.

[0100] (Appendix A7) The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ and selecting one of the above in accordance with a predetermined rule.

[0101] (Appendix A8) The information processing device according to any one of Appendices A1 to A7, wherein the input of the second model includes a latent variable and a behavior.

[0102] (Appendix A9) The information processing device according to any one of Appendices A1 to A8, further comprising: a second acquisition means for acquiring observations and actions; a calculation means for calculating a success probability of the action acquired by the second acquisition means using the observations acquired by the second acquisition means and the second model; and a second output means for generating control information for a control target based on the success probability calculated by the calculation means, and outputting the generated control information.

[0103] [Appendix B] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0104] (Appendix B1) At least one processor generates observation information o indicating an observation result at step t. ｔ an acquisition process for acquiring the observation information o acquired in the acquisition process using a first model that converts observations into latent variables; ｔ the latent variable z ｔ and the at least one processor (i) converts the first latent variable z converted in the conversion process in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection process for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction process for predicting the latent variable z ｔ and the latent variable z predicted in the prediction process. ｔand an update process of updating model parameters of at least one of the first model and the second model using the model parameter.

[0105] (Supplementary Note B2) The information processing method according to Supplementary Note B1, further comprising an output process in which the at least one processor outputs the model parameters updated in the update process.

[0106] (Appendix B3) The input of the first model is the observation information o ｔ and the latent variable z at step t-k (k is a natural number). ｔ－ｋ and the output of the first model is the latent variable z ｔ The information processing method according to Appendix B1 or B2, comprising:

[0107] (Appendix B4) In the selection process, the at least one processor ｔ－ｎ and the second latent variable z ｔ－ｎ and (b) selecting probabilistically one of the following:

[0108] (Appendix B5) In the selection process, the at least one processor ｔ－ｎ and the second latent variable z ｔ－ｎ and changing a probability of selecting either of the two as learning progresses.

[0109] (Appendix B6) In the selection process, the at least one processor ｔ－ｎ The information processing method according to Appendix B5, wherein the probability of selecting the first parameter is reduced as learning progresses.

[0110] (Appendix B7) In the selection process, the at least one processor ｔ－ｎ and the second latent variable z ｔ－ｎ and selecting one of the above in accordance with a predetermined rule.

[0111] (Supplementary Note B8) The information processing method according to any one of Supplementary Notes B1 to B7, wherein the input of the second model includes latent variables and behavior.

[0112] (Supplementary Note B9) The information processing method according to any one of Supplementary Notes B1 to B8, further comprising: a second acquisition process in which the at least one processor acquires observations and actions; a calculation process in which the at least one processor calculates a success probability of the action acquired in the second acquisition process using the observations acquired in the second acquisition process and the second model; and a second output process in which the at least one processor generates control information for a control object based on the success probability calculated in the calculation process, and outputs the generated control information.

[0113] [Appendix C] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0114] (Appendix C1) A program for causing a computer to function as an information processing device, the program causing the computer to: ｔ and a first model for converting observations into latent variables, and ｔ the latent variable z ｔ and (i) a first latent variable z converted by the conversion means in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables. ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection means for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction means for predicting the latent variable z ｔ and the latent variable z predicted by the prediction means ｔand an information processing program that causes the computer to function as an update unit that updates model parameters of at least one of the first model and the second model using the above.

[0115] (Supplementary Note C2) The information processing program according to Supplementary Note C1, further causing the computer to function as an output means for outputting the model parameters updated by the update means.

[0116] (Appendix C3) The input of the first model is the observation information o ｔ and the latent variable z at step t-k (k is a natural number). ｔ－ｋ and the output of the first model is the latent variable z ｔ The information processing program according to Appendix C1 or C2, comprising:

[0117] (Appendix C4) The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ and

[0118] (Appendix C5) The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ and changing a probability of selecting either of the first and second values ​​as learning progresses.

[0119] (Appendix C6) The selection means selects the first latent variable z ｔ－ｎ The information processing program according to appendix C5, wherein the probability of selecting the first parameter is reduced as learning progresses.

[0120] (Appendix C7) The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ and selecting one of the above in accordance with a predetermined rule.

[0121] (Appendix C8) The information processing program according to any one of appendices C1 to C7, wherein the input of the second model includes a latent variable and a behavior.

[0122] (Appendix C9) The information processing program according to any one of Appendices C1 to C8, further causing the computer to function as: a second acquisition means for acquiring observations and actions; a calculation means for calculating the success probability of the action acquired by the second acquisition means using the observations acquired by the second acquisition means and the second model; and a second output means for generating control information for a control target based on the success probability calculated by the calculation means, and outputting the generated control information.

[0123] [Appendix D] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0124] (Appendix D1) At least one processor generates observation information o indicating an observation result at step t. ｔ an acquisition process for acquiring the observation information o acquired in the acquisition process using a first model that converts the observation into a latent variable; ｔ the latent variable z ｔ and (i) in step t-n (n is a natural number equal to or greater than 1), converting the first latent variable z ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection process for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction process for predicting the latent variable z ｔ and the latent variable z predicted in the prediction process. ｔ and an update process for updating model parameters of at least one of the first model and the second model using the above.

[0125] (Supplementary Note D2) The information processing device according to Supplementary Note D1, wherein the at least one processor further executes an output process of outputting the model parameters updated in the update process.

[0126] (Appendix D3) The input of the first model is the observation information o ｔ and the latent variable z at step t-k (k is a natural number). ｔ－ｋ and the output of the first model is the latent variable z ｔ The information processing device according to Appendix D1 or D2,

[0127] (Appendix D4) In the selection process, the at least one processor ｔ－ｎ and the second latent variable z ｔ－ｎ and

[0128] (Appendix D5) In the selection process, the at least one processor ｔ－ｎ and the second latent variable z ｔ－ｎ and a probability of selecting either of the first and second values ​​is changed as learning progresses.

[0129] (Appendix D6) In the selection process, the at least one processor ｔ－ｎ The information processing device according to appendix D5, wherein the probability of selecting the first parameter is reduced as learning progresses.

[0130] (Appendix D7) In the selection process, the at least one processor ｔ－ｎ and the second latent variable z ｔ－ｎ and selecting one of the above in accordance with a predetermined rule.

[0131] (Appendix D8) The information processing device according to any one of appendices D1 to D7, wherein the input of the second model includes a latent variable and a behavior.

[0132] (Appendix D9) An information processing device described in any one of Appendices D1 to D8, wherein the at least one processor further executes: a second acquisition process that acquires observations and actions; a calculation process that calculates a success probability of the action acquired in the second acquisition process using the observations acquired in the second acquisition process and the second model; and a second output process that generates control information for a control object based on the success probability calculated in the calculation process and outputs the generated control information.

[0133] [Appendix E] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0134] (Appendix E1) Observation information o indicating the observation result at step t ｔ an acquisition process for acquiring the observation information o acquired in the acquisition process using a first model that converts the observation into a latent variable; ｔ the latent variable z ｔ and (i) in step t-n (n is a natural number equal to or greater than 1), converting the first latent variable z ｔ－ｎ and (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection process for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ a prediction process for predicting the latent variable z ｔ and the latent variable z predicted in the prediction process. ｔ and an information processing program for executing an update process of updating model parameters of at least one of the first model and the second model using the above-mentioned parameter.

[0135] 1, 1A, 1B, 1C Information processing device 11, 11A Acquisition unit 12, 12A, 12B Conversion unit 13 Selection unit 18C, 50A Output unit 14, 14A, 14B Prediction unit 15, 15A Update unit 16C Second acquisition unit 17C Calculation unit

Claims

1. Observation information o showing the observation result at step t ｔ and a first model for converting observations into latent variables, the first model being used to convert the observations into latent variables. ｔ Let z be the latent variable ｔ and (i) a first latent variable z converted by the conversion means in step t-n (n is a natural number equal to or greater than 1) as an input to a second model for predicting future latent variables from latent variables. ｔ－ｎ (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ A selection means for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ A prediction means for predicting a latent variable z ｔ and the latent variable z predicted by the prediction means ｔ and an update means for updating a model parameter of at least one of the first model and the second model using the model parameter.

2. The information processing apparatus according to claim 1, further comprising: output means for outputting the model parameters updated by said update means.

3. The input of the first model is the observation information o ｔ and the latent variable z at step t-k (k is a natural number). ｔ－ｋ and the output of the first model is the latent variables z ｔ The information processing device according to claim 1 , further comprising:

4. The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ The information processing device according to claim 1 , wherein either one of the above is selected probabilistically.

5. The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ The information processing apparatus according to claim 4 , wherein a probability of selecting either one of the above is changed as learning progresses.

6. The selection means selects the first latent variable z ｔ－ｎ The information processing apparatus according to claim 5 , wherein the probability of selecting the first parameter is decreased as learning progresses.

7. The selection means selects the first latent variable z ｔ－ｎ and the second latent variable z ｔ－ｎ 4. The information processing apparatus according to claim 1, wherein either one of the above is selected in accordance with a predetermined rule.

8. The information processing device according to any one of claims 1 to 7, wherein the input of the second model includes latent variables and actions.

9. An information processing device according to any one of claims 1 to 8, further comprising: a second acquisition means for acquiring observations and actions; a calculation means for calculating a success probability of the action acquired by the second acquisition means using the observations acquired by the second acquisition means and the second model; and a second output means for generating control information for a control object based on the success probability calculated by the calculation means, and outputting the generated control information.

10. At least one processor receives observation information o indicating an observation result at step t. ｔ an acquisition process for acquiring the observation information o acquired in the acquisition process using a first model that converts the observation into a latent variable, ｔ Let z be the latent variable ｔ and (i) converting the first latent variable z converted in the conversion process in step t-n (n is a natural number equal to or greater than 1) as an input to a second model that predicts future latent variables from latent variables, ｔ－ｎ (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ and a selection process for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ A prediction process in which the at least one processor predicts the latent variable z ｔ and the latent variable z predicted in the prediction process. ｔ and an update process of updating a model parameter of at least one of the first model and the second model using the model parameter.

11. A program for causing a computer to function as an information processing device, the program comprising: ｔ and a first model for converting observations into latent variables, the first model being used to convert the observations into latent variables. ｔ Let z be the latent variable ｔ and (i) a first latent variable z converted by the conversion means in step t-n (n is a natural number equal to or greater than 1) as an input to a second model for predicting future latent variables from latent variables. ｔ－ｎ (ii) adding a latent variable z ｔ－２ｎ The second latent variable z obtained by inputting ｔ－ｎ A selection means for selecting one of the latent variables z ｔ－ｎ By inputting into the second model, the latent variable z ｔ A prediction means for predicting a latent variable z ｔ and the latent variable z predicted by the prediction means ｔ and an information processing program that causes the program to function as an updating means for updating model parameters of at least one of the first model and the second model using the model parameters.

Images