Learning device, learning method, and program

Abstract

Provided is a learning device, wherein a model acquisition means acquires: a first prediction model that generates a first prediction feature amount that indicates a state of each object at the next time on the basis of an input object feature amount; and a second prediction model that generates a second prediction feature amount that indicates the state of each object at the next time on the basis of the input object feature amount and an image at the next time. An identification model training means trains an identification model that identifies an output from the first prediction model and an output from the second prediction model. A re-training means re-trains the first prediction model so as to reduce the accuracy of identification by the identification model.

Description

Learning device, learning method, and recording medium

[0001] The present disclosure relates to a technique for predicting future temporal transitions of objects in an image.

[0002] A world model is known as a technology that acquires representations of individual objects from unsupervised images of object positions, labels, etc., and predicts future temporal transitions. Patent Document 1 discloses a technology for performing robot learning by combining imitation learning and reinforcement learning while acquiring a world model through an expert and predicting the future.

[0003] Japanese Patent Application Laid-Open No. 2021-192141

[0004] When training a model to predict the temporal transition of objects in a video, training videos are used, but the performance of the model is evaluated based only on the difference from the training videos. Therefore, when the trained model includes probabilistic transitions in the future, it is unable to judge the likelihood of the outcome of the transition.

[0005] One objective of the present disclosure is to learn a prediction model that is capable of predicting plausible temporal transitions without being overly concerned with the learning videos.

[0006] In one aspect of the present disclosure, a learning device includes: a model acquisition means for acquiring a first prediction model that generates a first predicted feature that indicates a state of each object at the next time based on input object feature values; and a second prediction model that generates a second predicted feature that indicates a state of each object at the next time based on the input object feature values ​​and an image at the next time; a discriminative model training means for training a discriminative model that distinguishes between an output from the first prediction model and an output from the second prediction model; and a re-learning means for re-learning the first prediction model so that the discrimination accuracy of the discriminative model decreases.

[0007] In another aspect of the present disclosure, a computer-executed learning method includes: acquiring a first prediction model that generates a first predicted feature that indicates a state of each object at a next time based on input object feature values; and a second prediction model that generates a second predicted feature that indicates a state of each object at a next time based on the input object feature values ​​and an image at the next time; training a discriminative model that distinguishes between an output from the first prediction model and an output from the second prediction model; and re-training the first prediction model so that the discrimination accuracy of the discriminative model decreases.

[0008] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to execute the following processes: acquire a first prediction model that generates first predicted features indicating the state of each object at the next time based on input object features; and a second prediction model that generates second predicted features indicating the state of each object at the next time based on the input object features and an image at the next time; train a discrimination model that distinguishes between outputs from the first prediction model and the second prediction model; and re-train the first prediction model so that the discrimination accuracy of the discrimination model decreases.

[0009] 1 shows the concept of a learning device according to a first embodiment; FIG. 2 is a block diagram showing the hardware configuration of the learning device; FIG. 3 is a block diagram showing the functional configuration of a learning device for executing a first learning step; FIG. 4 is a block diagram showing the functional configuration of a learning device for executing a second learning step; FIG. 5 is a block diagram showing the functional configuration of a learning device for executing a third learning step; FIG. 6 is a main routine of learning processing by a learning device; FIG. 7 is a flowchart of learning processing of a transition model and an inference model; FIG. 8 is a flowchart of learning processing of a discriminative model; FIG. 9 is a flowchart of learning processing of a transition model; FIG. 10 is a schematic diagram showing an example of future prediction using a transition model and an inference model; FIG. 11 is a diagram showing a learning method for a discriminative model according to a first modified example; FIG. 12 is a block diagram showing the configuration of a learning device according to a second embodiment; FIG. 13 is a flowchart of processing by a learning device according to a second embodiment.

[0010] Preferred embodiments of the present disclosure will be described below with reference to the drawings. First Embodiment [Concept of Learning Device] FIG. 1 shows the concept of a learning device according to a first embodiment. The learning device 100 learns a prediction model that predicts future temporal transitions of objects included in input images. Such a prediction model is also called a world model. For example, in the case of input images captured of an environment in which a robot arm moves objects, the prediction model acquires representations of the objects included in the input images and predicts future temporal transitions of each object. Note that in the above example, the objects to be predicted include the robot arm itself, objects moved by the robot arm, and fixed objects placed in the captured environment.

[0011] In this embodiment, the learning device 100 basically learns a prediction model through the following three learning steps. (1) First Learning Step: The learning device 100 learns two prediction models as models for predicting future temporal transitions of objects. The first prediction model is a model (hereinafter also referred to as a "transition model") that outputs first object feature quantities indicating the state of each object at the next time based on input object feature quantities of each object. The second prediction model is a model (hereinafter also referred to as an "inference model") that outputs second object feature quantities indicating the state of each object at the next time based on input object feature quantities of each object and actual images at the next time. (2) Second Learning Step: The learning device 100 learns a discriminative model that distinguishes between outputs from the transition model and the inference model. (3) Third Learning Step: The learning device 100 re-learns the transition model so that the discriminative model can no longer distinguish between outputs from the transition model and the inference model.

[0012] By executing the above learning steps, the learning device 100 can generate a transition model that can accurately predict future temporal transitions of an object even if there is no image for the next time. Details of each learning step will be described later.

[0013] 2 is a block diagram showing the hardware configuration of learning device 100. As shown in the figure, learning device 100 includes an interface (IF) 12, a processor 13, a memory 14, a recording medium 15, a database (DB) 16, a display unit 17, and an input unit 18.

[0014] The IF 12 acquires an input image from an external device. The input image is a still image or a video image of an object in a certain environment. The IF 12 also outputs the transition model and inference model obtained by learning to the outside.

[0015] The processor 13 is a computer such as a CPU (Central Processing Unit) and executes a pre-prepared program to control the entire learning device 100. The processor 13 may be a CPU, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating Point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination thereof. The processor 13 executes the learning process described below.

[0016] The memory 14 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 14 stores various programs executed by the processor 13. The memory 14 is also used as a working memory while the processor 13 is executing various processes.

[0017] Recording medium 15 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or semiconductor memory, and is configured to be detachable from learning device 100. Recording medium 15 records various programs executed by processor 13. When learning device 100 executes various processes, the programs recorded on recording medium 15 are loaded into memory 14 and executed by processor 13.

[0018] If necessary, the DB 16 stores input images input during learning through the IF 12. The DB 16 may also store inference models and discrimination models generated during the learning process.

[0019] Display unit 17 is configured with, for example, a liquid crystal display, etc. Input unit 18 includes, for example, a keyboard, a mouse, etc. Display unit 17 and input unit 18 are used, for example, when an operator of learning device 100 performs necessary operation input.

[0020] [Functional Configuration] (First Learning Step) Figure 3 is a block diagram showing the functional configuration of the learning device 100a for executing the first learning step. In the first learning step, the learning device 100a includes a loss calculation unit 21a and a learning unit 22a. In the first learning step, the learning device 100a learns a transition model MA and an inference model MB. The transition model MA and the inference model MB are configured by neural networks. Note that although Figure 3 shows multiple transition models MA and multiple inference models MB, these are shown for convenience to illustrate that different input images are processed; in reality, there is one transition model MA and one inference model MB.

[0021] Now, assume that there is a time series of input images from time t-1 to time t+1. Here, t is a natural number, with 1 being the smallest unit between frames of the video. The frame rate of the video can be any value, but transitions may be estimated at n-frame intervals. In this case, for example, a transition prediction such as t ≥ t + n ≥ t + 2n is performed. More generally, a transition prediction such as t + s ≥ t + s + n ≥ t + s + 2n can be performed during learning using a random integer s (0 ≤ s < n).

[0022] When predicting a state from time t-1 to time t, the object features of each object at time t-1 predicted by the inference model MB are input to the transition model MA and the inference model MB. The object features are object-centric expressions that indicate the position, shape, etc. of each object, and can be, for example, feature vectors. Furthermore, an actual image at time t is input to the inference model MB and the loss calculation unit 21a.

[0023] The transition model MA predicts the transition of each object based on the input object feature values ​​at time t-1, calculates the probability distribution of the object's state at the next time t for each object (hereinafter referred to as the "state distribution"), and outputs this to the loss calculation unit 21a. The object's state distribution can be, for example, a vector including a feature vector indicating the object's state and a vector indicating the probability distribution of the object's state. As described above, the "state distribution" represents the state of each object using a probability distribution, and the "object feature value" is a deterministic state vector sampled based on this state distribution.

[0024] The transition model MB calculates object features for each object included in the input image at time t from the image. The transition model MA calculates the object state distribution at the next time t for each object based on the obtained object features and the image at time t, and outputs the distribution to the loss calculation unit 21a.

[0025] The loss calculation unit 21a receives as input the state distribution at time t generated by the transition model MA, the state distribution at time t generated by the inference model MB, and the actual image at time t. First, the loss calculation unit 21a calculates the inter-distribution distance between the state distribution at time t generated by the transition model MA and the state distribution at time t generated by the inference model MB. As the inter-distribution distance, for example, KL divergence (Kullback-Leibler divergence) can be used. Furthermore, the loss calculation unit 21a reconstructs the image at time t from the state distribution at time t generated by the inference model MB to generate a reconstructed image, and calculates the error between the reconstructed image and the input actual image at time t (hereinafter referred to as the "reconstruction error"). Then, the loss calculation unit 21a outputs the calculated inter-distribution distance and reconstruction error to the learning unit 22a.

[0026] In one example of the reconstruction error, an image can be expressed as a 3×W×H vector using RGB (3 channels), the number of horizontal pixels W, and the number of vertical pixels H, and the Euclidean distance between the vector of the correct image and the vector of the reconstructed image can be used as the reconstructed error.In another example, the negative value (value with a minus sign) of the log likelihood between a probability distribution in which the vector of the correct image is the mean value and the variance of each element is a given σ and the vector of the reconstructed image can be used as the reconstructed error.

[0027] The learning device 100a performs similar processing using the object features at time t predicted by the inference model MB and the image at time t+1. That is, the loss calculation unit 21a calculates the distribution distance between the state distribution at time t+1 generated by the transition model MA and the state distribution at time t+1 generated by the inference model MB. The loss calculation unit 21a also reconstructs the image at time t+1 from the state distribution at time t+1 generated by the inference model MB to generate a reconstructed image, and calculates the reconstruction error between the reconstructed image and the input actual image at time t+1. The loss calculation unit 21a then outputs the calculated distribution distance and reconstruction error to the learning unit 22a.

[0028] The learning unit 22a then learns the transition model MA and the inference model MB using the sum of the inter-distribution distance and the reconstruction error input from the two loss calculation units 21a as a loss. Specifically, the learning unit 22a updates the parameters of the transition model MA and the inference model MB so that the inter-distribution distance becomes smaller and the reconstruction error becomes smaller. As a result, the transition model MA and the inference model MB are learned so that the state distribution output by the transition model MA and the state distribution output by the inference model MB become closer to each other and the reconstructed image output by the inference model MB becomes closer to the actual image input to the loss calculation unit 21a. In this way, in the first learning step, the transition model MA and the inference model MB are learned using the input image.

[0029] In addition to the above-mentioned methods, other known world models such as ViMON, OP3, G-SWM, and GATSBI may also be used in the first learning step.

[0030] (Second Learning Step) Figure 4 is a block diagram showing the functional configuration of the learning device 100b for executing the second learning step. In the second learning step, the learning device 100b learns a discrimination model MC that distinguishes between the output of the transition model MA and the output of the inference model MB. The discrimination model MC is configured by a neural network. Note that in the second learning step, the discrimination model MC is learned using the transition model MA and the inference model MB learned in the first learning step. In other words, in the second learning step, the transition model MA and the inference model MB are not the targets of learning.

[0031] Now, assume that there is a time series of input images from time t-1 to time t+1. As shown in FIG. 4, object features of each object at time t-1 are input to the transition model MA and the inference model MB, and an image at time t is input to the inference model MB. The transition model MA generates a state distribution at time t from the object features at time t-1, reconstructs an image at time t from the state distribution at time t, and outputs it to the discriminative model MC. The inference model MB generates a state distribution at time t from the object features at time t-1 and the actual image at time t, reconstructs an image at time t from the state distribution at time t, and outputs it to the discriminative model MC. The image output from the transition model MA is an image predicted by the transition model MA. In contrast, the image output from the inference model MB is an image generated using an actual image.

[0032] The discriminative model MC discriminates between a reconstructed image input from the transition model MA (also referred to as "derived from the transition model") and a reconstructed image input from the inference model MB (also referred to as "derived from the inference model"). The discriminative model MC discriminates whether the input image is derived from the transition model or the inference model, and outputs the discrimination result to the loss calculation unit 21b. A correct answer label indicating whether the input image is derived from the transition model or the inference model is input to the loss calculation unit 21b. The loss calculation unit 21b outputs the error between the discrimination result of the discriminative model MC and the correct answer label as a loss to the learning unit 22b.

[0033] Similar processing is performed on the image at time t. That is, the transition model MA generates a state distribution at time t+1 from the object features at time t, reconstructs the image at time t+1 from the state distribution, and outputs it to the discriminative model MC. The inference model MB generates a state distribution at time t+1 from the object features at time t and the actual image at time t+1, reconstructs the image at time t+1 from the state distribution, and outputs it to the discriminative model MC. The discriminative model MC discriminates whether the input image is derived from the transition model or the inference model, and outputs the discrimination result to the loss calculation unit 21b. The loss calculation unit 21b outputs the error between the discrimination result of the discriminative model MC and the correct label as a loss to the learning unit 22b.

[0034] The learning unit 22b then updates the parameters of the discrimination model MC so that the sum of the losses input from the two loss calculation units 21b is reduced. In this manner, the discrimination model is trained. Note that in the above example, two loss calculation units 21b are used, but in practice, images at each time point can be input to one loss calculation unit 21b to calculate the loss at each time point and output the calculated loss to the learning unit 22b.

[0035] In the above example, the learning device 100b learns the discriminative model MC using reconstructed images (i.e., still images) input from the transition model MA and the inference model MB. Alternatively, the learning device 100b may learn the discriminative model MC based on a series of reconstructed images (i.e., moving images) output from the transition model MA and the inference model MB at multiple consecutive times. In this case, the loss calculation unit 21b may calculate the error between the moving images output from the transition model MA and the moving images output from the inference model MB as the loss.

[0036] Furthermore, the learning device 100c may train the discriminative model MC using object features generated by the transition model MA and the inference model MB instead of images. In this case, the loss calculation unit 21b trains the discriminative model MC by using the error between the classification result of the discriminative model MC and the correct label as the loss. Furthermore, the learning device 100c may train the discriminative model MC using object features output from the transition model MA and the inference model MB at multiple consecutive times, i.e., time-series object features.

[0037] 5 is a block diagram showing the functional configuration of the learning device 100c for executing the third learning step. In the third learning step, the learning device 100c re-learns the transition model MA learned in the first learning step using the inference model MB learned in the first learning step and the discrimination model MC learned in the second learning step. That is, in the third learning step, the inference model MB and the discrimination model MC are not targets of learning.

[0038] Assume that there is a time series of input images from time t-1 to time t+1. As shown in FIG. 5, object features of each object at time t-1 are input to the transition model MA and the inference model MB, and an image at time t is input to the inference model MB. The transition model MA generates a state distribution at time t from the object features at time t-1, reconstructs an image at time t from the state distribution at time t, and outputs the image to the discriminative model MC. The inference model MB generates a state distribution at time t from the object features at time t-1 and the actual image at time t, reconstructs an image at time t from the state distribution at time t, and outputs the image to the discriminative model MC. The discriminative model MC discriminates whether the input image originates from the transition model or the inference model, and outputs the discrimination result to the loss calculation unit 21c.

[0039] The loss calculation unit 21c receives a correct label indicating whether the input image is derived from the transition model or the inference model. The loss calculation unit 21c calculates a loss based on the classification result of the discriminative model MC and the correct label, and outputs the loss to the learning unit 22c. The learning unit 22c then retrains the transition model MA so that the classification accuracy of the discriminative model MC decreases. For example, the loss calculation unit 21c generates a negative value (a value with a negative sign) of the error between the classification result of the discriminative model MC and the correct label, or the reciprocal of the error from the correct label, as a loss, and the learning unit 22c updates the parameters of the transition model MA so as to reduce the loss. In this manner, the transition model MA is retrained.

[0040] [Learning Process] Next, we will explain the learning process performed by the learning device 100. Figure 6 shows the main routine of the learning process performed by the learning device 100. This process is realized by the processor 13 shown in Figure 2 executing a program prepared in advance and operating as each element shown in Figures 3 to 5.

[0041] The learning process is performed through the three learning steps described above. First, in the first learning step, the learning device 100a shown in Fig. 3 learns the transition model MA and the inference model MB (step S10). Next, in the second learning step, the learning device 100b shown in Fig. 4 learns the discrimination model MC (step S20). Finally, in the third learning step, the learning device 100c shown in Fig. 5 re-learns the transition model MA (step S30).

[0042] FIG. 7 is a flowchart of the learning process for the transition model and the inference model. First, the transition model MA acquires object features of each object, and the inference model MB acquires the object features of each object and an image at the next time (step S11). Next, the transition model MA predicts a state distribution at the next time based on the object features, and the inference model MB predicts a state distribution at the next time based on the object features and the image at the next time (step S12). Next, the loss calculation unit 21a calculates the inter-distribution distance between the state distributions output by the transition model MA and the inference model MB (step S13). The loss calculation unit 21a also reconstructs an image at the next time from the state distribution output by the inference model MB and calculates the reconstruction error with the actual image at the next time (step S14). Next, the learning unit 22a trains the transition model MA and the inference model MB so as to reduce the inter-distribution distance and the reconstruction error (step S15). The process then returns to the main routine of FIG. 6.

[0043] FIG. 8 is a flowchart of the discriminative model training process. First, the transition model MA acquires object features for each object, and the inference model MB acquires the object features and the image for the next time for each object (step S21). Next, the transition model MA predicts the state distribution for the next time based on the object features, and the inference model MB predicts the state distribution for the next time based on the object features and the image for the next time (step S22). Next, the transition model MA and the inference model MB output the prediction results to the discriminative model MC. Note that, as described above, the prediction results output by the transition model MA and the inference model MB may be still images or videos derived from each model, object features (object-centric representation), or time-series object features. The discriminative model MC outputs the classification results to the loss calculation unit 21b. The loss calculation unit 21b calculates the classification error between the classification result by the discriminative model MC and the correct label (step S23). Next, the learning unit 22b trains the discriminative model MC to reduce the classification error (step S24). Then, the process returns to the main routine of FIG.

[0044] FIG. 9 is a flowchart of the transition model retraining process. First, the transition model MA acquires object features for each object, and the inference model MB acquires the object features and the image at the next time for each object (step S31). Next, the transition model MA predicts the state distribution at the next time based on the object features, and the inference model MB predicts the state distribution at the next time based on the object features and the image at the next time (step S32). Next, the transition model MA and the inference model MB input their prediction results to the discriminative model MC. Note that, as described above, the prediction results output by the transition model MA and the inference model MB may be still images or videos derived from each model, object features (object-centric representation), or time-series object features. The discriminative model MC outputs the classification results to the loss calculation unit 21c. The loss calculation unit 21c calculates the loss using the classification results from the discriminative model MC and the correct label (step S33). Next, the learning unit 22c retrains the transition model MA so that the classification accuracy of the discriminative model MC decreases (step S34). Then, the process returns to the main routine of FIG. 6, and the learning process ends.

[0045] [Effects] As described above, in the first embodiment, a discriminative model MC that distinguishes between images derived from a transition model and images derived from an inference model is generated, and the transition model MA is re-trained so that discrimination by the discriminative model MC becomes difficult, i.e., so that discrimination accuracy decreases. As a result, the transition model MA can predict a plausible image at a certain time as a predicted image at that time. As a result, even if the prediction result of the transition model MA differs from the result of the inference model MB, even if a plausible future prediction is made taking into account probabilistic behavior, it is possible to prevent the transition from being judged as correct and to prevent results that look unnatural.

[0046] [Example of Future Prediction] Next, a description will be given of an example of future prediction using the transition model MA and the inference model MB learned by the above-described learning device 100. Fig. 10 schematically shows an example of future prediction using the transition model MA and the inference model MB.

[0047] Now, suppose images from time t0 to t2 are given. In this case, the prediction device predicts the state using the inference model MB from time t0 to t2, when images exist. That is, at time t0, the prediction device inputs the image at time t0 to the inference model MB. Furthermore, at time t1, the prediction device inputs the object feature quantities of each object at time t generated by the inference model MB and the actual image at time t1 to the inference model MB. Furthermore, at time t2, the prediction device inputs the object feature quantities of each object at time t1 generated by the inference model MB and the actual image at time t2 to the inference model MB.

[0048] On the other hand, after time t3, since no actual images exist, the prediction device makes predictions using the transition model MA. That is, at time t3, the prediction device inputs the object feature values ​​of each object at time t2 generated by the inference model MB into the transition model MA. Furthermore, at time t4, the prediction device inputs the object feature values ​​of each object at time t3 generated by the transition model MA into the transition model MA. In this way, future predictions can be made using the transition model MA even after time t3, when no actual images exist.

[0049] [Modifications] Next, modifications of the first embodiment will be described. The following modifications can be applied in appropriate combination. (Modification 1) In the above embodiment, when training the discriminant model MC, as shown in FIG. 4, images at each time point are input to the discriminant model MC, the loss is calculated, and the discriminant model MC is updated. Instead, the discriminant model MC may be trained using images that have passed through the transition model MA and the inference model MB multiple times at consecutive times.

[0050] 11 shows a method for training a discriminative model according to Modification 1. As shown in the figure, the object feature quantities of each object at time t-2 are input to the transition model MA and the inference model MB, the object feature quantities output by these are input to the transition model MA and the inference model MB, respectively, and the reconstructed image output by these is input to the discriminative model MC. The discriminative model MC outputs a discrimination result for the reconstructed image of the transition model MA and the inference model MB at time t, and the loss calculation unit 21b calculates the loss using this discrimination result. In this way, the discriminative model MC may be trained using the results of passing the object feature quantities at a certain time through the transition model MA and the inference model MB multiple times in succession.

[0051] The reason for performing such processing is as follows: When the transition model MA finally obtained by the learning device 100 is put into practical use, it is conceivable to predict future temporal transitions of an object over multiple time periods without any actual images, as described with reference to Fig. 10. Therefore, the above-described method makes it possible to train the discrimination model MC so as to be able to accurately discriminate the results of passing the transition model MA multiple times in succession.

[0052] This technique can also be applied when relearning the transition model MA after training the discrimination model MC. That is, when relearning the transition model MA, the learning device 100c may use the results of passing the transition model MA and the inference model MB multiple times in succession to relearn the transition model MA so that the discrimination accuracy of the discrimination model MC decreases.

[0053] (Variation 2) In the above learning process, the first to third learning steps are performed using all of the prepared input images, and the learning process is completed, as shown in Fig. 6. Alternatively, the prepared input images may be divided into predetermined units, and the first to third learning steps shown in Fig. 6 may be repeated for each unit.

[0054] (Variation 3) In the first learning step, some action may be input to the transition model MA and the inference model MB. For example, when the method of the above embodiment is applied to predicting an environment in which an object is moved using a robot arm, an action specifying the operation of the robot arm may be input to the transition model MA and the inference model MB at a certain time. This allows the transition model MA to predict future temporal transitions when the above action is performed.

[0055] (Variation 4) In the above embodiment, a still image or a video image is input to the transition model MA and the inference model MB, but this image does not have to be an image taken by a normal camera, and may be data generated by a sensor, etc. For example, a distance image measured using a depth camera or point cloud data generated using a lidar can also be used as the input image in this embodiment.

[0056] Furthermore, auxiliary data may be input to the transition model MA and the inference model MB in addition to the image. For example, audio data collected by a microphone, output data from sensors such as a weight sensor or a pressure sensor, etc. may be input to the transition model MA and the inference model MB along with the image. In this case, two methods for using the auxiliary data are possible. In the first method, the auxiliary data may be used as a target for transition prediction, similar to the image described above. That is, in the first learning step, the learning device 100a may calculate the distribution distances and reconstruction errors of the state distributions for the auxiliary data in addition to the image, and then learn the transition model MA and the inference model MB. On the other hand, in the second method, the auxiliary data may not be used as a target for transition prediction, but may be used solely as auxiliary data for image transition prediction. In this case, the auxiliary data may be treated in transition prediction in the same way as the action in the above-described variant 3.

[0057] [Application Example] The above-described embodiment can be applied to the fields of medicine and healthcare. For example, the above-described embodiment can be applied to transition prediction and control of a medical robot that assists in surgery or other procedures on a patient. In this case, the medical robot uses the transition model MA and inference model MB (hereinafter referred to as the "world model") learned by the learning device 100 of the embodiment to grasp the environment in the operating room, the patient's physical condition, the instruments and procedures required for surgery, and the like. Specifically, the medical robot can grasp the patient's internal condition in real time based on information obtained from cameras and sensors, and select the instruments required for surgery and adjust the procedures.

[0058] An example of a specific process when a surgical robot is used as a medical robot is as follows: (1) The surgical robot is equipped with cameras and sensors to grasp the internal state of the patient, the position of instruments required for surgery, and procedures in real time. (2) The world model learned according to the embodiment provides information required during surgery based on the information grasped by the surgical robot. (3) The surgical robot selects instruments required for surgery and adjusts procedures based on the information provided by the learned world model.

[0059] This allows surgical robots that use the world model to collaborate with doctors and nurses. Surgical robots can operate autonomously in cooperation with doctors and nurses by using a world model that can predict the movements of doctors and nurses. Surgical robots can also use predictive models to accurately recognize the patient's condition, allowing them to perform minimally invasive surgery.

[0060] 12 is a block diagram showing the configuration of a learning device according to the second embodiment. A learning device 70 according to the second embodiment includes a model acquisition unit 71, a discrimination model learning unit 72, and a re-learning unit 73.

[0061] 13 is a flowchart of processing by a learning device 70 according to the second embodiment. The model acquisition unit 71 acquires a first prediction model that generates a first predicted feature value indicating the state of each object at the next time based on input object feature values, and a second prediction model that generates a second predicted feature value indicating the state of each object at the next time based on the input object feature value and an image at the next time (step S71). The discriminative model training unit 72 trains a discriminative model that distinguishes between outputs from the first prediction model and the second prediction model (step S72). The re-training unit 73 re-trains the first prediction model so that the discrimination accuracy of the discriminative model decreases (step S73).

[0062] The learning device 70 of the second embodiment makes it possible to learn a model that can predict plausible temporal transitions.

[0063] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0064] (Supplementary Note 1) A learning device comprising: a model acquisition means for acquiring a first prediction model that generates a first predicted feature that indicates the state of each object at the next time based on input object feature values; and a second prediction model that generates a second predicted feature that indicates the state of each object at the next time based on the input object feature values ​​and an image at the next time; a discriminative model training means for training a discriminative model that distinguishes between an output from the first prediction model and an output from the second prediction model; and a re-training means for re-training the first prediction model so that the discrimination accuracy of the discriminative model decreases.

[0065] (Supplementary Note 2) The learning device according to Supplementary Note 1, wherein the discriminant model learning means learns the discriminant model using the first predicted feature amount and the second predicted feature amount.

[0066] (Supplementary Note 3) The learning device according to Supplementary Note 1, wherein the discriminant model training means trains the discriminant model using a first predicted image reconstructed from the first predicted feature and a second predicted image reconstructed from the second predicted feature.

[0067] (Supplementary Note 4) The learning device according to Supplementary Note 1, wherein the discriminant model learning means learns the discriminant model using a first predicted feature output by the first predictive model after repeated predictions multiple times and a second predicted feature output by the second predictive model after repeated predictions multiple times.

[0068] (Supplementary Note 5) The learning device according to Supplementary Note 1, wherein the re-learning means re-learns the first prediction model so that the discrimination accuracy of the discrimination model between a first predicted feature output by the first prediction model after repeated predictions multiple times and a second predicted feature output by the second prediction model after repeated predictions multiple times decreases.

[0069] (Supplementary Note 6) The learning device described in Supplementary Note 1, wherein the first prediction model predicts a first state distribution for each object at a next time and generates the first predicted feature based on the first state distribution, and the second prediction model predicts a second state distribution for each object at a next time and generates the second predicted feature based on the second state distribution.

[0070] (Supplementary Note 7) The learning device according to Supplementary Note 6, further comprising: a prediction model learning means for learning the first prediction model and the second prediction model; wherein the prediction model learning means learns the first prediction model and the second prediction model so that a distance between the first state distribution and the second state distribution is reduced and an error between an image reconstructed from the second predicted feature and an actual image at the next time is reduced; and wherein the model acquisition means acquires the first prediction model and the second prediction model learned by the prediction model learning means.

[0071] (Supplementary Note 8) The learning device according to Supplementary Note 1, wherein the object includes a patient and a medical instrument, the first prediction model outputs first predicted features indicating the states of the patient and the medical instrument at a next time, and the second prediction model outputs second predicted features indicating the states of the patient and the medical instrument at a next time.

[0072] (Supplementary Note 9) A learning method executed by a computer, comprising: acquiring a first prediction model that generates a first predicted feature that indicates a state of each object at a next time based on input object feature values; and a second prediction model that generates a second predicted feature that indicates a state of each object at a next time based on the input object feature values ​​and an image at the next time; learning a discrimination model that distinguishes between an output from the first prediction model and an output from the second prediction model; and re-learning the first prediction model so that the discrimination accuracy of the discrimination model decreases.

[0073] (Supplementary Note 10) A recording medium having recorded thereon a program that causes a computer to execute the following processes: acquire a first prediction model that generates first predicted features indicating the state of each object at the next time based on input object features; and a second prediction model that generates second predicted features indicating the state of each object at the next time based on the input object features and an image at the next time; learn a discrimination model that distinguishes between outputs from the first prediction model and the second prediction model; and re-learn the first prediction model so that the discrimination accuracy of the discrimination model decreases.

[0074] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0075] 13 Processor 21a, 21b, 21c Loss calculation unit 22a, 22b, 22c Learning unit 100a, 100b, 100c Learning device MA Transition model MB Inference model MC Discrimination model

Claims

1. a model acquisition means for acquiring a first prediction model that generates a first predicted feature quantity that indicates the state of each object at the next time based on an input object feature quantity, and a second prediction model that generates a second predicted feature quantity that indicates the state of each object at the next time based on the input object feature quantity and an image at the next time; a discriminative model training means for training a discriminative model that distinguishes between an output from the first prediction model and an output from the second prediction model; a re-learning means for re-learning the first prediction model so that the discrimination accuracy of the discrimination model decreases; A learning device comprising:

2. 2. The learning device according to claim 1, wherein the discriminant model learning means learns the discriminant model using the first predicted feature amount and the second predicted feature amount.

3. 2. The learning device according to claim 1, wherein the discriminant model learning means learns the discriminant model using a first predicted image reconstructed from the first predicted feature and a second predicted image reconstructed from the second predicted feature.

4. 2. The learning device according to claim 1, wherein the discriminant model learning means learns the discriminant model using a first predicted feature output by the first predictive model after repeated predictions a plurality of times and a second predicted feature output by the second predictive model after repeated predictions a plurality of times.

5. 2. The learning device according to claim 1, wherein the re-learning means re-learns the first prediction model so that the discrimination accuracy of the discrimination model between a first predicted feature output by the first prediction model after repeated predictions multiple times and a second predicted feature output by the second prediction model after repeated predictions multiple times decreases.

6. the first prediction model predicts a first state distribution of each object at a next time point, and generates the first predicted feature based on the first state distribution; The learning device according to claim 1 , wherein the second prediction model predicts a second state distribution of each object at a next time point, and generates the second predicted feature based on the second state distribution.

7. a prediction model learning means for learning the first prediction model and the second prediction model, the prediction model learning means learns the first prediction model and the second prediction model so that a distance between the first state distribution and the second state distribution becomes small and an error between an image reconstructed from the second predicted feature amount and an actual image at the next time point becomes small; The learning device according to claim 6 , wherein the model acquisition means acquires the first prediction model and the second prediction model learned by the prediction model learning means.

8. the object includes a patient and a medical instrument; the first prediction model outputs a first prediction feature indicating a state of the patient and the medical instrument at a next time; The learning device according to claim 1 , wherein the second prediction model outputs second prediction features that indicate the states of the patient and the medical instrument at a next time.

9. 1. A computer-implemented learning method comprising: obtaining a first prediction model that generates a first predicted feature amount that indicates a state of each object at a next time point based on an input object feature amount; and a second prediction model that generates a second predicted feature amount that indicates a state of each object at a next time point based on the input object feature amount and an image at the next time point; training a discriminative model that distinguishes between an output from the first predictive model and an output from the second predictive model; A learning method for re-learning the first prediction model so that the discrimination accuracy of the discriminative model decreases.

10. obtaining a first prediction model that generates a first predicted feature amount that indicates a state of each object at a next time point based on an input object feature amount; and a second prediction model that generates a second predicted feature amount that indicates a state of each object at a next time point based on the input object feature amount and an image at the next time point; training a discriminative model that distinguishes between an output from the first predictive model and an output from the second predictive model; A program that causes a computer to execute a process of re-learning the first prediction model so that the discrimination accuracy of the discrimination model decreases.