Information processing apparatus, information processing method, and information processing program

Abstract

To provide a technique capable of stably obtaining a correct physical quantity in a technique for converting a latent variable output from a world model into a physical quantity.SOLUTION: The information processing apparatus includes a relation deriving unit that derives, for an object to be inferred or predicted in a world model, a relation between physical quantities from a pair of latent variables corresponding to different times output from the world model, using a regression model that receives the pair of latent variables as input and outputs the relation between the physical quantities corresponding to the different times.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] A world model is known as a technique for inferring or predicting the state of an object included as a subject in a video image. Patent Document 1, for example, discloses a world model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2023-143222 Summary of the Invention [Problem to be solved by the invention]

[0004] In a world model, the state of an object to be inferred or predicted is represented (encoded) as a latent variable. Therefore, in order to perform information processing using a world model, it is necessary to convert the latent variables output from the world model into physical quantities such as position and orientation. However, in a configuration using a regression model that converts latent variables at a certain time into physical quantities at that time (particularly physical quantities relative to an absolute coordinate system), the regression result (physical quantity obtained by the conversion) may become unstable. For example, if the shape of an object is symmetric, the object coordinate system becomes unstable, resulting in an unstable regression result. The instability of the regression result manifests itself, for example, as discontinuity in the regression result (the difference between physical quantities corresponding to two times does not decrease even when the difference between two times is reduced and the difference between the states corresponding to those two times is reduced).

[0005] The present disclosure has been made in consideration of the above-described problems, and an exemplary purpose thereof is to provide a technology for converting latent variables output from a world model into physical quantities, which is capable of stably obtaining correct physical quantities. [Means for solving the problem]

[0006] An information processing device according to an exemplary aspect of the present disclosure includes a relationship derivation means for deriving a relationship between physical quantities from a pair of latent variables using a regression model that receives as input a pair of latent variables corresponding to each of different times output from a world model that represents the state of the object as a latent variable, and outputs a relationship between the physical quantities corresponding to each of the different times.

[0007] An information processing device according to an exemplary aspect of the present disclosure includes a model generation means for generating a regression model for an object included in a world model that expresses the state of the object as latent variables, which takes as input a pair of latent variables corresponding to each of different times output from the world model, and outputs a relationship between physical quantities corresponding to each of the different times, and the model generation means generates the regression model so that the correspondence between input and output of the regression model best approximates the correspondence between the pair of latent variables output from the world model and the relationship between the physical quantities obtained from a dataset.

[0008] An information processing method according to an exemplary aspect of the present disclosure includes a processor using a regression model that receives as input pairs of latent variables corresponding to different times output from a world model that represents the state of the object, and outputs relationships between physical quantities corresponding to each of the different times, to derive relationships between the physical quantities from the pairs of latent variables for the object included in the world model, the world model representing the state of the object as latent variables.

[0009] An information processing method according to an exemplary aspect of the present disclosure includes a processor generating a regression model for an object included in a world model that represents the state of the object as latent variables, the regression model taking as input pairs of latent variables corresponding to each of different times output from the world model and outputting relationships between physical quantities corresponding to each of the different times, wherein the processor generates the regression model so that the correspondence between input and output of the regression model best approximates the correspondence between the pairs of latent variables output from the world model and the relationships between the physical quantities obtained from a dataset.

[0010] An information processing program according to an exemplary aspect of the present disclosure causes a processor to execute a relationship derivation process for an object included in a world model that expresses the state of the object as latent variables, using a regression model that takes as input a pair of latent variables corresponding to each of different times output from the world model and outputs a relationship between the physical quantities corresponding to each of the different times, to derive the relationship between the physical quantities from the pair of latent variables.

[0011] An information processing program according to an exemplary aspect of the present disclosure causes a processor to execute a model generation process for generating a regression model for an object included in a world model that expresses the state of the object as latent variables, the model generation process taking as input pairs of latent variables corresponding to each of different times output from the world model and outputting relationships between physical quantities corresponding to each of the different times, wherein in the model generation process, the processor generates the regression model such that the correspondence between input and output of the regression model best approximates the correspondence between the pairs of latent variables output from the world model and the relationships between the physical quantities obtained from a dataset. [Effects of the Invention]

[0012] According to an exemplary aspect of the present disclosure, an exemplary effect is achieved in that a technology for converting latent variables output from a world model into physical quantities can be provided that is capable of stably obtaining correct physical quantities. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 3] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 4] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 5] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 6] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 7] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 8] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 9] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 10] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 11] FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to the present disclosure. [Figure 12] FIG. 10 is a schematic diagram illustrating the operation of a world model used by an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be exemplified. However, the present invention is not limited to the following exemplary embodiments, and various modifications are possible within the scope indicated in the claims. For example, embodiments obtained by appropriately combining the technologies (part or all of an object or method) adopted in the following exemplary embodiments may also be included in the scope of the present invention. Further, embodiments obtained by appropriately omitting part of the technologies adopted in the following exemplary embodiments may also be included in the scope of the present invention. Also, the effects mentioned in the following exemplary embodiments are merely examples of the effects expected in those exemplary embodiments and do not define the scope of the present invention. That is, embodiments that do not exhibit the effects mentioned in the following exemplary embodiments may also be included in the scope of the present invention.

[0015] 〔World Model〕 In the exemplary embodiments described below, the world model M0 is used. Therefore, before entering the description of the exemplary embodiments, the world model M0 will be described with reference to FIG. 12. FIG. 12 is a schematic diagram showing the operation of the world model M0.

[0016] The world model is a model that approximates the structure of the external world, constructed by machine learning using limited information of the external world (real world or virtual world). In the present disclosure, consider the world model M0 constructed by object-centered representation learning. In object-centered representation learning, using a moving image I including objects S1, S2, …, S m as subjects, a world model M0 is constructed that represents the state of the object S j as a latent variable. Here, m is an arbitrary natural number of 1 or more. The moving image I is composed of an image I1 corresponding to time t1, an image I2 (t1 < t2) corresponding to time t2, …, and an image I n corresponding to time t n (t nー1 < t n ). Here, n is an arbitrary natural number of 2 or more. Each image I iis a still image, also called a frame or a frame image. Here, i is a natural number between 1 and n. When the world model M0 is a model that approximates the structure of the real world, the moving image I may be, for example, a live-action image that represents the real world. When the world model M0 is a model that approximates the structure of a virtual world, the moving image I may be, for example, a CG (Computer Graphics) image that represents the virtual world. In the world model M0 constructed by object-centric representation learning, each object S included in the world model M0 j Regarding each time t i The state (physical quantities such as position and posture) in i (j) In the following, the objects S1, S2, ..., S m The set of, i.e., Objects S1, S2, …, S included in world model M0 m The set of objects {S1, S2, ..., S m}" is also written.

[0017] The world model M0 is i Image I corresponds to i and object S j latent variables for time t i-1 The latent variable Z corresponding to i-1 (j) So, the object S j latent variables for time t i The latent variable Z corresponding to i (j) Here, j is an arbitrary natural number between 1 and m. The world model M0 is a set of objects S j The above inferences for each time t included in the inference period can be performed in parallel. i The output of the world model M0 in i (1) , latent variable Z for object S2 i (2) , …, and object S m The latent variable Z i (m)The set of {Z i (j) |j=1,2,…,m}.

[0018] In addition, the world model M0 predicts the object S in the prediction period following the inference period. j latent variables for time t i-1 The latent variable Z corresponding to i-1 (j) From that object S j latent variables for time t i The latent variable Z corresponding to i (j) The world model M0 is derived by j In this case, the above predictions for each time t included in the prediction period can be performed in parallel. i The output of the world model M0 in i (1) , latent variable Z for object S2 i (2) , …, and object S m The latent variable Z i (m) The set of {Z i (j) |j=1,2,…,m}.

[0019] Examples of well-known world models include ViMON, OP3, G-SWM, and GATSBI. However, the world model M0 that can be used in the exemplary embodiments described below is not limited to these. Any world model that expresses the state of an object as a latent variable can be used in each of the exemplary embodiments described below.

[0020] As examples that do not limit the present invention, the world model M0 can be used, for example, for (1) inferring and predicting the movement of objects in virtual space in computer games, (2) inferring and predicting the movement of objects in real space in physical simulations, (3) inferring and predicting objects (e.g., obstacles) in real space in automatic driving or control of moving objects (automobiles, ships, aircraft, etc.), and (4) inferring and predicting objects (e.g., workpieces) in real space in control of robotic arms.

[0021] Exemplary Embodiment 1 A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0022] (Configuration of information processing device) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1.

[0023] As shown in FIG. 1, the information processing device 1 includes a relationship derivation unit 11.

[0024] The relation derivation unit 11 calculates the relationship between each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ (t i <t i’ ) corresponding latent variable Z i (j),Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) from the two times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) It is a means to derive the following. i ,t i’ may be two adjacent (consecutive) times, such as time t1 and time t2, or may be two non-adjacent (non-consecutive) times, such as t1 and t3.

[0025] The relation derivation unit 11 determines the pair (Z i (j) ,Z i’ (j) ) from the relation ρ i,i’ (j) To derive, we use a regression model M1. The input of the regression model M1 is the latent variable Z output from the world model M0. i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) The output of the regression model M1 is the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) Note that the two latent variables Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) is the sum of these two latent variables Z i (j) ,Z i’ (j) For example, two latent variables Zi (j) ,Z i’ (j) are represented by three-dimensional vectors, these two latent variables Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) may be input to the regression model M1 as a 6-dimensional vector obtained by concatenating two 3-dimensional vectors.

[0026] As a non-limiting example, at each time t i The physical quantity o corresponding to i (j) is the time t i Object S in j Position r j (t i ) in this case, the relation ρ i,i’ (j) is the time t i Object S in j Position r j (t i ) to time t i’ Object S in j Position r j (t i’ ) in this case, the position r j (t i ), position r j (t i’ ), and the relation ρ i,i’ (j) are each represented by a three-dimensional vector, and there is a relationship between these vectors, r j (t i’ )=ρ i,i’ (j) +r j (t i ) relationship holds.

[0027] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) is the time t iObject S in j Posture q j (t i ) in this case, the relation ρ i,i’ (j) is the time t i Object S in j Posture q j (t i ) to time t i’ Object S in j Posture q j (t i’ ) can be a change in posture to posture q j (t i ), posture q j (t i’ ), and the relation ρ i,i’ (j) are each represented by a quaternion, and there is a relationship between these quaternions, q j (t i’ )=ρ i,i’ (j) *q j (t i ) relationship holds.

[0028] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) is the time t i Object S in j Position r j (t i ) and posture q j (t i ) can be a combination of the relation ρ i,i’ (j) is the time t i Object S in j Position r j (t i ) to time t i’ Object S in j Position r j (t i’ ) and time t i Object S in j Attitude θ j (t i ) to time t i’ Object S inj Attitude θ j (t i’ ) in combination with a change in posture.

[0029] In addition, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) may be used as an input to the regression model M1. In this case, the relation derivation unit 11 uses this regression model M1 to derive the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) From the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) is derived.

[0030] Also, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j)-Z i (j) ) may be used as an input of the regression model M1. In this case, the relation derivation unit 11 uses this regression model M1 to derive the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j) ) to obtain the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) is derived.

[0031] (Flow of information processing method) The flow of the information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the information processing method S1.

[0032] The information processing method S1 includes a relationship derivation process S11 as shown in Fig. 2. The information processing method S1 is executed by, for example, the information processing device 1 or a computer.

[0033] The relation derivation process S11 is a process of deriving a relation between each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ (t i <t i’ ) corresponding latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j)) from the two times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Here, the two time t i ,t i’ may be two consecutive times such as time t1 and time t2, or may be two non-consecutive times such as t1 and t3. Note that the relationship derivation process S11 is executed by, for example, the relationship derivation unit 11 of the information processing device 1 or a processor of a computer.

[0034] In the relation derivation process S11, the pair (Z i (j) ,Z i’ (j) ) from the relation ρ i,i’ (j) To derive, we use a regression model M1. The input of the regression model M1 is the latent variable Z output from the world model M0. i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) The output of the regression model M1 is the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) Note that the two latent variables Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) is the sum of these two latent variables Z i (j) ,Z i’ (j) For example, two latent variables Z i (j) ,Zi’ (j) are represented by three-dimensional vectors, these two latent variables Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) may be input to the regression model M1 as a 6-dimensional vector obtained by concatenating two 3-dimensional vectors.

[0035] In addition, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) may be used as an input to the regression model M1. In this case, in the relation derivation process S11, the regression model M1 is used to calculate the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) From the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) is derived.

[0036] Also, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j)and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j) ) may be used as an input for the regression model M1. In this case, in the relation derivation process S11, the regression model M1 is used to calculate the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j) ) to obtain the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) is derived.

[0037] In FIG. 2, as an example that does not limit the present invention, when i′=i+1, that is, when two adjacent times t i ,t i+1 The physical quantity o corresponding to i (j) ,o i+1 (j) The relationship between i,i+1 (j) In this case, the derivation process S11 is repeated n-1 times to derive the relation ρ 1,2 (j) , the relationship ρ 2,3 (j) , …, ρ n-1,n (j) are derived sequentially.

[0038] In order to improve the stability of the regression, i and time t i’ It is preferable that the time interval between t i and time t i’ The longer the time interval between t and t, the more difficult it becomes to distinguish between pose changes of an object with rotational symmetry (for example, distinguishing between no rotation and a 180° rotation of an object with two-fold rotational symmetry). i ,t i+1 The physical quantity o corresponding to i (j) ,o i+1 (j) The relationship between i,i+1 (j) The configuration that derives is the best mode.

[0039] (Effects of information processing device and information processing method) In the information processing device 1 and the information processing method S1, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) Rather than a recursive structure, we use a different time t i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) This allows regression that is independent of the coordinate system that represents the physical quantity. This reduces the instability of the regression results. i ,t i’ The correct physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’(j) can be obtained stably.

[0040] Also, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) In the configuration in which the regression model is regressed, the problem of not being able to guarantee operation for non-learning objects (objects not included as subjects in the images used to learn the regression model) can arise because the object coordinate system is not defined. i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Since a regression structure is adopted, such a problem is unlikely to occur.

[0041] As an example that does not limit the present invention, the physical quantity o obtained by the information processing device 1 is i (j) ,o i’ (j) The relationship between i,i’ (j)The information processing device 1 can be used, for example, for (1) inferring and predicting the motion of an object in a virtual space in a computer game, (2) inferring and predicting the motion of an object in a real space in a physical simulation, (3) inferring and predicting an object (e.g., an obstacle) in a real space in the automatic driving or control of a moving object (such as an automobile, a ship, or an aircraft), and (4) inferring and predicting an object (e.g., a workpiece) in a real space in the control of a robot arm. When applied to the automatic driving of a moving object, the information processing device 1 can be used to predict the displacement and posture change of an obstacle (such as a person, an animal, or another moving object) and automatically drive the moving object so that it does not collide with the obstacle. When applied to the control of a robot arm, the information processing device 1 can be used to predict the displacement and posture change of a workpiece and control the robot arm so that a hand attached to the tip of the robot arm reaches the workpiece.

[0042] Exemplary Embodiment 2 A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0043] (Configuration of information processing device) The configuration of the information processing device 1A will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 1A.

[0044] As shown in FIG. 3, the information processing device 1A is obtained by adding a latent variable derivation unit 12 to the information processing device 1 (see Exemplary Embodiment 1).

[0045] The latent variable derivation unit 12 calculates the latent variable for each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) is a means to derive the above using the world model M0. i is included in the inference period, the latent variable derivation unit 12 i The corresponding image Ii and time t i-1 The latent variable Z corresponding to i-1 (j) and into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ is included in the inference period, at time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same way. Meanwhile, at time t i is included in the prediction period, the latent variable derivation unit 12 i-1 The latent variable Z corresponding to i-1 (j) By inputting this into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ If is included in the forecast period, then time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same way.

[0046] The relation derivation unit 11 of the information processing device 1A calculates the relation between two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) Pair (Z i (j) ,Zi’ (j) ) from the two times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) is derived.

[0047] (Flow of information processing method) The flow of the information processing method S1A will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the information processing method S1A.

[0048] 4, the information processing method S1A is obtained by adding a latent variable derivation process S12 to the information processing method S1 (see exemplary embodiment 1). Note that the information processing method S1A is executed by, for example, an information processing device 1A or a computer.

[0049] As shown in FIG. 4, the latent variable derivation process S12 is executed before the relation derivation process S11.

[0050] The latent variable derivation process S12 is a process of deriving latent variables for each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) This is a process to derive the above using the world model M0. i is included in the inference period, in the latent variable derivation process S12, i The corresponding image Ii and time t i-1 The latent variable Z corresponding to i-1 (j) and into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’is included in the inference period, at time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same way. Meanwhile, at time t i is included in the prediction period, in the latent variable derivation process S12, i-1 The latent variable Z corresponding to i-1 (j) By inputting this into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ is included in the inference period, at time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same manner. The latent variable derivation process S12 is executed by, for example, the latent variable derivation unit 12 of the information processing device 1A or by a processor of a computer.

[0051] In the relation derivation process S11 of the information processing method S1A, two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) from the two times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) is derived.

[0052] In FIG. 4, as an example that does not limit the present invention, when i′=i+1, that is, when two adjacent times t i ,t i+1 The physical quantity o corresponding to i (j) ,o i+1 (j) The relationship betweeni,i+1 (j) In this case, the relation derivation process S11 is repeated n-1 times to derive the relation ρ 1,2 (j) , the relationship ρ 2,3 (j) , …, ρ n-1,n (j) are derived sequentially.

[0053] (Effects of information processing device and information processing method) In the information processing device 1A and the information processing method S1A, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) Rather than a recursive structure, we use a different time t i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) This allows regression that is independent of the coordinate system that represents the physical quantity. This reduces the instability of the regression results. i ,t i’ The correct physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) can be obtained stably.

[0054] Also, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j)In the configuration in which the regression model is regressed, the problem of not being able to guarantee operation for non-learning objects (objects not included as subjects in the images used to learn the regression model) can arise because the object coordinate system is not defined. i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Since a regression structure is adopted, such a problem is unlikely to occur.

[0055] Exemplary Embodiment 3 A third exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0056] (Configuration of information processing device) The configuration of the information processing device 1B will be described with reference to Fig. 5. Fig. 5 is a block diagram showing the configuration of the information processing device 1B.

[0057] 5, the information processing device 1B is obtained by adding a physical quantity calculation unit 13, an initial physical quantity acquisition unit 14, and a coordinate transformation unit 15 to the information processing device 1 (see exemplary embodiment 1). Note that the information processing device 1B may further include a latent variable derivation unit 12, similar to the information processing device 1A (see exemplary embodiment 2).

[0058] The physical quantity calculation unit 13 calculates the physical quantity of each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j At time t j The physical quantity o corresponding to i (j) and the two times t j ,tj' corresponding physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) From this, at time t j’ The physical quantity o corresponding to i’ (j) The physical quantity calculation unit 13 calculates the physical quantity o corresponding to the time ti. i (j) , and the physical quantity o corresponding to the time ti′ calculated by the physical quantity calculation unit 13 i’ (j) are the physical quantities o1 corresponding to the initial time t1. (j) It is a relative physical quantity.

[0059] As a non-limiting example, at each time t i The physical quantity o corresponding to i (j) is the time t i Object S in j Position r j (t i ) in this case, the relation ρ i,i’ (j) is the time t i Object S in j Position r j (t i ) to time ti’ Object S in j Position r j (t i’ ) in this case, the position r j (t i ), position r j (t i’ ), and the relation ρ i,i’ (j) are each expressed by a three-dimensional vector. In this case, the physical quantity calculation unit 13 calculates the position r j Set (t1) to 0 (vector) and set the position r j (t2) is j (t2)=ρ 1,2 (j) +r j (t1) and calculate the position r j (t3) is j (t3)=ρ 2,3 (j) +r j (t2) and calculate..., position r j (t i’ ) to r j (t i’ )=ρ i’-1,i’ (j) +r j (t i’-1 ) is calculated according to the physical quantity calculation unit 13. i’ Position in r j (t i’ )=ρ i’-1,i’ (j) +ρ i’-2,i’-1 (j) +…+ρ 1,2 (j) is the position r at the initial time t1 j This is the relative position to (t1).

[0060] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) is the time t i Object S in j Posture q j (t i ) in this case, the relation ρ i,i’ (j)is the time t i Object S in j Posture q j (t i ) to time t i’ Object S in j Posture q j (t i’ ) can be a change in posture to posture q j (t i ), posture q j (t i’ ), and the relation ρ i,i’ (j) In this case, the physical quantity calculation unit 13 calculates the posture q j Set (t1) to 1 (quaternion) and set the posture q j (t2) as q j (t2)=ρ 1,2 (j) *q j (t1) and calculate the posture q j (t3) to q j (t3)=ρ 2,3 (j) *q j (t2) and calculate..., attitude q j (t i’ ) and q j (t i’ )=ρ i’-1,i’ (j) *q i’-1 Therefore, the physical quantity calculation unit 13 calculates the physical quantity according to the time t i’ Attitude q in j (t i’ )=ρ i’-1,i’ (j) *ρ i’-2,i’-1 (j) *…*ρ 1,2 (j) is the posture q at the initial time t1 j This is the relative attitude to (t1).

[0061] The initial physical quantity acquisition unit 14 acquires the initial physical quantity of each object S included in the object group {S1, S2, . . . , Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} jRegarding the physical quantity O1 corresponding to the initial time t1, (j) The initial physical quantity acquisition unit 14 acquires the physical quantity O1 corresponding to the initial time t1. (j) is an absolute physical quantity.

[0062] As a non-limiting example, at each time t i The physical quantity o corresponding to i (j) But at that time t i Object S in j Position r j (t i ), the initial physical quantity acquisition unit 14 calculates the object S at the initial time t1. j Absolute position R j Get (t1) (vector).

[0063] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) But at that time t i Object S in j Posture q j (t i ), the initial physical quantity acquisition unit 14 calculates the object S at the initial time t1. j Absolute posture Q j Get (t1) (quaternion).

[0064] The coordinate conversion unit 15 converts each object S included in the object group {S1, S2, ..., Sm} j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j Regarding the absolute physical quantity O1 corresponding to the initial time t1 acquired by the initial physical quantity acquisition unit 14, (j) The physical quantity calculation unit 13 calculates the time t j’ The relative physical quantity o corresponding to i’ (j) At time t j’ The absolute physical quantity O corresponding to i’ (j) It is a means to convert

[0065] As a non-limiting example, at each time t i The physical quantity o corresponding to i (j) But at that time t i Object S in j Position r j (t i ), the coordinate conversion unit 15 calculates the absolute position R1 at the initial time t1 acquired by the initial physical quantity acquisition unit 14. (j) The physical quantity calculation unit 13 calculates the time t j’ relative position r in i’ (j) At that time t j’ Absolute position R at i’ (j) =R1 (j) +r i’ (j) Convert to.

[0066] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) But at that time t i Object S in j Posture q j (t i ), the coordinate transformation unit 15 calculates the absolute attitude Q1 at the initial time t1 acquired by the initial physical quantity acquisition unit 14. (j) The physical quantity calculation unit 13 calculates the time t j’ The relative orientation q corresponding to i’ (j) At that time t j’ Absolute attitude Q i’ (j) =Q1 (j) *q i’ (j) Convert to.

[0067] The information processing device 1B (1) exclusively processes the time t i The relative physical quantity o corresponding to i (j) (2) may be configured to output only each time t i The absolute physical quantity O corresponding to i (j)(3) at each time t i The relative physical quantity o corresponding to i (j) and absolute physical quantity O i (j) and (4) at each time t i The relative physical quantity o corresponding to i (j) and absolute physical quantity O i (j) The physical quantity selected by the user may be output.

[0068] Exclusively at each time t i The relative physical quantity o corresponding to i (j) When outputting the initial physical quantity, the initial physical quantity acquisition unit 14 and the coordinate conversion unit 15 can be omitted from the information processing device 1B. i The relative physical quantity o corresponding to i (j) and absolute physical quantity O i (j) When outputting one of the physical quantities selected by the user, it is preferable to add a switching means for switching the physical quantity to be output to the physical quantity selected by the user to the information processing device 1B.

[0069] (Flow of information processing method) The flow of information processing method S1B will be described with reference to Fig. 6. Fig. 6 is a flow diagram showing the flow of information processing method S1B.

[0070] 6, the information processing method S1B is obtained by adding a physical quantity calculation process S13, an initial physical quantity acquisition process S14, and a coordinate transformation process S15 to the information processing method S1 (see exemplary embodiment 1). Similar to the information processing method S1A (see exemplary embodiment 2), the information processing method S1B may further include a latent variable derivation process S12. Note that the information processing method S1B is executed by, for example, an information processing device 1B or a computer.

[0071] 6, the physical quantity calculation process S13, the initial physical quantity acquisition process S14, and the coordinate conversion process S15 are executed after the relationship deriving process S11. However, the order of execution of the physical quantity calculation process S13 and the initial physical quantity acquisition process S14 is arbitrary. That is, the physical quantity calculation process S13 may be executed first, and then the initial physical quantity acquisition process S14, or the physical quantity calculation process S13 may be executed first, and then the initial physical quantity acquisition process S14. Alternatively, the physical quantity calculation process S13 and the initial physical quantity acquisition process S14 may be executed in parallel.

[0072] The physical quantity calculation process S13 calculates the physical quantity of each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j Regarding the time t calculated in the physical quantity calculation process S13 of the previous cycle, j The physical quantity o corresponding to i (j) and the two times t j ,tj' corresponding physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) From this, at time t j’ The physical quantity o corresponding to i’ (j) This is a process for calculating the physical quantity o corresponding to the time ti referred to in the physical quantity calculation process S13. i (j) , and the physical quantity o corresponding to the time ti′ calculated in the physical quantity calculation process S13 i’ (j) are the physical quantities o1 corresponding to the initial time t1. (j) The physical quantity calculation process S13 is executed by, for example, the physical quantity calculation unit 13 of the information processing device 1B or a processor of a computer.

[0073] The initial physical quantity acquisition process S14 is carried out by acquiring the physical quantity of each object S included in the object group {S1, S2, ..., Sm}. jAlternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j Regarding the physical quantity O1 corresponding to the initial time t1, (j) This is a process for acquiring the physical quantity O1 corresponding to the initial time t1 acquired in the initial physical quantity acquisition process S14. (j) is an absolute physical quantity. The initial physical quantity acquisition process S14 is executed by, for example, the initial physical quantity acquisition unit 14 of the information processing device 1B or by a processor of a computer.

[0074] The coordinate conversion process S15 converts the coordinates of each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j Regarding the absolute physical quantity O1 corresponding to the initial time t1 acquired in the initial physical quantity acquisition process S14, (j) The time t calculated in the physical quantity calculation process S13 is calculated using j’ The relative physical quantity o corresponding to i’ (j) At time t j’ The absolute physical quantity O corresponding to i’ (j) The coordinate conversion process S15 is executed by, for example, the coordinate conversion unit 15 of the information processing device 1B or by a processor of a computer.

[0075] In FIG. 6, as an example that does not limit the present invention, when i′=i+1, that is, when two adjacent times t i ,t i+1 The physical quantity o corresponding to i (j) ,o i+1 (j) The relationship between i,i+1 (j) In this case, the relation derivation process S11 is repeated n-1 times to derive the absolute physical quantity O2 (j) , absolute physical quantity O3 (j) , …, absolute physical quantity O n(j) are calculated sequentially.

[0076] The information processing method S1B is as follows: (1) exclusively for each time t i The relative physical quantity o corresponding to i (j) (2) may be configured to output only each time t i The absolute physical quantity O corresponding to i (j) (3) at each time t i The relative physical quantity o corresponding to i (j) and absolute physical quantity O i (j) and (4) at each time t i The relative physical quantity o corresponding to i (j) and absolute physical quantity O i (j) The physical quantity selected by the user may be output.

[0077] Exclusively at each time t i The relative physical quantity o corresponding to i (j) When outputting the initial physical quantity, the initial physical quantity acquisition process S14 and the coordinate transformation process S15 can be omitted from the information processing method S1B. i The relative physical quantity o corresponding to i (j) and absolute physical quantity O i (j) When the physical quantity selected by the user is to be output, it is preferable to add a switching process for switching the physical quantity to be output to the physical quantity selected by the user to the information processing method S1B.

[0078] (Effects of information processing method and information processing device) In the information processing device 1B and the information processing method S1B, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding toi (j) Rather than a recursive structure, we use a different time t i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) This allows regression that is independent of the coordinate system that represents the physical quantity. This reduces the instability of the regression results. As a result, i’ The correct physical quantity o corresponding to i’ (j) ,O i’ (j) can be obtained stably.

[0079] Also, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) In the configuration in which the regression model is regressed, the problem occurs that the operation cannot be guaranteed for non-learning objects (objects not included as subjects in the images used for learning the regression model) because the object coordinate system is not defined. i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Since a regression structure is adopted, such a problem is unlikely to occur.

[0080] As an example that does not limit the present invention, the physical quantity o obtained by the information processing device 1B is i’ (j) ,O i’ (j) The information processing device 1 can be used, for example, for (1) inferring and predicting the motion of objects in virtual space in computer games, (2) inferring and predicting the motion of objects in real space in physical simulations, (3) inferring and predicting objects (e.g., obstacles) in real space in the automated driving or control of moving objects (automobiles, ships, aircraft, etc.), and (4) inferring and predicting objects (e.g., workpieces) in real space in the control of robotic arms. When applied to the automated driving of moving objects, the information processing device 1 can be used to predict the position and posture of obstacles (people, animals, other moving objects, etc.) and automatically drive the moving object so that it does not collide with the obstacle. When applied to the control of robotic arms, the information processing device 1 can be used to predict the position and posture of a workpiece and control the robotic arm so that a hand attached to the tip of the robotic arm reaches the workpiece.

[0081] Exemplary Embodiment 4 A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0082] (Configuration of information processing device) The configuration of the information processing device 2 will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the configuration of the information processing device 2.

[0083] As shown in FIG. 7, the information processing device 2 includes a model generation unit 21.

[0084] The model generation unit 21 generates a model of each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ (t i <t i’ ) corresponding latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) from the two times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) This is a means for generating a regression model M1 for deriving the following. i ,t i’ may be two adjacent (consecutive) times, such as time t1 and time t2, or may be two non-adjacent (non-consecutive) times, such as t1 and t3.

[0085] To generate the regression model M1, the model generation unit 21 generates a regression model M1 by using the regression model M1 for each object S included in the object group {S1, S2, . . . , Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For each time ti, the physical quantity o i (j) The model generation unit 21 uses a data set DS including the following: The model generation unit 21 generates ... i ,t i’ The latent variable Z corresponding to i (j) ,Z i’(j) Pair (Z i (j) ,Z i’ (j) ) and the two time points t obtained from the dataset DS. i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) A regression model M1 is trained to best approximate the correspondence between

[0086] The physical quantities corresponding to each time ti that make up the data set DS are relative physical quantities o i (j) or an absolute physical quantity O i (j) Two times t i ,t i’ The absolute physical quantity O corresponding to i (j) ,O i’ (j) The relationship between two times t i ,t i’ The relative physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Because it matches.

[0087] To improve the stability of the regression, it is preferable to generate a regression model M1 whose input-output correspondence relationship satisfies the following condition (1): Furthermore, to improve the stability of the regression and further reduce accumulated errors, it is preferable to generate a regression model M1 whose input-output correspondence relationship satisfies the following condition (2) in addition to the following condition (1):

[0088] Condition (1): The input-output correspondence of the regression model M1 is i ,t i+1 The latent variable Z corresponding to i (j) ,Z i+1(j) Pair (Z i (j) ,Z i+1 (j) ) and the two time points t obtained from the dataset DS i ,t i+1 The physical quantity o corresponding to i (j) ,o i+1 (j) The relationship between i,i+1 (j) The correspondence between is best approximated.

[0089] Condition (2): The input-output correspondence of the regression model M1 is i ,t i+k The latent variable Z corresponding to i (j) ,Z i+k (j) Pair (Z i (j) ,Z i+k (j) ) and the two time points t obtained from the dataset DS i ,t i+k The physical quantity o corresponding to i (j) ,o i+k (j) The relationship between i,i+k (j) Here, k is a natural number equal to or greater than 2.

[0090] As a non-limiting example, at each time t i The physical quantity o corresponding to i (j) is the time t i Object S in j Position r j (t i ) in this case, the relation ρ i,i’ (j) is the time t i Object S in j Position r j (t i ) to time t i’ Object S in j Position r j (ti’ ) in this case, the position r j (t i ), position r j (t i’ ), and the relation ρ i,i’ (j) are each represented by a three-dimensional vector, and there is a relationship between these vectors, r j (t i’ )=ρ i,i’ (j) +r j (t i ) holds. Therefore, the model generating unit 21 calculates, for example, ρ i,i’ (j) =r j (t i’ )-r j (t i ), the relationship ρ i,i’ (j) can be calculated.

[0091] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) is the time t i Object S in j Posture q j (t i ) in this case, the relation ρ i,i’ (j) is the time t i Object S in j Posture q j (t i ) to time t i’ Object S in j Posture q j (t i’ ) can be a change in posture to posture q j (t i ), posture q j (t i’ ), and the relation ρ i,i’ (j) are each represented by a quaternion, and there is a relationship between these quaternions, q j (t i’ )=ρ i,i’ (j) *qj (t i ) holds. Therefore, the model generating unit 21 calculates, for example, ρ i,i’ (j) =q j (t i’ ) / q j (t i ), the relationship ρ i,i’ (j) can be calculated.

[0092] As an example that does not limit the invention, i The physical quantity o corresponding to i (j) is the time t i Object S in j Position r j (t i ) and posture q j (t i ) can be a combination of the relation ρ i,i’ (j) is the time t i Object S in j Position r j (t i ) to time t i’ Object S in j Position r j (t i’ ) and time t i Object S in j Attitude θ j (t i ) to time t i’ Object S in j Attitude θ j (t i’ ) in combination with a change in posture.

[0093] In addition, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) ,Z i’ (j)The difference between Z i’ (j) -Z i (j) may be used as an input to the regression model M1. In this case, the model generation unit 21 uses the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) From the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) Generate a regression model M1 to derive

[0094] Also, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j) ) may be used as an input for the regression model M1. In this case, the model generation unit 21 uses the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j)) to obtain the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) Generate a regression model M1 to derive

[0095] (Flow of information processing method) The flow of the information processing method S2 will be described with reference to Fig. 8. Fig. 8 is a flow diagram showing the flow of the information processing method S2.

[0096] The information processing method S2 includes a model generation process S21 as shown in Fig. 8. The information processing method S2 is executed by, for example, the information processing device 2 or a computer.

[0097] The model generation process S21 is a process of generating a model for each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ (t i <t i’ ) corresponding latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) from the two times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) This is a process for generating a regression model M1 for deriving the following. i ,t i’may be two adjacent (consecutive) times such as time t1 and time t2, or may be two non-adjacent (non-consecutive) times such as t1 and t3. The model generation process S21 is executed by, for example, the model generation unit 21 of the information processing device 2 or a processor of a computer.

[0098] In the model generation process S21, in order to generate the regression model M1, each object S included in the object group {S1, S2, ..., Sm} is j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For each time ti, the physical quantity o i (j) In the model generation process S21, the input / output correspondence of the regression model M1 is calculated based on the data set DS obtained from the world model M0 at two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) and the two time points t obtained from the dataset DS i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) A regression model M1 is trained to best approximate the correspondence between

[0099] The physical quantities corresponding to each time ti that make up the data set DS are relative physical quantities o i (j) or an absolute physical quantity O i (j) Two times t i ,t i’ The absolute physical quantity O corresponding to i (j) ,O i’ (j)The relationship between two times t i ,t i’ The relative physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Because it matches.

[0100] Also, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) may be used as an input for the regression model M1. In this case, in the model generation process S21, the latent variable Z i (j) ,Z i’ (j) The difference between Z i’ (j) -Z i (j) From the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) Generate a regression model M1 to derive

[0101] Also, the latent variable Z i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) as the input of the regression model M1, we use the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i(j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j) ) may be used as an input for the regression model M1. In this case, in the model generation process S21, the latent variable Z i (j) and the latent variable Z i’ (j) and difference Z i’ (j) -Z i (j) With the pair (Z i (j) ,Z i’ (j) ,Z i’ (j) -Z i (j) ) to obtain the physical quantity o i (j) ,o i’ (j) The relationship between i,i’ (j) Generate a regression model M1 to derive

[0102] (Effects of information processing device and information processing method) In the information processing device 2 and the information processing method S2, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) Rather than generating a regression model that regresses , we use a different time t i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j)This makes it possible to generate a regression model M1 that is independent of the coordinate system that represents the physical quantity. This reduces the instability of the regression results when using a regression model.

[0103] Also, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) In the configuration in which the regression model is regressed, the problem of not being able to guarantee operation for non-learning objects (objects not included as subjects in the images used to learn the regression model) can arise because the object coordinate system is not defined. i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Since a regression structure is adopted, such a problem is unlikely to occur.

[0104] Exemplary Embodiment 5 A fifth exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiments will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0105] (Configuration of information processing device) The configuration of the information processing device 2A will be described with reference to Fig. 9. Fig. 9 is a block diagram showing the configuration of the information processing device 2A.

[0106] As shown in FIG. 9, the information processing device 2A is obtained by adding a latent variable derivation unit 22 to the information processing device 2 (see Exemplary Embodiment 4).

[0107] The latent variable derivation unit 22 calculates the latent variable for each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) is a means to derive the above using the world model M0. i is included in the inference period, the latent variable derivation unit 12 i The corresponding image Ii and time t i-1 The latent variable Z corresponding to i-1 (j) and into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ is included in the inference period, at time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same way. Meanwhile, at time t i is included in the prediction period, the latent variable derivation unit 12 i-1 The latent variable Z corresponding to i-1 (j) By inputting this into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ If is included in the forecast period, then time t i’ The latent variable Z corresponding toi’ (j) is obtained in the same way.

[0108] The model generation unit 21 of the information processing device 2A determines whether the correspondence between the input and output of the regression model M1 is correct at two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) and the two time points t obtained from the dataset DS i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) A regression model M1 is trained to best approximate the correspondence between

[0109] (Flow of information processing method) The flow of the information processing method S2A will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the information processing method S2A.

[0110] 10, the information processing method S2A is obtained by adding a latent variable derivation process S22 to the information processing method S2 (see exemplary embodiment 4). Note that the information processing method S2A is executed by, for example, an information processing device 2A or a computer.

[0111] As shown in FIG. 10, the latent variable derivation process S22 is executed before the model generation process S21.

[0112] The latent variable derivation process S22 is a process of deriving a latent variable for each object S included in the object group {S1, S2, ..., Sm}. j Alternatively, a specific object S selected from the group of objects {S1, S2, ..., Sm} j For two different times t i ,t i’ The latent variable Z corresponding toi (j) ,Z i’ (j) This is a process to derive the above using the world model M0. i is included in the inference period, in the latent variable derivation process S12, i The corresponding image Ii and time t i-1 The latent variable Z corresponding to i-1 (j) and into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ is included in the inference period, at time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same way. Meanwhile, at time t i is included in the prediction period, in the latent variable derivation process S12, i-1 The latent variable Z corresponding to i-1 (j) By inputting this into the world model M0, i The latent variable Z corresponding to i (j) Also, at time t i’ is included in the inference period, at time t i’ The latent variable Z corresponding to i’ (j) is obtained in the same manner. The latent variable derivation process S22 is executed by, for example, the latent variable derivation unit 22 of the information processing device 2A or by a processor of a computer.

[0113] In the model generation process S21 of the information processing method S2A, the correspondence between the input and output of the regression model M1 is calculated based on the correlation between the input and output of the regression model M1 at two different times t i ,t i’ The latent variable Z corresponding to i (j) ,Z i’ (j) Pair (Z i (j) ,Z i’ (j) ) and the two time points t obtained from the dataset DSi ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) A regression model M1 is trained to best approximate the correspondence between

[0114] (Effects of information processing device and information processing method) In the information processing device 2 and the information processing method S2, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) Rather than generating a regression model that regresses , we use a different time t i ,t i’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) This makes it possible to generate a regression model M1 that is independent of the coordinate system that represents the physical quantity. This reduces the instability of the regression results when using a regression model.

[0115] Also, at time t i The latent variable Z corresponding to i (j) From that time t i The physical quantity o corresponding to i (j) In the configuration in which the regression model is regressed, the problem of not being able to guarantee operation for non-learning objects (objects not included as subjects in the images used to learn the regression model) can arise because the object coordinate system is not defined. i ,ti’ Latent variable Z for i (j) ,Z i’ (j) From those times t i ,t i’ The physical quantity o corresponding to i (j) ,o i’ (j) The relationship between i,i’ (j) Since a regression structure is adopted, such a problem is unlikely to occur.

[0116] [Software implementation example] Some or all of the functions of the information processing devices 1, 1A, 1B, 2, and 2A (hereinafter also referred to as "each of the above devices") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0117] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 11. Figure 11 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0118] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0119] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0120] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0121] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0122] Furthermore, the functions of each of the devices may be realized by a single processor provided in a single computer, by multiple processors provided in a single computer working in cooperation, or by multiple processors provided in each of multiple computers working in cooperation. Furthermore, the programs for causing each of the devices to realize the functions may be stored in a single memory provided in a single computer, or may be distributed and stored in multiple memories provided in a single computer, or may be distributed and stored in multiple memories provided in each of multiple computers.

[0123] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0124] (Appendix 1) a relation derivation means for deriving a relation between physical quantities from a pair of latent variables using a regression model that receives as input a pair of latent variables corresponding to each of different times output from the world model and outputs a relation between physical quantities corresponding to each of the different times, for an object included in the world model that expresses the state of the object as a latent variable; Information processing device.

[0125] (Appendix 2) further comprising latent variable derivation means for deriving the latent variables using the world model; the relationship deriving means derives a relationship between the physical quantities from the pair of latent variables derived by the latent variable deriving means; 2. The information processing device according to claim 1.

[0126] (Appendix 3) a physical quantity calculation unit that calculates the physical quantity corresponding to the later of the different times from the physical quantity corresponding to the earlier of the different times and the relationship of the physical quantities derived by the relationship derivation unit; 2. The information processing device according to claim 1.

[0127] (Appendix 4) the physical quantity is the position of the object, the relationship deriving means derives a vector representing a displacement from a position of the object at an earlier one of the different times to a position of the object at a later one of the different times; the physical quantity calculation means calculates a vector representing the position of the object at the later of the different times by adding a vector representing the position of the object at the earlier of the different times and the vector representing the displacement derived by the relationship deriving means. 4. The information processing device according to claim 3.

[0128] (Appendix 5) the physical quantity is the attitude of the object, the relationship deriving means derives a quaternion representing a change in posture from the posture of the object at an earlier one of the different times to the posture of the object at a later one of the different times; the physical quantity calculation means calculates a quaternion representing the attitude of the object at a later one of the different times by multiplying a quaternion representing the attitude of the object at an earlier one of the different times by a quaternion representing the attitude change derived by the relationship deriving means. 4. The information processing device according to claim 3.

[0129] (Appendix 6) a coordinate conversion means for converting the relative physical quantity at the later of the different times calculated by the physical quantity calculation means into the absolute physical quantity corresponding to the later of the different times, with reference to the absolute physical quantity corresponding to an initial time; 6. The information processing device according to any one of Supplementary Notes 3 to 5.

[0130] (Appendix 7) a model generation means for generating a regression model for an object included in a world model that expresses the state of the object as a latent variable, the regression model receiving as input a pair of latent variables corresponding to each of different times output from the world model, and outputting a relationship between physical quantities corresponding to each of the different times; the model generation means generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing device.

[0131] (Appendix 8) further comprising latent variable derivation means for deriving the latent variables using the world model; the model generation means generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables acquired from the latent variable derivation means and the physical quantities acquired from a dataset. 8. The information processing device according to claim 7.

[0132] (Appendix 9) a processor inputs pairs of latent variables corresponding to different times output from a world model, the pairs corresponding to different times being output from the world model, for an object included in the world model, which expresses a state of the object as latent variables, and derives a relationship between the physical quantities from the pairs of latent variables using a regression model that outputs a relationship between the physical quantities corresponding to the different times; Information processing methods.

[0133] (Appendix 10) a processor generating a regression model for an object included in a world model that expresses a state of the object as a latent variable, the regression model receiving as input pairs of latent variables corresponding to each of different times output from the world model, and receiving as output relationships between physical quantities corresponding to each of the different times; the processor generates the regression model such that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing methods.

[0134] (Appendix 11) causing the processor to execute a relationship derivation process for an object included in a world model that expresses a state of the object as a latent variable, using a regression model that receives as input pairs of latent variables corresponding to different times output from the world model and outputs relationships between physical quantities corresponding to the different times, to derive relationships between the physical quantities from the pairs of latent variables; Information processing program.

[0135] (Appendix 12) causing the processor to execute a model generation process for generating a regression model for an object included in a world model that expresses the state of the object as a latent variable, the regression model taking as input pairs of latent variables corresponding to each of different times output from the world model, and outputting relationships between physical quantities corresponding to each of the different times; In the model generation process, the processor generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing program. (Appendix 13) at least one processor, For an object included in a world model that expresses the state of the object as a latent variable, a relationship derivation process is performed in which pairs of latent variables corresponding to each of different times output from the world model are used as input, and a regression model is used in which relationships between physical quantities corresponding to each of the different times are output, to derive relationships between the physical quantities from the pairs of latent variables. Information processing device.

[0136] The information processing device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes.

[0137] (Appendix 14) at least one processor, For an object included in a world model that expresses the state of the object as a latent variable, a model generation process is executed to generate a regression model that receives as input pairs of latent variables corresponding to each of different times output from the world model and outputs relationships between physical quantities corresponding to each of the different times; In the model generation process, the at least one processor generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing device.

[0138] The information processing device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes. [Explanation of symbols]

[0139] 1, 1A, 1B Information processing equipment 11 Relation derivation part 12 Latent variable derivation part 13 Physical quantity calculation section 14 Initial physical quantity acquisition part 15 Coordinate conversion section 2, 2A Information Processing Equipment 21 Model Generation Unit 22 Latent variable derivation part C1 processor C2 Memory

Claims

1. a relation derivation means for deriving a relation between physical quantities from a pair of latent variables using a regression model that receives as input a pair of latent variables corresponding to each of different times output from the world model and outputs a relation between physical quantities corresponding to each of the different times, for an object included in the world model that expresses the state of the object as a latent variable; Information processing device.

2. further comprising latent variable derivation means for deriving the latent variables using the world model; the relationship deriving means derives a relationship between the physical quantities from the pair of latent variables derived by the latent variable deriving means; The information processing device according to claim 1 .

3. a physical quantity calculation unit that calculates the physical quantity corresponding to the later of the different times from the physical quantity corresponding to the earlier of the different times and the relationship of the physical quantities derived by the relationship derivation unit; The information processing device according to claim 1 .

4. the physical quantity is the position of the object, the relationship deriving means derives a vector representing a displacement from a position of the object at an earlier one of the different times to a position of the object at a later one of the different times; the physical quantity calculation means calculates a vector representing the position of the object at the later of the different times by adding a vector representing the position of the object at the earlier of the different times and the vector representing the displacement derived by the relationship deriving means. The information processing device according to claim 3 .

5. the physical quantity is the attitude of the object, the relationship deriving means derives a quaternion representing a change in posture from the posture of the object at an earlier one of the different times to the posture of the object at a later one of the different times; the physical quantity calculation means calculates a quaternion representing the attitude of the object at a later one of the different times by multiplying a quaternion representing the attitude of the object at an earlier one of the different times by a quaternion representing the attitude change derived by the relationship deriving means. The information processing device according to claim 3 .

6. a coordinate conversion means for converting the relative physical quantity at the later of the different times calculated by the physical quantity calculation means into the absolute physical quantity corresponding to the later of the different times, with reference to the absolute physical quantity corresponding to an initial time; The information processing device according to claim 3 .

7. a model generation means for generating a regression model for an object included in a world model that expresses the state of the object as a latent variable, the regression model receiving as input a pair of latent variables corresponding to each of different times output from the world model, and outputting a relationship between physical quantities corresponding to each of the different times; the model generation means generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing device.

8. further comprising latent variable derivation means for deriving the latent variables using the world model; the model generation means generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables acquired from the latent variable derivation means and the physical quantities acquired from a dataset. The information processing device according to claim 7 .

9. a processor inputs pairs of latent variables corresponding to different times output from a world model, the pairs corresponding to different times being output from the world model, for an object included in the world model, which expresses a state of the object as latent variables, and derives a relationship between the physical quantities from the pairs of latent variables using a regression model that outputs a relationship between the physical quantities corresponding to the different times; Information processing methods.

10. a processor generating a regression model for an object included in a world model that expresses a state of the object as a latent variable, the regression model receiving as input pairs of latent variables corresponding to each of different times output from the world model, and receiving as output relationships between physical quantities corresponding to each of the different times; the processor generates the regression model such that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing methods.

11. causing the processor to execute a relationship derivation process for an object included in a world model that expresses a state of the object as a latent variable, using a regression model that receives as input pairs of latent variables corresponding to different times output from the world model and outputs relationships between physical quantities corresponding to the different times, to derive relationships between the physical quantities from the pairs of latent variables; Information processing program.

12. causing the processor to execute a model generation process for generating a regression model for an object included in a world model that expresses the state of the object as a latent variable, the regression model taking as input pairs of latent variables corresponding to each of different times output from the world model, and outputting relationships between physical quantities corresponding to each of the different times; In the model generation process, the processor generates the regression model so that a correspondence relationship between inputs and outputs of the regression model best approximates a correspondence relationship between the pair of latent variables output from the world model and the physical quantities acquired from a dataset. Information processing program.

Images