Identifier, converter, identification method, and program

Abstract

To enable support of decision making using a world model, this identifier includes: an acquisition unit for acquiring a plurality of latent variables corresponding to a plurality of objects, respectively, by inputting an input image including the plurality of objects as subjects to a world model machine-learned so as to receive input of an image to output a latent variable of an object included as a subject in the image; an identification unit for identifying an object corresponding to each of the plurality of latent variables; and an output unit for outputting the identification result.

Description

Classifier, transformer, classification method, and program

[0001] The present disclosure relates to a classifier, a transformer, a classification method, and a program.

[0002] There is known a technology for a world model that is trained using unlabeled images (including video sequences containing multiple images) and predicts the future of objects included as subjects in the input images.

[0003] For example, Patent Document 1 describes a machine learning model that is trained to predict distances to line pixels corresponding to lines in an input image, and describes that predictions from the machine learning model are used to update a world model.

[0004] Japan Special Table No. 2021-532512

[0005] When an input image containing multiple subjects is input to a world model, the latent variables output from the world model are not associated with which objects they correspond to. Therefore, when converting the latent variables output from the world model into physical quantities in real space, it is necessary to match the physical quantities corresponding to which objects the latent variables output from the world model correspond to. Similarly, when inversely converting physical quantities in real space into latent variables, it is necessary to match the physical quantities corresponding to which objects the latent variables correspond to. However, Patent Document 1 does not consider a configuration for identifying which object a latent variable corresponds to from a latent variable output from the world model.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and one exemplary purpose thereof is to provide a technology that can identify, from a latent variable output from a world model, which object the latent variable corresponds to.

[0007] An example aspect of the present disclosure provides a classifier that receives an image as input, and is machine-learned to output latent variables of objects included as subjects in the image, by inputting an input image containing multiple objects as subjects, thereby acquiring multiple latent variables corresponding to each of the multiple objects; a classification means that identifies objects corresponding to each of the multiple latent variables; and an output means that outputs the classification results obtained by the classification means.

[0008] An identification method according to an exemplary aspect of the present disclosure includes an acquisition process in which at least one processor acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process in which the at least one processor identifies objects corresponding to each of the plurality of latent variables; and an output process in which the at least one processor outputs an identification result from the identification process.

[0009] A program according to an exemplary aspect of the present disclosure is a program that causes a computer to function as a classifier, and the program causes the computer to execute an acquisition process of acquiring multiple latent variables corresponding to each of the multiple objects by inputting an input image containing multiple objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image, an identification process of identifying objects corresponding to each of the multiple latent variables, and an output process of outputting the identification results of the identification process.

[0010] According to one exemplary aspect of the present disclosure, an exemplary effect is achieved in that it is possible to identify, from a latent variable output from a world model, which object the latent variable corresponds to.

[0011] FIG. 1 is a block diagram showing the configuration of a classifier according to the present disclosure. FIG. 2 is a flow diagram showing the flow of a classification method according to the present disclosure. FIG. 3 is a block diagram showing the configuration of a converter according to the present disclosure. FIG. 4 is a diagram showing an example of a classification result according to the present disclosure. FIG. 5 is a flow diagram showing the flow of a learning method according to the present disclosure. FIG. 6 is a flow diagram showing the flow of a conversion method according to the present disclosure. FIG. 7 is a diagram showing an overview of a medical robot using a classifier according to the present disclosure. FIG. 8 is a block diagram showing the configuration of a computer functioning as a classifier and a converter according to the present disclosure.

[0012] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0013] [First Exemplary Embodiment] A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is a basic form for each of the exemplary embodiments described below. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0014] (Configuration of Classifier) ​​The configuration of the classifier 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the classifier 1. As shown in Fig. 1, the classifier 1 includes an acquisition unit 11, a classification unit 12, and an output unit 13. The acquisition unit 11, the classification unit 12, and the output unit 13 are components that respectively realize an acquisition means, a classification means, and an output means in the exemplary embodiment.

[0015] The acquisition unit 11 acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image containing the plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of the objects included as subjects in the image. The acquisition unit 11 supplies the acquired plurality of latent variables to the identification unit 12.

[0016] Here, the world model may be a world model that outputs latent variables for the background in addition to the objects included as subjects in the image. In the case of such a world model, the acquisition unit 11 acquires multiple latent variables corresponding to each of the multiple objects and the background. Furthermore, the identification unit 12, which will be described later, identifies the objects and the background corresponding to each of the multiple objects and the background.

[0017] In this disclosure, "image" also includes a moving image that includes multiple images.

[0018] A "world model" is a model trained by machine learning using unlabeled images, which predicts the future of the external world by modeling the external world. When an image of the external world is input, the world model outputs latent variables for each of the multiple objects included as subjects in the image. The latent variables are also referred to as latent representations or representations. As an example, the latent variables are represented by vectors or distribution functions, but are not limited to these.

[0019] Specific configurations of the world model include, but are not limited to, a VAE (Variational Autoencoder), a GAN (Generative Adversarial Network), an RNN (Recurrent Neural Network), a CNN (Convolutional Neural Network), a GNN (Graph Neural Network), a Transformer, and combinations thereof.

[0020] The identification unit 12 identifies an object corresponding to each of the multiple latent variables acquired by the acquisition unit 11. The identification unit 12 supplies the identification result to the output unit 13.

[0021] The output unit 13 outputs the classification result obtained by the classification unit 12 .

[0022] (Effect of Classifier) ​​As described above, the classifier 1 employs a configuration including an acquisition unit 11 that acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image, a classification unit 12 that identifies objects corresponding to each of the plurality of latent variables acquired by the acquisition unit 11, and an output unit 13 that outputs the classification results obtained by the classification unit 12.

[0023] Therefore, the classifier 1 has the effect of being able to identify, from a latent variable output from the world model, which object the latent variable corresponds to.

[0024] (Flow of Identification Method) The flow of identification method S1 will be described with reference to Fig. 2. Fig. 2 is a flow chart showing the flow of identification method S1. As shown in Fig. 2, identification method S1 includes an acquisition process S11, an identification process S12, and an output process S13.

[0025] In the acquisition process S11, the acquisition unit 11 acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image including the plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of the objects included as subjects in the image. The acquisition unit 11 supplies the acquired plurality of latent variables to the identification unit 12.

[0026] (Identification Process S12) In the identification process S12, the identification unit 12 identifies an object corresponding to each of the multiple latent variables acquired by the acquisition unit 11. The identification unit 12 supplies the identification result to the output unit 13.

[0027] (Output Process S13) In the output process S13, the output unit 13 outputs the classification result obtained by the classification unit 12 in the classification process S12.

[0028] (Effects of the Classification Method) As described above, the classification method S1 employs a configuration including an acquisition process S11 in which the acquisition unit 11 acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image, a classification process S12 in which the classification unit 12 identifies an object corresponding to each of the plurality of latent variables acquired by the acquisition unit 11, and an output process S13 in which the output unit 13 outputs the classification result by the classification unit 12 in the classification process S12. Therefore, the classification method S1 can achieve the same effects as the above-described classifier 1.

[0029] Second Exemplary Embodiment A second exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be denoted by the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technical means shown in each drawing referenced to describe this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0030] (Overview of Converter) Converter 2 is a device that converts latent variables output from a world model, which are latent variables of multiple objects included as subjects in an input image, into physical quantities of the multiple objects. Converter 2 also converts the physical quantities of the multiple objects into the latent variables of the multiple objects. In the present disclosure, converting physical quantities into latent variables is also referred to as "inverse conversion." Converter 2 may also convert (or inverse convert) the size and shape of an object (for example, the lengths of three sides of a rectangular parallelepiped) instead of the physical quantities.

[0031] Furthermore, the transformer 2 identifies objects corresponding to each of the latent variables of the objects in order to perform transformation or inverse transformation. In other words, the transformer 2 includes a classifier 1A corresponding to the classifier 1 described above.

[0032] In the present disclosure, the total number of latent variables z output by the world model is set to N. Also, the total number of types of objects that the converter 2 can convert or inversely convert is set to M. Furthermore, the latent variable z of object n included as a subject in the input image is set to latent variable z ｎ , N latent variables z ｎ The set of latent variables {z ｎ} ｎ＝１～Ｎ It is expressed as:

[0033] For example, the latent variable of the object "robot" is the latent variable z ｒｏｂｏｔIn addition, n may be identification information (ID) for distinguishing an object from other objects. For example, the latent variable first output from the world model is expressed as the latent variable z １ , and then the output latent variable is the latent variable z ２ , which may also be expressed as:

[0034] Furthermore, the latent variable z of the object n is converted by the converter 2 into the physical quantity o as the physical quantity o ｎ and N physical quantities o ｎ The set of physical quantities {o ｎ} ｎ＝１～Ｎ That is, the physical quantity o ｎ is the physical quantity converted by the converter 2, and is the actual physical quantity of the object n (O ｎ ) is different from

[0035] Furthermore, the classification result l output from the classifier 1A is a latent variable z ｎ The classification result l corresponding to ｎ , N classification results l ｎ The set of the identification results {l ｎ} ｎ＝１～Ｎ It is expressed as:

[0036] (Configuration of Converter) The configuration of the converter 2 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the converter 2. The converter 2 includes a control unit 20, a communication unit 28, and a storage unit 29.

[0037] The communication unit 28 is an interface that transmits and receives data via a network. For example, the communication unit 28 outputs data output from the control unit 20 to another device, or outputs data output from another device to the control unit 20. Examples of data that the communication unit 28 outputs to another device include, but are not limited to, a classification result, a latent variable after transformation, and a physical quantity after inverse transformation. Examples of data that the communication unit 28 outputs to the control unit 20 include, but are not limited to, an image. Examples of the communication unit 28 include, but are not limited to, communication chips for various communication standards such as wireless communication standards of mobile data communication networks (e.g., 3G standard, 4G standard, or 5G standard).

[0038] Furthermore, instead of or in addition to the communication unit 28, the converter 2 may include a data input unit that accepts input of data and a data output unit that outputs data.

[0039] The storage unit 29 stores data referenced by the control unit 20. Examples of data stored in the storage unit 29 include physical quantities {O ｍ} ｍ＝１～Ｍ Examples of the storage unit 29 include, but are not limited to, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0040] (Control unit 20) The control unit 20 controls each component included in the converter 2. As shown in Fig. 3, the control unit 20 also includes an acquisition unit 11A, a classification unit 12A, an output unit 13A, a learning unit 21, and a conversion unit 22. In this exemplary embodiment, the acquisition unit 11A, the classification unit 12A, the output unit 13A, the learning unit 21, and the conversion unit 22 respectively function as an acquisition means, a classification means, an output means, a learning means, and a conversion means.

[0041] The acquisition unit 11A acquires data. For example, the acquisition unit 11A inputs an input image stored in the storage unit 29 into a world model M1 that has been machine-learned to receive an image as an input and output latent variables of objects included as subjects in the image, thereby acquiring latent variables {z ｎ} ｎ＝１～Ｎ Get.

[0042] As another example, the acquisition unit 11A acquires an input image from another device via the communication unit 28. The acquisition unit 11A stores the acquired input image in the storage unit 29.

[0043] The world model M1 may be stored in the storage unit 29. In this case, parameters that define the world model M1 may be stored in the storage unit 29. Furthermore, the world model M1 may be stored in a storage device other than the storage unit 29 (for example, an external server, etc.).

[0044] The identification unit 12A identifies an object corresponding to a latent variable z. As an example, the identification unit 12A inputs a latent variable z and applies a latent variable {z ｎ} ｎ＝１～Ｎ By inputting each of the ｎ} ｎ＝１～Ｎ The recognition unit 12A recognizes the objects corresponding to each of the recognition results {l ｎ} ｎ＝１～Ｎ to the output unit 13A. With this configuration, the identification unit 12A can suitably identify the object corresponding to the latent variable z. A specific example of the identification model M2 will be described later.

[0045] The output unit 13A outputs data. As one example, the output unit 13A outputs the classification result l by the classification unit 12A to the storage unit 29, thereby storing the classification result l in the storage unit 29. As another example, the output unit 13A outputs the classification result l by the classification unit 12A to another device via the communication unit 28.

[0046] As described above, the converter 2 includes an acquisition unit 11A, a classification unit 12A, and an output unit 13A, which respectively correspond to the acquisition unit 11, the classification unit 12, and the output unit 13 included in the above-described classifier 1. That is, as described above, the converter 2 includes a classifier 1A corresponding to the classifier 1. An example of the classification result 1 output by the classifier 1A will be described later.

[0047] The learning unit 21 trains a machine learning model by machine learning. As one example, the learning unit 21 trains a discrimination model M2 by machine learning. As another example, the learning unit 21 trains a conversion model M3 or a conversion model M4, which will be described later, by machine learning. As yet another example, the learning unit 21 trains the discrimination model M2 and a conversion model M3 or a conversion model M4, which will be described later, by machine learning. Examples of processing executed by the learning unit 21 will be described later.

[0048] In this way, the converter 2 can also be said to be a learning device including the acquisition unit 11A, the identification unit 12A, and the learning unit 21. The learning device performs machine learning to generate at least one of the identification model M2, the conversion model M3, and the conversion model M4.

[0049] The learning unit 21 may train a machine learning model using multiple types of objects, physical quantities of multiple objects, or physical quantities of multiple sizes. As one example, the learning unit 21 may train a machine learning model using the position, orientation, velocity, and angular velocity of an object as physical quantities. As another example, the learning unit 21 may train a machine learning model using the position of a certain object (e.g., a rectangular parallelepiped object) and the position of another object (e.g., a robot). As yet another example, the above two examples may be combined to train a machine learning model. As yet another example, the physical quantities of a specific object may be used to train a machine learning model (e.g., multiple transformation models M3) specialized for the specific object.

[0050] Furthermore, when the learning unit 21 trains the discrimination model M2 by machine learning, the learning unit 21 may train the discrimination model M2 without using a dataset including an object to be identified and an object different from the object to be identified that is indistinguishable from the object to be identified. In other words, the learning unit 21 may train the discrimination model M2 by machine learning using a dataset including only one object to be identified. For example, if the discrimination model M2 identifies whether an object is a robot, the learning unit 21 does not use a dataset including multiple robots with the same shape as the robot to be identified (robots that are different from the robot to be identified but indistinguishable from the robot to be identified) for the machine learning of the discrimination model M2. This does not apply if the learning unit 21 has trained the discrimination model M2 sufficiently. In other words, the learning unit 21 may train the discrimination model M2 by sufficiently learning two objects that were previously indistinguishable from the two objects, so that the two objects can be distinguished from each other.

[0051] Further, as an example, the learning unit 21 may use the classification result {l ｎ} ｎ＝１～Ｎ , multiple latent variables {zｎ} ｎ＝１～Ｎ , and the correct physical quantity {O ｍ} ｍ＝１～Ｍ By using the above, at least one of the discriminant model M2, the conversion model M3, and the conversion model M4 is trained. With this configuration, the training unit 21 can suitably train at least one of the discriminant model M2, the conversion model M3, and the conversion model M4.

[0052] The conversion unit 22 converts a plurality of latent variables {z ｎ} ｎ＝１～Ｎ The physical quantities of multiple objects {o ｎ} ｎ＝１～Ｎ Alternatively, the conversion unit 22 converts the physical quantities of the plurality of objects {o ｍ} ｍ＝１～Ｍ multiple latent variables {z ｍ} ｍ＝１～Ｍ With this configuration, the converter 2 can convert the latent variable z output from the world model into a physical quantity o, or inversely convert the physical quantity o into the latent variable z output from the world model.

[0053] As an example, the conversion unit 22 inputs a latent variable z and an identification result l, and converts the latent variable {z ｎ} ｎ＝１～Ｎ and the classification result {l ｎ} ｎ＝１～Ｎ By inputting ｎ} ｎ＝１～Ｎ physical quantity {o ｎ} ｎ＝１～Ｎ In this case, the conversion performed by the conversion model M3 is expressed as f in the following equation (1): f(z) → o ... (1) If the latent variable z is a probability distribution, a reparameterization trick or the like is used to sample from the probability distribution and obtain a vectorized value. may be used as an input to f.

[0054] As another example, the conversion unit 22 inputs the physical quantity o and the identification result l, and converts the physical quantities {o} of the plurality of objects stored in the storage unit 29 into a conversion model M4 that has been machine-learned to output a latent variable z of the object corresponding to the physical quantity o. ｍ} ｍ＝１～Ｍ and the classification result {l ｍ} ｍ＝１～Ｍ By inputting ｍ} ｍ＝１～Ｍ The latent variable {z ｍ} ｍ＝１～Ｍ In this case, the inverse transformation performed by the transformation model M4 is －１ It is expressed as follows: －１ (o) → z ... (2) If z is a probability distribution expression, f －１ The output of can be parameters to express the distribution. For example, if it is composed of multiple normal distributions of z, the mean and variance of each component are output.

[0055] In this way, by using the conversion model M3 or the conversion model M4, the conversion unit 22 can suitably convert the latent variable z output from the world model into a physical quantity o, or suitably inversely convert the physical quantity o into the latent variable z output from the world model.

[0056] The conversion model M3 and the conversion model M4 may be stored in the storage unit 29. In this case, parameters that define the conversion model M3 and the conversion model M4 may be stored in the storage unit 29. Furthermore, the conversion model M3 and the conversion model M4 may be stored in a storage device other than the storage unit 29 (for example, an external server, etc.).

[0057] (Example 1 of Classification Result) An example of the classification result l output by the classifier 1A will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the classification result l.

[0058] As an example, when the discrimination unit 12A of the discriminator 1A uses a discrimination model M2, the discrimination model M2 may be a discrimination model M2A that has been machine-learned to output accuracy information indicating the likelihood that an input latent variable z is a latent variable corresponding to a target object. In other words, in this example, the number M of types of target objects is M=1.

[0059] For example, the identification unit 12A uses the latent variables {z ｎ} ｎ＝１～Ｎ Each of these is input to the discriminant model M2A, and the latent variables {z ｎ} ｎ＝１～Ｎ and obtain accuracy information indicating the likelihood that each of the input latent variables corresponds to the target object "robot." As an example, the identification unit 12A may convert the accuracy information using a sigmoid function to obtain an identification result l (0≦l≦1). In this case, the closer the value of the identification result l is to 1, the more likely it is that the input latent variable corresponds to the target object "robot." In addition, in this case, if the identification result l is greater than a predetermined threshold, the identification unit 12A may set the identification result l to 1.

[0060] For example, as shown in the upper part of FIG. 4, the acquisition unit 11A acquires the latent variable z ｎ When the latent variable z ｎ The discrimination unit 12A then converts the accuracy information output from the discrimination model M2A using a sigmoid function to obtain the discrimination result l ｎ (0≦l ｎ The output unit 13A calculates the calculated classification result l ｎ Output.

[0061] With this configuration, the classifier 1A can suitably determine whether or not the acquired latent variable z is a latent variable z corresponding to the target object.

[0062] (Example 2 of Classification Result) Another example of the classification result when the number M of types of target objects is M>=1 will be described. In this example, the classification model M2A uses a plurality of latent variables {z ｎ} ｎ＝１～Ｎand outputs the classification results for each of the M target objects.

[0063] In this example, the identification unit 12A extracts the latent variable z ｎ For each of the latent variables, probability information indicating the likelihood that it is a latent variable corresponding to the target object m is obtained. In this configuration, the classification unit 12A converts each piece of probability information by softmax entropy and outputs it as a classification result.

[0064] Here, the latent variable z ｎ The identification result of ｎ，ｍ For example, when M=3, the latent variable z ｎ The classification results indicating whether each of the target objects corresponds to the target object 1, the target object 2, and the target object 3 are ｎ、１ , l ｎ、２ , and l ｎ、３ Furthermore, the latent variable z ｎ does not correspond to any of the target objects 1, 2, and 3. ｎ、０ It is expressed as:

[0065] (Example 3 of Classification Result) Another example of the classification result 1 output by the classifier 1A will be described with reference again to FIG.

[0066] As another example, when the classifier 1A uses the discrimination model M2, the discrimination model M2 uses a plurality of input latent variables {z ｎ} ｎ＝１～Ｎ The classification model M2B may be machine-learned to output accuracy information indicating the likelihood that any one of the above is a latent variable corresponding to the target object. In other words, in this example, the number M of types of target objects is M>=1.

[0067] For example, the identification unit 12A uses the latent variables {z ｎ} ｎ＝１～Ｎ Among them, the latent variable z １ ~ latent variable z ３ is input to the discriminant model M2B, and the latent variable z １ ~ latent variable z ３As an example, the discriminant model M2B acquires certainty information indicating the likelihood that any of the latent variables z １ ~ latent variable z ３ and outputs certainty information indicating the certainty that the latent variables in a predetermined order (predetermined positions) are latent variables corresponding to the target object "robot."

[0068] For example, when the identification model M2B outputs accuracy information indicating the likelihood that the third latent variable is a latent variable corresponding to the target object “robot,” the identification unit 12A outputs accuracy information indicating the likelihood that the third latent variable is a latent variable corresponding to the target object “robot.” １ ~ latent variable z ３ As an example, the classification unit 12A converts each of the six pieces of accuracy information by softmax entropy, and obtains six classification results l as shown in the lower part of FIG. １２３ , l １３２ , l ２１３ , l ２３１ , l ３１２ , and l ３２１ may be set as the identification result.

[0069] Here, the identification result l ｉｊｋ is the latent variable z ｉ The first is the latent variable z ｊ The second is the latent variable z ｋ The third is the latent variable, and the likelihood that the latent variables in a given order are latent variables corresponding to the target object is shown. ｉｊｋ is the third z ｋ indicates the likelihood that the latent variable corresponds to the target object "robot". １２３ is the latent variable z ３ indicates the likelihood that the latent variable corresponds to the target object "robot", and l ３２１ is the latent variable z １ indicates the likelihood that is a latent variable corresponding to the target object "robot."

[0070] For example, the latent variable z ３corresponds to the target object “robot”, and the discrimination model M2B outputs certainty information indicating the likelihood that the third latent variable corresponds to the target object “robot”, the discrimination result l ２１３ and the identification result l ２１３ The sum of the values ​​of will be close to 1, and the other classification results will be 0.

[0071] With this configuration, the classifier 1A can calculate the latent variables {z ｎ} ｎ＝１～Ｎ It is possible to suitably identify which of the latent variables z corresponds to the target object.

[0072] (Example 4 of Classification Result) Still another example of the classification result 1 output by the classifier 1A will be described with reference again to FIG.

[0073] As another example, when the classifier 1A uses the discrimination model M2, the discrimination model M2 uses a plurality of input latent variables {z ｎ} ｎ＝１～Ｎ may be a discriminative model M2C that has been machine-learned to output certainty information indicating the certainty that each of the latent variables M1 and M2C corresponds to each of the plurality of target objects. In other words, in this example, the number M of types of target objects is M>1.

[0074] For example, the identification unit 12A uses the latent variables {z ｎ} ｎ＝１～Ｎ Among them, the latent variable z １ ~ latent variable z ３ is input to the discriminant model M2C, and the latent variable z １ ~ latent variable z ３ are latent variables corresponding to the target object "robot 1", the target object "robot 2", and the target object "object", respectively. １ ~ latent variable z ３ The order of the latent variables in the predetermined positions is swapped, and certainty information indicating the certainty that the latent variables in the predetermined positions are latent variables corresponding to the target object "robot 1," the target object "robot 2," and the target object "object" is output.

[0075] For example, when the identification model M2C outputs certainty information indicating the certainty that the first latent variable corresponds to the target object “robot 1”, the second latent variable corresponds to the target object “robot 2”, and the third latent variable corresponds to the target object “object”, the identification unit 12A outputs certainty information indicating the certainty that the first latent variable corresponds to the target object “robot 1”, the second latent variable corresponds to the target object “robot 2”, and the third latent variable corresponds to the target object “object”, as in the above example. １ ~ latent variable z ３ The classification unit 12A converts each of the six pieces of accuracy information by soft max entropy, and obtains six classification results l as shown in the lower part of FIG. １２３ , l １３２ , l ２１３ , l ２３１ , l ３１２ , and l ３２１ may be set as the identification result.

[0076] For example, the latent variable z １ corresponds to the target object "Robot 2", and the latent variable z ２ corresponds to the target object "Robot 1", and the latent variable z ３ In this case, if the identification model M2C outputs accuracy information indicating the likelihood that the first latent variable corresponds to the target object "robot 1", the second latent variable corresponds to the target object "robot 2", and the third latent variable corresponds to the target object "object", the identification result l ２１３ The value of will be close to 1, and the other classification results will be 0.

[0077] With this configuration, the classifier 1A can calculate the latent variables {z ｎ} ｎ＝１～Ｎ It is possible to suitably identify which target object each of the latent variables z corresponds to.

[0078] (Example in which the learning unit 21 learns the discriminative model M2) An example in which the learning unit 21 learns the discriminative model M2 will be described.

[0079] The learning unit 21 trains the discrimination model M2 by machine learning so that it receives a latent variable z as input and outputs a discrimination result l that discriminates an object corresponding to the latent variable z. More specifically, the learning unit 21 trains the discrimination model M2 by machine learning using, as training data, a pair of the latent variable z and data indicating whether the latent variable z is a latent variable of the target object m.

[0080] (Example 1 in which the learning unit 21 learns the discriminative model M2 and the conversion model M3 or M4) An example of a configuration in which the learning unit 21 learns the discriminative model M2 and the conversion model M3 or M4 will be described. Note that in this example, if the discriminative model M2 has already been machine-learned, the learning unit 21 may machine-learn the conversion model M3 or M4 using the following method.

[0081] As an example, when N>M=1, the classification result {l ｎ} ｎ＝１～Ｎ is a set of multiple latent variables {z ｎ} ｎ＝１～Ｎ The classification result {l ｎ} ｎ＝１～Ｎ In other words, a case will be described in which the above-described discriminative model M2 is a discriminative model M2A that outputs certainty information indicating the likelihood that the input latent variable z is a latent variable corresponding to the target object.

[0082] In this case, the learning unit 21 calculates the loss using the following formulas (3) and (4): where the identification result l is a value converted by the above-mentioned sigmoid function, and is the identification result l (0≦l≦1).

[0083] Equation (3) is the loss for training the discriminative model M2 and the transformation model M3, and f represents the transformation model M3. That is, the training unit 21 trains the discriminative model M2 and the transformation model M3 by minimizing the loss calculated by equation (3). The loss calculated by equation (3) is minimized when the discrimination result ln by the discriminative model M2 indicates that the latent variable zn has been identified as the latent variable of the object n, and when the latent variable zn has been converted into the correct physical quantity O by the transformation model M3.

[0084] Equation (4) is the loss for training the discriminant model M2 and the transformation model M4, and f －１ represents the transformation model M4. That is, the learning unit 21 trains the discriminant model M2 and the transformation model M4 by minimizing the loss calculated by the formula (4). The loss calculated by the formula (4) is the discrimination result l by the discriminant model M2. ｎ is the latent variable z ｎ is identified as the latent variable of object n, and the correct physical quantity O is converted into the latent variable z ｎ If it can be converted to, it will be the smallest.

[0085] With this configuration, the learning unit 21 can preferably train the discrimination model M2 and the transformation model M3 or M4. Furthermore, with this configuration, the learning unit 21 trains the discrimination model M2 and the transformation model M3 or M4 using the correct physical quantity O. Therefore, with this configuration, the learning unit 21 can train the discrimination model M2 and the transformation model M3 or M4 without using correct data in which latent variables are associated with physical quantities.

[0086] (Example 2 in which the learning unit 21 learns the discriminative model M2 and the conversion model M3 or M4) Another example of a configuration in which the learning unit 21 learns the discriminative model M2 and the conversion model M3 or M4 will be described. Note that in this example as well, if the discriminative model M2 has already been machine-learned, the learning unit 21 may machine-learn the conversion model M3 or M4 using the following method.

[0087] As another example, when N>=M, the classification result {l ｎ} ｎ＝１～Ｎ For each target object m, a latent variable {z ｎ} ｎ＝１～Ｎ To address the problem of assigning only one latent variable among ｎ} ｎ＝１～Ｎ In other words, the above-mentioned discrimination model M2 is a discrimination model based on accuracy information indicating how each latent variable {z ｎ} ｎ＝１～Ｎ Hereinafter, a case will be described in which the discrimination model M2B is machine-learned to output accuracy information indicating the likelihood that any of the above is a latent variable corresponding to the target object m.

[0088] For example, when N = 3 and M = 1, the latent variable z_ ｎ is a vector, the latent variable z １ , latent variable z ２ , latent variable z ３ The vectors connected in this order are input to the discrimination model M2, and the discrimination result l ｉｊｋ is the third latent variable z ｋ is a latent variable corresponding to the target object m. In this case, the learning unit 21 calculates the loss using the following equations (5) and (6). Here, the classification result l ｉｊｋ is a value converted by the soft max entropy described above, and satisfies the following formula (7).

[0089] Equation (5) is the loss for training the discriminative model M2 and the transformed model M3, and f represents the transformed model M3. That is, the training unit 21 trains the discriminative model M2 and the transformed model M3 by minimizing the loss calculated by Equation (5). The loss calculated by Equation (5) is the difference between the discrimination result l by the discriminative model M2 and the transformed model M3. ｉｊｋ is the third latent variable z ｋ is identified as a latent variable of the target object m, and the latent variable z ｎIf it can be converted into the correct physical quantity O, it will be minimum.

[0090] Equation (6) is the loss for training the discriminative model M2 and the transformation model M4, and f －１ represents the transformation model M4. That is, the learning unit 21 trains the discriminant model M2 and the transformation model M4 by minimizing the loss calculated by the formula (6). The loss calculated by the formula (6) is the discrimination result l by the discriminant model M2. ｉｊｋ is the third latent variable z ｋ is identified as a latent variable of the target object m, and the correct physical quantity O is converted into the latent variable z ｎ If it can be converted to, it will be the smallest.

[0091] This configuration allows the learning unit 21 to suitably learn the conversion model M3 and the conversion model M4. Also, with this configuration, the learning unit 21 can learn the identification model M2 and the conversion model M3 or M4 without using ground truth data that associates latent variables with physical quantities.

[0092] (Third Example of Learning Executed by Learning Unit 21) A description will now be given of yet another example of the configuration in which the learning unit 21 learns the conversion model M3 and the conversion model M4.

[0093] As another example, when N=M and the classification result by the classification unit 12A is a classification result of a plurality of latent variables {z ｎ} ｎ＝１～Ｎ In order to deal with the problem of assigning each of the latent variables to the target object m one-to-one, the case where the classification result is based on the probability information indicating how to correctly assign each latent variable will be described. In other words, the above-mentioned classification model M2 is used to classify the input multiple latent variables {z ｎ} ｎ＝１～Ｎ is a discriminative model M2C that has been machine-learned to output probability information indicating the likelihood that each of the latent variables is a latent variable corresponding to each of a plurality of target objects.

[0094] In this case, the learning unit 21 adds only one classification result l to the above-mentioned equations (5) and (6).ｉｊｋ The classification result l that is smallest when is 1 ｉｊｋ The loss is calculated by adding the following equation (8) which means the entropy for

[0095] That is, the learning unit 21 learns the conversion model M3 by minimizing the loss calculated by adding equation (8) to equation (5).

[0096] Furthermore, the learning unit 21 learns the conversion model M4 by minimizing the loss calculated by adding the formula (8) to the formula (6).

[0097] This configuration allows the learning unit 21 to preferably learn the conversion model M3 and the conversion model M4.

[0098] (Example 4 of Learning Executed by Learning Unit 21) A description will be given of yet another example of a configuration in which the learning unit 21 learns the conversion model M3 and the conversion model M4 when M > 1. In this example, the case where the discriminative model M2 is the discriminative model M2B or the discriminative model M2C will be described.

[0099] For example, when N = 3 and M = 2, the latent variable z ｎ is a vector, the latent variable z １ , latent variable z ２ , latent variable z ３ The vectors connected in this order are input to the discrimination model M2, and the discrimination result l ｉｊｋ is the second latent variable z ｊ is the target object m １ and the third latent variable z ｋ is the target object m ２ In this case, the learning unit 21 calculates the loss using the following equations (9) and (10).

[0100] In formula (9), f １ is the target object m １ represents a conversion model M3 that converts the latent variables of the target object m ２ represents a conversion model M3 that converts the latent variables of f into physical quantities.１ －１ is the target object m １ represents a conversion model M4 that converts the physical quantity of f into a latent variable, ２ －１ is the target object m ２ represents a conversion model M4 that converts the physical quantity O １ is the target object m １ is the correct physical quantity, and the physical quantity O ２ is the target object m ２ is the correct physical quantity.

[0101] Even in this configuration, the learning unit 21 can preferably learn the conversion model M3 and the conversion model M4. Also, even in this configuration, the learning unit 21 can learn the identification model M2 and the conversion model M3 or M4 without using ground truth data in which latent variables are associated with physical quantities.

[0102] (Learning Method Flow 1) The flow of the learning method S2 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the learning method S2. Note that the following describes a learning method for the identification model M2 and the conversion model M3 that converts the latent variable z into the physical quantity o.

[0103] (Step S21) In step S21, the acquisition unit 11A inputs an input image into the world model M1 to obtain a plurality of latent variables {z ｎ} ｎ＝１～Ｎ The acquisition unit 11A acquires the acquired multiple latent variables {z ｎ} ｎ＝１～Ｎ is stored in the storage unit 29.

[0104] (Step S22) In step S22, the identification unit 12A adds a plurality of latent variables {z ｎ} ｎ＝１～Ｎ By inputting each of the ｎ} ｎ＝１～Ｎ The recognition unit 12A recognizes the objects corresponding to each of the recognition results {l ｎ} ｎ＝１～Ｎ is supplied to the output section 13A.

[0105] (Step S23) In step S23, the output unit 13A outputs the classification result {l ｎ} ｎ＝１～Ｎ is output to the storage unit 29 and stored in the storage unit 29.

[0106] (Step S24) In step S24, the conversion unit 22 converts the plurality of latent variables {z ｎ} ｎ＝１～Ｎ By inputting each of the following, multiple latent variables {z ｎ} ｎ＝１～Ｎ Each of these is expressed as a set of multiple latent variables {z ｎ} ｎ＝１～Ｎ The physical quantities of the objects corresponding to each of the ｎ} ｎ＝１～Ｎ The conversion unit 22 converts the physical quantity {o ｎ} ｎ＝１～Ｎ is stored in the storage unit 29.

[0107] (Step S25) In step S25, the learning unit 21 performs the learning on the classification results {l ｎ} ｎ＝１～Ｎ , the physical quantity {o ｎ} ｎ＝１～Ｎ , and the correct physical quantity {O ｍ} ｍ＝１～Ｍ The loss is calculated by referring to the above.

[0108] (Step S26) In step S26, the learning unit 21 trains the discriminative model M2 and the conversion model M3 by referring to the loss calculated in step S25.

[0109] (Learning Method Flow 2) The flow of the learning method for the identification model M2 and the conversion model M4 that converts the physical quantity o into the latent variable z will be described.

[0110] (Steps S21 to S23) Steps S21 to S23 are the same as the processes described above, and therefore the explanation will be omitted.

[0111] (Step S24) In step S24, the conversion unit 22 converts the correct physical quantity {Oｍ} ｍ＝１～Ｍ By inputting each of the above, multiple physical quantities {O ｍ} ｍ＝１～Ｍ are respectively expressed as multiple latent variables {z ｍ} ｍ＝１～Ｍ The conversion unit 22 converts (inversely converts) the latent variable {z ｍ} ｍ＝１～Ｍ is stored in the storage unit 29.

[0112] (Step S25) In step S25, the learning unit 21 performs the learning on the classification results {l ｎ} ｎ＝１～Ｎ , the latent variables {z ｍ} ｍ＝１～Ｍ , and the latent variables {z ｎ} ｎ＝１～Ｎ The loss is calculated by referring to the above.

[0113] (Step S26) In step S26, the learning unit 21 trains the discriminative model M2 and the conversion model M4 by referring to the loss calculated in step S25.

[0114] (Flow of Conversion Method) The flow of conversion method S3 will be described with reference to Fig. 6. Fig. 6 is a flow chart showing the flow of conversion method S3.

[0115] (Step S31) In step S31, the acquisition unit 11A inputs an input image to the world model M1 to obtain a plurality of latent variables {z ｎ} ｎ＝１～Ｎ The acquisition unit 11A acquires the acquired multiple latent variables {z ｎ} ｎ＝１～Ｎ is stored in the storage unit 29.

[0116] (Step S32) In step S32, the identification unit 12A adds a plurality of latent variables {z ｎ} ｎ＝１～Ｎ By inputting each of the ｎ} ｎ＝１～Ｎ The recognition unit 12A recognizes the objects corresponding to each of the recognition results {lｎ} ｎ＝１～Ｎ The classification unit 12A outputs argmax{l ｎ} ｎ＝１～Ｎ As described above, the classification result is output as a classification result l indicating whether the latent variable zn corresponds to one or more target objects m. ｎ、ｍ In this case, in step S32, the classification unit 12A calculates argmax_m(l ｎ、ｍ ) is output.

[0117] (Step S33) In step S33, the output unit 13A outputs the classification result argmax{l ｎ} ｎ＝１～Ｎ is output to the storage unit 29 and stored in the storage unit 29.

[0118] (Step S34) In step S34, the conversion unit 22 converts the plurality of latent variables {z ｎ} ｎ＝１～Ｎ and the classification result argmax{l ｎ} ｎ＝１～Ｎ By inputting ｎ} ｎ＝１～Ｎ Each of these is expressed as a set of multiple latent variables {z ｎ} ｎ＝１～Ｎ The physical quantities of the objects corresponding to each of the ｎ} ｎ＝１～Ｎ The conversion unit 22 converts the physical quantity {o ｎ} ｎ＝１～Ｎ is stored in the storage unit 29.

[0119] Alternatively, in step S34, the conversion unit 22 converts the physical quantities {o ｎ} ｎ＝１～Ｎ and the classification result argmax{l ｎ} ｎ＝１～Ｎ By inputting the above, the physical quantities of multiple objects {o ｎ} ｎ＝１～Ｎ are respectively expressed as multiple latent variables {z ｎ} ｎ＝１～ＮThe conversion unit 22 converts (inversely converts) the latent variable {z ｎ} ｎ＝１～Ｎ is stored in the storage unit 29.

[0120] (Effect of Converter 2) As described above, in Converter 2, the multiple latent variables {z ｎ} ｎ＝１～Ｎ The latent variable z output from the world model M1 is not associated with a specific object. However, the converter 2 can identify which object each of the multiple latent variables z output from the world model M1 corresponds to. Therefore, the converter 2 can identify which object each of the multiple latent variables z output from the world model M1 corresponds to, based on the latent variable z output from the world model, and then convert the latent variable z into a physical quantity o, or inversely convert the physical quantity o back into the latent variable z.

[0121] [Application Example] An application example using the classifier 1A will be described with reference to Fig. 7. Fig. 7 is a diagram showing an overview of a medical robot rb using the classifier 1A. Note that this application example may be applied to the configuration of the first exemplary embodiment.

[0122] 7 uses a machine-learned world model to grasp the environment in the operating room, the patient's physical condition, and the instruments and procedures required for surgery. The world model is a machine-learned model that receives inputs of sensor information indicating images and changes in the environment and outputs information required for surgery.

[0123] For example, the medical robot rb acquires images from a camera that photographs the operating room and acquires sensor information from a sensor that detects changes in the operating room environment. Next, the medical robot rb inputs the acquired images and sensor information into a world model to acquire information necessary for the surgery. Here, the classifier 1A identifies to which object the latent variables, which are information necessary for the surgery and output from the world model, correspond. Next, the medical robot rb grasps the internal condition of the patient in real time based on the output of the world model and the classification results of the classifier 1A, and optimizes the selection of instruments and procedures necessary for the surgery.

[0124] In this way, the medical robot rb using the classifier 1A and the world model can predict the movements of doctors, nurses, etc. Therefore, the medical robot rb can autonomously execute optimized operations in cooperation with doctors, nurses, etc.

[0125] Furthermore, the medical robot rb using the classifier 1A and the world model can accurately recognize the condition of the patient, which allows the medical robot rb to perform surgery with minimal invasiveness.

[0126] [Example of Software Implementation] Some or all of the functions of the classifiers 1, 1A, and converter 2 (hereinafter also referred to as "each of the above devices") may be implemented by hardware such as an integrated circuit (IC chip), or by software.

[0127] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 8. Figure 8 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0128] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to function as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0129] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0130] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, a mouse, a display, and a printer.

[0131] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0132] [Appendix A] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0133] (Appendix A1) A classifier comprising: an acquisition means for acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; a classification means for identifying an object corresponding to each of the plurality of latent variables; and an output means for outputting a classification result by the classification means.

[0134] (Appendix A2) The classifier according to Appendix A1, wherein the classification means identifies an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into a classification model that has been machine-trained to use a latent variable as an input and output a classification result that identifies an object corresponding to the latent variable.

[0135] (Supplementary Note A3) The classifier according to Supplementary Note A2, wherein the discriminative model is a model that is machine-learned to output accuracy information indicating the likelihood that an input latent variable is a latent variable corresponding to a target object.

[0136] (Supplementary Note A4) The classifier according to Supplementary Note A2, wherein the discrimination model is a model trained by machine learning to output certainty information indicating the likelihood that any one of a plurality of input latent variables is a latent variable corresponding to a target object.

[0137] (Appendix A5) The classifier according to Appendix A2, wherein the discrimination model is a model trained by machine learning to output accuracy information indicating the likelihood that each of a plurality of input latent variables is a latent variable corresponding to each of a plurality of target objects.

[0138] (Appendix A6) A converter comprising: the classifier according to appendix A1 or A2; and a conversion means for converting the plurality of latent variables into physical quantities of the plurality of objects, respectively, with reference to the classification result, or converting the physical quantities of the plurality of objects into a plurality of latent variables, respectively.

[0139] (Appendix A7) The converter according to Appendix A6, wherein the conversion means converts the plurality of latent variables into the physical quantities of the plurality of objects by inputting the plurality of latent variables and the identification result by the identification means into a conversion model that has been machine-learned to take latent variables and identification results as inputs and output physical quantities of the object corresponding to the latent variables, or converts the physical quantities of the plurality of objects into the plurality of latent variables by inputting the physical quantities of the plurality of objects and the identification result by the identification means into a conversion model that has been machine-learned to take physical quantities of the object and identification results as inputs and output latent variables of the object corresponding to the physical quantities.

[0140] (Supplementary Note A8) The converter according to Supplementary Note A7, further comprising: a learning unit that performs machine learning on the identification model and the conversion model using a classification result by the classification means, the plurality of latent variables, and a correct physical quantity.

[0141] (Supplementary Note A9) The converter according to Supplementary Note A8, wherein the identification result by the identification means is an identification result based on probability information indicating a probability that each of the plurality of latent variables is a latent variable corresponding to a target object.

[0142] (Supplementary Note A10) The converter according to Supplementary Note A8, wherein the identification result by the identification means is an identification result based on probability information indicating a probability that any one of the plurality of latent variables is a latent variable corresponding to a target object.

[0143] (Supplementary Note A11) The converter according to Supplementary Note A8, wherein the identification result by the identification means is an identification result based on probability information indicating a probability that each of the plurality of latent variables is a latent variable corresponding to each of the plurality of target objects.

[0144] (Appendix A12) A converter comprising: an acquisition means for acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification means for identifying an object corresponding to each of the plurality of latent variables; a conversion means for converting the plurality of latent variables into physical quantities of the plurality of objects, respectively, or converting the physical quantities of the plurality of objects into a plurality of latent variables, respectively, with reference to an identification result by the identification means; and an output means for outputting the physical quantities or latent variables converted by the conversion means.

[0145] (Appendix A13) An acquisition means for acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification means for identifying an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into an identification model that has been machine-learned to take a latent variable as input and output an identification result that identifies the object corresponding to the latent variable; and a conversion means for converting the plurality of latent variables into physical quantities of the plurality of objects by inputting the plurality of latent variables and the identification result by the identification means into a conversion model that has been machine-learned to take a latent variable and the identification result as input and output the physical quantity of the object corresponding to the latent variable, or a conversion means for converting the physical quantities of the plurality of objects into a plurality of latent variables by inputting the physical quantities of the plurality of objects and the identification result by the identification means into a conversion model that has been machine-learned to take a physical quantity of an object and the identification result as input and output the latent variable of the object corresponding to the physical quantity. a learning means for performing machine learning on the discrimination model and the transformation model using a discrimination result by the discrimination means, the plurality of latent variables, and a correct physical quantity.

[0146] [Appendix B] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0147] (Appendix B1) An identification method including: an acquisition process in which at least one processor acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process in which the at least one processor identifies an object corresponding to each of the plurality of latent variables; and an output process in which the at least one processor outputs an identification result from the identification process.

[0148] (Appendix B2) The identification method according to Appendix B1, wherein in the identification process, the at least one processor identifies an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into an identification model that has been machine-trained to use the latent variables as input and output an identification result that identifies the object corresponding to the latent variable.

[0149] (Supplementary Note B3) The identification method according to Supplementary Note B2, wherein the identification model is a model that is machine-learned to output accuracy information indicating the likelihood that an input latent variable is a latent variable corresponding to a target object.

[0150] (Supplementary Note B4) The identification method according to Supplementary Note B2, wherein the identification model is a model trained by machine learning to output accuracy information indicating the likelihood that any one of a plurality of input latent variables is a latent variable corresponding to a target object.

[0151] (Appendix B5) The identification method according to Appendix B2, wherein the identification model is a model trained by machine learning to output accuracy information indicating the likelihood that each of a plurality of input latent variables is a latent variable corresponding to each of a plurality of target objects.

[0152] (Appendix B6) A conversion method including: the identification method according to Appendix B1 or B2; and a conversion process in which the at least one processor refers to the identification result and converts the plurality of latent variables into physical quantities of the plurality of objects, respectively, or converts the physical quantities of the plurality of objects into a plurality of latent variables, respectively.

[0153] (Appendix B7) The conversion method described in Appendix B6, wherein in the conversion process, the at least one processor: converts the multiple latent variables into the physical quantities of the multiple objects by inputting the multiple latent variables and the identification results from the identification process into a conversion model that has been machine-learned to take latent variables and identification results as inputs and output physical quantities of the object corresponding to the latent variables, or converts the physical quantities of the multiple objects into multiple latent variables by inputting the physical quantities of the multiple objects and the identification results from the identification process into a conversion model that has been machine-learned to take physical quantities of the object and the identification results as inputs and output latent variables of the object corresponding to the physical quantities.

[0154] (Appendix B8) The conversion method according to Appendix B7, further including a learning process in which the at least one processor performs machine learning on the identification model and the conversion model using an identification result in the identification process, the plurality of latent variables, and a correct physical quantity.

[0155] (Supplementary Note B9) The conversion method according to Supplementary Note B8, wherein the classification result in the classification process is a classification result based on probability information indicating a probability that each of the plurality of latent variables is a latent variable corresponding to a target object.

[0156] (Supplementary Note B10) The conversion method according to Supplementary Note B8, wherein the classification result in the classification process is a classification result based on probability information indicating a probability that any one of the plurality of latent variables is a latent variable corresponding to a target object.

[0157] (Supplementary Note B11) The conversion method according to Supplementary Note B8, wherein the classification result in the classification process is a classification result based on probability information indicating a probability that each of the plurality of latent variables is a latent variable corresponding to each of the plurality of target objects.

[0158] (Appendix B12) A conversion method including: an acquisition process in which at least one processor acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process in which the at least one processor identifies an object corresponding to each of the plurality of latent variables; a conversion process in which the at least one processor converts the plurality of latent variables into physical quantities of the plurality of objects, respectively, or converts the physical quantities of the plurality of objects into a plurality of latent variables, respectively, with reference to an identification result in the identification process; and an output process in which the physical quantities or latent variables after conversion by the conversion means are output.

[0159] (Supplementary Note B13) An acquisition process in which at least one processor acquires a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process in which the at least one processor identifies an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into an identification model that has been machine-learned to take a latent variable as input and output an identification result that identifies the object corresponding to the latent variable; and a conversion process in which the at least one processor converts the plurality of latent variables into physical quantities of the plurality of objects by inputting the plurality of latent variables and an identification result in the identification process into a conversion model that has been machine-learned to take a latent variable and an identification result as input and output a physical quantity of the object corresponding to the latent variable, or a conversion process in which the at least one processor converts the physical quantities of the plurality of objects into a plurality of latent variables by inputting the physical quantities of the plurality of objects and the identification result in the identification process into a conversion model that has been machine-learned to take a physical quantity of the object and the identification result as input and output a latent variable of the object corresponding to the physical quantity. a learning process in which the at least one processor performs machine learning to develop the discrimination model and the transformation model using a classification result from the classification process, the plurality of latent variables, and a correct physical quantity.

[0160] [Appendix C] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0161] (Appendix C1) A program that causes a computer to function as a classifier, the program causing the computer to execute: an acquisition process that acquires multiple latent variables corresponding to each of the multiple objects by inputting an input image containing multiple objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; a classification process that identifies objects corresponding to each of the multiple latent variables; and an output process that outputs the classification results of the classification process.

[0162] (Appendix C2) The program according to Appendix C1, wherein in the identification process, an object corresponding to each of the plurality of latent variables is identified by inputting each of the plurality of latent variables into an identification model that has been machine-trained to take the latent variables as input and output an identification result that identifies the object corresponding to the latent variables.

[0163] (Supplementary Note C3) The program according to Supplementary Note C2, wherein the discriminative model is a model trained by machine learning to output accuracy information indicating the likelihood that an input latent variable is a latent variable corresponding to a target object.

[0164] (Supplementary Note C4) The program according to Supplementary Note C2, wherein the discriminative model is a model trained by machine learning to output accuracy information indicating the likelihood that any one of a plurality of input latent variables is a latent variable corresponding to a target object.

[0165] (Appendix C5) The program according to Appendix C2, wherein the discriminative model is a model trained by machine learning to output accuracy information indicating the likelihood that each of a plurality of input latent variables is a latent variable corresponding to each of a plurality of target objects.

[0166] (Appendix C6) A program that causes the computer to execute the acquisition process, identification process, and output process described in Appendix C1 or C2, and a conversion process that converts the plurality of latent variables into physical quantities of the plurality of objects, respectively, by referring to the identification result, or converts the physical quantities of the plurality of objects into a plurality of latent variables, respectively.

[0167] (Appendix C7) The program described in Appendix C6, wherein in the conversion process, the multiple latent variables are converted into the physical quantities of the multiple objects by inputting the multiple latent variables and the identification results by the identification means into a conversion model that has been machine-learned to take latent variables and identification results as inputs and output physical quantities of the object corresponding to the latent variables, or the physical quantities of the multiple objects are converted into the multiple latent variables by inputting the physical quantities of the multiple objects and the identification results by the identification means into a conversion model that has been machine-learned to take physical quantities of the object and identification results as inputs and output latent variables of the object corresponding to the physical quantities.

[0168] (Supplementary Note C8) The program according to Supplementary Note C7, further executing a learning process for machine learning the identification model and the transformation model using an identification result in the identification process, the plurality of latent variables, and a correct physical quantity.

[0169] (Supplementary Note C9) The program according to Supplementary Note C8, wherein the classification result in the classification process is a classification result based on accuracy information indicating a likelihood that each of the plurality of latent variables is a latent variable corresponding to a target object.

[0170] (Supplementary Note C10) The program according to Supplementary Note C8, wherein the classification result in the classification process is a classification result based on accuracy information indicating a likelihood that any one of the plurality of latent variables is a latent variable corresponding to a target object.

[0171] (Supplementary Note C11) The program according to Supplementary Note C8, wherein the classification result in the classification process is a classification result based on accuracy information indicating a likelihood that each of a plurality of latent variables is a latent variable corresponding to each of a plurality of target objects.

[0172] (Appendix C12) A program that causes a computer to function as a converter, the program executing: an acquisition process that acquires multiple latent variables by inputting an input image containing multiple objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process that identifies objects corresponding to each of the multiple latent variables; a conversion process that converts the multiple latent variables into physical quantities of the multiple objects, respectively, or converts the physical quantities of the multiple objects into multiple latent variables, respectively, by referring to the identification results of the identification process; and an output process that outputs the physical quantities or latent variables converted by the conversion means.

[0173] (Appendix C13) A program that causes a computer to function as a learning device, comprising: an acquisition process of acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process of identifying an object corresponding to each of a plurality of latent variables by inputting each of the plurality of latent variables into an identification model that has been machine-learned to take a latent variable as input and output an identification result that identifies the object corresponding to the latent variable; and a conversion process of converting the plurality of latent variables into physical quantities of the plurality of objects by inputting the plurality of latent variables and the identification result in the identification process into a conversion model that has been machine-learned to take a latent variable and the identification result as input and output the physical quantity of the object corresponding to the latent variable, or a conversion process of converting the physical quantities of the plurality of objects into a plurality of latent variables by inputting the physical quantities of the plurality of objects and the identification result in the identification process into a conversion model that has been machine-learned to take a physical quantity of an object and the identification result as input and output the latent variable of the object corresponding to the physical quantity. a learning process for machine learning the identification model and the conversion model using the identification result in the identification process, the plurality of latent variables, and correct physical quantities.

[0174] [Appendix D] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0175] (Appendix D1) A classifier comprising at least one processor, the at least one processor performing an acquisition process of acquiring a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process of identifying an object corresponding to each of the plurality of latent variables; and an output process of outputting an identification result from the identification process.

[0176] (Appendix D2) The classifier according to Appendix D1, wherein the at least one processor, in the classification process, identifies an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into a classification model that has been machine-trained to use the latent variables as input and output a classification result that identifies the object corresponding to the latent variables.

[0177] (Appendix D3) The classifier according to Appendix D2, wherein the discrimination model is a model trained by machine learning to output accuracy information indicating the likelihood that an input latent variable is a latent variable corresponding to a target object.

[0178] (Appendix D4) The classifier according to Appendix D2, wherein the discrimination model is a model trained by machine learning to output certainty information indicating the likelihood that any one of a plurality of input latent variables is a latent variable corresponding to a target object.

[0179] (Appendix D5) The classifier according to Appendix D2, wherein the discrimination model is a model trained by machine learning to output accuracy information indicating the likelihood that each of a plurality of input latent variables is a latent variable corresponding to each of a plurality of target objects.

[0180] (Appendix D6) A converter comprising: the classifier according to appendix D1 or D2; and at least one processor, wherein the at least one processor performs a conversion process of converting the plurality of latent variables into physical quantities of the plurality of objects, respectively, or converting the physical quantities of the plurality of objects into a plurality of latent variables, respectively, with reference to the classification result.

[0181] (Appendix D7) The converter described in Appendix D6, wherein the at least one processor, in the conversion process, converts the multiple latent variables into the physical quantities of the multiple objects by inputting the multiple latent variables and the identification result by the identification means into a conversion model that has been machine-learned to take the latent variables and the identification result as input and output the physical quantities of the object corresponding to the latent variables, or converts the physical quantities of the multiple objects into multiple latent variables by inputting the physical quantities of the multiple objects and the identification result by the identification means into a conversion model that has been machine-learned to take the physical quantities of the object and the identification result as input and output the latent variables of the object corresponding to the physical quantities.

[0182] (Appendix D8) The converter according to Appendix D7, wherein the at least one processor further executes a learning process for machine learning the identification model and the conversion model using an identification result in the identification process, the plurality of latent variables, and a correct physical quantity.

[0183] (Appendix D9) The converter according to appendix D8, wherein the classification result in the classification process is a classification result based on probability information indicating a probability that each of the plurality of latent variables is a latent variable corresponding to a target object.

[0184] (Appendix D10) The converter according to appendix D8, wherein the classification result in the classification process is a classification result based on probability information indicating a probability that any one of the plurality of latent variables is a latent variable corresponding to a target object.

[0185] (Appendix D11) The converter according to appendix D8, wherein the classification result in the classification process is a classification result based on probability information indicating a probability that each of the plurality of latent variables is a latent variable corresponding to each of the plurality of target objects.

[0186] (Appendix D12) A converter comprising at least one processor, the at least one processor performing: an acquisition process for acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process for identifying an object corresponding to each of the plurality of latent variables; a conversion process for converting the plurality of latent variables into physical quantities of the plurality of objects, respectively, or converting the physical quantities of the plurality of objects into a plurality of latent variables, respectively, with reference to an identification result in the identification process; and an output process for outputting the physical quantities or latent variables after the conversion process.

[0187] (Appendix D13) A system including at least one processor, wherein the at least one processor comprises: an acquisition process of acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image including a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process of identifying an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into an identification model that has been machine-learned to take a latent variable as input and output an identification result that identifies the object corresponding to the latent variable; and a conversion process of converting the plurality of latent variables into physical quantities of the plurality of objects by inputting the plurality of latent variables and the identification result in the identification process into a conversion model that has been machine-learned to take a latent variable and the identification result as input and output the physical quantity of the object corresponding to the latent variable, or a conversion process of converting the physical quantities of the plurality of objects into a plurality of latent variables by inputting the physical quantities of the plurality of objects and the identification result in the identification process into a conversion model that has been machine-learned to take a physical quantity of the object and the identification result as input and output the latent variable of the object corresponding to the physical quantity. a learning process for machine learning the discrimination model and the transformation model using a discrimination result in the discrimination process, the plurality of latent variables, and a correct physical quantity.

[0188] [Appendix E] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0189] (Appendix E1) A non-transient recording medium having recorded thereon a program for executing the following: an acquisition process for acquiring multiple latent variables corresponding to each of the multiple objects by inputting an input image containing multiple objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process for identifying an object corresponding to each of the multiple latent variables; and an output process for outputting the identification results of the identification process.

[0190] (Appendix E2) A non-transient recording medium having recorded thereon a program that executes the following: an acquisition process for acquiring multiple latent variables corresponding to each of the multiple objects by inputting an input image containing multiple objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process for identifying an object corresponding to each of the multiple latent variables; a conversion process for converting the multiple latent variables into physical quantities of the multiple objects, respectively, or converting the physical quantities of the multiple objects into multiple latent variables, respectively, with reference to the identification results of the identification process; and an output process for outputting the physical quantities or latent variables after the conversion process.

[0191] (Appendix E3) An acquisition process in which an input image containing multiple objects as subjects is input into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image, thereby acquiring multiple latent variables corresponding to each of the multiple objects; an identification process in which an object corresponding to each of the multiple latent variables is identified by inputting each of the multiple latent variables into an identification model that has been machine-learned to take a latent variable as input and output an identification result that identifies the object corresponding to the latent variable; and a conversion process in which the multiple latent variables are converted into physical quantities of the multiple objects by inputting the multiple latent variables and the identification result in the identification process into a conversion model that has been machine-learned to take a latent variable and the identification result as input and output the physical quantity of the object corresponding to the latent variable, or a conversion process in which the physical quantities of the multiple objects are converted into multiple latent variables by inputting the physical quantities of the multiple objects and the identification result in the identification process into a conversion model that has been machine-learned to take a physical quantity of the object and the identification result as input and output the latent variable of the object corresponding to the physical quantity. a learning process for machine learning the identification model and the conversion model using the identification result in the identification process, the plurality of latent variables, and correct physical quantities; and a non-transitory recording medium on which a program for executing the above is recorded.

[0192] 1, 1A Classifier 2 Converter 11, 11A Acquisition unit 12, 12A Classification unit 13, 13A Output unit 21 Learning unit 22 Conversion unit l Classification result M1 World model M2 Classification model M3, M4 Conversion model n Object o Physical quantity rb Medical robot z Latent variable

Claims

1. A classifier comprising: an acquisition means for acquiring a plurality of latent variables corresponding to each of a plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects contained as subjects in the image; an identification means for identifying objects corresponding to each of the plurality of latent variables; and an output means for outputting an identification result by the identification means.

2. The classifier according to claim 1, wherein the classification means identifies an object corresponding to each of the plurality of latent variables by inputting each of the plurality of latent variables into a classification model that has been machine-trained to use the latent variables as input and output a classification result that identifies an object corresponding to the latent variable.

3. The classifier according to claim 2, wherein the discrimination model is a model trained by machine learning so as to output accuracy information indicating the likelihood that an input latent variable is a latent variable corresponding to a target object.

4. The classifier according to claim 2, wherein the discrimination model is a model trained by machine learning so as to output probability information indicating the likelihood that any one of a plurality of input latent variables is a latent variable corresponding to a target object.

5. The classifier according to claim 2, wherein the discrimination model is a model trained by machine learning so as to output probability information indicating the likelihood that each of a plurality of input latent variables is a latent variable corresponding to each of a plurality of target objects.

6. A converter comprising: a classifier according to claim 2; and a conversion means for converting the plurality of latent variables into physical quantities of the plurality of objects, respectively, by referring to the classification result, or for converting the physical quantities of the plurality of objects into a plurality of latent variables, respectively.

7. The converter according to claim 6, wherein the conversion means converts the multiple latent variables into the physical quantities of the multiple objects by inputting the multiple latent variables and the identification result by the identification means into a conversion model that has been machine-learned to take latent variables and identification results as inputs and output physical quantities of the object corresponding to the latent variables, or converts the physical quantities of the multiple objects into multiple latent variables by inputting the physical quantities of the multiple objects and the identification result by the identification means into a conversion model that has been machine-learned to take physical quantities of an object and identification results as inputs and output latent variables of the object corresponding to the physical quantities.

8. The converter according to claim 7, further comprising: a learning means for machine learning the discrimination model and the conversion model using a discrimination result by the discrimination means, the plurality of latent variables, and a correct physical quantity.

9. The converter according to claim 8, wherein the classification result by said classification means is a classification result based on probability information indicating the likelihood that each of said plurality of latent variables is a latent variable corresponding to a target object.

10. The converter according to claim 8, wherein the classification result by said classification means is a classification result based on probability information indicating the likelihood that any one of said plurality of latent variables is a latent variable corresponding to a target object.

11. The converter according to claim 8, wherein the classification result by said classification means is a classification result based on probability information indicating the likelihood that each of a plurality of latent variables is a latent variable corresponding to each of a plurality of target objects.

12. A classification method comprising: an acquisition process in which at least one processor acquires a plurality of latent variables corresponding to each of the plurality of objects by inputting an input image containing a plurality of objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; a classification process in which the at least one processor identifies objects corresponding to each of the plurality of latent variables; and an output process in which the at least one processor outputs a classification result obtained by the classification process.

13. A program that causes a computer to function as a classifier, the program causing the computer to execute an acquisition process that acquires multiple latent variables corresponding to each of the multiple objects by inputting an input image containing multiple objects as subjects into a world model that has been machine-learned to take an image as input and output latent variables of objects included as subjects in the image; an identification process that identifies objects corresponding to each of the multiple latent variables; and an output process that outputs the identification results of the identification process.

Images