Estimating reliability of control data

Abstract

Reliability of control data is provided for estimation. The invention relates to a computer-implemented method (600) of estimating reliability of control data of a computer-controlled system interacting with an environment. The control data is inferred from model inputs by a machine-learnable control model trained on a training dataset. The model inputs comprise at least one directional vector extracted from sensor data and associated with a component of the computer-controlled system or an object in the environment. The reliability is estimated using a generative model trained to generate synthetic model inputs representative of the training dataset by applying an inverse of the generative model to the model inputs to determine a likelihood of the model inputs being generated according to the generative model. The generative model comprises a coupling layer comprising a circular transformation and one or more of an unconditional rotation and a conditional rotation.

Description

Technical Field

[0001] This invention relates to a computer-implemented method for estimating the reliability of control data in a computer control system interacting with its environment, and a corresponding system. The invention further relates to a computer-implemented method for generating synthetic direction data instances, and a corresponding system. The invention also relates to a computer-readable medium. Background Technology

[0002] Robots (and more generally, computer control systems) are increasingly being automatically trained to learn new skills or adapt their behavior to unseen conditions. Because real-world interactions between robots and their environments are often costly (and potentially dangerous if humans are nearby), it is important to perform this training in a safe, fast, and data-efficient manner.

[0003] A technique for training robots is presented in “Bayesian Optimization Meets Riemannian Manifolds in Robot Learning” by N. Jaquier et al. (available at https: / / arxiv.org / abs / 1910.04998 and incorporated herein by reference). This technique is geometrically perceptive, leveraging the fact that parameters representing a robot and / or its interaction with the environment are not typically unrestricted independent numerical values ​​but instead have geometric meaning. For example, parameters describing a robot may include, for instance, one or more orientations or directions of corresponding links in a robot arm. Orientation and direction are examples of orientation vectors that can be represented as points on a (unit) sphere, such as a sphere in 4-dimensional space for orientation, or a sphere in 3D for direction. Besides representing orientation or direction in computer-controlled systems, orientation vectors have various other technical uses as described herein, such as in computational biology.

[0004] One of the main challenges when deploying a trained computer control system in practice is handling unexpected situations effectively. It is often difficult to collect a training set that represents the entire set of operating conditions that might be encountered in practice. Therefore, when deployed, the system may get stuck in situations it did not encounter during training. Due to the lack of training examples, there is a risk that the robot may perform harmful actions; for example, harmful to the robot itself, to the environment it interacts with (e.g., in the case of manufacturing the robot, it may include the object being manufactured), or to people near the robot. Generally, one way to handle such discrepancies between training and use is by performing anomaly detection. Using anomaly detection, the robot can detect that it is in a state inconsistent with the dataset on which it was originally trained. Therefore, in this situation, the reliability of the trained model may be expected to be low. Based on such detection, the robot may, for example, stop, its operation may switch to a safe mode, or human intervention may be initiated. Summary of the Invention

[0005] The object of this invention is to provide a technique for estimating the reliability of control data inferred from a machine-learnable control model, which provides more accurate results for control data inferred from direction vectors, such as those representing the position or orientation of a relevant object. Using such reliability estimation, computer-controlled systems can be controlled more reliably. Another object of this invention is to provide a technique for generating synthetic data representing a training dataset, which provides more accurate results for data including direction vectors (e.g., model inputs to a machine-learnable control model). A further object of this invention is to provide a model that can be used both efficiently and accurately for estimating reliability and for generating synthetic data. Specifically, the object is to provide an accurate model that is parallelizable and therefore can be evaluated for, for example, multi-core processors or other parallel execution environments.

[0006] According to a first aspect of the invention, as defined in claims 1 and 13 respectively, a computer-implemented method and corresponding system for estimating the reliability of control data are described. According to another aspect of the invention, as defined in claims 9 and 14 respectively, a computer-implemented method and corresponding system for generating synthetic direction data instances are described. According to yet another aspect of the invention, as defined in claim 15, a computer-readable medium is described. Embodiments are defined in the dependent claims.

[0007] Various aspects involve instances of orientation data, such as model inputs used to control models. Such instances of orientation data may include at least one orientation vector associated with a physical object. An orientation vector may represent a property or combination of properties of the physical object, and can be represented as an orientation, such as a point on a hypersphere. For example, an orientation vector may represent the object's orientation (e.g., direction of movement) or orientation. Here, the term "orientation" is used in a physical sense, for example, as an imaginary rotation (combined with an optional imaginary translation) required to move an object from a reference placement to its current position.

[0008] For example, the physical object can be a robotic arm, in which case the orientation data instance can include one or more orientation vectors representing the orientation of a corresponding link of the robotic arm and / or a orientation vector representing the direction of movement of the corresponding link. More generally, the physical object can be a component of a computer control system or an object in the environment with which the system interacts; for example, the orientation data instance can include orientation vectors representing one or more orientations in the environment for a path navigation task. As another example, the physical object can be a protein or other biological structure, where the orientation data instance represents the orientation or direction of a corresponding component of the biological structure (e.g., the backbone of a protein). The orientation data instance can, for example, include one or more directions and / or one or more orientations, and / or other orientation vectors, such as up to or at least five, up to or at least ten, or up to or at least twenty orientation vectors.

[0009] In addition to representing the direction associated with a physical object, orientation data instances (e.g., model inputs to a control model) typically also represent additional information about the physical object. For example, an orientation vector itself can represent not only direction but also magnitude, such as velocity. However, in other embodiments, orientation vectors are used exclusively to represent direction, rather than magnitude. In any case, orientation data instances often include additional data besides one or more orientation vectors. For example, an orientation data instance can represent the state of a robot or other computer control system. In addition to one or more orientation vectors, such a state can include data representing one or more of the following: force, torque, stiffness, inertia, maneuverability, and attitude of the system or the object interacting with it. Generally, one or more orientation vectors and / or other data included in the orientation vectors can be extracted from sensor data of the physical object(s) to which the orientation data instance relates.

[0010] In various aspects, generative models are utilized to estimate the reliability of machine-learnable control models used for such directional data. Generally, generative models can be trained to generate directional data instances representing datasets. The dataset can include, for example, multiple directional data instances as described above, extracted from sensor data representing one or more physical objects corresponding to direction vectors.

[0011] In some respects, given an instance of orientation data, a generative model is used to determine the likelihood of generating that orientation data based on the generative model. Combined with a control model trained on the same dataset, the likelihood can indicate the consistency of the orientation data instance with that dataset, and thus provide an estimate of the reliability of the control data inferred by the control model from that orientation data instance. In other respects, the generative model is used to generate synthetic orientation data instances that represent the dataset. In both cases, it is beneficial for the generative model to generate orientation data instances that are more representative of the dataset; for example, it is beneficial for the generative model to learn a probability distribution over orientation data instances that are more representative of the training dataset.

[0012] Generally, generative models can generate oriented data instances from latent feature vectors by applying one or more training layers. Layers can transform input vectors into output vectors deterministically or stochastically, depending on the set of parameters trained during the training of the generative model. Layers are also called transformations or mappings. Constructing generative models by composing layers and training such models through log-likelihood optimization is known, for example, from D. Rezende et al., “Variational Inference with Normalizing Flows” (available at https: / / arxiv.org / abs / 1505.05770 and incorporated herein by reference) and D. Nielsen et al., “SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows” (available at https: / / arxiv.org / abs / 2007.02731 and incorporated herein by reference). For example, generative models as described herein can be implemented as normalized flows or SurVAE flows. Therefore, generative models can, for example, enable accurate likelihood assessment, efficient inference / data synthesis, and latent space representation learning that is useful for downstream tasks.

[0013] Interestingly, the generative model described in this paper uses a coupling layer specifically designed to accurately represent transformations for directional data. The coupling layer transforms the input direction vector into an output direction vector. The coupling layer can map an input direction vector located on a hypersphere to an output direction vector located on the same hypersphere; for example, the coupling layer can represent a norm-preserving mapping. Therefore, the coupling layer can act on the direction represented by the input direction vector.

[0014] For example, the input direction vector and / or the output direction vector can represent directions in 3D (e.g., as a 3D sphere). Points on the surface), orientation in 3D (e.g., as a hypersphere) (Points on the input direction vector) or a cascade of multiple such directions and orientations. The coupling layer can act on the input direction vector, thereby preserving other elements of the representation of the direction data instance.

[0015] To transform an input direction vector into an output direction vector, the coupling layer can apply unconditional and / or conditional rotations to the input direction vector. Unconditional rotations can rotate the input direction vector based on the parameters of the generative model (or apply the coupling layer's transformation to the input direction vector as a result). Conditional rotations can rotate a first subset of the input direction vector's Cartesian coordinates based on another set of Cartesian coordinates that do not intersect with the first subset, such as residual coordinates. For example, in... k In 3D space, rotation can correspond to multiplication. Matrices. Rotations can be norm-preserving. Generally, conditional and / or unconditional rotations can be deterministic or random.

[0016] In addition to unconditional and / or conditional rotations, the coupling layer may further include circular transformations (e.g., rotations already applied to the vector) acting independently on one or more circular slices (e.g., Cartesian coordinate pairs) of the input direction vector. Generally, such a Cartesian coordinate pair represents a point on a circle with a certain radius (this does not need to be a single point). A circular transformation can transform a point on that circle to another point on the same circle (e.g., the same point if the transformation is an identity for that particular point). For example, by parameterizing the point by angle and radius, a circular transformation can transform the angle but not the radius. Circular transformations can also be deterministic or stochastic. Interestingly, as a result of preserving the radius of the circle corresponding to the Cartesian coordinate pair, the radius of the overall direction vector being transformed can also be preserved.

[0017] The circular transformation can be parameterized, for example, by generating parameters of the model and / or additional Cartesian coordinates that do not correspond to the circular slice. Preferably, the circular transformation is an invertible function with an invertible inverse, such as a differential homeomorphism. The corresponding circular transformation can be applied to corresponding, preferably pairwise disjoint Cartesian coordinate pairs, for example, to each pair of Cartesian coordinates to which a conditional rotation has been applied. The circular transformation can be parameterized based on coordinates unaffected by the conditional rotation.

[0018] The inventors have discovered that using the coupling layer that combines rotation and circular transformation allows for the representation of a wide range of transformations on orientation data. Rotation is a particularly natural and important transformation to apply to orientation data, making its conditional and / or unconditional application particularly useful. Interestingly, functionality not corresponding to higher-dimensional rotations can also be achieved by combining this with transformations of circular slices. Nevertheless, through construction, the norm of the transformed orientation vector can be preserved, and accordingly, flexible and powerful transformations on the orientation vector are obtained. In various embodiments, the generative model combines multiple coupling layers in parallel and / or sequentially, allowing for the flexible learning of more complex transformations and deriving more complex probability distributions on the generated orientation data. Therefore, anomalies in measured orientation data can be detected more accurately, and, for example, synthetic orientation data instances that are more representative of previously measured orientation data instances can be generated.

[0019] A key advantage of using rotation and circular transformations is that it allows the coupling layer to be used not only in the generation direction, such as generating synthetic direction data, but also in the inverse direction, sometimes referred to as the inference direction. For example, the inverse of the coupling layer can be computed efficiently and accurately, thus allowing the computation of precise inverses without the need for stochastic approximations or numerically stable inverse techniques.

[0020] In particular, since the corresponding circular transformations can act independently on the corresponding Cartesian coordinate pairs, the inverse of the generative model can be computed by inverting the corresponding circular transformation individually. Circular transformations can be parameterized based on other coordinates and model parameters. Interestingly, since these values ​​are unaffected by the transformation, these parameters can be computed from the output of the circular transformation in the same way they are computed from the input of the circular transformation. Furthermore, independent transformations of circular slices can produce Jacobians with a triangular structure, and thus can be inverted in an accurate, efficient, and numerically stable manner. The inverse Jacobian can be used for log-likelihood calculations. Therefore, the use of generative models in the inference direction is improved.

[0021] Similarly, since conditional rotations act on a subset of Cartesian coordinates parameterized based on other coordinates and / or model parameters, and these values ​​are unaffected by the conditional rotations, the parameters used in conditional rotations can also be determined from their outputs in the same way they are computed from their inputs. Furthermore, since the inverses of both conditional and unconditional translations are also translations, they can be efficiently inverted.

[0022] Therefore, constructing coupling layers using rotations and circular transformations can efficiently and accurately perform both density evaluation (e.g., for determining consistency and training generative models) and sampling (e.g., for generating synthetic orientation data). In particular, it avoids the use of autoregressive flow-based iterative processes, which have increased computational complexity for high-dimensional data and the property that they cannot simultaneously perform density evaluation and sampling from model density efficiently (e.g., autoregressive flows either go from a uniform space to the model data space or vice versa, and depending on the choice, sampling or density evaluation may be inefficient).

[0023] Another advantage of the proposed coupling layer—which helps improve accuracy—is the avoidance of singularities. For example, a trainable mapping on a hypersphere, proposed by D. Rezende et al. in “Normalizing Flows on Tori and Spheres” (available at https: / / arxiv.org / abs / 2002.02428 and incorporated herein by reference), utilizes a cylinder. However, because cylinders and spheres are topologically inequivalent, the mapping between them is not differentially homeomorphic, and in particular, singularities exist, for example, at poles. Interestingly, the coupling layer proposed in this paper allows such singularities to be avoided. For example, a differentially homeomorphic mapping can be obtained, and thus a continuous model density. In particular, the reliability of approaching such singularities is improved. This is particularly important when determining consistency, as this is intended to reliably handle outlier cases.

[0024] Optionally, applying a circular transformation can include mapping Cartesian coordinate pairs to angles; applying a circular homeomorphism to the angles; and mapping the angles back to Cartesian coordinates. Similarly, the inverse of the circular transformation can be applied by mapping Cartesian coordinates to angles, applying the inverse of the circular homeomorphism, and mapping the result back to Cartesian coordinates. This helps ensure that the resulting circular transformation is invertible. This is more complex for ensuring, for example, that the transformation acts directly on a 2D plane. Likewise, the likelihood contribution of the circular transformation can be efficiently calculated based on the mapping between angles and coordinates and the likelihood contribution of the circular homeomorphism.

[0025] Optionally, the circular homeomorphism can be parameterized based on a trained parametric model (e.g., a neural network). As input, the parametric model can use the size of the circle and / or Cartesian coordinates to which no circular transformation has been applied. For example, the Cartesian coordinates of a direction vector can be partitioned into one or more pairs, along with a set of remaining Cartesian coordinates. In such cases, a circular transformation can be applied to the corresponding pairs, where parameters are determined by evaluating the trained parametric model using the size of the circle and / or one, more, or all of the remaining coordinates. Using only the Cartesian coordinates to which no circular transformation has been applied by the coupling layer has the advantage that the corresponding circular transformations can be evaluated in parallel, since the parameter computation of one circular transformation does not require that the transformation of another circular slice has already been performed.

[0026] Alternatively, unconditional rotations and / or conditional rotations can be achieved by using multiple Householder reflections. Householder reflections allow both rotations and their inverses to be implemented efficiently. Therefore, for example, better results can be obtained compared to working with rotation angles. In various embodiments, the number of Householder reflections is chosen to be equal to the dimension of the vector to which they are applied, for example, the overall direction vector or a subset of coordinates to which the conditional rotation is applied. This, in principle, allows for the representation of any rotation. However, it is also possible to use fewer Householder reflections, which can result in a more data-efficient model, for example, reducing overfitting.

[0027] Optionally, the orientation vector of the orientation data instance can represent the orientation of a physical object in three-dimensional space. Such orientation can be represented by a unit quaternion, for example, by four real values ​​whose squares sum to one. In some embodiments, the input and output orientation vectors of the coupling layer can each represent an orientation. In some embodiments, the input and output orientation vectors of the coupling layer can include multiple unit quaternions, for example, representing a cascade of multiple orientations (e.g., two, three, up to or at least five, or up to or at least ten orientations).

[0028] In various aspects, generative models are used to estimate the reliability of control data inferred from model inputs by a machine-learnable control model. This reliability can be estimated based on the consistency between the model inputs and the training dataset of the control model. For this purpose, model inputs can be obtained, for example, extracted from sensor data, such as by determining the orientation and / or orientation of a physical object, such as a component of the system to be controlled or an object in its environment. For example, sensor data can be image or video data capturing physical objects.

[0029] To more generally determine the consistency between the model input and the training dataset, or the consistency between oriented data instances and the dataset, the inverse of the generative model can be applied to the obtained oriented data instances to determine the likelihood (e.g., log-likelihood) of the oriented data instances generated according to the generative model. This typically involves applying the inverse of the coupling layer, for example, obtaining the output orientation vector and using that inverse to determine the corresponding input orientation vector (e.g., deterministically if the inverse is deterministic, or determined by sampling if the inverse is stochastic). Determining the likelihood can involve determining the determinant of the inverse of the Jacobian or other types of likelihood contribution of the coupling layer, which is itself known for normalized or SurVAE streams. Interestingly, through the construction of the coupling layer, accurate likelihood estimation can be achieved.

[0030] Based on the determined likelihood, a consistency value can be output, indicating the consistency between the oriented data instances and the dataset on which the generative model was trained. For example, if the dataset is the training dataset for another machine learning model, the consistency value can be used as a measure of the reliability of oriented data instances as model inputs to that other machine learning model. For example, the log-likelihood itself, or values ​​derived from it, such as binary labels, can be output. Due to the use of coupling layers as described in this paper, the generative model better represents the dataset and thus obtains a more accurate consistency value.

[0031] Optionally, the orientation vectors can represent the orientation of the links of the robot arm; for example, orientation data instances can include multiple such orientation vectors representing the corresponding orientations. The determined consistency values ​​can be used as a reliability measure of the control model used to control the robot arm, for example, by operating the robot in a safe mode or stopping the robot when the consistency value indicates inconsistency. For example, orientation data instances in the dataset can represent the normal state of the robot and / or its environment, indicating inconsistencies in abnormal situations that require appropriate action.

[0032] Optionally, at least if the reliability metric indicates sufficient reliability, a control model or other additional machine learning model can be applied to the directional data instance. The consistency value can be used to determine how to use the model output of another machine learning model—for example, whether to discard it, or assign it higher or lower weights. For instance, if the log-likelihood is below a threshold, the model output can be discarded, and / or an alarm can be issued. If the model output is sufficiently reliable, the computer-controlled system can then be controlled accordingly.

[0033] Robotic systems are examples of such computer-controlled systems; for example, this could be a robotic arm as described above, but various other types of robots are also possible, and orientation data is also involved, such as drones or humanoid robots. For example, a robot can automatically perform one or more tasks, for example, under the control of an external device or an embedded controller. Further examples of computer control systems to which the described technology can be applied are vehicles and their components, home appliances, power tools, manufacturing machines, personal assistants, access control systems, drones, nanorobots, and heating control systems.

[0034] Optionally, the synthetic orientation data instances generated by the generative model, or the orientation data instances to which the inverse of the generative model is applied, can represent proteins or other biological structures. For example, orientation data instances can represent candidate protein conformations. In particular, in this case, orientation vectors can represent the orientation or direction of components of a protein or other biological structure. Using the generative model, candidate protein conformations can be generated. The candidate can then be checked for validity automatically and / or manually, for example, by using an energy function as known per se. It is also possible to use the inverse of the generative model to check candidates generated in other ways to be consistent with a dataset of valid conformations, for example, in lieu of other validity checks or in addition to other validity checks. In either case, if a validity is found, the protein can be synthesized automatically, for example.

[0035] Alternatively, the generative model can be used to train the generative model based on the training dataset. In this case, directional data instances can be obtained by retrieving them from the training dataset. The inverse of the generative model can be used to determine the likelihood of the directional data instances from the training dataset, and the determined likelihood can be maximized. Thus, a generative model representing the training dataset can be obtained.

[0036] Generative models are applied in various ways to generate synthetic orientation data instances based on the generative model. Because the generative model includes coupling layers as described in this paper, the generated instances more accurately represent the dataset on which the generative model was trained.

[0037] Optionally, the generative model can be repeatedly applied to obtain multiple instances of generated orientation data. These multiple instances of orientation data can be used as training and / or testing data in training another machine learning model (e.g., a control model for a computer control system). Synthetic data can be used for data augmentation, for example, to train another machine-learnable model on data from large datasets and / or in situations where training data is difficult to obtain, such as dangerous or rare situations in which a computer control system interacts with its environment. For example, the generated data can be labeled manually or automatically, or the other machine learning model can be trained in an unsupervised manner. By using synthetic data, a better-trained machine-learnable model can be obtained without performing additional real-world physical measurements and / or interactions.

[0038] Optionally, input instances can be obtained to another machine learning model, and this other machine learning model can be applied to the input instances to determine its output. For example, the output can be used as control data in a computer control system. Because the other machine learning model is trained on better training data, the quality of its output is improved; for example, the control of the system is improved.

[0039] Those skilled in the art will appreciate that two or more of the above embodiments, implementations and / or optional aspects of the present invention can be combined in any manner deemed useful.

[0040] Based on this specification, those skilled in the art can make modifications and variations to any system and / or any computer-readable medium, which correspond to the modifications and variations described in the corresponding computer implementation method. Attached Figure Description

[0041] These and other aspects of the invention are clear from the embodiments described below and will be illustrated with reference to the embodiments described below. In the accompanying drawings,

[0042] Figure 1 A system for training and / or using generative models is shown;

[0043] Figure 2 A system for estimating the reliability of control data is shown;

[0044] Figure 3 A system for estimating the reliability of control data for a robotic arm is shown;

[0045] Figure 4 A detailed example of a coupling layer is shown;

[0046] Figure 5a A detailed example of applying a coupling layer to two four-dimensional orientation vectors, which, for example, represent the orientation of a physical object, is shown.

[0047] Figure 5b A detailed example of applying a coupling layer to a direction vector, such as representing the orientation of a physical object, is shown;

[0048] Figure 6 A computer-based method for estimating reliability is shown;

[0049] Figure 7 A computer-implemented method for generating synthetic orientation data instances is shown;

[0050] Figure 8 A computer-readable medium containing data is shown.

[0051] It should be noted that items with the same reference numerals in different figures have the same structural features and the same function, or the same signal. Where the function and / or structure of such items have already been explained, there is no need to repeat the explanation in the detailed description. Detailed Implementation

[0052] Figure 1 A system 100 is shown for training generative model 040 and / or generating synthetic orientation data instances based on generative model 040.

[0053] System 100 may include a data interface 120 for accessing model data 040 representing the generated model. Generally, model data 040 may include parameters for evaluating the model in the generation direction, for example, to generate a data instance in the synthesis direction, and / or parameters for evaluating the model in the inference direction, for example, parameters for generating the inverse of the model, to determine, for example, consistency with the dataset.

[0054] For example, parameters can include parameters of one or more neural networks, such as parameters used for coupling layer transformations that determine the model's parameters. Neural networks are also known as artificial neural networks. Examples include deep neural networks and convolutional neural networks. In this case, the parameter set can include the weights of the neural network nodes. Parameters can also include parameters of the probability distribution used in the coupling layers, etc. Depending on the specific application, a variety of known neural network architectures and other types of machine-learnable models can be used.

[0055] The parameters used for the generation and inference directions typically overlap, but do not need to be identical. For example, the transformation of the generative model may be deterministic in one direction and random in another, in which case the number of parameters used for the random direction is usually larger. For example, the number of parameters used for the generation and / or inference directions may be at least 10,000, at least 100,000, or at least 1,000,000.

[0056] The generative model can be trained by system 100, or it can have been previously trained on a training dataset. In both cases, the orientation data instances from the training dataset can include at least one orientation vector associated with the physical object extracted from sensor data representing the physical object.

[0057] When system 100 applies generative model 040 based on a generative model previously trained on a training dataset, the model data 040 accessed by the system may only include parameters for the generation direction, for example, excluding parameters for the inference direction. When system 100 trains the generative model on a training dataset, model data 040 typically includes parameters for both the generation and inference directions.

[0058] When the generative model is trained by system 100, the data interface can also be used to access the training dataset 030 on which the model is trained. For example, the training dataset may include at least 100, at least 10,000, or at least 1,000,000 orientation data instances. Therefore, data interface 120 can act as an input interface for obtaining orientation data instances from the training dataset 030 for training the generative model. The generative model trained by system 100 can be used according to the methods described herein, for example, by system 100 itself or by... Figure 1 Different instances of the system generate synthetic direction data instances, the Figure 1 Different instances of the system are configured to generate directional data instances based on the model, but are not trained. Alternatively or additionally, the generative model trained by system 100 can be used, for example, according to the methods described herein. Figure 2 The system 200 determines the consistency of directional data instances, for example, as a reliability metric for the control model.

[0059] Also Figure 1 As illustrated, the input interface can be comprised of a data storage interface 120, which can access data 030 and 040 from data storage device 021. For example, data storage interface 120 can be a memory interface or permanent storage interface—such as a hard disk or SSD interface—but can also be a LAN, local area network, or wide area network interface—such as a Bluetooth, Zigbee, or Wi-Fi interface, or an Ethernet or fiber optic interface. Data storage device 021 can be an internal data storage device of system 100—such as a hard disk or SSD—but can also be an external data storage device—such as a network-accessible data storage device. In some embodiments, data 030 and 040 can each be accessed from different data storage devices, for example, via different subsystems of data storage interface 120. Each subsystem can have the type described above for data storage interface 120.

[0060] System 100 may further include a processor subsystem 140. Processor subsystem 140 may be configured to train generative model 040 during operation of system 100 by maximizing a determined likelihood of directional data instances from dataset 030. This likelihood may be determined by processor subsystem 140 by inversely applying the generative model to the directional data instances to determine the likelihood of generating directional data instances according to the generative model. Training may include outputting the likelihood as a consistency value indicating the consistency of the directional data instances with the dataset, and using the likelihood as a training signal for training generative model 040.

[0061] Generative models can generate directional data instances by transforming the input direction vector into the output direction vector based on the coupling layer.

[0062] As part of applying the inverse of the generative model, the processor subsystem 140 can be configured to apply the inverse of the coupling layer. This can include applying the inverse of a circular transformation. This inverse can be applied to Cartesian coordinate pairs representing points on a circle. This inverse can transform said points to other points on the circle. The circular transformation can be defined based on the parameters of the generative model and / or additional Cartesian coordinates. Applying the inverse of the coupling layer can further include applying the inverse of unconditional rotation and / or conditional rotation. The inverse of unconditional rotation can rotate the direction vector according to the parameters of the generative model. The inverse of conditional rotation can rotate the first set of Cartesian coordinates based on a second set of Cartesian coordinates.

[0063] Instead of training a generative model, or in addition to training a generative model, processor subsystem 140 can be configured to apply generative model 040 to generate synthetic orientation data instances during operation of system 100. This application may include applying a coupling layer to the input orientation vector. The application of the coupling layer may include applying unconditional rotations and / or conditional rotations. Unconditional rotations can rotate the orientation vector based on parameters of the generative model. Conditional rotations can rotate a first set of Cartesian coordinates based on a second set of Cartesian coordinates. The application of the coupling layer may further include applying a circular transformation to Cartesian coordinate pairs representing points on a circle. This transformation can transform said points to other points on the circle. The circular transformation may be defined based on parameters of the generative model and / or additional Cartesian coordinates.

[0064] System 100 may further include an output interface. The output interface may be used to output training data representing the learned (or “trained”) model, and / or to output generated synthetic orientation data instances or data based thereon.

[0065] For example, processor subsystem 140 can be configured to repeatedly apply a generative model to obtain multiple generated instances of directional data during operation of system 100, and to use the multiple instances of directional data as training and / or testing data to train an additional machine learning model (e.g., a control model). The additional machine learning model can be output using an output interface. Alternatively or additionally, model inputs to the additional machine learning model can be obtained, and the additional machine learning model can be applied to the model inputs to determine the output of the additional machine learning model (e.g., control data), wherein the output of the additional machine learning model is output by the output interface.

[0066] For example, also Figure 1 As illustrated in the diagram, the output interface can be comprised of a data interface 120, which in these embodiments is an input / output (“IO”) interface through which output data can be stored in a data storage device 021. For example, model data 040 defining an “untrained” model can be at least partially replaced by model data of a trained model during or after training, because model parameters such as the weights of the neural network and other types of parameters can be adapted to reflect training on the training data 030. This also... Figure 1 The same reference numeral 040 is used to refer to both untrained and trained model data in the illustration. In other embodiments, trained model data may be stored separately from the model data defining the "untrained" model. In some embodiments, the output interface may be separate from the data storage interface 120, but generally it may have the type described above for the data storage interface 120. In the case of output control data, the output interface may also be an actuator interface, such as reference numeral 040. Figure 2 The subject of discussion.

[0067] Figure 2 A system 200 for determining the consistency between orientation data instances and a dataset is illustrated. In this example, the orientation data instances are model inputs to a machine-learnable control model used to control a computer control system. Consistency can provide an estimate of the reliability of the machine-learnable control model for the model inputs. System 200 can be a control system that controls a computer control system. In addition to applying a generative model, system 200 can also train a generative model, for example, by... Figure 1 The system is combined.

[0068] System 200 may include a data interface 220 for accessing trained model data 040 representing a trained generative model, wherein the trained generative model can be generated by... Figure 1The system 100 generates, or as described elsewhere. The generative model can be trained to generate directional data instances representing a dataset. This generation may include transforming the input direction vector into an output direction vector according to a coupling layer. See also [reference needed]. Figure 1 The model data 040 discussed may include at least the parameters required to apply the generative model in the inference (inverse) direction, but does not need to include any additional parameters required to apply the generative model in the generative direction.

[0069] For example, also Figure 2 As illustrated in the diagram, the data interface can be composed of a data storage interface 220, which can access data 040 from the data storage device 022. Generally, the data interface 220 and the data storage device 022 can have the same characteristics as the referenced data interface 220. Figure 1 The same type is described for data interface 120 and data storage device 021. Although not shown in the figure, the data interface can also be used to access model data representing a machine-learnable control model.

[0070] The data interface can also be used as an input interface to obtain orientation data instances, which will be used as model input for another machine-learnable control model. The orientation data instance may include at least one orientation vector associated with a physical object, such as a component of a computer control system or an object in a computer control system environment. The orientation vector may have been previously extracted from sensor data representing the physical object. The figure illustrates the case where the input interface is sensor interface 260, where orientation data instance 223 is extracted by sensor interface 260 from sensor data 224, as discussed in more detail below. Orientation data instances can also be obtained via another type of input interface.

[0071] System 200 may further include a processor subsystem 240, which can be configured to, during operation of system 200, inversely apply the generative model 040 to the orientation data instance 223 to determine the likelihood of generating the orientation data instance according to the generative model. Based on the likelihood, processor subsystem 240 can output a consistency value indicating the consistency of the orientation data instance with the dataset, and thus, in this case, estimate the reliability of the control data that will be inferred by the machine-learnable control model using the orientation data as model input.

[0072] This application may include applying the inverse of the coupling layer. Applying the inverse of the coupling layer may include applying the inverse of a circular transformation. This inverse can be applied to Cartesian coordinate pairs representing points on a circle. This inverse can transform said points to other points on the circle. The circular transformation can be defined based on the parameters of the generative model and / or additional Cartesian coordinates. Applying the inverse of the coupling layer may further include applying the inverse of unconditional rotation and / or conditional rotation. The inverse of unconditional rotation can rotate the direction vector according to the parameters of the generative model. The inverse of conditional rotation can rotate the first set of Cartesian coordinates based on a second set of Cartesian coordinates.

[0073] To understand, and to target Figure 1 The same considerations and implementation options apply to processor subsystem 240 as to processor subsystem 140. It will be further understood that, unless otherwise indicated, the same considerations apply to processor subsystem 240. Figure 1 The same considerations and implementation options as System 100 can generally be applied to System 200.

[0074] Figure 2 Various optional components of system 200 are further illustrated. For example, in some embodiments, system 200 may include sensor interface 260 for direct access to sensor data 224 acquired by sensor 072 in environment 082. The direction vector of direction data instance 223 associated with a physical object of the computer control system or its environment 082 can be extracted from sensor data 224. Sensors may be positioned within environment 082, but may also be positioned remotely, for example, if one or more quantities can be measured remotely. Sensor 072 may, but does not need to, be part of system 200. Sensor 072 may have any suitable form, such as an image sensor, lidar sensor, radar sensor, pressure sensor, temperature sensor, etc. Sensor interface 260 may have any suitable form corresponding in type to the sensor type, including but not limited to low-level communication interfaces, such as I2C or SPI-based data communication, or data storage interfaces of the type described above for data interface 220.

[0075] In some embodiments, system 200 may include actuator interface 280 for providing control data 226 to actuators (not shown) in environment 082. Control data 226 may be generated by processor subsystem 240 to control the actuator based on determined consistency values, for example, by using consistency values ​​to determine whether to use control data from a machine-learnable control model, such as control data from a model that is only usable if sufficiently reliable. The actuator may be part of system 200. For example, the actuator may be an electrical, hydraulic, pneumatic, thermal, magnetic, and / or mechanical actuator. Specific but non-limiting examples include electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servo mechanisms, solenoids, stepper motors, etc. This type of control reference... Figure 3 Describe the robot system.

[0076] In other embodiments ( Figure 2 (Not shown in the image), system 200 may include an output interface to a presentation device such as a display, light source, speaker, vibration motor, etc., which can be used to generate a sensorily perceptible output signal. This output signal can be generated based on a determined reliability metric. For example, a control signal inferred from a machine-learnable control model can be output along with the determined reliability metric.

[0077] Generally speaking, each system described in this specification—including but not limited to— Figure 1 System 100 and Figure 2 System 200 may be embodied as a single device or apparatus, such as a workstation or server. The device may be an embedded device. The device or apparatus may include one or more microprocessors executing appropriate software. For example, the processor subsystem of the corresponding system may be embodied by a single central processing unit (CPU), and also by a system or combination of such CPUs and / or other types of processing units. The software may have been downloaded and / or stored in a corresponding memory, such as volatile memory like RAM, or non-volatile memory like flash memory. Alternatively, the processor subsystem of the corresponding system may be implemented in the device or apparatus as programmable logic, such as a field-programmable gate array (FPGA). Generally, each functional unit of the corresponding system may be implemented as a circuit. The corresponding system may also be implemented in a distributed manner, such as involving different devices or apparatuses, such as distributed local or cloud-based servers. In some embodiments, system 200 may be part of a vehicle, robot, or similar physical entity, and / or may represent a control system configured to control the physical entity.

[0078] Figure 3The above example illustrates that system 200, together with robotic arm 300, is part of a robotic system that serves as a control system for controlling robotic arm 300. Although the robotic arm is used as an example, those skilled in the art will understand that the system can also control other types of computer control systems, such as (semi-)autonomous vehicles, building control systems, etc.

[0079] The robotic arm 300 can function in environment 082, such as as part of a manufacturing production line. As an illustrative example, the robotic arm in this figure has two links 301 and 302. Other numbers of links are also possible, such as one, three, or at least four. Controlling the robotic arm 300 may include connections between the links controlling the robotic arm to move the robotic arm in a given direction.

[0080] For example, in normal operation, system 200 can control robot arm 300 according to a machine-learnable control model, such as a strategy trained using reinforcement learning based on a training dataset. The machine-learnable control model can take orientation data instance DD 375 as model input. Orientation data instance DD 375 includes orientation vectors DV1 370 and DV2 371, which indicate the orientation of links 301 and 302 of the robot arm. Orientation data instance DD may include additional data, such as measured forces and torques.

[0081] In this setup, collecting training data for anomalous situations can be challenging. Therefore, in these anomalous situations, the machine learning model may become less reliable. Furthermore, there is an increased risk of the robotic arm performing unintended actions in these anomalous situations, such as damaging the robotic arm or objects in its environment.

[0082] Therefore, system 200 can use a generative model trained on the same training dataset as the machine-learnable control model to determine a consistency value. The consistency value can indicate the consistency of the orientation data instance with the training dataset, and thus the reliability of the machine-learnable control model for the orientation data instance can be estimated. At least if the consistency value indicates sufficient consistency and therefore sufficient reliability of the orientation data instance with the training dataset, the control model can be applied to the orientation data instance, and its output can be used to control the robotic arm 300. If the consistency value does not indicate sufficient consistency, the control of the robotic arm can be adapted to reduce the risk of performing undesirable actions, for example, by stopping the robotic arm, switching to a safe mode, assigning less importance to the output of the control model, etc.

[0083] Various detailed examples of coupling layers are described below. Generally, a generative model may include one or more such coupling layers.

[0084] Oriented data instances can be generated based on latent feature vectors. Applying a generative model may include obtaining latent feature vectors and determining synthetic orientation data instances from them (e.g., deterministically or randomly). The generative model may define a prior distribution from which the distribution of latent feature vectors is assumed, and thus, random synthetic orientation data instances can be generated by sampling the latent feature vectors according to the prior distribution. Interestingly, it may also be possible to adapt the latent feature vectors to influence the generated orientation data instances, for example, controlling the presence of certain aspects in the generated orientation data instances.

[0085] For example, generative models can generate directional data instances using the following methods:

[0086] - Obtain latent feature vectors, for example, by sampling latent feature vectors based on prior probability distributions;

[0087] - Apply an initial part (e.g., trainable) of the generative model to the latent feature vector to obtain an intermediate representation of the input, including the input direction vector to the coupling layer;

[0088] - Apply a coupling layer to the input direction vector to obtain the output direction vector as part of the output intermediate representation, for example, keeping the portion of the input intermediate representation that is not input to the coupling layer unchanged;

[0089] - Apply the final (e.g., trainable) part of the generative model to the output intermediate representation to determine the orientation of the data instance.

[0090] The initial and final parts of the generative model are optional: the coupling layer can also be directly applied to the latent feature vectors and / or the output of the coupling layer can be directly included in the generated orientation data instance. For example, the orientation vector included in the orientation data instance can be the output orientation vector of the coupling layer, or the output orientation vector can provide multiple orientation vectors included in the orientation data instance.

[0091] The coupling layer can use the portion of the intermediate representation that is not input to the coupling layer as input to the parametric model to determine the parameters of the circular transformation or conditional rotation of the coupling layer.

[0092] The intermediate representations of input and output can also represent data that does not represent direction vectors, for example, data not from a hypersphere or a manifold topologically equivalent to a hypersphere. For instance, the intermediate representation could include data from a Euclidean manifold, data from a symmetric positive definite matrix manifold, and / or data from a special Euclidean group, which could, for example, be used to represent the state of a robotic system. This additional data can be transformed by other uncoupled layers of the generative model, which is known in itself.

[0093] Generative models can include multiple coupled layers. For example, the output direction vector of one coupled layer can be used as the input of another coupled layer, for example, in the order of at least three, at least five, or at least ten coupled layers. Multiple coupled layers can be executed in parallel in the sense that they are applied to different parts of the same intermediate representation.

[0094] For example, in one embodiment, one or more subsequent coupling layers can be applied to a cascade of multiple individual orientation vectors, such as multiple 3D or 4D vectors, for example, at least two, at least four, or at least ten such individual orientation vectors. Then, one or more subsequent coupling layers can be applied individually to each individual orientation vector. In this way, the generative model can first apply more global transformations to the overall orientation data instance, and then generate aspects of individual orientation vectors separately.

[0095] The training of the generative model described in this paper can be performed using stochastic methods such as stochastic gradient descent, for example using the Adam optimizer disclosed in Kingma and Ba’s “Adam: A Method for Stochastic Optimization” (available at https: / / arxiv.org / abs / 1412.6980, and incorporated herein by reference). The objective function to be optimized can include the sum of the log-likelihoods of items in the training dataset generated by the generative model. As is known, such optimization methods can be heuristic and / or reach local optima. Training can be performed on an instance-by-instance basis or, for example, in batches of up to or at least 64 or up to or at least 256 instances. From the perspective of training efficiency, it is advantageous to use a generative model that can be subject to gradient-based optimization, such that the generative model is, for example, continuous and / or differentiable in its parameter set.

[0096] For example, in an embodiment, the generative model can be a normalized flow. The normalized flow can be trained to approximate the inherent probability distribution of real-world data represented by the training dataset. As an example, the MNIST dataset, as known in the art, consists of 28×28 pixel images of handwritten digits, and therefore can correspond to... The inherent probability distribution defined above Preprocessing steps, such as dequantization, can be taken to process discrete data and achieve continuous density. The distribution can be learned by changing the formula based on the variables. Let... X It is the domain of the data space generated by the generative model; Z It is the latent space of the latent feature vectors, for example, having the same dimension; p Z It is a probability distribution on Z, and It is a mapping, such as a differentiable mapping and a differentiable inverse mapping. As is known by itself, in a normalized flow, the variable changes the formula.

[0097]

[0098] It can have of The analytical form of the distribution is provided; for example, the formula for changing variables can represent the distribution. p Z exist f For potential space Z How does it change under the influence of [the system / mechanism]?

[0099] Therefore, in normalized flows and similar models, computing the likelihood of data generated from the model can include evaluating the Jacobian determinant of the inverse model, for example, by evaluating the inverse of the Jacobian determinant of the model. This can further include evaluating the Jacobian determinant of the inverse coupling layer, for example, by evaluating the inverse of the Jacobian determinant of the coupling layer. Interestingly, as discussed elsewhere, the components of the coupling layer provided herein have a triangular structure, thereby allowing these determinants to be computed in an efficient and numerically stable manner. For example, in some embodiments, the computational complexity of computing the determinant is reduced from... Reduce to This is because the determinant can be calculated as the sum of its diagonal elements.

[0100] distributed p Z It can be a probability distribution with fixed parameters or parameterized by the parameters of the generative model, for example, if Z If it is Euclidean space, then the distribution p Z Is it an unrelated Gaussian, or if Z If it is dense, then the distribution is... p Z It is a uniform distribution. (Function) f It can be parameterized by various machine-learnable functions, such as one or more deep or other neural networks. Therefore, using the normalized flow, it can be based on a fundamental distribution. p Z For complex distributions p X Modeling.

[0101] In this embodiment, the generative model is a SurVAE stream. A SurVAE stream is similar to a normalized stream but can include non-bijective layers, such as generative surjective layers, inferred surjective layers, or stochastic layers. The coupling layer can be a bijective layer of the SurVAE stream, in which case it is applied similarly to that in a normalized stream. Specifically, also in a SurVAE stream, calculating the likelihood of the generated data can involve calculating the Jacobian determinant of the coupling layer, as discussed above.

[0102] Generally, direction vectors included in direction data instances can be represented in various ways. For example, a direction vector can be represented by a set of Cartesian coordinates. For instance, a direction vector representing a direction in 3-dimensional space can be represented as having… 3D vector For example, represented as a sphere Points on a circle. Similarly, directions in 2D space can be represented as circles. Points on the hypersphere. Generally, the direction vector corresponding to a direction in k dimensions can be represented as (k-1)-hypersphere. The point on top.

[0103] The techniques described in this paper can be applied to higher-order directional data, such as directional vectors representing directions in at least 5, 10, or 20 dimensions. It is not necessary to restrict Cartesian coordinates to having a certain norm; for example, directional vectors can represent quantities in other ways, or vectors with different norms but the same directional components can be considered equivalent.

[0104] There is no need to use Cartesian coordinates; for example, a direction vector can be represented by one or more angles. In particular, orientation can be represented in various ways. For example, as is known per se, orientation in three-dimensional space can be represented as a 3-sphere. Points on the y-axis are represented by a special orthogonal group. Elements, represented by axes and angles, etc.

[0105] Figure 4 A detailed, but non-limiting, example of a coupling layer for use in a generative model as described herein is shown. The coupling layer transforms the input direction vector DV 410 into the output direction vector DV”475.

[0106] As shown in this example, the input direction vector DV can represent a point on the hypersphere, for example, D -1-dimensional hypersphere A point on the y-axis can be represented as a point with a specific norm (e.g., norm 1). D A dimensional vector. The output direction vector "DV" can be points on the same hypersphere, such as those with the same norm. D Dimensional vector. Throughout the description of this graph, it is assumed that the direction vector is represented by a set of Cartesian coordinates. Although the actual representation used to store the direction vector may differ, for example, at the beginning, end, and / or throughout the coupling layer, the direction vector may be represented as a set of angles, or as a combination of a vector and one or more angles, etc.

[0107] Typically, the input direction vector DV and the output direction vector DV" have the same dimension. For example, the dimension of the input and / or output direction vectors can be: 3, for representing direction in 3D space; 4, for representing orientation in 3D space, corresponding to a unit quaternion, etc. For example, the input and / or output direction vectors can also be a concatenation of one or more 2D direction vectors and / or one or more 3D direction vectors and / or one or more 4D direction vectors. The overall dimension of the input and / or output direction vectors can be, for example, at least 10 or at least 20.

[0108] As shown in the figure, the coupling layer can involve conditional and / or unconditional rotations, as well as a circular transformation applied to the input direction vector. In this example, the unconditional rotation URot 420 and the conditional rotation CRot 440 are then applied to the input direction vector DV, followed by the circular transformation CTrans 460 to achieve the output direction vector DV. However, it is also possible to skip one of the two rotations, or to perform the unconditional rotation URot, the conditional rotation CRot, and the circular transformation CTrans in any order other than that shown in the figure.

[0109] Unconditional rotation (URot) can rotate the direction vector DV based on the parameters of the generative model to update the direction vector DV'430. Unconditional rotation typically preserves the dimension of the direction vector DV. Unconditional rotation can be achieved using quantitative methods. D This is achieved through Householder reflection. Householder reflection of coordinates is an efficient, realizable, trainable, and invertible representation of the rotation operator. The unconditional rotation URot can be viewed as an arbitrary permutation of the data dimensions.

[0110] For example, in mathematics, the Householder reflection can be represented as , where parameter set It is learned, but fixed. An unconditional rotation URot can be implemented as... The transformation is defined as:

[0111] .

[0112] For example, the number of Householder reflections D It can be equal to the dimension of the direction vector DV. This has the advantage that any rotation of the direction vector DV can be represented in this way. However, it is also possible to use fewer Householder reflections to reduce the number of parameters in the model.

[0113] Rotations based on Householder reflections allow for the evaluation of the inverse of unconditional rotations in an efficient and numerically stable manner. Because R It is an orthogonal matrix, so its inverse is...R It can be achieved by transposing (e.g.) This is obtained by [method name], and the transformation does not cause a change in volume because [reason]. .

[0114] A conditional rotation CRot can be based on a second set of coordinates SSC 436 of the direction vector DV', such as the remaining coordinates or a subset thereof, and a subset of coordinates FSC 435 of the rotation direction vector DV'. Therefore, for the first subset of coordinates, the conditional rotation CRot can result in a direction vector with updated coordinates FSC'450, while the remaining coordinates, such as the second set of coordinates SSC, remain unchanged.

[0115] The parameters of a conditionally rotated CRot may include model parameters of the generating model and / or parameters determined by applying a trained parametric model (e.g., a neural network or similar) to a second set of coordinates, SSC.

[0116] Similar to the unconditional rotation URot, the conditional rotation CRot can also be achieved using multiple Householder reflections. For example, Householder reflections can have their own trained parameter models, or they can share a common parameter model. Preferably, the number of Householder reflections is the same as the number of parameters in the first subset FSC, but it can be fewer to save parameters. (Identifying the parameter model) A conditional rotation CRot can be mathematically defined as:

[0117] .

[0118] The figure further illustrates the circular transformation CTrans applied to the corresponding Cartesian coordinate pairs (also known as "circular slices") of the direction vector. For example, the circular transformation CTrans can be applied to one, at least two, at least three, or at least five corresponding circular slices.

[0119] The figure illustrates coordinate pairs PC1 455 to PCd 456, which are transformed into corresponding coordinate pairs PC1' 470 to PCd' 471. As shown in the figure, the coordinate pairs are preferably coordinates of a subset FSC' that has been subjected to the conditional rotation CRot transformation and divided into pairs. Thus, in this case, the first set of coordinates, FSC, is chosen to contain an even number of coordinates. However, this is not mandatory; it is also possible to take a subset of the first set of coordinates, FSC', and / or coordinates that have not been subjected to the conditional rotation CRot transformation (if used). There is also often a set of coordinates that has not been subjected to the circular transformation and is therefore invariant; in this case, this is the second set of coordinates, SSC, which has also not been subjected to the conditional rotation CRot transformation. The transformed pairs of coordinates PC1' and the unchanged coordinates SSC together can form the output direction vector DV.

[0120] For example, mathematically, given a subset FSC' of direction vector coordinates, it is possible to observe that subset. Located on a circle with the following radius

[0121] .

[0122] After dividing the coordinates into the first subset FSC' and the second subset SSC After that, among them d If the number is even, the first subset FSC' can be further partitioned into two independent pairs of coordinates PCi, for example, by applying the following function:

[0123]

[0124] in The marking has a radius r A circle.

[0125] The circular transformation CTrans acts on the coordinate pair PCi representing a point by transforming a point on a circle with a certain radius to another point on the same circle (e.g., with the same radius). The transformation applied to the coordinate pair PCi can be defined based on the parameters of the generative model and / or by applying a trainable parameter determination model to another Cartesian coordinate. The parameters and / or parameter determination model are typically different for the corresponding coordinate pairs, although the parameters can be shared between them. As shown in the figure, the other coordinates used can be a subset of the parameters SSC that have not been transformed by the circular transformation CTrans. This has the advantage that the order in which different coordinate pairs PCi are transformed is irrelevant, thus allowing for parallelization, and that when the circular transformation is inverted, the parameters can be based on these same unaffected parameter SSCs.

[0126] The transformation applied to the coordinate system PCi can generally be any differential homeomorphism on the circle, which is known in itself. Examples of such differential homeomorphisms are known from Rezende et al., “Normalizing Flows on Tori and Spheres”, in particular from Section 2.1 (incorporated hereby by reference).

[0127] Specifically, a transformation for the coordinate pair PCi can be applied by mapping the coordinate pair PCi to an angle, applying a circular homeomorphism to that angle, and mapping the resulting angle back to obtain the transformed coordinate pair PCi'. For example, a circular neural spline flow or Möbius transformation can be used, as known, for example, from Rezende et al. Performing the transformation based on the angle helps ensure that the function is monotonically increasing and therefore a homeomorphism. Furthermore, a homeomorphism applied to a circle can be defined as another homeomorphism. Combinations, for example, through function composition. Or by taking, for example, parameters that can be trained. convex combination By taking a function defined on the angle, we can ensure that convex combinations are still diffeomorphic.

[0128] For example, in mathematics, the transformation of coordinate pairs can be achieved as follows:

[0129]

[0130] in

[0131]

[0132] It is embedded in In A bijection between the Cartesian coordinates and their parameterized coordinates, for example

[0133] and .

[0134] For example, It could be a neural spline flow or a Möbius transformation, or another circular differential homeomorphism.

[0135] It can be noted, for example, that for calculating the log-likelihood, the volume change caused by switching between parameterized angles and Cartesian coordinates depends only on the radius of the circular slice, where Therefore, the mappings from angle to angle can cancel each other out, since the radius does not change under the flow transformation. The volume change due to the flow is known and can be efficiently computed. Interestingly, because the circular slices can be transformed independently, the Jacobian of the overall transformation can be triangular and therefore can be computed in an efficient and numerically stable manner.

[0136] Interestingly, using the coupling layer proposed in this paper, it is possible to efficiently compute the output direction vector DV” from the input direction vector DV in the generation direction, and also to efficiently compute the input direction vector DV from the output direction vector DV” in the inference direction. In particular, the computation of the inverse can be combined with the computation of the likelihood (e.g., log-likelihood) of the direction data generated according to the generative model. For example, the computation of this likelihood can involve computing the Jacobian determinant of the coupling layer, as described in this paper.

[0137] Specifically, in the example shown in the figure, the output direction vector DV” can be partitioned into Cartesian coordinate pairs PCi’ and a second set of coordinates SSC. The corresponding Cartesian coordinate pairs PCi’ can be transformed by applying the inverse of the circular transformation CTrans to obtain the Cartesian coordinate pairs PCi, while the second set of coordinates SSC remains unchanged. Interestingly, the parameters of the circular transformation CTrans can be based on the model parameters and / or the second set of parameters SSC, for example, they can be computed from the input to the circular transformation (in the generation direction) or from its output (in the inference direction) with the same result. Furthermore, the order in which the corresponding pairs PCi’ are transformed by the inverse may be irrelevant, and therefore the inverse can be parallelized.

[0138] Given a coordinate pair PCi and a second set of coordinates SSC, the inverse of the conditional rotation CRot can be applied. The parameters of the conditional rotation can be determined based on the model parameters and / or the second set of coordinates unaffected by the conditional rotation CRot, thus allowing the parameters to be determined again in the generation and inference directions in the same manner. Based on the parameters, the inverse rotation can be applied to the combined coordinate pair FSC' to obtain the first set of coordinates FSC. Then, the inverse of the unconditional rotation URot can be applied to the combination of the first set of coordinates FSC and the second set of coordinates SSC to obtain the input direction vector DV.

[0139] As those skilled in the art will understand, this inverse can be applied to variations of the coupling layer shown in the figure, wherein, for example, the order of transformations URot, CRot, and CTrans is different; one of rotations URot and CRot is skipped; or the unaffected set of coordinates differs between conditional rotation CRot and circular transformation CTrans.

[0140] Below is a detailed pseudocode example of the algorithm, which can be used to evaluate the coupling layer shown in the figure in the generation direction.

[0141]

[0142] Figure 5aA detailed example of applying a coupling layer to two four-dimensional orientation vectors, which, for example, represent the orientation of a physical object, is shown.

[0143] In this example, the input direction vector of the applied coupling layer is an 8-dimensional input direction vector, which is a concatenation of two 4-dimensional direction vectors, DV1 510 and DV2 511. In this example, the first direction vector DV1 is transformed, while the second direction vector DV2 remains unchanged, but can be used to parameterize the transformation of the first direction vector DV1.

[0144] In this coupling layer, the conditional rotation CRot 540 can rotate the Cartesian coordinates of the first direction vector DV1, thereby resulting in an updated first direction vector DV1'. The conditional rotation CRot can be as referenced. Figure 4 For example, conditional rotations can be parameterized based on the parameters of the generative model and / or based on the second direction vector DV2.

[0145] The resulting updated direction vector DV1' can then be divided into two circular slices, PC1 555 and PC2 556, each with Cartesian coordinates. The circular transform CTrans 560 can be applied to the corresponding circular slices to obtain the corresponding output points PC1'570 and PC2'571, which have the same radius as the input circular slice PCi. The circular transform CTrans can be referenced as follows. Figure 4 The parameters can be specifically parameterized by the parameters of the generated model and / or by the second direction vector DV2 without the application of conditional rotation or circular transformation.

[0146] As shown in the figure, output points PC1' and PC2' can together form a four-dimensional direction vector DV1", for example, a direction vector located on the same hypersphere as the original first direction vector DV1. The updated first direction vector DV1" and the original second direction vector DV2 can together form an eight-dimensional output direction vector of the coupling layer, for example, located on the same hypersphere as the input direction vector formed by DV1 and DV2.

[0147] Interestingly, in this example, not only can the input direction vector be preserved in a norm-preserving manner, but the first and second direction vectors DV1 and DV2 can also preserve their norms. This is advantageous if they represent different directional quantities, although in such cases it is not absolutely necessary for the coupling layer to guarantee the preservation of the norms of individual direction vectors; for example, even without such guarantees, consistency checks can yield meaningful results.

[0148] As those skilled in the art will understand, this example can be generalized to cases where, for example, multiple 4-dimensional direction vectors are used to parameterize the transformation of the first direction vector DV1 and / or the first and second direction vectors DVi have other dimensions.

[0149] Figure 5b A detailed example of applying a coupling layer is shown. In this example, the input direction vector of the coupling layer is a four-dimensional vector DV, for example, representing the orientation of a physical object. The coupling layer can be norm-preserving; for example, an input direction vector on a 4-dimensional hypersphere can be mapped to an output direction vector on the same 4-dimensional hypersphere.

[0150] The unconditional rotation URot 520 can be applied to the input direction vector DV to obtain the updated direction vector DV'531. For example, the unconditional rotation URot can be referenced as follows. Figure 4 As stated above.

[0151] The updated direction vector DV' can be partitioned into a first set of coordinates FSC 535—which represents the coordinate pairs of the circular slice—and a second set of coordinates SSC 536—for example, the remaining two coordinates. The first set of coordinates FSC can then be transformed by a conditional rotation CRot transformation 541, resulting in an updated first circular slice FSC' 550. The conditional rotation can be as follows: Figure 4 In the same way, and can be parameterized by the second set of coordinates, SSC.

[0152] The updated first circular slice FSC' can then be transformed using the circular transform CTrans 561 to obtain the circular slice FSC'. The circular transform CTrans can be as follows: Figure 4 As in the example, and can be parameterized by the second set of coordinates, SSC. The circular slice determined by the circular transformation CTrans and the second set of coordinates, SSC, can together form the output direction vector DV of the coupling layer.

[0153] As will be understood by those skilled in the art, many variations of this example are possible, for example, unconditional rotation URot or conditional rotation CRot may be skipped, and operations URot, CRot, CTrans, etc. may be performed in different orders.

[0154] Figure 6 A block diagram of a computer-implemented method 600 for determining the consistency of orientation data instances with a dataset is shown. For example, the orientation data instance can be a model input for a machine-learnable control model, based on which the control model can infer control data for controlling a computer-controlled system interacting with its environment. In this case, consistency with the training dataset of the machine-learnable control model can be used as an estimate of the reliability of the control data. Method 600 can correspond to... Figure 1 System 100 or Figure 2 The operation of system 200. However, this is not a limitation, because method 600 can also be performed using another system, apparatus or device.

[0155] Method 600 may include accessing model data representing the generative model in an operation entitled “Accessing the Generative Model”. The generative model may be trained, for example, prior to or as part of method 600, to generate synthetic orientation data representing the dataset.

[0156] Method 600 may include obtaining 620 instances of direction data in an operation entitled "Obtaining Direction Data". The direction data may include at least one direction vector. The direction vector may be associated with a physical object, such as a component of a computer control system or an object in its environment. The direction vector may be extracted, for example, prior to or as part of method 600, from sensor data representing the physical object.

[0157] Method 600 may include, in an operation entitled "Applying an Inverse Generative Model," applying 630 the inverse of the generative model to the orientation data instance to determine the likelihood of the orientation data instance generated according to the generative model. Method 600 may further include, in an operation entitled "Determining a Consistency Value," determining 640 a consistency value based on the determined likelihood. The consistency value may indicate the consistency between the orientation data and the dataset; for example, the consistency value may be a measure of the reliability of control data inferred from a machine-learnable model. The generative model may include a coupling layer that transforms the input orientation vector into an output orientation vector.

[0158] The coupling layer may include a circular transformation and one or more of unconditional and conditional rotations. Operation 630 may include applying the inverse of the coupling layer.

[0159] To apply the inverse of the coupling layer, method 600 may include applying the inverse of the 636 circular transformation in an operation entitled "Applying the Inverse Circular Transformation". The inverse can be applied to Cartesian coordinate pairs representing points on a circle. The inverse can transform the points to other points on the circle. The circular transformation can be defined based on parameters of the generating model and / or additional Cartesian coordinates.

[0160] To apply the inverse of the coupling layer, method 600 may further include the application of the inverse of the unconditional rotation of 632 in the operation entitled "Applying the Inverse Unconditional Rotation". Instead of operation 632 or other than operation 632, method 600 may include the application of the inverse of the conditional rotation of 634 in the operation entitled "Applying the Inverse Conditional Rotation". The inverse of the unconditional rotation can rotate the direction vector according to the parameters of the generative model. The inverse of the conditional rotation can rotate the first set of Cartesian coordinates based on the second set of Cartesian coordinates.

[0161] Figure 7 A block diagram of a computer-implemented method 700 for generating synthetic orientation data instances based on a generative model is shown. The synthetic orientation data instances may include at least one orientation vector associated with a physical object. Method 700 may correspond to... Figure 1 The operation of system 100. However, this is not a limitation, because method 700 can also be performed using another system, apparatus or device.

[0162] Method 700 may include accessing model data representing the generative model in an operation entitled “Accessing the Generative Model”. The generative model may be trained, for example, prior to or as part of method 700, to generate directional data instances representing the training dataset.

[0163] Method 700 may include applying a generative model 730 in an operation entitled "Applying a Generative Model". Operation 730 may include applying a coupling layer to the input orientation vector. Method 700 may further include outputting an instance of the synthesized orientation data generated 740 in an operation entitled "Outputting the Generated Data".

[0164] The generative model may include a coupling layer that transforms the input direction vector into an output direction vector. The coupling layer may include a circular transformation and one or more of unconditional and conditional rotations.

[0165] To apply the coupling layer, method 700 may include application of unconditional rotation 732 in the operation titled "Applying Unconditional Rotation". Instead of operation 732, or in addition to operation 732, method 700 may include application of conditional rotation 734 in the operation titled "Applying Conditional Rotation". Unconditional rotation can rotate the direction vector according to the parameters of the generative model. Conditional rotation can rotate the first set of Cartesian coordinates based on the second set of Cartesian coordinates.

[0166] To apply the coupling layer, method 700 may further include, in the operation entitled "Applying a Circular Transformation," applying a circular transformation 736 to Cartesian coordinate pairs representing points on the circle. This transformation may transform the points to other points on the circle. The circular transformation may be defined based on parameters of the generating model and / or additional Cartesian coordinates.

[0167] To understand, generally speaking Figure 6 Method 600 and Figure 7 The operations of method 700 can be combined and performed in any suitable order (e.g., sequential, simultaneous, or a combination thereof), which, where applicable, conforms to a specific order, for example, necessary by the input / output relationship. Some or all of the methods can also be combined; for example, method 700 using a generative model can be subsequently applied to the trained model trained according to method 600, or method 600 can be used first to train a generative model on a training dataset and then used to determine the consistency of obtained oriented data instances that are not from the training dataset.

[0168] One or more methods can be implemented on a computer as computer-implemented methods, dedicated hardware, or a combination of both. For example... Figure 8 As illustrated, instructions for a computer (e.g., executable code) may be stored on a computer-readable medium 800, for example, in the form of a series of machine-readable physical symbols 810 and / or as a series of elements with different electrical (e.g., magnetic) or optical properties or values. The executable code may be stored in a transient or non-transient manner. Examples of computer-readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Figure 8 An optical disc 800 is shown. Alternatively, a computer-readable medium 800 may include temporary or non-temporary model data 810 representing a generative model as described herein, such as parameters including one or more coupling layers.

[0169] Examples, embodiments, or optional features—whether or not indicated as non-limiting—should not be construed as limiting the claimed invention.

[0170] It should be noted that the embodiments mentioned above are illustrative and not limiting of the invention, and those skilled in the art will be able to devise many alternative embodiments.

[0171] In the claims, any reference marks placed between parentheses should not be construed as limiting the claims. The use of the verb "comprising" and its variations does not exclude the presence of elements or steps other than those recited in the claims. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Expressions such as "at least one of" when preceding a list or group of elements indicate the selection of all elements or any subset of elements from that list or group. For example, the expression "at least one of A, B, and C" should be understood to include only A, only B, only C, both A and B, both A and C, both B and C, or all A, B, and C. The invention can be implemented by means of hardware comprising several different elements, and by means of a suitably programmed computer. In a device claim enumerating several components, several of these components can be embodied by the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously.

Claims

1. A computer-implemented method (600) of estimating a reliability of control data of a computer-controlled system interacting with an environment, wherein the control data is inferred from model inputs by a machine-learnable control model trained on a training dataset, the method comprising: - accessing (610) model data representing a generative model, wherein the generative model is trained to generate synthetic model inputs representing the training dataset of the control model, - obtaining (620) a model input, wherein the model input comprises at least one direction vector extracted from sensor data and associated with a component of the computer-controlled system or an object in the environment; - applying (630) an inverse of the generative model to the model input to determine a likelihood of the model input being generated according to the generative model, and determining (640) a reliability measure of the control data of the machine-learnable model based on the likelihood, - wherein the generative model comprises a coupling layer transforming an input direction vector into an output direction vector, the coupling layer comprising a circular transformation (636) and one or more of an unconditional rotation (632) and a conditional rotation (634), wherein: - the circular transformation transforms a pair of Cartesian coordinates representing a point on a circle to a further point on the circle, wherein the circular transformation is defined based on parameters of the generative model and / or a further pair of Cartesian coordinates; - the unconditional rotation rotates the direction vector according to parameters of the generative model; - the conditional rotation rotates a first set of Cartesian coordinates based on a second set of Cartesian coordinates.

2. The method (600) of claim 1, wherein the circular transformation comprises mapping the pair of Cartesian coordinates to an angle; applying a circular differential homeomorphism to the angle; and mapping the angle to the Cartesian coordinates.

3. The method (600) of claim 2, comprising determining parameters of the circular differential homeomorphism by applying a trained parametric model to a size of the circle and the Cartesian coordinates without applying the circular transformation.

4. The method (600) of any preceding claim, wherein the unconditional rotation and / or the conditional rotation each comprises a plurality of Householder reflections.

5. The method (600) of any preceding claim, wherein the direction vector corresponds to a unit quaternion representing an orientation of the component or object, and wherein the output direction vector corresponds to one or more unit quaternions.

6. The method (600) of any preceding claim, further comprising, at least if the reliability measure indicates sufficient reliability, applying the control model to the model input and using control data inferred by the control model to control the computer-controlled system.

7. The method (600) of claim 6, wherein the direction vector represents an orientation of a link of a robotic arm, the control data being for controlling the robotic arm.

8. The method (600) of any of claims 1 to 5, comprising training the generative model by obtaining model inputs from the training dataset and maximizing the determined likelihood.

9. A computer-implemented method (700) of training a machine-learnable control model, wherein the control model is for controlling a computer-controlled system that interacts with an environment, the method comprising repeatedly applying a generative model to obtain a plurality of synthetic model inputs, and training the control model to infer control data to control the computer-controlled system, wherein the training uses the plurality of synthetic model inputs as training and / or test data, the method further comprising: - accessing (710) model data representing a generative model, wherein the generative model is trained to generate synthetic directional data instances representing a training data set of the control model, wherein the training data set comprises training model inputs of the control model, wherein directional vectors of the training model inputs have been extracted from sensor data acquired by a sensor and are associated with components of the computer-controlled system or objects in the environment; - wherein the generative model comprises a coupling layer that transforms an input directional vector into an output directional vector, the coupling layer comprising a circular transformation (736) and one or more of an unconditional rotation (732) and a conditional rotation (734), wherein: - the circular transformation transforms a Cartesian coordinate pair representing a point on a circle to a further point on the circle, wherein the circular transformation is defined based on parameters of the generative model and / or further Cartesian coordinates; - the unconditional rotation rotates a directional vector according to parameters of the generative model; - the conditional rotation rotates a first set of Cartesian coordinates based on a second set of Cartesian coordinates.

10. The method (700) of claim 9, further comprising obtaining a model input of the control model and applying the control model to the model input to infer control data of the computer-controlled system.

11. The method (700) of claim 9, wherein the generative model is configured to generate synthetic directional data instances representing candidate protein conformations, the directional vectors representing orientations or directions of backbone components of the protein.

12. A system (100, 200) for estimating a reliability of control data of a computer-controlled system that interacts with an environment, wherein the control data is inferred from model inputs by a machine-learnable control model that is trained on a training data set, the system comprising: - a data interface (120, 220) for accessing model data representing a generative model, wherein the generative model is trained to generate synthetic model inputs representing a training data set of the control model, - an input interface (120, 260) for obtaining a model input, wherein the model input comprises at least one directional vector that is extracted from sensor data and is associated with components of the computer-controlled system or objects in the environment; - a processor subsystem (140) configured to apply an inverse of the generative model to the model input to determine a likelihood of the model input being generated according to the generative model and to determine a reliability measure of the control data of the machine-learnable model based on the likelihood, wherein the generative model comprises a coupling layer that transforms an input directional vector into an output directional vector, the coupling layer comprising a circular transformation and one or more of an unconditional rotation and a conditional rotation, wherein: - the circle transformation transforms a pair of Cartesian coordinates representing a point on a circle to a further point on the circle, wherein the circle transformation is defined based on parameters of the generative model and / or a further pair of Cartesian coordinates; - the unconditional rotation rotates a direction vector in accordance with parameters of the generative model; - the conditional rotation rotates a first set of Cartesian coordinates based on a second set of Cartesian coordinates.

13. A system (100) for training a machine-learnable control model, wherein the control model is for controlling a computer-controlled system that interacts with an environment, the system comprising: - a data interface (120) for accessing model data representing a generative model, wherein the generative model is trained to generate synthetic direction data instances representing a training data set, wherein the training data set comprises training model inputs of the control model, wherein direction vectors of the training model inputs have been extracted from sensor data acquired by a sensor and are associated with a component of the computer-controlled system or an object in the environment; - a processor subsystem (140) configured to repeatedly apply the generative model to obtain a plurality of synthetic model inputs and to train the control model using the plurality of synthetic model inputs as training and / or test data to infer control data for controlling the computer-controlled system, wherein the generative model comprises a coupling layer that transforms an input direction vector into an output direction vector, the coupling layer comprising a circle transformation and one or more of an unconditional rotation and a conditional rotation, wherein: - the circle transformation transforms a pair of Cartesian coordinates representing a point on a circle to a further point on the circle, wherein the circle transformation is defined based on parameters of the generative model and / or a further pair of Cartesian coordinates; - the unconditional rotation rotates a direction vector in accordance with parameters of the generative model; - the conditional rotation rotates a first set of Cartesian coordinates based on a second set of Cartesian coordinates.

14. A computer-readable medium (800) comprising transitory or non-transitory data (810) representing one or more of: - instructions that, when executed by a processor system, cause the processor system to perform a computer-implemented method according to any one of claims 1 to 8 and / or according to any one of claims 9 to 11; - model data representing a generative model for use in a computer-implemented method according to any one of claims 1 to 8.

Images