system dynamics global stability modeling

Abstract

System dynamics globally stable modeling of dynamics models. A system and computer-implemented method for training a dynamics model to learn dynamics of a physical system is provided. In particular, a dynamics model can be learned to be able to infer future states of the physical system and / or its environment based on current states of the physical system and / or its environment. The learned dynamics model is inherently globally stable. That is, instead of learning the dynamics model and trying to verify its stability separately, the learnable dynamics model includes a learnable Lyapunov function that is jointly learned with the nominal dynamics of the physical system. Thus, the learned dynamics model is highly suitable for real-life applications in which the physical system can assume states unseen during training because the learned dynamics model is inherently globally stable.

Description

Technical Field

[0001] This invention relates to machine learning systems and computer-implemented methods for training dynamic models to learn the dynamics of physical systems. The invention further relates to trained dynamic models. The invention further relates to control or monitoring systems and computer-implemented methods for using machine learning modeling of the dynamics of physical systems to control or monitor physical systems. The invention further relates to a computer-readable medium comprising instructions for performing one of the computer-implemented methods. Background Technology

[0002] In many real-world applications, learning dynamic models of physical systems is desirable. This is especially desirable when the physical system can only be observed as a highly unstructured environment—for example, as a frame from a video camera—rather than as physically relevant quantities such as angles, angular velocities, etc. In such examples, neural networks are promising schemes for learning transformations from the observation space of the physical system's dynamics to a potential, physically meaningful low-dimensional space, and can also be referred to as machine learning dynamic models.

[0003] An example of such a real-world application is the manufacture of electronic components, where two parts may need to be connected by first joining copper wires to one component, then bending the wires into place, and finally joining the copper wires to a second component. The bending of the wires can be performed by a robotic arm. To adapt this process to the individual components, the bending process cannot be hard-coded, but is preferably learned based on sensor input. Typically, sensors measuring the physically relevant low-dimensional states of the system (e.g., angles, angular velocities, joint positions, etc.) cannot be added to the production system (if the production system is already in place) because they must be built into the machine itself. In contrast, adding external sensors (such as, for example, a camera system) to the manufacturing station can be quite explicit. This video data can be used by the camera system to learn the bending process of the copper wires.

[0004] A disadvantage is that learning the dynamics of a robotic arm from video data using neural networks is very difficult because the long-term behavior of a robotic arm, such as its stability at certain equilibrium points, cannot typically be automatically incorporated into a neural network. The same applies to modeling the dynamics of other types of physical systems from observed states of the physical system, including but not limited to robots, vehicles, buildings, household appliances, power tools, manufacturing machines, etc.

[0005] In other words, when using general-purpose neural networks to model the dynamics of physical systems, there are almost no guarantees regarding the behavior of the learned neural network. For example, determining the stability properties of a learned neural network is difficult. This has led to the need for extremely large datasets to train such neural networks in order to ensure that the learning dynamics of the neural network exhibit correct long-term behavior.

[0006] While some recent work [1], [2], [3] has begun to consider the stability properties of neural networks, it has typically done so by implementing stability "gently" as an additional loss term on the training data. Therefore, such an approach is hardly capable of describing the stability of a learned neural network for unseen states, which refer to the states of the physical system that were not "seen" during training. Disadvantageously, such an approach is not optimally suited for real-world applications where such unseen states may arise.

[0007] References

[0008] [1] Yinlam Chow, Ofir Nachum, Edgar Duenez-Guzman and MohammadGhavamzadeh. A lyapunov-based approach to safe reinforcement learning . NIPS2018, https: / / arxiv.org / abs / 1805.07708

[0009] [2] Spencer M Richards, Felix Berkenkamp and Andreas Krause. The lyapunov neural network: Adaptive stability certification for safe learning of dynamic systems arXiv preprint, https: / / arxiv.org / abs / 1808.00924

[0010] [3] Andrew J Taylor, Victor D Dorobantu, Hoang M Le, Yisong Yue and Aaron D Ames. Episodic learning with control lyapunov functions for uncertainty robotic systems arXiv preprint, https: / / arxiv.org / abs / 1903.01577. Summary of the Invention

[0011] Ideally, the dynamics model should be trained to learn the dynamics of the physical system, such as the forward dynamics of a robotic arm, in a way that ensures the global stability of the physical system's dynamics.

[0012] According to a first aspect of the invention, a machine learning system and a corresponding computer-implemented method as defined in claims 1 and 10 are provided, respectively, for training a dynamics model to learn the dynamics of a physical system. According to another aspect of the invention, a computer-implemented method and a corresponding control or monitoring system as defined in claims 11 and 13 are provided, respectively, for controlling or monitoring a physical system using machine learning modeling of the dynamics of the physical system. According to another aspect of the invention, a computer-readable medium as defined in claim 9 is provided, comprising transient or non-transient data representing model data defining the learned dynamics model. According to another aspect of the invention, as defined in claim 12, a computer-readable medium is provided, comprising data representing instructions arranged to cause a processor system to perform the computer-implemented method of claim 10 or 11.

[0013] The above measures can involve training a dynamics model to learn the dynamics of a physical system. Specifically, a dynamics model can be learned to infer the future states of the physical system and / or its environment based on their current states. Such states of the physical system can be represented in various ways. For example, in the above example of a robotic arm, the state of the robotic arm can be represented as angles, angular velocities, joint positions, etc., or generally represented by known physical quantities. The training data can contain direct observations of such known physical quantities. In other examples, the state of the physical system can be observed in a high-dimensional observation space, for example as video data, which can be transformed to a latent, physically meaningful low-dimensional space using transformations. In this case, the state of the physical system, and therefore the training data, can be represented as latent samples in the obtained low-dimensional space, and can, but does not need to, correspond to known physical quantities. These known transformations are learned through machine learning, for example using variational autoencoders (VAEs). In some examples, the transformations can be learned by the dynamics model itself, since the training data can include, for example, indirect observations of physically meaningful quantities in a high-dimensional observation space.

[0014] It should be noted that, generally speaking, in many cases, the dynamics of a physical system can be learned from its environment, in addition to learning the dynamics from the state of the physical system itself. For example, the steering dynamics of an autonomous vehicle can be learned from a video camera installed in the vehicle and pointing towards the road ahead.

[0015] To train a dynamic model, time series of states that define the physical system and / or its environment can be accessed. The training data. Using such training data to train a dynamical model to learn the dynamics of a physical system is inherently known. That is, the training data can provide time-series state pairs, where each pair consists of the current state and a future state of the physical system, where the former is the input to the dynamical model during training, and the latter is the target output of the dynamical model during training.

[0016] The above measures can be achieved using Lyapunov functions. To ensure the learning dynamics model The global stability of a system. That is, the stability of any nonlinear system can be shown by the existence of a Lyapunov function, which is a specific type of function that aligns the system along the dynamical... The descent in the function value of the trajectory is associated with different types of stability. A dynamical system is globally stable if and only if it has a continuously differentiable positive definite function. (Right now The positive definite function Along The trajectory decreases, that is... In the above time-discrete variants, the initial state... and They can jointly constrain state evolution. Or in discrete time This state evolution can be represented as The trajectory, and along The decreasing nature of all trajectories can mean that for any And any It maintains: or .

[0017] Given a learning dynamics model, a Lyapunov function can also be learned, for example, using a neural network. However, instead of learning the dynamics model and attempting to verify its stability solely through the (learned) Lyapunov function, the above approach jointly learns the dynamics model and the Lyapunov function, where, according to the Lyapunov function, the dynamics are inherently constrained to be stable everywhere in the state space. Therefore, the learned dynamics model is stable even for states not seen during training. Compared to known schemes for learning dynamics models, the above approach provides an inherently globally stable learned dynamics model, rather than merely being incentivized to be globally stable during training, for example, through a loss function. The above approach provides a learned dynamics model highly suitable for real-world applications where, because the learned dynamics model is inherently guaranteed to be stable, the physical system can assume states not seen during training.

[0018] In some examples, the dynamics model can be integrated into a deep architecture, such as an architecture in which a variational autoencoder transforms observations from sensor data into a latent, physically meaningful low-dimensional space of the system dynamics of the physical system, and this architecture can be used as input to the learned dynamics model to obtain a prediction of the future state of the physical system. This prediction can be used in various control or monitoring applications, such as controlling a robotic arm to bend wires, controlling the steering and / or braking of an autonomous vehicle, etc.

[0019] Optionally, the processor subsystem is configured to learn as a dynamic model ( The nominal dynamics of a physical system ( ) ) to satisfy as given by the Lyapunov function ( The projection of a function on a function subject to Lyapunov conditions ( ) ), to jointly learn dynamic models and Lyapunov functions, based on state-time series ( To learn the nominal dynamics of the physical system ( Here, nominal dynamics This can represent system dynamics, which are learned directly and "naively" from training data before projection, and can be learned, for example, through a neural network. During training, the final dynamics... Then you can The projection onto the dynamic set is evaluated, the projection making It becomes a Lyapunov function that ensures global asymptotic stability.

[0020] Optionally, projection ( () is an orthogonal projection onto the half-space. For any (e.g., unstable) nominal dynamics... and a given positive definite function ( , ,for , ), can be done by Projected to To ensure global asymptotic stability, where Orthogonal projections on a half-space can be indicated. For example, automatic differentiating tools can be used to define... This allows for the explicit completion of tasks within... The limitations include .therefore, It can be explicitly trained, similar to the training of any other dynamic model.

[0021] Optionally, the dynamic model ( This includes a neural network used for evaluating time-series state pairs ( Study the nominal dynamics of physical systems. Besides machine-learnable Lyapunov functions. In addition, the dynamic model can include methods for learning nominal dynamics. Therefore, the dynamic model can consist of two different neural networks.

[0022] Optionally, machine-learnable Lyapunov functions ( At least in part, it is composed of an input convex neural network (ICNN). Lyapunov function. It can also be learned and represented through neural networks. This is achieved by specifically selecting the input to a convex neural network. The input convex neural network Implementation In its input The condition that the middle is convex ensures It has only one global optimum, which is one of the properties of the Lyapunov function that the input convex neural network is to learn.

[0023] Optionally, the input is a convex neural network (ICNN). This includes smoothed variants of the Modified Linear Unit (ReLU) for each activation function. Using smoothed variants of ReLU ensures... It is continuously differentiable. For example, linear ReLU can be modified to provide... The quadratic function in the equation.

[0024] Optionally, the Lyapunov function ( ) is shifted to the equilibrium point of the Lyapunov function ( Establish the zero value of the Lyapunov function at ( ) Furthermore, the learning of the Lyapunov function involves using a quadratic regularization term to ensure the strict positive definiteness of the Lyapunov function. Positive definiteness is another property of the Lyapunov function that the input convex neural network learns and can guarantee through the above measures.

[0025] Optionally, the processor subsystem is configured to, when in training data ( Learning input convex neural networks (ICNN) on ) When training data ( Applying continuously differentiable and invertible functions ( And the output of a continuously differentiable and invertible function is used as the input to the input convex neural network. Reversibility can ensure Mapping a subset of (which may be convex by constraint) to a composite function The adjacent region is thus ensured, thereby guaranteeing that there are no local optima in the combinatorial function, and therefore There is only one optimal value.

[0026] Optionally, the output device is an actuator associated with a physical system, and the processor subsystem is configured to provide the actuator with a future state inferred from the physical system. The control data is used to control physical systems. For example, actuators can be used to control vehicles, such as autonomous or semi-autonomous vehicles, robots, manufacturing machines, buildings, etc.

[0027] Alternatively, the control or monitoring system is one of the following: a vehicle control system, a robot control system, a manufacturing control system, or a building control system. Such control systems greatly benefit from the use of dynamic models that have been learned to be globally stable through joint learning with Lyapunov functions.

[0028] Optionally, the computer-implemented method or system for training a machine-learnable dynamics model to learn the dynamics of a physical system may further include any computer-implemented method or system described herein for using the learned dynamics model, for example, for the control or monitoring of a physical system.

[0029] Those skilled in the art will appreciate that two or more of the above-mentioned embodiments, implementations and / or optional aspects of the present invention can be combined in any manner deemed useful.

[0030] Modifications and variations of any system, any computer-implemented method, or any computer-readable medium may be made by those skilled in the art based on this description, corresponding to the modifications and variations described in another of the entities. Attached Figure Description

[0031] These and other aspects of the invention will become clear from the embodiments and drawings, and will be further illustrated with reference to the embodiments and the accompanying drawings, which are described below as examples, in which...

[0032] Figure 1 A system for training a learnable dynamics model to learn the dynamics of a physical system based on training data including the state of the physical system and / or its environment is shown, wherein the dynamics model includes learnable Lyapunov functions to establish the global stability of the learned dynamics model;

[0033] Figure 2 A computer implementation method for training a learnable dynamic model to learn the dynamics of a physical system based on training data is shown.

[0034] Figures 3A-3C Each figure shows a contour plot of the Lyapunov function of a stable dynamical system, where Figure 3A The diagram illustrates the trajectory of the Lyapunov function. Figure 3B The diagram shows It has components that are not in the half-space. And this component can be subtracted. To obtain ,as well as Figure 3C The diagram shows It exists in the half-space and returns unchanged;

[0035] Figure 4A The diagram illustrates the modified linear unit activation function, while Figure 4B The illustration shows a smooth variant of the modified linear unit activation function;

[0036] Figure 5A A contour plot of the dynamics of a simple damped pendulum, as simulated according to first principles, is shown. Figure 5B The contour plot of the learned stable dynamics model is shown, and Figure 5C The contour plot of the learned Lyapunov function is shown;

[0037] Figure 6A The prediction errors for the angular position and angular velocity of the 8-bar pendulum as a function of the time step are shown, while Figure 6B The average prediction error over 1000 time steps is shown as a function of the number of links in the pendulum.

[0038] Figure 7A The structure of the video texture generation network is shown, where the encoder... and decoder A variational autoencoder is formed, and a stable dynamic model is established. With decoder They are trained together to predict the next frame in the video texture;

[0039] Figure 7B The trajectory projected from left to right onto a two-dimensional plane is shown in three different runs for a dynamic model with stability constraints, as well as for a dynamic model in the form of a general neural network without stability constraints.

[0040] Figure 8 A system is shown that uses a learned dynamics model for control or monitoring;

[0041] Figure 9 The system is shown as part of a (semi-)autonomous vehicle;

[0042] Figure 10 This demonstrates a computer-based implementation of using machine learning modeling of physical system dynamics to control or monitor physical systems; and

[0043] Figure 11 A computer-readable medium containing data is shown.

[0044] It should be noted that the figures are purely illustrative and not drawn to scale. In each figure, elements corresponding to those already described may have the same reference numerals.

[0045] Reference number list

[0046] The following list of reference numerals is provided to facilitate the interpretation of the drawings and should not be construed as limiting the claims.

[0047] 20 sensors

[0048] 22 cameras

[0049] 40 Actuators

[0050] 42 Electric motor

[0051] 60 Environment

[0052] 80 (semi) autonomous vehicles

[0053] 100 Systems for Training Dynamic Models

[0054] 160 Processor Subsystem

[0055] 180 Data Storage Interface

[0056] 190 Data storage devices

[0057] 192 training data

[0058] 194 Model Data

[0059] 196 Model Data

[0060] 200 Methods for Training Dynamic Models

[0061] 210 Access training data

[0062] 220 Access model data that constrains the dynamics model

[0063] 230 Learning Dynamic Models Based on Training Data

[0064] 240 Joint Learning Using Dynamical Models of Lyapunov Functions

[0065] Contour plots of Lyapunov functions for stable dynamical systems (300-304)

[0066] 400 Modified Linear Unit Activation Function

[0067] 410 Smooth variant of the modified linear unit activation function

[0068] 500. Stream plots, such as those used in first-principles simulations.

[0069] 510 Learning the dynamic streamline diagram of the dynamic model

[0070] 520 Contour plot of Lyapunov functions after study

[0071] 600 Prediction error for an 8-bar pendulum as a function of time step

[0072] 610 Time Step

[0073] 620 Prediction Error

[0074] 630 is the average error over 1000 time steps as a function of the number of links.

[0075] 640 links

[0076] 650 Average prediction error

[0077] 660 Simple Learning Dynamics Model

[0078] 670 Stable learning dynamics model

[0079] Structure of a 700-bit video texture generation network

[0080] 710 Stable dynamic model operation 1

[0081] 720 Stable dynamic model running 2

[0082] 730 Stable dynamic model running for 3 days

[0083] 740 Naïve Dynamics Model

[0084] 800 Systems for control or monitoring using learned dynamics models

[0085] 820 Sensor Data Interface

[0086] 822 Sensor Data

[0087] 840 Actuator Interface

[0088] 842 Control Data

[0089] 860 processor subsystem

[0090] 880 Data Storage Interface

[0091] 890 Data Storage Device

[0092] 900 Methods for control or monitoring using learned dynamics models

[0093] 910 Accessing Sensor Data

[0094] 920 Access to model data that constrains the learning dynamics model

[0095] 930 Using models to infer future states from observed current states

[0096] 940 Output data based on inferred future states

[0097] 1000 computer-readable media

[0098] 1010 Non-temporary data. Detailed Implementation

[0099] The following is for reference. Figure 1 and Figure 2 The training of a learnable dynamic model is described, wherein the learnable dynamic model includes learnable Lyapunov functions to establish the global stability of the learned dynamic model, and then references... Figure 3A-5C A more detailed description of the dynamics model and its training is provided, followed by reference. Figures 6A-7B The application of the dynamic model is described, including comparisons with unsteady dynamic models, and references are made. Figure 8-10 Describe the use of learning dynamics models for the control or monitoring of physical systems, such as (semi-)autonomous vehicles.

[0100] Figure 1System 100 is shown, which is used to train a dynamic model to learn the dynamics of a physical system by learning to infer the future state of the physical system and / or its environment based on the current state of the physical system and / or its environment.

[0101] System 100 may include an input interface for accessing time-series training data 192 representing the state of a physical system and / or its environment, and model data 194 constraining a machine-learnable dynamics model including a machine-learnable Lyapunov function. For example, also as Figure 1 As illustrated, the input interface can be comprised of a data storage interface 180, which can access training data 192 and model data 194 from a data storage device 190. For example, the data storage interface 180 can be a memory interface or a permanent storage interface, such as a hard disk or SSD interface; but it can also be a personal area network (PAN), local area network (LAN), or wide area network (WAN) interface, such as a Bluetooth, Wi-Fi, or Ethernet or fiber optic interface. The data storage device 190 can be an internal data storage device of system 100, such as a hard disk drive or SSD, but it can also be an external data storage device, such as a network-accessible data storage device. In some embodiments, training data 192 and model data 194 can each be accessed from different data storage devices, for example, via different subsystems of the data storage interface 180. Each subsystem can have the type described above for the data storage interface 180.

[0102] System 100 may further include a processor subsystem 160, which may be configured to learn a dynamical model based on time-series state pairs during operation of system 100, in order to learn and infer future states of the physical system and / or its environment based on the current state, wherein the learning is constrained to provide a globally stable modeling of the dynamics of the physical system by jointly learning the dynamical model and a Lyapunov function, such that the value of the learned Lyapunov function decreases along all state trajectories inferred from the learned dynamical model. (Refer to...) Figure 3A-5C The dynamic model and its training are further described.

[0103] System 100 may further include an output interface for outputting training model data 196 representing the learned (or “trained”) dynamics model. For example, also as Figure 1As illustrated, the output interface can be comprised of a data storage interface 180, wherein, in these embodiments, the interface is an input / output (“IO”) interface, through which training model data 196 can be stored in the data storage device 190. For example, model data 194 defining an “untrained” dynamics model can be at least partially replaced by model data 196 of a trained dynamics model during or after training, because the parameters of the dynamics model—such as the weights of a neural network and other types of parameters—can be adapted to reflect training on the training data 192. This also... Figure 1 The figures are illustrated by reference numerals 194 and 196, which refer to the same data records on data storage device 190. In other embodiments, training model data 196 may be stored separately from model data 194 that defines an "untrained" dynamic model. In some embodiments, the output interface may be separate from the data storage interface 180, but may generally have the type described above for data storage interface 180.

[0104] Figure 2 A computer-implemented method 200 is shown for training a dynamical model to learn the dynamics of a physical system by learning to infer the future states of the physical system and / or its environment based on the current state of the physical system and / or its environment. Method 200 may correspond to... Figure 1 The system 100 operation, but does not need to correspond to Figure 1 The operation of system 100 can correspond to the operation of another type of system, device or equipment, or it can correspond to a computer program.

[0105] The computer-implemented method 200 is shown to include: in a step entitled "Accessing Training Data", accessing 210 time-series training data representing the states of a physical system and / or its environment; in a step entitled "Accessing Model Data that Qualifies the Dynamics Model", accessing 220 model data that qualified a machine-learnable dynamics model including a machine-learnable Lyapunov function; and in a step entitled "Learning the Dynamics Model Based on Training Data", learning 230 the dynamics model based on time-series state pairs to infer future states of the physical system and / or its environment based on current states, wherein the learning step 230 includes, in a step or sub-step entitled "Joint Learning of the Dynamics Model with Lyapunov Function", constraining learning to provide a globally stable modeling of the dynamics of the physical system by jointly learning 240 the dynamics model and the Lyapunov function, such that the value of the learned Lyapunov function decreases along all state trajectories inferred from the learned dynamics model.

[0106] The following example describes machine-learnable dynamics models, including their training, in more detail. However, practical implementations of machine-learnable dynamics models and their training can be carried out in various other ways, such as on the basis of similar mathematical concepts. For example, instead of using neural networks, Gaussian processes or another type of machine-learnable model can be used to learn the nominal dynamics of a physical system and to learn Lyapunov functions.

[0107] In short, we will now study continuous-time dynamical systems, but also apply necessary adaptations to discrete-time dynamical systems. That is, given time... The state of being The time derivative of this state with respect to a certain function Modeled as

[0108] (1).

[0109] For such a dynamic system (or one with control input) (for Modeling the temporal evolution of the corresponding system is a fundamental problem with applications in reinforcement learning, control, prediction, and many other settings. Neural networks have long been the natural choice for modeling these functions due to their representativeness. However, when using general neural networks to model the dynamics in this setting, there are few guarantees about the behavior of the learned system. For example, it is very difficult to make any statement about the stability properties of the learned model (informally, the tendency of the system to remain within some invariant bounded set). While some recent work has begun to consider the stability properties of neural networks, see [1, 2, 3] mentioned above, such work typically does so by ("gently") implementing stability as an additional loss term on the training data. Therefore, such work can hardly describe the stability of the system in unseen states.

[0110] This specification describes a scheme for learning provably stable neural network dynamics over the global state space. To do this, system dynamics and a Lyapunov function can be learned jointly. This stability can be a hard constraint imposed on the model. Thus, unlike recent schemes, stability may not be implemented via an imposed loss function, but rather directly built into the model's dynamics (e.g., even a model randomly initialized within the described model class will be provably stable everywhere in the state space). For this purpose, a Lyapunov function can be provided, which can be based on the input convex neural network, and which can ensure global exponential stability to equilibrium points while still considering expressive dynamics.

[0111] Using these methods, see the following example for reference. Figures 5A-6B Explain learning such asn The physical model of the linkage pendulum is presented, along with its dynamics, and substantial improvements are shown on top of a general network. It is also shown how such a dynamic model can be integrated into larger network systems to learn dynamics over complex output spaces. In particular, see reference... Figures 7A-7B This demonstrates how to combine a model with a variational autoencoder (VAE) to learn dynamic “video textures.”

[0112] The following mainly considers autonomous dynamics. (for The settings of the initial state. Although the techniques described in this specification can also be applied to dynamics with control, for ease of explanation, the following discussion focuses on the autonomous case. If for any initial state... We have with and The system is then constrained to be globally asymptotically stable (for simplicity, at the equilibrium point). (surroundings); if there is any and However, only for If applicable, then It is locally asymptotically stable, among which It is some bounded set containing the origin. Similarly, if for some constants... For any (Correspondingly, for) If we have the following formula (2), then It is globally (and correspondingly locally) exponentially stable (e.g., "exponentially fast" converges to equilibrium).

[0113] (2).

[0114] Lyapunov theory establishes a connection between the various types of stability mentioned above and the descent of a specific type of function, known as the Lyapunov function. Specifically, let... It is a continuously differentiable positive definite function, that is, for , and Lyapunov analyzed that, It is stable (based on the above constraints) if and only if we can find a certain function. As mentioned above, this causes the value of the function to follow the path defined by... The generated trajectory decreases. Formally, this is the time derivative. The condition is the condition of the following equation (3).

[0115] (3)

[0116] This condition should apply to all. Or for all This applies to ensure global or local stability accordingly. Similarly, it applies if and only if there exists a positive definite... When the following equation (4) holds, It is globally asymptotically stable.

[0117] ,in (4).

[0118] These conditions imply that various forms of stability are relatively explicit, but the opposite is shown (any stable system for a given...). Both (which must comply with this property) are relatively more complex. However, we mainly focus on the "simpler" one of these two directions because the goal is to implement conditions that ensure stability.

[0119] In short, for any (unstable) nominal dynamics and a given positive definite function ( , ,for , We can do this by... Projected to To ensure global asymptotic stability, where An orthogonal projection onto a half-space. For example, automatic differentiating tools can be used to limit / achieve this. This allows for the explicit completion of tasks within... The limitations include Overall, regarding Neural networks can be explicitly trained, much like training any other function.

[0120] The following is also for reference Figure 3A-7B A more detailed explanation of the above is provided. The intuition behind at least some of the techniques described in this specification can be summarized as follows: instead of learning the dynamic function and attempting to verify its stability individually via the Lyapunov function, the dynamic model and the Lyapunov function are learned jointly, whereby, according to the Lyapunov function, the modeling of the dynamics can be inherently constrained to be stable (stable everywhere in the state space). Specifically, following the principles mentioned above, let… The term "nominal" dynamic model refers to, for example, a dynamic model learned directly on training data without incorporating Lyapunov functions, and lets... It is a positive definite function: for , and Then, in order to (provably and globally) ensure that the dynamic function is stable, It can be projected such that it satisfies the following conditions

[0121] (9)

[0122] That is, one can define dynamics as

[0123] (10)

[0124] in Can be marked Time The orthogonal projection onto the half-space, where the second equation originates from the analytic projection of the point onto the half-space. This can be constrained, for example, using an automatic differentiator. Therefore, gradient Items can be included Within the limitations, obtain targeted Composite neural networks (e.g., including those targeting) The first neural network and the second neural network targeting the Lyapunov function), the second neural network targeting the Lyapunov function A composite neural network can be trained just like any other function.

[0125] Figures 3A-3C The above content is illustrated in diagrams. Figures 3A-3C Each of the figures shows contour plots of the Lyapunov functions of a stable dynamical system, 300-304, where... Figure 3A The trajectory of the Lyapunov function is illustrated using arrows pointing in the direction of its increment. Let... . Figure 3B The diagram shows It has components that are not in the half-space. And this component can be subtracted. To obtain ,as well as Figure 3C The diagram shows It exists in half-space and returns unchanged.

[0126] As previously indicated, It may need to be positive definite, but in addition, Preferably, it can be free from any local optima other than 0. This is due to the Lyapunov diminishing condition: one can expect to guarantee the equilibrium point. The stability, however, the diminishing conditions imposed on the dynamics may mean You can follow The trajectory decreases. If... If a location has a local optimum far from the origin, then the dynamics may theoretically, but practically, not necessarily be trapped in that location; this can be achieved through... The term becomes zero to reveal itself, which may cause the dynamics to become undefined at the optimum. The above conditions can be addressed through design decisions as described below.

[0127] Lyapunov function It can be learned and represented using neural networks. To ensure... The nature of the requirements can be considered in light of the following considerations. More specifically, one might prefer to ensure (i) It has only one (globally) optimal (assuming without loss of generality) (ii) It is positive definite, for example, for , and (iii) It is continuously differentiable.

[0128] First, this can be achieved by utilizing an input convex neural network (ICNN). express To implement property (i), the input convex neural network (ICNN) It can be defined by the following formula (11):

[0129] (11)

[0130] in It can be from input to Real-valued weight mapping for layer activation; It can activate the previous layer. Positive weights mapped to the next layer; It can be a deviation from the real value; and It can be a convex, monotonically non-decreasing nonlinear activation, such as ReLU or its smoothed variants. In the case of this formula, exist The network is convex, and any convex function can be approximated by such a network.

[0131] Secondly, what is desirable is to ensure It is positive definite. Represented using ICNN. Make It is convex, meaning that there exists only one optimal state, but it is not mandatory. There are multiple ways to ensure For example, people can find The global optimum is found by shifting it or removing the bias term. However, shifting requires finding the optimum during training, which can be computationally expensive, while removing the bias may impose restrictions on the categories of variables that can be approximated. As an alternative, It can be shifted to ensure Additionally, some quadratic regularization terms can be added to ensure positive definiteness, for example as:

[0132] (12)

[0133] in It can be a positive, convex, and non-decreasing function, where ,and It is a small constant, for example, chosen to make .

[0134] Third, it is desirable, for example, to use in Figure 4B The smooth version 410 of ReLU activation shown in the figure ensures It is continuously differentiable, and the smoothed version 410 of the ReLU activation uses... Quadratic function substitution in, for example, Figure 4A The linear ReLU 400 is shown in the figure.

[0135] (13).

[0136] Alternatively, distortion of the input space can be used to make the input space more responsive to the input space. The input is first mapped by any continuously differentiable and invertible function. That is, while convexity ensures that the Lyapunov function has no local optima, this may be a sufficient but not necessary condition, and in fact, requiring a strongly convex Lyapunov function might impose too stringent requirements on the dynamics of learning. For this reason, the above discussion of the ICNN function... The input can optionally be any continuously differentiable and invertible function. For example, use the following equation (14) as a Lyapunov function.

[0137] (14)

[0138] Reversibility can ensure The subsets of (which, by constraint, are convex sets) can be mapped to the composite function. The adjacent regions are defined to ensure that there are no local optima in the composite function.

[0139] The above dynamic model can be found in the reference Figure 5A-7B The description provides an explanation using several example problems. Figures 5A-5C Involving, especially n Physically based dynamical modeling of a link-pendulum system. Damped stiffness. n The state of the connecting rod pendulum Each link can be used angular position and angular velocity To describe. In the following text, It can be Fully connected neural networks, and It possesses the properties previously described. ICNN utilizes data generated by the symbolic algebra solver sympy. To train the model.

[0140] exist Figure 5A In the case of a simple damped pendulum (i.e., where...), The simulated dynamics under the condition of ) are shown in streamline diagram 500, while Figure 5B A streamline diagram 510 corresponding to the dynamics learned from the dynamic model is shown, and in which... Figure 5C The contour plot 520 of the learned Lyapunov function is shown. As can be seen, even over long time periods, the dynamics can be learned to accurately predict the motion of the pendulum.

[0141] In change n When considering the time frame of the simulation, the learned dynamics can also be quantitatively evaluated. Figure 6A Graph 600 is shown, which illustrates the angular position of the 8-bar pendulum as a function of time step 610. and angular velocity The prediction error is 620, while Figure 6B Plot 630 shows the average prediction error 650 over 1000 time steps as a function of the multiple links 640 of the pendulum. Plots 600 and 630 both show the errors for both a "simple" prior art model 660 and a learning dynamics model 670, as described in this specification. While both the simple model and the stable model show increasing average errors at the beginning of the trajectory, the dynamics model 670 is able to capture the contraction in the physical system (implied by energy conservation) and in fact shows decreasing errors toward the end of the simulation (both the real and simulated dynamics are stable). In contrast, the error in the simple model increases.

[0142] Figures 7A-7B Another example involves a dynamic model, as described in this specification, that can be used for stable video texture generation, for example, by using a variational autoencoder (VAE) to learn the encoding of an image and a stable dynamic model to learn the dynamics in the encoding space. Given a frame sequence... The overall network can be in time A frame is fed at a time and can be trained to... and The reconstructed frame can be obtained from a given mean. and variance encoder Potential state and decoder Let's define a VAE. The overall network can be trained to minimize both the standard VAE loss (reconstruction error plus the KL divergence term) and the reconstruction loss for the next predicted state. or more precisely at The evolution of the underlying dynamics is modeled. In other words, as... Figure 7A The diagram illustrates the structure of a video texture generation network, where the encoder... and decoder A variational autoencoder is formed, and a stable dynamic model is established therein. With decoder Together they are trained to predict the next frame in the video texture, and the overall network can be trained to minimize:

[0143] .

[0144] The overall network can be trained on pairs of consecutive frames sampled from the video. To generate video textures, the encoding of individual frames can be used to seed the dynamics model, and the dynamics model can be numerically integrated to obtain the trajectory. The VAE decoder can then convert each step of the trajectory into a frame.

[0145] Figure 7B The diagram illustrates trajectories projected from left to right onto a two-dimensional plane, with three different runs for a dynamical model with stability constraints, and a dynamical model in the form of a general neural network without stability constraints. In this example, the true latent space is 320-dimensional, and the trajectories can be projected onto a two-dimensional plane for display. For the unstable model, the dynamics diverge rapidly and produce static images, while for the stable model, it is possible to generate different (stable) trajectories that consistently produce realistic images over long time intervals.

[0146] Figure 8 A system 800 is illustrated that uses machine learning modeling of physical system dynamics to control or monitor a physical system. System 800 may include an input interface 880 for accessing training model data 196, which represents data that can be generated by... Figure 1 System 100 or Figure 2 Method 200 generates or, as described elsewhere, a learning dynamics model. For example, also as Figure 8As illustrated in the diagram, the input interface can be composed of a data storage interface 880, which can access the training model data 196 from the data storage device 890. Generally, the input interface 880 and the data storage device 890 can have the same characteristics as the reference... Figure 1 The same type is described for input interface 180 and data storage device 190. Figure 8 Further illustrated is a data storage device 892, which includes sensor data 822 representing observations of the current state of the physical system. As described elsewhere, in some embodiments, the sensor data 822 may also be received directly from the sensor 20 via the sensor interface 820 or via another type of interface, rather than being accessed from the data storage device 890 via the data storage interface 880.

[0147] System 800 may further include a processor subsystem 860, which may be configured to, during operation of system 800, infer the future state of the physical system based on observations of the current state of the physical system, such as those obtained from sensor data 822, and provide output data 842 to an output device using an output interface as described elsewhere, so as to enable control or monitoring of the physical system based on the inferred future state, as also described elsewhere in this specification.

[0148] Will understand, and used for Figure 1 The same considerations and implementation options apply to the processor subsystem 860. It will be further understood that, unless otherwise stated, the same considerations and implementation options apply to the processor subsystem 860. Figure 1 The same considerations and implementation options as System 100 can generally be applied to System 800.

[0149] Figure 8 Various optional components of system 800 are further illustrated. For example, in some embodiments, system 800 may include a sensor data interface 820 for direct access to sensor data 822 acquired by sensor 20 in environment 60. Sensor 20 may, but does not need to, be part of system 800. Sensor 20 may have any suitable form, such as an image sensor, lidar sensor, radar sensor, pressure sensor, container temperature sensor, etc. In some embodiments, sensor data 822 may be sensor measurements of different physical quantities, as it may be obtained from two or more different sensors sensing different physical quantities. Sensor data interface 820 may have any suitable form corresponding in type to the sensor type, including but not limited to: low-level communication interfaces, such as I2C or SPI-based data communication; or data storage interfaces of the type described above for data storage interface 880.

[0150] In some embodiments, system 800 may include actuator interface 840 for providing control data 842 to actuator 40 in environment 60. Such control data 842 may be generated by processor subsystem 860 to control actuator 40 based on one or more inferred future states of a physical system, which may be generated using a learned dynamics model. For example, the actuator may be an electric, hydraulic, pneumatic, thermal, magnetic, and / or mechanical actuator. Specific but non-limiting examples include electric motors, electroactive polymers, hydraulic cylinders, piezoelectric actuators, pneumatic actuators, servo mechanisms, solenoids, stepper motors, etc. This type of control reference... Figure 9 A description is given for (semi)autonomous vehicles.

[0151] In other embodiments ( Figure 8 (Not shown in the image) System 800 may include an output interface to a presentation device such as a display, light source, speaker, vibration motor, etc., which may be used to generate a sensory-perceptible output signal based on one or more inferred future states of the physical system. The sensory-perceptible output signal may directly indicate the inferred future state of the physical system, but may also represent a derived sensory-perceptible output signal, for example, for use in guidance, navigation, or other types of control of the physical system.

[0152] Generally speaking, including but not limited to Figure 1 System 100 and Figure 8 Each system described in this specification of System 800 may be embodied as a single device or apparatus, such as a workstation or server. The device may be an embedded device. The device or apparatus may include one or more microprocessors executing appropriate software. For example, the processor subsystem of the corresponding system may be embodied by a single central processing unit (CPU), and also by a system or combination of such CPUs and / or other types of processing units. The software may have been downloaded and / or stored in a corresponding memory, such as volatile memory like RAM, or non-volatile memory like flash memory. Alternatively, the processor subsystem of the corresponding system may be implemented in the device or apparatus in the form of programmable logic, for example as a field-programmable gate array (FPGA). Generally, each functional unit of the corresponding system may be implemented in the form of circuits. The corresponding system may also be implemented in a distributed manner, for example involving different devices or apparatuses, such as distributed local or cloud-based servers. In some embodiments, System 800 may be part of a vehicle, robot, or similar physical entity, and / or may represent a control system configured to control the physical entity.

[0153] Figure 9 The above example illustrates a system 800 as a control system for a (semi-)autonomous vehicle 80 operating in environment 60. The autonomous vehicle 80 can be autonomous, as it may include an autonomous driving system or a driver assistance system, the latter also referred to as a semi-autonomous system. The autonomous vehicle 80 can, for example, incorporate system 800 to control its steering and braking based on sensor data obtained from a video camera 22 integrated into the vehicle 80. For example, system 800 can control electric motor 42 to perform (regenerative) braking in case the autonomous vehicle 80 anticipates a collision with a traffic participant. System 800 can control steering and / or braking to avoid a collision with a traffic participant. For this purpose, system 800 can infer the future state of the vehicle relative to its environment (including traffic participants) based on sensor data obtained from the video camera. If the anticipated future state of the vehicle, such as its future position relative to traffic participants, leads to a collision, system 800 can take appropriate action.

[0154] Figure 10 A computer-implemented method 900 for controlling or monitoring a physical system using machine learning modeling of physical system dynamics is illustrated. Method 900 can correspond to... Figure 8 The system 800 can be used for operation, but alternatively it can be performed using or by any other system, apparatus or device.

[0155] Method 900 is shown to include: in a step entitled "Accessing Sensor Data", accessing 910 sensor data representing the current state of the physical system; and in a step entitled "Accessing Model Data that Defines the Learning Dynamics Model", accessing 920 model data that defines the dynamics model of machine learning as described elsewhere in this specification, said model data being, for example, from... Figure 1-2 The system 100 or method 200 obtains, in the step entitled "Inferring Future State from Observed Current State Using Model", 930 the future state of the physical system is inferred based on the observation of the current state of the physical system, such as from sensor data, and in the step entitled "Providing Output Data Based on Inferred Future State", 940 output data is provided to the output device using the output interface to the output device used in the control or monitoring of the physical system, so that the physical system can be controlled or monitored based on the inferred future state.

[0156] It will be understood that, generally speaking, the corresponding Figure 2 and Figure 10 The operations or steps of the computer-implemented methods 200 and 800 can be performed in any suitable order, such as sequentially, simultaneously, or a combination thereof, which, where applicable, is subject to a particular order, for example, necessary by an input / output relationship.

[0157] Each method, algorithm, or pseudocode described in this specification can be implemented on a computer as a computer-implemented method, dedicated hardware, or a combination of both. If further... Figure 11 As illustrated, computer instructions, such as executable code, can be stored on a computer-readable medium 1000, for example, in the form of a series of machine-readable physical tags 1010 and / or as a series of elements with different electrical (e.g., magnetic) or optical properties or values. The executable code can be stored in a transient or non-transitory manner. Examples of computer-readable media include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Figure 11 An optical disc 1010 is shown. In an alternative embodiment, the computer-readable medium 1000 may include training model data 1010 that defines a dynamic model of machine learning as described elsewhere in this specification.

[0158] Examples, embodiments, or optional features—whether or not indicated as non-limiting—should not be construed as limiting the invention as claimed.

[0159] According to the abstract of the specification, a system and computer implementation method for training a dynamical model to learn the dynamics of a physical system are provided. Specifically, a dynamical model can be learned to infer the future state of the physical system and / or its environment based on the current state of the physical system and / or its environment. The learned dynamical model is inherently globally stable. That is, instead of learning the dynamical model and attempting to verify its stability independently, the learnable dynamical model includes a learnable Lyapunov function, which is jointly learned along with the nominal dynamics of the physical system. Therefore, the learned dynamical model is highly suitable for real-world applications where, due to the inherent global stability of the learned dynamical model, the physical system can assume states not observed during training.

[0160] It should be noted that the embodiments mentioned above are illustrative rather than limiting of the invention, and those skilled in the art will be able to devise many alternative embodiments without departing from the scope of the appended claims. Any reference marks placed between parentheses in the claims should not be construed as limiting the claims. The use of the verb “comprising” and its variations does not exclude the presence of elements or stages other than those stated in the claims. The article “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. Expressions such as “at least one of…” when preceding a list or group of elements indicate the selection of all elements or any subset of elements from that list or group. For example, the expression “at least one of A, B, and C” should be understood to include only A, only B, only C, both A and B, both A and C, both B and C, or all A, B, and C. The invention can be implemented by means of hardware comprising several different elements, and by means of a suitably programmed computer. In device claims enumerating several components, several of these components can be embodied by the same hardware. The only fact is that the inclusion of certain measures in mutually distinct dependent claims does not indicate that a combination of these measures cannot be used advantageously.

Claims

1. A machine learning system (100) for training a dynamic model to learn the dynamics of a physical system by learning to infer the future states of the physical system and / or its environment based on the current state of the physical system and / or its environment, said machine learning system comprising: - Input interface (180), which is used to access: - A time series representing the state of a physical system and / or its environment Training data (192); - Machine-learnable dynamic models Given limited model data (194), the machine-learnable dynamic model Including machine-learnable Lyapunov functions ; - Processor subsystem (160), which is configured to be based on time-series state pairs To learn the dynamic model so that it can be based on the current state To learn how to infer the future state of a physical system and / or its environment The learning is constrained to provide a globally stable model of the dynamics of the physical system by jointly learning the dynamical model and the Lyapunov function, such that the value of the learned Lyapunov function follows all state trajectories inferred from the learned dynamical model. Decreasing ; The processor subsystem (160) is configured to learn as a dynamic model. The nominal dynamics of the physical system To satisfy the condition as given by the Lyapunov function Projection on a function with restricted Lyapunov conditions To jointly learn dynamic models and Lyapunov functions, based on state-time series. To learn the nominal dynamics of the physical system ;and The projection It is an orthogonal projection onto the half-space.

2. The machine learning system (100) according to claim 1, wherein, Dynamics model Includes a neural network, which is used to base time series state pairs To learn the nominal dynamics of physical systems .

3. A machine learning system (100) for training a dynamic model to learn the dynamics of a physical system by learning to infer the future states of the physical system and / or its environment based on the current state of the physical system and / or its environment, said machine learning system comprising: - Input interface (180), which is used to access: - A time series representing the state of a physical system and / or its environment Training data (192); - Machine-learnable dynamic models Given limited model data (194), the machine-learnable dynamic model Including machine-learnable Lyapunov functions ; - Processor subsystem (160), which is configured to be based on time-series state pairs To learn the dynamic model so that it can be based on the current state To learn how to infer the future state of a physical system and / or its environment The learning is constrained to provide a globally stable model of the dynamics of the physical system by jointly learning the dynamical model and the Lyapunov function, such that the value of the learned Lyapunov function follows all state trajectories inferred from the learned dynamical model. Decreasing ; Among them, machine-learnable Lyapunov functions At least in part, it is composed of an input convex neural network ICNN. express.

4. The machine learning system (100) according to claim 3, wherein, Input convex neural network ICNN, Includes smooth variants of the Modified Linear Unit (ReLU) for each activation function.

5. The machine learning system (100) according to claim 3 or 4, wherein, Lyapunov function The shift is made at the equilibrium point of the Lyapunov function. Establish the zero value of the Lyapunov function. Furthermore, the learning of Lyapunov functions involves using a quadratic regularization term to ensure the strict positive definiteness of Lyapunov functions.

6. The machine learning system (100) according to claim 3 or 4, wherein, The processor subsystem (160) is configured to operate on training data. Learning input convex neural networks (ICNNs) At that time, on the training data Applying continuously differentiable and invertible functions Furthermore, the output of a continuously differentiable and invertible function is used as the input to the convex neural network. .

7. A computer-readable medium (1000) comprising a function including a machine-learnable Lyapunov function. Machine-learnable dynamics models Limited temporal or non-temporal model data (1010), where the learning dynamic model is based on the current state. Inferring the future state of a physical system and / or its environment The learning of the dynamics model has been constrained to provide a globally stable model of the physical system's dynamics by jointly learning the dynamics model and the Lyapunov function, such that the value of the learned Lyapunov function follows all state trajectories inferred from the learned dynamics model. Decreasing ,in, Machine-learnable Lyapunov functions At least in part, it is composed of an input convex neural network ICNN. express.

8. A computer-implemented method (200) for training a dynamics model to learn the dynamics of a physical system by learning to infer the future states of the physical system and / or its environment based on the current state of the physical system and / or its environment, the method comprising: - Visit (210, 220): - A time series representing the state of a physical system and / or its environment The training data; - Machine-learnable dynamic models Given limited model data, the machine-learnable dynamic model Including machine-learnable Lyapunov functions ; -Based on time series state pairs To learn the (230) dynamic model so as to be based on the current state To learn how to infer the future state of a physical system and / or its environment The learning mentioned therein includes: - Constraint learning provides a globally stable model of the dynamics of a physical system by jointly learning the (240) dynamic model and the Lyapunov function, such that the value of the learned Lyapunov function follows all state trajectories inferred from the learned dynamic model. Decreasing ; Among them, machine-learnable Lyapunov functions At least in part, it is composed of an input convex neural network ICNN. express.

9. A computer-implemented method (900) for controlling or monitoring a physical system using machine learning modeling of physical system dynamics, the method comprising: - Visits (910, 920): - Represents the current state of the physical system The sensor data observed; - Dynamic models for machine learning The dynamic model of machine learning is defined by the limited model data. Including machine-learnable Lyapunov functions ; The learning dynamics model is based on the current state. Inferring the future state of a physical system and / or its environment The learning of the dynamics model has been constrained to provide a globally stable model of the physical system's dynamics by jointly learning the dynamics model and the Lyapunov function, such that the value of the learned Lyapunov function follows all state trajectories inferred from the learned dynamics model. Decreasing ; - Based on observations of the current state of the physical system, such as those obtained from sensor data, infer (930) the future state of the physical system. ; - Use the output interface to the output device used in the control or monitoring of the physical system to provide (940) output data to the output device so that the physical system can be controlled or monitored based on the inferred future state; Among them, machine-learnable Lyapunov functions At least in part, it is composed of an input convex neural network ICNN. express.

10. A computer-readable medium (1000) comprising temporary or non-temporary data (1010) representing instructions arranged to cause a processor system to perform a computer-implemented method according to claim 8 or 9.

11. A control or monitoring system (800) that uses machine learning modeling of physical system dynamics to control or monitor a physical system, said control or monitoring system comprising: - Input interfaces (820, 880), which are used to access: - Represents the current state of the physical system Observed sensor data (822); - Dynamic models for machine learning The dynamic model of machine learning is based on the limited model data (196). Including machine-learnable Lyapunov functions ; The learning dynamics model is based on the current state. Inferring the future state of a physical system and / or its environment The learning of the dynamics model has been constrained to provide a globally stable model of the physical system's dynamics by jointly learning the dynamics model and the Lyapunov function, such that the value of the learned Lyapunov function follows all state trajectories inferred from the learned dynamics model. Decreasing ; - Output interface (840) to output devices (40, 42), which is used in the control or monitoring of physical systems; - The processor subsystem (860) is configured as follows: - Inferring the future state of a physical system based on observations of its current state, such as those obtained from sensor data. ; - Use the output interface to provide output data to the output device (842) so that the physical system can be controlled or monitored based on the inferred future state; Among them, machine-learnable Lyapunov functions At least in part, it is composed of an input convex neural network ICNN. express.

12. The control or monitoring system (800) according to claim 11, wherein, The output device is an actuator (40, 42) associated with a physical system, and wherein the processor subsystem (860) is configured to provide the actuator with a future state inferred from the physical system. The control data (842) is used to control the physical system.

13. The control or monitoring system (800) according to claim 12, wherein the system is one of the following: -Vehicle control system; - Robot control system; - Manufacturing control systems; and - Building control system.

Images