Reduced-order modeling and control of high-dimensional physical systems using neural network models.

Abstract

A system and method are provided for training a neural network to control the operation of a system having nonlinear dynamics represented by a partial differential equation (PDE). The method includes collecting digital representations of time series data representing instances of the system's function space and measurements of the system's state of operation. Co-location points corresponding to solutions to the PDE are generated. The neural network is trained using training data including the collected time series data and the co-location points to train parameters of a nonlinear operator. The neural network has an autoencoder architecture including an encoder and a decoder. The encoder encodes each instance of the training data into a latent space, a linear operator propagates the encoded instance into the latent space using a transformation determined by the parameters of the nonlinear operator, and the decoder decodes the transformed encoded instances of the training data to minimize a hybrid loss function.

Description

[Technical Field]

[0001] The present disclosure relates generally to system modeling, prediction, and control, and more particularly to a system and method for reduced-order modeling and control of high-dimensional physical systems using neural network models. [Background technology]

[0002] Control theory in control systems engineering is a subfield of mathematics that deals with the control of continuously operating dynamic systems in engineering processes and machines. The goal is to develop control policies to control such systems with control actions in an optimal manner without delay or overshoot, and to ensure control stability. Summary of the Invention [Problem to be solved by the invention]

[0003] Traditionally, some methods for controlling systems are based on techniques that enable a model-based design framework in which system dynamics and constraints can be directly considered. Such methods can be used in many applications to control systems, such as dynamic systems of various complexities. Examples of such systems can include production lines, automobile engines, robots, numerically controlled machining, motors, satellites, and generators.

[0004] Furthermore, a model of the system's dynamics, or a model of the system, describes the system's dynamics using differential equations. However, in many situations, a system's model may be nonlinear, difficult to design, difficult to use in real time, or inaccurate. Examples of such cases are abundant in certain applications, such as robotics, building controls such as heating, ventilation, and air conditioning (HVAC) systems, gas leak detection, smart grids, factory automation, transportation, self-tuning machines, and traffic networks. In addition, even when a nonlinear model is available, designing an optimal controller for the system's control can be an inherently difficult task.

[0005] Furthermore, in the absence of an accurate model of a dynamic system, some control methods utilize operational data generated by the dynamic system to construct feedback control policies that stabilize system dynamics or embed quantifiable control-related performance. Typically, various types of methods for controlling a system that utilize operational data can be used. In one embodiment, a control method may first build a model of the system and then utilize the model to design a controller. However, such control methods result in a black-box design of the control policy, which directly maps the system's state to control commands. However, such control policies are not designed taking into account the physical properties of the system.

[0006] In another embodiment, the control method may construct a control policy directly from data without an intermediate model-building step of the system. A drawback of such a control method is that the model-building step may require a large amount of data. In addition, the controller is calculated from an estimated model, for example, according to the certainty equivalence principle, but in practice, the model estimated from the data may not capture the physical properties of the system's dynamics. Therefore, many of the system's control techniques may not be available with the constructed model of the system.

[0007] Therefore, there is a need for a method and system for controlling the system in an optimal manner to address the above-mentioned problems. [Means for solving the problem]

[0008] The present disclosure provides a computer-implemented method and system for reduced-order modeling and control of high-dimensional physical systems using neural network models.

[0009] An objective of some embodiments is to train a neural network model so that the trained neural network model can be utilized to control the behavior of a system having nonlinear dynamics described by a partial differential equation (PDE). The neural network model has an autoencoder architecture that may include an encoder, a linear predictor (e.g., a linear operator), and a decoder. In some embodiments, the linear predictor may be based on a Koopman operator.

[0010] In other embodiments, the neural network model has an autoencoder architecture that may include an encoder, a nonlinear predictor (such as a nonlinear operator), and a decoder. In some implementations, the nonlinear predictor may be based on one of an intrusive or non-intrusive model of the high-dimensional dynamics of the system. For example, the linear or nonlinear operator may be based on a reduced-order model (ROM).

[0011] To generate an intrusive ROM, a reduced solution of the system can be found by solving a reduced-order model, i.e., by projecting the original model into a reduced space. In some embodiments, a proper orthogonal decomposition (POD)-Galerkin projection method can be used to generate the intrusive ROM. In such cases, singular value decomposition (SVD) can be applied to a snapshot matrix of data of the system's dynamics, and a POD basis can be extracted. Furthermore, the intrusive ROM can be constructed by applying Galerkin projection.

[0012] Traditionally, POD-Galerkin projection-based model reduction is intrusive because the numerical implementation of the reduced model requires access to discretized PDE operators. The intrusive nature of such ROM techniques limits the scope of traditional model reduction methods. The main drawback of such intrusive methods is that they require access to the full model. Furthermore, when proprietary software is used to solve PDEs where the details of the governing equations of the system's dynamics are unknown, traditional intrusive model reduction methods may not be applicable because the solver may typically not be available at the time of work.

[0013] To generate a non-intrusive ROM, a reduced model can be learned from snapshots, i.e., either from numerical approximations or measurements of the dynamic system's state, when the discretized system's operators are unavailable. In one embodiment, machine learning models can be used to analyze the underlying processes that generate the non-intrusive ROM.

[0014] Another objective of some embodiments is to generate a model of the system's dynamics that captures the physics of the system's behavior. In this way, embodiments simplify the system model design process while retaining the benefits of having a model of the system when designing a control application.

[0015] Accordingly, one embodiment discloses a computer-implemented method for training a neural network model for controlling the operation of a system having nonlinear dynamics represented by a partial differential equation (PDE). The neural network includes a nonlinear operator of the system's dynamics represented in a latent space by a parameterized ordinary differential equation (ODE) with parameters determined by training. The computer-implemented method may include collecting digital representations of time series data indicative of measurements of the system's operation at different time instances. The computer-implemented method may further include generating collocation points corresponding to solutions of the PDE that represent the nonlinear dynamics for a set of initial and boundary conditions on the state of the system's operation and constraints on the system's operation that evolve from the boundary conditions according to the PDE. The computer-implemented method may further include training the neural network using training data including the collected time series data and the collocation points to train the parameters of the nonlinear operator. The neural network includes an autoencoder architecture including an encoder and a decoder, where the encoder is configured to encode each instance of training data into a latent space, the nonlinear operator is configured to propagate the encoded instances of training data through the latent space using a transformation determined by parameters of the nonlinear operator, and the decoder is configured to decode the transformed encoded instances of training data to minimize a hybrid loss function that includes a data-driven loss between the neural network's decoding and the collected time-series data, and a physical information loss between the neural network's decoding and the solution of a PDE at the collocation points. In one embodiment, the nonlinear operator may be a reduced-order model. Such a ROM may be useful for accurately representing systems with nonlinear dynamics. Such a low-order nonlinear operator or predictor may be designed to match desired properties, such as linearity, reduced order, stability, nonlinearity, etc., while obeying the physical laws of the system.

[0016] In some embodiments, generating the parameterized ODE may be based on one or more model reduction techniques. The one or more model reduction techniques may include at least one of proper orthogonal decomposition (POD)-Galerkin projection or dynamic mode decomposition (DMD). To be suitable for control, ROM training methods need to find low-dimensional manifolds and dynamics that can both yield high-accuracy predictions and long-term stability. ROMs are generally projection-based, such as dynamic mode decomposition (DMD) and POD, which can transform the trajectories of a high-dimensional dynamic system into an appropriate and optimal low-dimensional subspace. One challenge with POD methods is their intrusiveness, i.e., the need for access to the solver code. To overcome this drawback, operator inference approaches utilize singular value decomposition (SVD)-based model reduction, employing lifting to fit the system's latent space dynamics data to polynomials, e.g., quadratic models. However, such polynomial models may have limited expressive power (e.g., up to quadratic in the case of lift-and-learn approaches), and custom-tuned SVD-based optimization techniques may be required.

[0017] In some embodiments, to overcome these challenges, an autoencoder-based reduced-order model can be used as a nonlinear ROM technique that can yield accurate and stable ROM. However, autoencoder-based ROM requires a dataset that densely covers the hypothetical infinite-dimensional phase portrait of the dynamical system. The large data demands severely limit the use of such models in physics applications where data acquisition can be expensive.

[0018] In some embodiments, the method may further include controlling the system by using a linear or nonlinear control law. Accordingly, some embodiments recognize that a model of the system can be represented by a nonlinear reduced-order model. For example, where a complete physics-based model of the system is typically captured by a PDE, the ROM can be represented by an ordinary differential equation (ODE). ODEs can represent the system's dynamics as a function of time, but with less accuracy than representing the dynamics using a PDE. Additionally or alternatively, some embodiments use a model of the system determined by data-driven adaptation in various model-based predictive control techniques, such as model predictive control (MPC). Such embodiments may enable leveraging MPC's ability to consider constraints in controlling the system. However, linearity may not adequately represent the complex dynamics of systems with multiple basins of attraction. In dynamic systems, a "basin of attraction" is a set of all starting points or initial conditions, usually close to each other, that reach the same final state, which may be called equilibrium, as the system evolves over time. Nonlinear model reduction methods, on the other hand, may use projection techniques to derive low-cost approximation models of nonlinear systems.

[0019] In one embodiment, the residual factors of the PDE are based on Lie operators. The method further includes performing an eigendecomposition on the Lie operators. The residual factors of such Lie operators can be used to learn the dynamics of either linear or nonlinear models in latent space.

[0020] In some embodiments, the digital representation of the time series data can be obtained using computational fluid dynamics (CFD) simulation or experimentation. CFD simulation and experimentation are high-fidelity calculations for obtaining the digital representation of the time series data. CFD simulation or experimentation allows for improved accuracy and speed of complex simulation scenarios, such as transonic or turbulent fluid flows, for various applications of systems, such as heating, ventilation, and air conditioning (HVAC) applications for describing airflow.

[0021] In some embodiments, the nonlinear operator is based on a reduced-order model. While the field of model reduction is mature for linear systems, reducing nonlinear models can be challenging. For example, a POD method can be used to reduce nonlinear models. To improve the computational efficiency of the resulting reduced model, the POD method can be combined with a sparse sampling method (also called "hyper-reduction"). For example, the sparse sampling method can include missing point estimation (MPE), empirical interpolation (EIM), discrete empirical interpolation (DEIM), Gappy POD, or Gauss-Newton with approximated tensors (GNAT). Other methods for nonlinear model reduction use a data-driven approach via dynamic mode decomposition (DMD) and operator inference. Recently, certain input-independent model reduction methods, such as balanced truncation and the iterative rational Krylov algorithm (IRKA), have been extended to second-order bilinear systems.

[0022] In one embodiment, the parameters of the nonlinear operator are determined based on a stochastic approach, which may assume that the measurement data of the system is random with a probability distribution that depends on the parameters of interest.

[0023] In another embodiment, the nonlinear operator is based on a continuous-time dynamical system. In some embodiments, the nonlinear operator can be approximated in latent space using data-driven approximation techniques. Data-driven approximation techniques can be generated using numerical or experimental snapshots.

[0024] In some embodiments, nonlinear operators can be approximated using deep learning techniques, which allow the original dynamics of the system to be embedded into significantly lower-order nonlinearities. Deep learning techniques for nonlinear approximation can be successful in long-term dynamic prediction and control of the system.

[0025] In some embodiments, the parameters of the nonlinear operator can be fine-tuned in real time based on a set of expected measurements and the output of the neural network. Once the ROM is trained, the neural network can be used to reconstruct the original dynamics of the system. Such ROM outputs can be projected to appropriate measurement outputs based on a measurement model. Furthermore, differences between actual measurements (such as a set of expected measurements) and the output of the neural network can be used to fine-tune the parameters of the ROM.

[0026] In some embodiments, the generation of the collocation points may be based on a subset of the set of initial and boundary conditions having a structure that reduces the complexity of solving the PDE and a function space of the system that satisfies the subset of initial and boundary conditions. In some embodiments, the structure of the subset of initial and boundary conditions includes at least one of a sinusoidal function, a harmonic function, a periodic function, or an exponential function. The collocation points may be samples extracted from a domain of the system's function space such that, in the case of a PDE, the collocation points also satisfy boundary conditions or other constraints associated with the system. Advantageously, the generation of the collocation points is computationally inexpensive compared to computing snapshots in a CFD calculation.

[0027] In some embodiments, estimations and control commands for controlling the operation of the system may be generated. In one or more embodiments, the generation of estimations and control commands for controlling the operation of the system is based on data-driven or optimization-based control and estimation techniques. Such techniques may be advantageous for controlling dynamic systems. For example, model-based control and estimation techniques enable a model-based design framework in which system dynamics and constraints may be directly considered.

[0028] In some embodiments, the generation of inferences and control commands for controlling the operation of the system is based on data-driven control and inference techniques, the goal of which is to design a control policy for the system from data and to control the system using the data-driven control policy.

[0029] Another embodiment discloses a training system for training a neural network to control the operation of a system having nonlinear dynamics represented by a partial differential equation (PDE). The neural network may include a nonlinear operator of the system's dynamics represented in a latent space by a parameterized ordinary differential equation (ODE) with parameters determined by training. The training system may include at least one processor and a memory having stored thereon instructions that, when executed by the at least one processor, cause the training system to collect digital representations of time series data indicative of instances of the system's function space and corresponding measurements of states of operation of the system at different time instances. The training system may further generate collocation points corresponding to a solution of the PDE that represents the nonlinear dynamics for a set of initial and boundary conditions on the states of operation of the system and constraints on the system's operation that evolve from the boundary conditions according to the PDE. The training system may further train the neural network using training data including the collected time series data and the collocation points to train the parameters of the nonlinear operator. The neural network has an autoencoder architecture including an encoder and a decoder, where the encoder is configured to encode each instance of the training data into a latent space, the nonlinear operator is configured to propagate the encoded instances of the training data into the latent space using a transformation determined by parameters of the nonlinear operator, and the decoder is configured to decode the transformed encoded instances of the training data to minimize a hybrid loss function including a data-driven loss between the decoding of the neural network and the collected time series data and a physical information loss between the decoding of the neural network and the solution of a PDE at the collocation points.

[0030] Yet another embodiment discloses a non-transitory computer-readable storage medium containing a processor-executable program for executing a method for controlling the operation of a system having nonlinear dynamics represented by a partial differential equation (PDE). The neural network may include a nonlinear operator of the system's dynamics represented in a latent space by a parameterized ordinary differential equation (ODE) with parameters determined by training. The method may include collecting digital representations of time series data indicative of measurements of the system's operation at different time instances. The method may further include generating collocation points corresponding to solutions of the PDE that represent the nonlinear dynamics for a set of initial and boundary conditions on the state of the system's operation and constraints on the system's operation that evolve from the boundary conditions according to the PDE. The method may further include training the neural network using training data including the collected time series data and the collocation points to train the parameters of the nonlinear operator. The neural network includes an autoencoder architecture including an encoder and a decoder, where the encoder is configured to encode each instance of the training data into a latent space, the nonlinear operator is configured to propagate the encoded instances of the training data into the latent space using a transformation determined by parameters of the nonlinear operator, and the decoder is configured to decode the transformed encoded instances of the training data to minimize a hybrid loss function including a data-driven loss between the decoding of the neural network and the collected time series data and a physical information loss between the decoding of the neural network and the solution of a PDE at the collocation points.

[0031] Nonlinearities (or nonlinear operators) in the latent space play an important role in developing non-intrusive ROMs. For example, if the original system may contain several regions of attraction, a finite-dimensional approximation of a linear operator in the latent space may not be able to find various equilibria of the system. Furthermore, linear operators that can approximate the Koopman operator may not be compressed in the latent space. The advantage of nonlinear operators in the latent space is that, on the one hand, an integrated model with several regions of attraction can be constructed, and, on the other hand, the nonlinearity can result in a ROM that may be more accurate for a given reduction order (or a lower-order model for a given accuracy).

[0032] In some embodiments, a nonlinear ROM can be generated. In other embodiments, a lift-and-learn approach can be utilized for nonlinear model reduction, enabled by data-driven learning of the reduced model through a structure-preserving lens of projection. A lift-and-learn approach typically has two key components: data-driven learning of the reduced model via operator inference, and variable transformation and lifting to reveal the system polynomial structure. However, such nonlinear model reduction can employ an inductive bias, for example, assuming the ROM model should be quadratic or polynomial, which may not be the most efficient scenario.

[0033] In this disclosure, a non-intrusive ROM generation is proposed that may not require access to the original finite element or finite volume solvers to determine the dynamics of the system.

[0034] The present disclosure is further described in the following detailed description with reference to several drawings, in which like reference numerals represent like parts throughout the several views of the drawings, shown as non-limiting examples of exemplary embodiments of the present disclosure. The drawings shown are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the presently disclosed embodiments. [Brief explanation of the drawings]

[0035] [Figure 1A] FIG. 1 is a block diagram of two stages for training a neural network in an offline stage to be used in an online stage to control the operation of a system, according to one embodiment of the present disclosure. [Figure 1B] FIG. 1 is a schematic diagram of an architecture of a nonlinear operator of a neural network, according to some embodiments of the present disclosure. [Figure 1C] FIG. 1 is a block diagram for fine-tuning parameters of a nonlinear operator in real time to train a neural network, according to some embodiments of the present disclosure. [Figure 1D] FIG. 1 illustrates a flowchart for generating co-location points according to an embodiment of the present disclosure. [Figure 2A] FIG. 1 is a schematic diagram of the principles used to control the operation of a system according to some embodiments of the present disclosure. [Figure 2B] FIG. 1 shows a flowchart illustrating an exemplary method for approximating a nonlinear operator, according to some embodiments of the present disclosure. [Figure 2C] FIG. 1 is a schematic diagram of a neural network autoencoder architecture according to some embodiments of the present disclosure. [Figure 3] FIG. 1 is a block diagram of an apparatus for controlling the operation of a system according to some embodiments of the present disclosure. [Figure 4] FIG. 1 illustrates a flowchart of principles for controlling the operation of a system according to some embodiments of the present disclosure. [Figure 5] FIG. 1 is a block diagram illustrating generation of a reduced-order model according to some embodiments of the present disclosure. [Figure 6] FIG. 1 is a schematic diagram of a neural network according to some embodiments of the present disclosure. [Figure 7A] FIG. 1 illustrates an input of a digital representation in an encoder of a neural network model, according to some embodiments of the present disclosure. [Figure 7B]FIG. 1 illustrates the propagation of an encoded digital representation into a latent space by a nonlinear operator of a neural network model, according to some embodiments of the present disclosure. [Figure 7C] FIG. 1 illustrates the decoding of a transformed encoded digital representation by a neural network model decoder, according to some embodiments of the present disclosure. [Figure 8] FIG. 1 is an exemplary diagram of a real-time implementation of an apparatus for controlling the operation of an air conditioning system, according to some embodiments of the present disclosure. [Figure 9] 1 is an exemplary diagram of a real-time implementation of an apparatus for reconstructing gas distribution from a camera, according to some embodiments of the present disclosure. FIG. [Figure 10] 1 shows a flowchart illustrating a method for training a neural network according to some embodiments of the present disclosure.

[0036] In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, devices and methods are shown in block diagram form solely to avoid obscuring the disclosure. It is contemplated that various changes may be made to the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter as set forth in the appended claims.

[0037] As used in this specification and claims, the terms "for example," "for example," and "such as," as well as the verbs "comprise," "have," and "include," and each of their other verb forms, when used in conjunction with a list of one or more components or other items, should be construed as open-ended, meaning that the list should not be considered as excluding other additional components or items. The term "based on" means based at least in part on. Furthermore, it should be understood that the style and terminology used herein are for purposes of description and should not be regarded as limiting. Any headings used herein are for convenience only and have no legal or limiting effect.

[0038] Specific details are provided in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements of the disclosed subject matter may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail so as not to obscure the embodiments. Furthermore, like reference numbers and names in the various drawings indicate like elements.

[0039] In describing embodiments of the disclosure, the following definitions apply throughout this disclosure: "Control system" or "controller" may refer to a device or set of devices for managing, directing, supervising, or regulating the behavior of other devices or systems. A control system may be implemented in either software or hardware and may include one or more modules. A control system that includes a feedback loop may be implemented using a microprocessor. A control system may be an embedded system.

[0040] "Air conditioning system" or heating, ventilation, and air conditioning (HVAC) system may refer to a system that uses a vapor compression cycle to move a refrigerant through the system's components based on the principles of thermodynamics, fluid mechanics, and / or heat transfer. Air conditioning systems span a broad range of systems, from systems that provide only outside air to building occupants, to systems that control only the building's temperature, to systems that control both temperature and humidity.

[0041] A "Central Processing Unit (CPU)" or "Processor" may refer to a computer or a component of a computer that reads and executes software instructions. Further, a processor may be referred to as "at least one processor" or "one or more processors."

[0042] 1A shows a two-stage block diagram 100A for training a neural network model in an offline stage for use in an online stage to control the operation of a system, according to one embodiment of the present disclosure. Block diagram 100A may include two stages, such as offline stage 102 and online stage 104. Block diagram 100A depicts the control and estimation of a large-scale system, such as a system with nonlinear dynamics described by partial differential equations (PDEs), using a two-stage apparatus, i.e., offline stage 102 and online stage 104.

[0043] The offline stage 102 (or stage I) may include a neural network 106. The neural network 106 has an autoencoder architecture. The neural network 106 includes an autoencoder 108, which includes an encoder and a decoder. The neural network 106 further includes a nonlinear operator 110 of the system's dynamics, represented in a latent space by a parameterized ordinary differential equation (ODE) with parameters determined by training the neural network 106. The offline stage 102 may further include a computational fluid dynamics (CFD) simulation or experiment module 112, a differential equation 114 for representing the system's nonlinear dynamics, a digital representation 116 of time-series data, and collocation points 118. The time-series data may represent the temporal evolution of a vector field, such as a wind field, that evolves over time due to initial and boundary conditions or external forcing. Alternatively, the time-series data may represent the temporal evolution of the spatial distribution of the density of an entity, such as a liquid, a crowd, a flock of birds, or a school of fish. The online stage 104 (or stage II) may include a data assimilation module 120 and a control unit 122 for controlling the system.

[0044] In the offline stage 102, offline tasks for system control and estimation may be performed to derive a nonlinear operator 110 (or nonlinear predictor). In some embodiments, the nonlinear operator 110 may be based on a reduced-order model (ROM). For example, the ROM may be represented by a neural ordinary differential equation (NODE) ​​operator. Such a ROM may be referred to as a latent space model. Generally, it is desirable for the dimensions of the latent space to be significantly smaller than the input. Details of the architecture of the nonlinear operator 110 for representing a ROM in latent space are further provided, for example, in FIG. 1B.

[0045] Typically, data for the development of the latent space model (represented by the nonlinear operator 110) may be generated by performing high-fidelity CFD simulations and experiments using the CFD simulation or experimentation module 112.

[0046] Generally, CFD refers to a branch of fluid mechanics that utilizes numerical analysis and data structures to analyze and solve problems involving fluid flow. For example, computers can be used to perform the calculations necessary to simulate the freestream flow of fluids and the interaction of fluids (such as liquids and gases) with surfaces defined by boundary conditions. Furthermore, various software has been designed to improve the accuracy and speed of complex simulation scenarios associated with transonic or turbulent flows that may occur in system applications, such as HVAC applications for describing airflow within a system. Furthermore, initial validation of such software is typically performed using equipment such as a wind tunnel. Additionally, previously performed analytical or empirical analyses of specific problems related to the airflow associated with a system can be used for comparison in CFD simulations.

[0047] In some embodiments, the digital representation 116 of the time series data is obtained using a CFD simulation or experiment module 112. The CFD simulation or experiment module 112 may output a data set, such as the digital representation 116 of the time series data, which may be utilized to develop a latent space model (or nonlinear operator 110). The nonlinear operator 110 may be constructed for several trajectories generated by the CFD simulation. In an exemplary scenario, an HVAC system may be installed in a room. The room may be in various scenarios, such as with open windows and closed doors. CFD simulations may be performed for the room with closed windows, open windows, one occupant, two occupants, or multiple occupants. In such cases, the autoencoder 108 may be valid for all such conditions associated with the room. Tasks such as the CFD simulation may be performed in the offline stage 102.

[0048] In some embodiments, collocation points 118 associated with the function space of the system may be generated based on the PDE, the digital representation 116 of the time series data, and the nonlinearly transformed encoded digital representation (e.g., the output of the nonlinear operator 110). The neural network 106 may be trained based on the generated collocation points 118. Specifically, the neural network 106 may be trained based on the difference between the predicted value of the nonlinear operator 110 and a dataset, such as the digital representation 116 of the time series data plus a physical information portion, i.e., based on the differential equations 114 for representing the nonlinear dynamics of the system that generates the collocation points 118.

[0049] Additionally, the output of the neural network 106 may be utilized by a data assimilation module 120 of the online stage 104. The data assimilation module 120 may output, for example, a reconstructed model of temperature and velocity in a region, such as a room, associated with a system, such as an HVAC system. The reconstructed model of temperature and velocity may be utilized by a control unit 122. The control unit 122 may generate control commands to control the operation (e.g., airflow) of the system, such as an HVAC system.

[0050] The data assimilation module 120 utilizes the process of data assimilation, which refers to the assimilation of accurate information from sensors with potentially inaccurate model information. For example, sensors may be installed in a room to monitor specific sensory data. Examples of sensory data installed in a room for HVAC applications include, but are not limited to, thermocouple readings, thermal camera measurements, speed sensor data, and humidity sensor data. The information from the sensors may be assimilated by the data assimilation module 120.

[0051] Typically, data assimilation refers to a mathematical field that may attempt to optimally combine predictions (usually in the form of a numerical model) with observations associated with a system. Data assimilation may be used for a variety of purposes, such as to find an optimal state estimate of a system, to determine initial conditions for a numerical predictive model of a system, to interpolate sparse observation data using knowledge of the observed system, and to identify numerical parameters of a model from observed experimental data. Depending on the purpose, different solution methods may be used.

[0052] It should be noted that the offline stage 102 and the online stage 104 are examples of developing a simplified robust neural network 106, which can then be used by the control unit 122 to estimate and control a system with nonlinear dynamics. Typically, system estimation and control involves estimating values ​​of parameters of the nonlinear operator 110 based on measured empirical data, which may have a random component. The parameters describe the underlying physical setting, such that the parameter values ​​can affect the distribution of the measured data. Furthermore, an estimator, such as the control unit 122, attempts to approximate the unknown parameters using the measured data. Generally, two approaches to approximation are possible. According to one embodiment, the parameters of the nonlinear operator 110 for approximation can be determined based on a stochastic approach. A stochastic approach may assume that the measured data of the system is random, with a probability distribution that depends on the parameters of interest. In some embodiments, the parameters of the nonlinear operator 110 for approximation can be determined based on a set membership approach, which may assume that the measured data vector belongs to a set that depends on the parameter vector.

[0053] In some embodiments, poor out-of-distribution performance can pose severe challenges when utilizing the ROM of the nonlinear operator 110, especially when it is difficult for a user to obtain data that covers the entire distribution of possible data inputs. In an exemplary scenario, in an HVAC application, data may be collected from a room with two windows, but not from one room for each possible number of windows. In such thermal fluid applications, experiments may be performed for specific parameters, but conducting experiments for all parameters may be difficult. In such situations, it may be necessary to embed knowledge of the physics into a model of the system to improve extrapolation performance.

[0054] For example, some embodiments use symbolic regression to determine the underlying structure of a nonlinear dynamic system from data. In other embodiments, symbolic regression can be used with graph neural networks (GNNs) to extract explicit physical relationships in a system, facilitating a sparse latent representation. Symbolic representations extracted from GNNs generally generalize better to out-of-distribution data than GNNs. However, symbolic regression also suffers from excessive computational cost and can be prone to overfitting. Typically, symbolic regression is a type of regression analysis that explores a space of mathematical expressions to find a model that best fits a given dataset in terms of both accuracy and simplicity. No specific model is provided as a starting point for symbolic regression. Instead, initial expressions are formed by randomly combining mathematical constructs such as mathematical operators, analytical functions, constants, and state variables.

[0055] Furthermore, GNNs are a class of artificial neural networks for processing data that can be represented as graphs. In general, certain existing neural network architectures can be interpreted as GNNs that operate on well-defined graphs. Convolutional neural networks, in the context of computer vision, can be considered as GNNs applied to graphs structured as pixel grids. Transformers, in the context of natural language processing, can be considered as GNNs applied to complete graphs with nodes representing words in a sentence.

[0056] It should be noted that incorporating knowledge of the physical information pieces or differential equations associated with the system may reduce the need for large training data sets, such as the digital representation 116 of the time series data, to identify the nonlinear operator 110. Furthermore, because the neural network 106 performs operator learning, it becomes possible for the neural network 106 to predict beyond the training horizon, which may be further used for compressed sensing, estimation, and control of the system.

[0057] Another example of incorporating physics into ROM is the use of parametric models in latent space, for example, through the use of sparse identification of nonlinear dynamics (SINDy), which relies on the fact that most dynamical systems of interest have relatively few nonlinear dynamical terms among the family of possible terms (i.e., polynomial nonlinearities). Such methods may utilize sparsity promotion techniques to find models that automatically balance sparsity in some terms with model accuracy. For example, some embodiments may use a chain-rule-based loss that links latent space derivatives to observable space derivatives for co-training the autoencoder 108 and the latent dynamics of the nonlinear operator 110. However, such chain-rule-based losses may be sensitive to noise in the data, especially when time derivatives with finite differences need to be evaluated. Collocation-based enforcement of physics, i.e., projecting candidate functions into the governing equations to enforce chain rules instead of finite differences, may address such numerical challenges. Furthermore, neural ODEs (NODEs) can be utilized to fit any nonlinear model (e.g., network) as a latent space dynamics model (or nonlinear operator 110), greatly expanding the set of models for latent dynamics that can be efficiently trained.

[0058] The computer-implemented method of the present disclosure uses an autoencoder to perform nonlinear model reduction with NODEs in latent space to model complex nonlinear dynamics. Such methods are used to reduce data demands and improve the overall predictive stability of the system under demanding training conditions. To achieve this goal, a numerical collocation method is used to embed knowledge from known governing equations into the latent space dynamics of the ROM, as illustrated in Figure 1B.

[0059] As illustrated in FIG. 1B, the nonlinear operators 110 of the neural network 106 may be represented by NODEs.

[0060] FIG. 1B illustrates a schematic diagram 100B of an architecture of a nonlinear operator 110 represented by a NODE, according to some embodiments of the present disclosure. The schematic diagram 100B illustrates a nonlinear ROM for a latent space. The schematic diagram 100B may include a data-driven loss model 124, a physical information loss model 126, and a hybrid model 128. The data-driven loss model 124 is a data-driven loss (L data ) 130. The physical information loss model 126 represents the physical information loss (L physics ) 132. The hybrid model 128 shows the hybrid loss (L hybrid )134.

[0061] A method for training the neural network 106 may include collecting digital representations of time series data 116. The digital representations of time series data 116 represent instances of a system's function space and corresponding measurements of the system's state of operation at different time instances. Collection of the digital representations of time series data 116 by a CFD simulation or experiment module 112 is further described, for example, in FIG. 1A.

[0062] The method may further include generating collocation points 118 corresponding to solutions of a PDE that represent the nonlinear dynamics for a set of initial and boundary conditions at states of operation of the system and constraints on the system's operation that evolve from the boundary conditions according to the PDE. In some embodiments, the generation of collocation points 118 may be based on a subset of the set of initial and boundary conditions having a structure that reduces the complexity of solving the PDE. The generation of collocation points 118 may further be based on a function space of the system that satisfies the subset of initial and boundary conditions. Details of the generation of collocation points 118 are further provided, for example, in FIG. 1D .

[0063]

number

[0064]

Number

[0065]

Number

[0066]

Number

number

[0072]

number

[0073]

number

[0074]

number

[0075] In particular, NODE is a nonlinear operator (h θ )110. NODE is a neural network model that generalizes standard layer-by-layer propagation to continuous depth models. Starting from the observation that forward propagation in neural networks is equivalent to one step of discretization of an ODE, models can be constructed and effectively trained via ODEs. In addition to providing a novel family of architectures, particularly for reversible density models and continuous time series, NODE may further provide improved memory efficiency in supervised learning tasks.

[0076] In the numerical analysis of ODEs, the Runge-Kutta method is a family of implicit and explicit iterative methods, including the Euler method, used in time discretization for the approximate solution of systems of nonlinear equations such as ODEs. In mathematics and computer science, the Euler method (also known as the forward Euler method) is a first-order numerical method for solving ODEs with given initial values. The Euler method is the basic explicit method for the numerical integration of ODEs and is the simplest Runge-Kutta method.

[0077] Furthermore, residual networks (ResNets) can be utilized to train deep networks. ResNets can be used to approximate ODEs and provide a training approach for the neural network 106. Given the network parameters, any residual network can be considered an explicit Euler discretization of a particular ODE, and any numerical ODE solver can be used to evaluate the output layer of the neural network 106. Adjoint methods can further be used to efficiently backpropagate (and thus train) the neural network 106. Such methods can further be used to train time-continuous normalized flows. In such cases, using a continuous formulation can avoid the computation of the Jacobian determinant, which is one of the main bottlenecks in the normalized flows of systems. Furthermore, NODEs can be used to model latent dynamics in time series modeling, easily addressing irregularly sampled data.

[0078] The training task for learning the autoencoder 108 and neural network 106, which together comprise the ROM, the digital representation 116 of the time series data, and the collocation points 118, is to use a data-driven loss (L data )130 and physical information loss (L physics ) 132. The nonlinear operator 110 is trained and h θ It is represented as a NODE indicated by

[0079]

number

[0080] We propose colocation points 118 that differ from traditional colocation points in terms of sample space. For example, instead of sampling from the spatiotemporal domain as is traditional, our colocation points 118 are sampled from an appropriate function space. The selection of colocation points 118 is further explained in FIG. 1D.

[0081] FIG. 1C illustrates the fine tuning of the parameters of the neural network 106 to minimize the hybrid loss 134.

[0082] 1C shows a block diagram 100C for fine-tuning the parameters of a nonlinear operator 110 in real time to train a neural network 106. The block diagram 100C may include a digital representation 116 of time series data, collocation points 118, the neural network 106, and a data assimilation module 120. The block diagram 100C may further include a measurement 136 based on the output of the neural network 106 and a measurement 138 based on the output of the system.

[0083]

number

[0084] Note that a measurement model shows the relationship between sensor data outputs and the state variables of the system. The simplest model is linear, and such a relationship can be represented by a measurement matrix. The state trajectory can be measured during online functioning of the system. For example, a device (or training system) may include an input interface configured to acquire measurement data, such as system speed and temperature data, from sensors located within an HVAC system. In such a case, the measurement matrix has a size corresponding to the number of sensors multiplied by the number of original dynamics, n, and the entries of the measurement matrix are "0" where no sensors are located and "1" where sensors are located in the physical domain of the system.

[0085] The generation of collocation points 118 utilized in training neural network 106 is further described in FIG. 1D.

[0086] 1D shows a flowchart 100D for generating co-location points 118 according to one embodiment of the present disclosure. Flowchart 100D may include steps 140, 142, 144, 146, 148, 150, and 152. Fewer or more steps may be provided. Additionally, one or more steps may be combined or separated without departing from the scope of the disclosure.

[0087] In general, naive selection of collocation points 118 (or samples) can lead to inaccurate latent space dynamics, even when the dimensions of the latent space are large, i.e., r > n. In some embodiments, greedy selection of collocation points 118 or samples can obtain significantly fewer degrees of freedom than full-order simulation while obtaining greater accuracy than naive uniform sampling. Furthermore, in some embodiments related to hyper-reduction of a model, stochastic sampling can eliminate errors due to uniform sampling. To better control hyper-reduction errors, a greedy algorithm for selecting collocation points 118 is proposed, which can augment collocation points 118 set to meet the system's target residual.

[0088] In step 140, the method may include obtaining differential equations 114 for a governing model of the system. Details of obtaining differential equations 114 are further provided in, for example, FIG. 1A.

[0089] In step 142, the method may include randomly selecting collocation points 118. The selected collocation points 118 are utilized for physical information loss 132. To robustly select the collocation points 118 used in physical information loss 132, a balance between computational speed and accuracy may be necessary. In one example, “N” samples may be used, with computational speed scaling approximately linearly with the number of samples used. To maximize computational speed, as few samples as possible may be selected. Given a target accuracy, a greedy algorithm may select the fewest number of collocation points 118 from the appropriate function space of the full-order PDE solution to achieve the target accuracy.

[0090] In step 144, the neural network 106 may be executed, and a residual of physical information loss 132 may be calculated for each of the selected collocation points 118. Based on the randomly selected collocation points 118, the neural network 106 may be executed. Based on the output of the neural network 106, a residual of physical information loss 132 may be calculated for each of the selected collocation points 118. In every iteration, the greedy algorithm may add one collocation point to the sample set of N″ samples, reducing the error of the latent space dynamics associated with the physical information loss 132.

[0091] In step 146, the method may include checking a convergence criterion. For example, it may be necessary to check the convergence of the error of the latent space dynamics associated with the physics information loss 132. The convergence criterion may be defined as the average of the individual residuals of the physics information loss 132.

[0092] In step 148, the method may include terminating the sampling of the co-location points 118. For example, the selection of random co-location points 118 may be terminated when a desired error convergence is achieved. In one example, the sampling of the co-location points 118 may be terminated when the error convergence falls below a predetermined threshold error.

[0093] In step 150, the method may include sampling the collocation points 118. Sampling of the collocation points 118 may continue until a desired error convergence is achieved, for example, until the error convergence exceeds a predetermined threshold error. In such a case, the greedy algorithm loops over the “Q” collocation points 118 with the largest individual residuals of physical information loss 132.

[0094] In step 152, the method may include adding each sample of the "Q" collocation points 118 to the sample set. Based on a determination that the desired convergence has not been achieved, the "Q" collocation points 118 are successively added to the sample set. The greedy algorithm is repeated until the target accuracy is met. For example, the loop continues from step 144 to step 152 until the target accuracy is met.

[0095] It should be noted that compared to conventional naive uniform sampling approaches, the proposed greedy algorithm approach yields significantly more accurate results. Such sampling can be performed either before or after the calculation of the trajectories obtained from the CFD simulation or experimental module 112, which are then required for the calculation of the data-driven losses 130.

[0096] Control of the system based on the output of the neural network 106 is further described in FIG. 2A , which shows a schematic diagram 200A of principles used to control the operation of a system, according to some embodiments of the present disclosure. Schematic diagram 200A depicts a controller 202 and a system 204. System 204 may be a system having nonlinear dynamics. Controller 202 may include a nonlinear operator 110 and a control unit 206 in communication with nonlinear operator 110. Control unit 206 is similar to control unit 122 of FIG. 1A .

[0097] The controller 202 may be configured to control continuously operating dynamic systems, such as systems 204 in engineering processes and machines. Hereinafter, the terms “controller” and “device” may be used interchangeably and have the same meaning. Hereinafter, the terms “continuously operating dynamic system” and “system” may be used interchangeably and have the same meaning. Examples of systems 204 include, but are not limited to, HVAC systems, light detection and ranging (LIDAR) systems, condensing units, production lines, self-tuning machines, smart grids, automotive engines, robots, numerically controlled machining, motors, satellites, generators, and transportation networks. In some embodiments, the controller 202 may be configured to generate estimates and control commands for controlling the operation of the system. For example, the controller 202 may develop control policies, such as estimates and control commands, to control the system 204 using control actions in an optimal manner without delay or overshoot in the system 204 and ensure control stability.

[0098] In some embodiments, the generation of estimations and control commands for controlling the operation of the system may be based on model-based control and estimation techniques. For example, the control unit 206 may be configured to generate control commands for controlling the system 204 based on at least one of model-based control and estimation techniques or optimization-based control and estimation techniques. For example, the optimization-based control and estimation techniques may be model predictive control (MPC) techniques.

[0099] Model-based control and estimation techniques can be advantageous for controlling dynamic systems such as system 204. For example, MPC techniques can enable a model-based design framework in which the dynamics and constraints of system 204 can be directly considered. MPC techniques can develop control commands for controlling system 204 based on a latent space model or a model of nonlinear operators 110. The nonlinear operators 110 of system 204 refer to the dynamics of system 204 described using nonlinear differential equations.

[0100] In some embodiments, control unit 206 may be configured to generate estimations and control commands for controlling system 204 based on data-driven control and estimation techniques. Data-driven control and estimation techniques may utilize operational data generated by system 204 to develop feedback control policies that stabilize system 204. For example, states of system 204 measured during operation of system 204 may be provided as feedback for controlling system 204.

[0101]

number

[0102] In mathematics and computer algebra, automatic differentiation (AD) (also known as algorithmic differentiation, computational differentiation, autodifferentiation, or simply autodiff) is a set of techniques for evaluating the derivative of a function specified by a computer program. AD ​​exploits the fact that any computer program, no matter how complex, performs a sequence of elementary arithmetic operations (such as addition, subtraction, multiplication, and division) and elementary functions (such as exponentials, logarithms, sines, and cosines). By repeatedly applying the chain rule to such operations, derivatives of any order can be calculated automatically, up to working precision, using at most a small constant number of arithmetic operations than the original computer program.

[0103] Typically, the use of operational data to design control policies or control commands is referred to as data-driven control and estimation techniques. Data-driven control and estimation techniques may be utilized to design control policies from data, which may then be used to control the system 204. Furthermore, in contrast to such data-driven control and estimation techniques, some embodiments may use operational data to design models, such as the nonlinear operator 110. Data-driven models, such as the nonlinear operator 110, may be used to control the system 204 using various model-based control methods. Furthermore, data-driven control and estimation techniques may be utilized to determine an actual model of the system 204 from the data, i.e., to determine a model that may be used to estimate the behavior of the system 204 having nonlinear dynamics. In one example, the model of the system 204 may be determined from data that may capture the dynamics of the system 204 using differential equations. Furthermore, a model with physics-based PDE model accuracy may be learned from the operational data.

[0104] Furthermore, to simplify the computations for model generation, ordinary differential equations (ODEs) of the nonlinear operator 110 may be formulated to describe the dynamics of the system 204. In some embodiments, the parameterized ODEs may be generated based on one or more model reduction techniques. For example, the one or more model reduction techniques may include at least one of a proper orthogonal decomposition (POD)-Galerkin projection method or a dynamic mode decomposition (DMD) method. Furthermore, the ODE may be part of a PDE, for example, describing boundary conditions. However, in some embodiments, the ODE may not reproduce the actual dynamics of the system 204 (i.e., the dynamics described by the PDE) in the case of uncertainty conditions. Examples of uncertainty conditions may be when the boundary conditions of the PDE may change over time or when one of the coefficients included in the PDE may change.

[0105] An exemplary method for approximating the nonlinear operator 110 is further described in FIG. 2B.

[0106] FIG. 2B shows a flowchart 200B illustrating an exemplary method for approximating the nonlinear operator 110 according to some embodiments of the present disclosure. Flowchart 200B may include steps 208, 210, 212, and 214. Fewer or more steps may be provided. Additionally, one or more steps may be combined or separated without departing from the scope of the disclosure. Flowchart 200B illustrates an example of a data-driven ROM using POD and Galerkin projection. In such cases, POD ROM derivation requires snapshots, which may be state parameter values ​​over a period of time. Furthermore, finite volume or finite element methods may be used to solve the original PDE or large-scale system to find the snapshots. The basis functions may be described as a singular value decomposition (SVD) of the snapshot matrix.

[0107] In step 208, a finite volume model or a finite element model may be obtained. "Finite volume" may refer to a small volume that surrounds each node point on a mesh. The finite volume method (FVM) is a method for expressing and evaluating PDEs in the form of algebraic equations. Furthermore, the finite element method (FEM) may be used to numerically solve differential equations that arise in engineering and mathematical modeling.

[0108] In step 210, snapshots can be determined based on a finite volume model or a finite element model. In finite volume methods, the divergence theorem is used to convert volume integrals in partial differential equations, which may contain divergence terms, into surface integrals. These terms are then evaluated as fluxes at the surface of each finite volume. Such methods can be conservative because the flux entering a given volume is identical to the flux leaving an adjacent volume. Another advantage of FVM is that it is easily formulated to allow for unstructured meshes. Such methods are used in many computational fluid dynamics packages.

[0109] Furthermore, typical target problem domains include the traditional fields of structural analysis, heat transfer, fluid flow, mass transport, and electromagnetic potential. FEM is a general numerical method for solving partial differential equations (i.e., some boundary value problems) in two or three spatial variables. To solve the problem, FEM subdivides a large system into smaller, simpler parts called finite elements. These finite elements can be achieved by a specific spatial discretization in the spatial dimensions, which is realized by constructing a mesh of objects with a finite number of points, i.e., the numerical domain of the solution. The finite element method formulation of a boundary value problem ultimately results in a system of algebraic equations. This method approximates an unknown function over the domain. The simple equations that model these finite elements are then assembled into a larger system of equations that models the entire problem. FEM then approximates the solution by minimizing the associated error function via the calculus of variations.

[0110] In step 212, basis functions may be applied to the snapshots. In linear algebra, SVD is the factorization of a real or complex matrix. It generalizes the eigendecomposition of a square normal matrix with an orthonormal eigenbasis to any matrix. Specifically, the SVD of an m×n complex matrix M is M=UΣV * where U is an m×m complex unitary matrix, Σ is an m×n rectangular diagonal matrix with non-negative real numbers on the diagonal, V is an n×n complex unitary matrix, and V * is the conjugate transpose of V. Such a decomposition exists for any complex matrix. If M is real, U and V can be guaranteed to be real orthogonal matrices, and in this context SVD is often denoted.

[0111] In step 214, a reduced-order model can be obtained. For example, the nonlinear operator 110 can be approximated to obtain the reduced-order model. If the governing field equations are known, Galerkin methods can be used to derive a system of ordinary differential equations for evolving the time-dependent amplitudes. The tools of proper orthogonal decomposition (POD) and Galerkin projection provide systematic methods for generating reduced-order models from data. The central idea of ​​POD is to determine a nested family of subspaces of increasing (finite) dimension that optimally spans the data, in the sense that the error of the projection onto each subspace is minimized. Galerkin projection then determines the dynamics on each subspace by orthogonal projection of the governing equations.

[0112]

number

[0113] However, ROM solutions can lead to unstable solutions that do not reproduce the physical properties of the original PDE model, which has a viscosity term that always stabilizes the solution, i.e., bounded over bounded-time support (e.g., the solution may diverge on finite-time support). Furthermore, POD-based model reduction methods constrain states to evolve in a linear subspace (linear trial subspace), which imposes fundamental limitations on the efficiency and accuracy of the resulting ROM. Such linear trial subspaces also exist in other model reduction methods, such as balanced truncation, rational interpolation, and reduction-based methods. To address these limitations, some embodiments of the present disclosure propose a data-driven, non-intrusive model reduction framework that uses autoencoder network methods to address the linear trial subspace problem. Such deep learning-based models project the original high-dimensional dynamical system into a nonlinear subspace and predict the nonlinear dynamics.

[0114] An example of the use of deep learning techniques (or neural network models 106) to approximate the nonlinear operator 110 for model reduction is further provided in FIG. 2C.

[0115] 2C shows a schematic diagram 200C of a neural network autoencoder architecture according to some embodiments of the present disclosure. For example, a deep neural network model may be utilized to learn the nonlinear basis and the nonlinear operator 110 using snapshot data. The schematic diagram 200C includes an autoencoder 108. The autoencoder 108 includes an encoder 216, a decoder 218, and a nonlinear operator 220. The nonlinear operator 220 may be identical to the nonlinear operator 110 of FIG. 1A. The schematic diagram 200C further includes a nonlinear operator 220 and a nonlinear operator 222.

[0116]

number

[0117] Furthermore, within the latent space of the autoencoder 108, such as the nonlinear operator 220, the dynamics of the system 204 are constrained to be represented by NODEs.

[0118] Typically, the autoencoder 108 can be trained in several ways: Usually, the training dataset X is arranged as a three-dimensional (3D) tensor, whose dimensions are the number of sequences (with different initial states), the number of snapshots, and the dimensionality of the measurements, respectively.

[0119] A block diagram of an apparatus for controlling the operation of system 204 is further illustrated in FIG.

[0120] 3 shows a block diagram 300 of an apparatus 302 for controlling operation of the system 204 according to some embodiments of the present disclosure. The block diagram 300 may include the apparatus 302. The apparatus 302 may include an input interface 304, a processor 306, a memory 308, and storage 310. The storage 310 may further include a model 310a, a controller 310b, an update module 310c, and a control command module 310d. The apparatus 302 may further include a network interface controller 312 and an output interface 314. The block diagram 300 may further include a network 316, a state trajectory 318, and an actuator 320 associated with the system 204.

[0121] The device 302 includes an input interface 304 and an output interface 314 for connecting the device 302 to other systems and devices. In some embodiments, the device 302 may include multiple input interfaces and multiple output interfaces. The input interface 304 is configured to receive a state trajectory 318 of the system 204. The input interface 304 includes a network interface controller (NIC) 312 adapted to connect the device 302 to a network 316 through a bus. Furthermore, the device 302 receives the state trajectory 318 of the system 204 via the network 316, either wirelessly or wired.

[0122] The state trajectory 318 may be a plurality of states of the system 204 that define the actual behavior of the dynamics of the system 204. For example, the state trajectory 318 may serve as a reference continuous state space for controlling the system 204. In some embodiments, the state trajectory 318 may be received from real-time measurements of some of the states of the system 204. In some other embodiments, the state trajectory 318 may be simulated using a PDE that describes the dynamics of the system 204. In some embodiments, the shape of the received state trajectory 318 may be determined as a function of time. The shape of the state trajectory 318 may represent the actual pattern of behavior of the system 204.

[0123] The device 302 further includes a memory 308 for storing instructions executable by the processor 306. The processor 306 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 308 may include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable memory system. The processor 306 is connected to one or more input / output devices via a bus. Furthermore, the stored instructions implement methods for controlling the operation of the system 204. The memory 308 may be further expanded to include storage 310. The storage 310 may be configured to store 310 a model 310 a, a controller 310 b, an update module 310 c, and a control command module 310 d.

[0124] The controller 310b may be configured to store instructions in the storage 310 for execution by the processor 306 to execute one or more modules. Additionally, the controller 310b manages each module of the storage 310 to control the system 204.

[0125] Additionally, in some embodiments, the update module 310c may be configured to update gains associated with the model of the system 204. The gains may be determined by reducing the error between the state of the system 204 estimated using the model 310a and the actual state of the system 204. In some embodiments, the actual state of the system 204 may be a measured state. In some other embodiments, the actual state of the system 204 may be a state estimated using a PDE that describes the dynamics of the system 204. In some embodiments, the update module 310c may update the gains using extremum searching. In some other embodiments, the update module 310c may update the gains using a Gaussian process-based optimization technique.

[0126] The control command module 310d may be configured to determine control commands based on the model 310a. The control command module 310d may control the operation of the system 204. In some embodiments, the operation of the system 204 may be subject to constraints. Furthermore, the control command module 310d determines the control commands using predictive model-based control techniques while enforcing the constraints. The constraints include state constraints in the continuous state space of the system 204 and control input constraints in the continuous control input space of the system 204.

[0127] The output interface 314 is configured to send control commands to actuators 322 of the system 204 to control the operation of the system 204. Some examples of the output interface 314 may include a control interface that sends the control commands to control the system 204.

[0128] The control of system 204 is further explained in FIG.

[0129] 4 shows a flowchart 400 of principles for controlling the operation of the system 204, according to some embodiments of the present disclosure. The flowchart 400 may include steps 402, 404, and 406.

[0130] In some embodiments, the system 204 may be modeled from the laws of physics. For example, the dynamics of the system 204 may be represented by mathematical equations using the laws of physics.

[0131]

number

[0132]

number

[0133]

number

[0134] In some embodiments, such abstract dynamics can be obtained from the numerical discretization of nonlinear partial differential equations (PDEs), which typically require many n state dimensions.

[0135] In some embodiments, a physics-based high-dimensional model of the system 204 needs to be solved to control the operation of the system 204 in real time. For example, in the case of an HVAC system, the Boussinesq equation needs to be solved to control the airflow dynamics and temperature in a room. In some embodiments, the physics-based high-dimensional model of the system 204 includes a large number of equations and variables that may be complex to solve. For example, solving the physics-based high-dimensional model in real time requires more computing power. Therefore, the physics-based high-dimensional model of the system 204 may be simplified.

[0136] In step 404, apparatus 302 is provided to generate a reduced-order model to reproduce the dynamics of system 204 such that apparatus 302 efficiently controls system 204. In some embodiments, apparatus 302 may simplify a high-order physics-based model using model reduction techniques to generate the reduced-order model. In some embodiments, the model reduction techniques reduce the dimensionality of the high-order physics-based model (e.g., variables of a PDE) such that the reduced-order model can be used in real time to predict and control system 204. Generating a reduced-order model for controlling system 204 is further described in detail with reference to FIG. 5 . In step 406, apparatus 302 predicts and controls system 204 using the reduced-order model in real time.

[0137] The generation of a reduced order model such as nonlinear operator 110 is further described in FIG.

[0138] 5 shows a block diagram 500 illustrating the generation of a reduced-order model according to some embodiments of the present disclosure. The nonlinear operator 110 is a reduced-order model. The block diagram 500 depicts an architecture including a digital representation 116 of time-series data and a neural network 106. The autoencoder 108 of the neural network 106 includes an encoder 216, a decoder 218, and a nonlinear operator 220. The block diagram 500 further depicts an output 502 of the neural network 106.

[0139] Snapshots of CFD simulations or experiments are the data required for an autoencoder, such as the neural network model autoencoder 108 depicted in Figure 6. The latent space is governed by a nonlinear ODE that is trained based on both the snapshots of data and model information using a DSC equation, such as Equation 14.

[0140] Furthermore, for a given time-dependent differential equation (e.g., an ODE or PDE), there may be a set of feasible initial conditions. Some embodiments define feasible initial conditions as those that can be classified into the domain of the system dynamics f.

[0141] Typically, the domain of a function is the set of inputs that the function accepts. More precisely, given a function f:X → Y, the domain of f is X. The domain can also be part of the function's definition rather than a property of the function. In such cases, X and Y are both subsets of R, and the function f can be graphed in a Cartesian coordinate system. In such cases, the domain is represented on the x-axis of the graph as the projection of the function onto the x-axis of the graph.

[0142] In some embodiments, the generation of collocation points 118 is based on a subset of the set of initial and boundary conditions that has a structure that reduces the complexity of solving the PDE, and a function space of the system that satisfies the subset of initial and boundary conditions. The collocation points 118 may be samples extracted from the domain of the system dynamics f such that, in the case of the PDE, the collocation points 118 may satisfy the initial and boundary conditions.

[0143] In some embodiments, the subset of initial and boundary conditions may include at least one of a sinusoidal function, a harmonic function, a periodic function, or an exponential function. For example, if the boundary conditions of the system dynamics f are periodic, then the collocation points 118 should be periodic. If the boundary conditions are Dirichlet, i.e., if the system dynamics f is equal to a particular value at its boundary point, then the collocation points 118 should also be equal to such a value at the corresponding boundary point. Advantageously, the collocation points 118 may be evaluated significantly more computationally inexpensively than computing snapshots. The snapshots may be generated either by a simulator or experiments, and the collocation points 118 may be generated by simply sampling from a space of feasible functions.

[0144] Furthermore, a function space is a set of functions between two defined sets. Often, the domain and / or codomain may have addition properties that can be inherited by the function space. For example, the set of functions from any set X to a vector space has a natural vector space structure given by pointwise addition and scalar multiplication. In other scenarios, the function space may inherit a topological or metric structure.

[0145] The autoencoder 108 may receive the digital representation 116 of the time series data and the collocation points 118 to be projected onto the differential equation. The encoder 216 encodes the digital representation into a latent space. The nonlinear predictor 220 may propagate the encoded digital representation into the latent space using a transformation determined by the values ​​of the parameters of the nonlinear operator 220. Further, the decoder 218 may decode the transformed encoded digital representation. The output 502 of the transformed encoded digital representation may be a reconstructed snapshot or a decoded, linearly transformed encoded digital representation.

[0146] The basic neural network implemented for the autoencoder 108 architecture is illustrated in FIG.

[0147] FIG. 6 shows a schematic diagram 600 of a neural network 106 according to some embodiments of the present disclosure. The neural network 106 may be a network or circuit of artificial neurons or nodes. Thus, the neural network 106 is an artificial neural network used to solve artificial intelligence (AI) problems. The connections of biological neurons are modeled in the artificial neural network as weights between nodes. Positive weights reflect excitatory connections, while negative weight values ​​represent inhibitory connections. All inputs 602 of the neural network 106 may be modified by the weights and summed. Such activity is called a linear combination. Finally, an activation function controls the amplitude of the output 604 of the neural network 106. For example, the acceptable range of the output 604 is typically from 0 to 1, or from -1 to 1. Artificial networks may be used for predictive modeling, adaptive control, and other applications in which the artificial network may be trained via a training dataset. Self-learning from experience may occur within the network, allowing it to draw conclusions from complex and seemingly unrelated sets of information. The architecture of the blocks of the autoencoder 106 is illustrated in Figures 7A, 7B and 7C.

[0148] 7A shows a diagram 700A depicting the input of a digital representation in an encoder 216 (such as an autoencoder 108) of a neural network 106, according to some embodiments of the present disclosure. The diagram 700A includes the encoder 216, a snapshot 702, a collocation point 118, and a final layer 704 of the encoder 216.

[0149] The input of the encoder 216 can be either a snapshot 702 or a collocation point 118. The snapshot 702 can be, for example, a digital representation 116 of time-series data. The encoder 216 obtains the values ​​of the snapshot 702 or the collocation point 118. The encoder 216 outputs to a latent space or a nonlinear operator 220 through a final layer 704 of the encoder 216. The digital representation 116 of the time-series data can be collected to indicate measurements of the behavior of the system 204 at different time instances. Furthermore, for training a neural network 106 (such as the autoencoder 108) with an autoencoder architecture, the encoder 216 can encode the digital representation into a latent space. The encoding process is model reduction.

[0150] 7B shows a diagram 700B depicting the propagation of the encoded digital representation into a latent space by the nonlinear operator 220 of the neural network 106, according to some embodiments of the present disclosure. The diagram 700B includes the final layer 704 of the encoder 220, the linear operator 220, and the final iteration 706 of the nonlinear operator 220 or latent space model. The nonlinear operator 220 is represented as a NODE, and h θ It is shown as follows.

[0151] The nonlinear operator 220 is configured to propagate the coded digital representation into the latent space using a linear transformation determined by the values ​​of the parameters of the nonlinear operator 220. The output of the final iteration 706 of the nonlinear operator 220 is sent to the decoder 218 of the neural network 106. The process of propagating the coded digital representation into the latent space is called reduced-order model propagation or time integration.

[0152] 7C shows a diagram 700C depicting the decoding of the transformed encoded digital representation by the decoder 218 of the neural network 106, according to some embodiments of the present disclosure. The diagram 700C includes the decoder 218, a final iteration 706 of the nonlinear operator 220, and an output 708 of the decoder 218.

[0153] The decoder 218 forwards the input and provides an output 708. The decoder 218 is configured to decode the transformed coded digital representation to produce the output 708. The output 708 is a decoded, linearly transformed coded digital representation, such as a reconstructed snapshot as depicted in Figure 5. The process of decoding is a reconstruction of the snapshot.

[0154]

number

[0155] An exemplary scenario for a real-time implementation of the device 302 for controlling the operation of the system 204 is further described in FIG.

[0156] 8 shows an example diagram 800 of a real-time implementation of an apparatus 302 for controlling the operation of a system 204, according to some embodiments of the present disclosure. The example diagram 800 includes a room 802, a door 804, a window 806, a ventilation unit 808, and a set of sensors 810.

[0157] In the exemplary scenario, system 204 is an air conditioning system. Exemplary diagram 800 shows a room 802 having a door 804 and at least one window 806. The temperature and airflow of room 802 are controlled by device 302 via the air conditioning system through a ventilation unit 808. A set of sensors 810, such as sensor 810a and sensor 810b, is disposed in room 802. At least one airflow sensor, such as sensor 810a, is used to measure the speed of airflow at a given point in room 802, and at least one temperature sensor, such as sensor 810b, is used to measure the temperature of the room. It should be noted that other types of setups are also possible, such as, for example, a room with multiple HVAC units or a house with multiple rooms.

[0158] Another exemplary scenario for real-time implementation of the device 302 for imaging greenhouse gas leaks is further described in FIG.

[0159] 9 shows an example diagram 900 of a real-time implementation of the apparatus 302 for reconstructing the distribution of a gas from a camera. For example, the apparatus 302 is implemented to image a greenhouse gas leak 902. The example diagram 900 may include a mid-infrared light source 904, a single-pixel camera 906, and a neural network 106. The single-pixel camera 906 may include two objective lenses, such as lens 908 and lens 912. The single-pixel camera 906 may further include a digital micromirror device (DMD) 910 and a photodetector 914.

[0160] DMD 910 receives light from the gas scene and projects it through lens 912 onto photodetector 914. DMD 910 changes the orientation of its mirrors at a high frequency, resulting in a series of measurements 920 of the gas scene at every instance of time.

[0161] The series of measurements 920 is then compared to measurements 918 using the DMD configuration of the gas reconstruction generated from the trained neural network 106. The gas distribution reconstruction 916 is refined to minimize the difference between the series of measurements 920 from the mid-infrared (MIR) sensor and the gas reconstruction measurements 918.

[0162] A system 204, such as an air conditioning system, may be described by a physics-based model called the Boussinesq equation, as exemplarily shown in FIG. 4. However, the Boussinesq equation includes infinite dimensions to solve the Boussinesq equation for controlling the air conditioning system. Data assimilation may be further added to the ODE model. The model optimally reproduces the dynamics (e.g., airflow dynamics) of the air conditioning system. Furthermore, in some embodiments, the model of airflow dynamics relates airflow values ​​(e.g., airflow velocity) to the temperature of the air-conditioned room during operation of the air conditioning system. Furthermore, the device 302 optimally controls the air conditioning system to generate a regulated airflow.

[0163] 10 shows a flowchart 1000 depicting a method for training a neural network 106 according to some embodiments of the present disclosure. Flowchart 1000 may include steps 1002, 1004, and 1006. Fewer or more steps may be provided. Additionally, one or more steps may be combined or separated without departing from the scope of the disclosure.

[0164] In step 1002, the method may include collecting digital representations of time series data 116. The digital representations of time series data 116 represent instances of a system's function space and corresponding measurements of the system's state of operation at different time instances. Collection of the digital representations of time series data 116 by a CFD simulation or experiment module 112 is further described, for example, in FIG. 1A.

[0165] In step 1004, the method may include generating collocation points 118 corresponding to solutions of a PDE that represent the nonlinear dynamics for a set of initial and boundary conditions at the state of operation of the system and constraints on the operation of the system that evolve from the boundary conditions according to the PDE. In some embodiments, the generation of collocation points 118 may be based on a subset of the set of initial and boundary conditions that has a structure that reduces the complexity of solving the PDE. The generation of collocation points 118 may be further based on a function space of the system that satisfies the subset of initial and boundary conditions. Details of the generation of collocation points 118 are further provided, for example, in FIG. 1D .

[0166]

number

[0167] The above description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. Various changes are contemplated that may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter as set forth in the appended claims.

[0168] Specific details are provided in the following description to provide a thorough understanding of the embodiments. However, it will be understood by those skilled in the art that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements of the disclosed subject matter may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. Furthermore, like reference numbers and names in the various drawings indicate like elements.

[0169] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any specifically described process may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to a return of the function to the calling function or the main function.

[0170] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automatic implementation may be performed or at least assisted by machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor(s) may perform the necessary tasks.

[0171] The various methods or processes outlined herein may be coded as software executable on one or more processors using any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and may be compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0172] Each embodiment is described as a process, which is depicted as a flowchart, flow diagram, data flow diagram, structure diagram, or block diagram. While a flowchart depicts operations as a sequential process, many of the operations can be performed in parallel or simultaneously. Additionally, the order of operations may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any process specifically described may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to a return of the function to the calling function or the main function.

[0173] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automatic implementation may be performed or at least assisted by machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor(s) may perform the necessary tasks.

[0174] Numerous modifications and other embodiments of the disclosures described herein will come to mind to one skilled in the art to which the disclosures described herein pertain having the benefit of the teachings presented in the above description and the associated drawings. It should be understood that the disclosure is not limited to the particular embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, while the above description and the associated drawings describe exemplary embodiments in the context of certain illustrative combinations of elements and / or functions, it should be recognized that alternative embodiments may provide various combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, combinations of elements and / or functions other than those expressly described above are also contemplated, for example, as may be set forth in some of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

1. 1. A computer-implemented method for training a neural network for controlling the operation of a system having nonlinear dynamics represented by partial differential equations (PDEs), the neural network including nonlinear operators of the dynamics of the system represented in a latent space by parameterized ordinary differential equations (ODEs) having parameters determined by the training, the method comprising: collecting digital representations of time series data indicative of instances of a function space of the system and corresponding measurements of states of the operation of the system at different time instances; generating collocation points corresponding to a solution of the PDE that represents the nonlinear dynamics for a set of initial and boundary conditions at the state of operation of the system and constraints on the operation of the system that evolve from the boundary conditions according to the PDE; training the neural network using training data including the collected time series data and the collocation points to train the parameters of the nonlinear operator, the neural network having an autoencoder architecture including an encoder and a decoder, the encoder configured to encode each instance of the training data into a latent space, the nonlinear operator configured to propagate the encoded instances of the training data into the latent space using a transformation determined by the parameters of the nonlinear operator, and the decoder configured to decode the transformed encoded instances of the training data to minimize a hybrid loss function including a data-driven loss between the decoding of the neural network and the collected time series data and a physical information loss between the decoding of the neural network and a solution of the PDE at the collocation points.

2. The generation of the collocation points comprises: a subset of the set of initial and boundary conditions having a structure that reduces the complexity of solving the PDE; The method of claim 1 , wherein the function space of the system satisfies the subset of initial and boundary conditions.

3. The method of claim 2 , wherein the structure of the subset of initial and boundary conditions comprises at least one of a sinusoidal function, a harmonic function, a periodic function, or an exponential function.

4. The method of claim 1 , wherein the parameters of the nonlinear operator are determined based on a probabilistic approach.

5. The method of claim 1 , wherein the nonlinear operator is based on a continuous-time dynamical system.

6. The method of claim 1 , further comprising fine-tuning the parameters of the nonlinear operator in real time based on a set of expected measurements and an output of the neural network.

7. The method of claim 1 , further comprising generating estimates and control commands for controlling the operation of the system.

8. The method of claim 7 , wherein the generation of the estimation and control commands for controlling the operation of the system is based on model-based control and estimation techniques.

9. The method of claim 7 , wherein the generation of the estimation and control commands for controlling the operation of the system is based on optimization-based control and estimation techniques.

10. The method of claim 7 , wherein the generation of the estimation and control commands for controlling the operation of the system is based on data-driven control and estimation techniques.

11. 10. The method of claim 1, further comprising generating the parameterized ODE based on one or more model reduction techniques, wherein the one or more model reduction techniques include at least one of a proper orthogonal decomposition (POD)-Galerkin projection method or a dynamic mode decomposition (DMD) method.

12. The method of claim 1 , further comprising obtaining a digital representation of the time series data based on the use of computational fluid dynamics (CFD) simulations and experiments.

13. 1. A training system for training a neural network for controlling the operation of a system having nonlinear dynamics represented by partial differential equations (PDEs), the neural network including nonlinear operators of the dynamics of the system represented in a latent space by parameterized ordinary differential equations (ODEs) having parameters determined by the training, the training system comprising at least one processor and a memory having instructions stored thereon, the instructions, when executed by the at least one processor, causing the training system to: collecting digital representations of time series data indicative of instances of a function space of the system and corresponding measurements of states of the operation of the system at different time instances; generating collocation points corresponding to a solution of the PDE that represents the nonlinear dynamics for a set of initial and boundary conditions at the state of operation of the system and constraints on the operation of the system that evolve from the boundary conditions according to the PDE; and training the neural network using training data including the collected time series data and the collocation points to train the parameters of the nonlinear operator, the neural network having an autoencoder architecture including an encoder and a decoder, the encoder configured to encode each instance of the training data into a latent space, the nonlinear operator configured to propagate the encoded instances of the training data into the latent space using a transformation determined by the parameters of the nonlinear operator, and the decoder configured to decode the transformed encoded instances of the training data to minimize a hybrid loss function including a data-driven loss between the decoding of the neural network and the collected time series data and a physical information loss between the decoding of the neural network and a solution of the PDE at the collocation points.

14. The generation of the collocation points comprises: a subset of the set of initial and boundary conditions having a structure that reduces the complexity of solving the PDE; The training system according to claim 13, which is based on the function space of the system that satisfies the subset of the initial conditions and boundary conditions.

15. The training system according to claim 14, wherein the structure of the subset of the initial conditions and boundary conditions includes at least one of a sine function, a harmonic function, a periodic function, or an exponential function.

16. The training system according to claim 13, wherein the parameters of the non-linear operator are determined based on a probabilistic approach.

17. The training system according to claim 13, wherein the non-linear operator is based on a continuous-time dynamic system.

18. The training system according to claim 13, further configured to fine-tune the parameters of the non-linear operator in real time based on a set of predicted measurements and the output of the neural network.

19. The training system according to claim 13, further configured to generate estimation and control commands for controlling the operation of the system.

20. A non-transitory computer-readable storage medium including a program executable by a processor for performing a method for controlling the operation of a system having non-linear dynamics represented by a partial differential equation (PDE), wherein the neural network includes a non-linear operator of the dynamics of the system represented in a latent space by a parameterized ordinary differential equation (ODE) having parameters determined by training, and the method includes collecting a digital representation of time series data indicating an instance of the function space of the system and corresponding measurements of the state of the operation of the system at different time instances; generating collocation points corresponding to a solution of the PDE representing the non-linear dynamics with respect to a set of initial conditions and boundary conditions in the state of the operation of the system and constraints in the operation of the system developed from the boundary conditions according to the PDE; and training the neural network using training data including the collected time series data and the collocation points to train the parameters of the nonlinear operator, the neural network having an autoencoder architecture including an encoder and a decoder, the encoder configured to encode each instance of the training data into a latent space, the nonlinear operator configured to propagate the encoded instances of the training data into the latent space using a transformation determined by the parameters of the nonlinear operator, and the decoder configured to decode the transformed encoded instances of the training data to minimize a hybrid loss function including a data-driven loss between the decoding of the neural network and the collected time series data and a physical information loss between the decoding of the neural network and a solution of the PDE at the collocation points.

Images