Simulation Training

Abstract

The learning model (125) is used to run the simulator (210B) to generate starting values ​​for the nodes in the simulator (210B). After running the simulation, the cost for this run is determined. If the cost is within a threshold, the learning model results are used as starting values ​​for the optimizer (205B). The optimizer (205B) is then used to generate starting values ​​for the nodes in the simulator (210B). The optimizer (205B) is then run repeatedly such that for each iteration, the results of running the optimizer (205B) are used as training inputs (215B) to the learning model (125).

Description

[Technical field]

[0001] The present disclosure relates to machine learning techniques, and more particularly to machine learning techniques that use a simulation model to train other machine learning techniques on the same problem. Summary of the Invention [Means for solving the problem]

[0002] This Summary is provided to introduce in a simplified form a selection of concepts that are more fully described below in the Detailed Description section. This Summary does not identify essential features or features of the claimed subject matter. The invention innovation is defined by the claims, and in the event of a conflict between this Summary and the claims, the claims shall control.

[0003] Generally speaking, the techniques described herein describe training a learning model using an optimizer.

[0004] In an embodiment, a computer enabled learning model training system is disclosed, comprising: 1. a processor, a memory in operative communication with the processor, computing code associated with the processor configured to create a simulator trainer, an optimizer for determining initial node values ​​for the simulator, the simulator including nodes having values, the optimizer using as input an input time series from time t=(-n) to time t=(0) and outputting an output time series from time t=(-n) to time t=(0) for the nodes, a reverser for reversing the input time series from time t=(0) to t=(-n) to generate an inverted input time series, and for reversing the output time series from time t=(0) to t=(-n), and a learning model using the inverted input time series as a training input and using a selected value from the output time series at t=(-n) as a ground truth for a cost function associated with the learning model. In an embodiment, the simulator is a heterogeneous neural network.

[0005] In an embodiment, the system further comprises a cost function determiner that uses the selected node values ​​from the output time series as input to a cost function, the cost function determiner further comprising a cost function that uses the ground truth as input to the cost function.

[0006] In an embodiment, the costs derived from the cost function are used by the optimizer to determine subsequent initial node values.

[0007] The embodiment further includes an iterator that repeatedly runs the optimizer, simulator, and learning model until a halting state is reached.

[0008] In an embodiment, when a stop state is reached, the initial node values ​​are used as input to a state estimation simulation.

[0009] In an embodiment, a state estimation simulation is run from time t=(-n) to time t=(0), and then a state simulation is run from time t(0) to t(m). The state simulation produces outputs that can be used to generate control sequences, which are used to execute the device modeled by the state simulation.

[0010] In an embodiment, the learning model is a neural network.

[0011] In an embodiment, the neural network is a recurrent neural network.

[0012] In an embodiment, a computer-enabled method for training a learning model using an optimizer model implemented in a computing system is disclosed. The method includes one or more processors and one or more memories coupled to the one or more processors, the one or more memories including computer-executable instructions for causing the computing system to perform operations including running the optimizer to determine initial simulator node values, running the simulator using inputs and the initial simulator node values ​​to generate simulator outputs, comparing selected node values ​​from the simulator outputs to desired node values ​​to determine costs, inverting the selected node values ​​to generate inverse selected node values, inverting the simulator inputs to generate inverse simulator inputs, using the inverse selected node values ​​and the inverse simulator inputs as training inputs to the learning model, and running the learning model.

[0013] In an embodiment, the act of executing the learning model produces the inverse time series as the learning model output.

[0014] In an embodiment, the learned model output at time t(-n) is compared to the initial simulator node values ​​in a cost function.

[0015] In an embodiment, the costs are derived from a cost function, and the costs are used for backpropagation within the learning model.

[0016] In an embodiment, the simulator is a heterogeneous neural network.

[0017] In an embodiment, the input includes weather data over time.

[0018] In an embodiment, the selected node value is the temperature of the area inside the space in which the simulator is modeled.

[0019] In an embodiment, the act of inverting the input of the simulator includes the act of inverting a time series originally from t=(-n) to time=(0) to time t=(0) to t=(-n) to generate a reversed time series.

[0020] In an embodiment, a computer readable storage medium is disclosed having instructions configured thereon that, when executed by one or more processors, perform a method of training a simulator, the method including running an optimizer to determine initial simulator node values, running the simulator using inputs and the initial simulator node values ​​to generate simulator outputs, comparing selected node values ​​from the simulator outputs to desired node values ​​to determine costs, inverting the selected node values ​​to generate inverse selected node values, inverting the simulator inputs to generate inverse simulator inputs, using the inverse selected node values ​​and the inverse simulator inputs as training inputs to a learning model, and running the learning model.

[0021] In an embodiment, for the learning model cost function, the learning model output at time t = (-n) is compared to the simulator output at time t = (0), and the cost derived from the learning model cost function is used for backpropagation within the learning model.

[0022] In an embodiment, the learning model is a recurrent neural network.

[0023] These and other aspects of the present invention will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. The following description sets forth various embodiments of the invention and numerous specific details thereof, given by way of example only and not by way of limitation. Many substitutions, modifications, additions, or rearrangements may be made within the scope of these embodiments, and the embodiments are intended to include all such substitutions, modifications, additions, or rearrangements.

[0024] Non-limiting and non-exhaustive embodiments of the present embodiments are described with reference to the following figures, in which like reference numbers refer to like parts throughout the various views unless otherwise specified: [Brief description of the drawings]

[0025] [Figure 1] The relationships between the inputs and outputs of the simulator and the learning model during the training process that can implement the embodiments disclosed herein are disclosed. [Diagram 2] We disclose an interaction between a learning model and a simulator that can implement the embodiments disclosed herein, once the learning model is fully trained. [Figure 2A] Interactions between initial value simulation, state estimation simulation, and state simulation are disclosed that may implement the embodiments disclosed herein. [Figure 2B] The present disclosure provides an interaction between an optimizer and a heterogeneous node system that can implement the embodiments disclosed herein. [Diagram 3] A computing system is disclosed that can be implemented in conjunction with the disclosed embodiments. [Figure 4]A distributed computing system capable of implementing the embodiments disclosed herein is disclosed. [Diagram 5] An optimizer loop, along with inputs and outputs, in which the embodiments disclosed herein can be implemented, is disclosed. [Figure 6] The states of the optimizer, along with the outputs, are disclosed, which may implement the embodiments disclosed herein. [Figure 7] A learning model loop, along with inputs and outputs, capable of implementing the embodiments disclosed herein is disclosed. [Figure 8] A heterogeneous node loop, together with inputs and outputs, in which the embodiments disclosed herein can be implemented, is disclosed. [Figure 9] Training of an RNN is disclosed that can implement the embodiments disclosed herein. [Figure 10] A trained RNN capable of implementing the embodiments disclosed herein is disclosed. [Figure 11A] A flowchart is disclosed that describes a method for training a learning model using an optimizer that can implement embodiments disclosed herein. [Figure 11B] A flowchart is disclosed that describes a method for training a learning model using an optimizer that can implement embodiments disclosed herein. [Figure 12] An exemplary simulator node capable of implementing the embodiments disclosed herein is disclosed. [Figure 13] Exemplary learning model inputs and outputs are disclosed that can implement the embodiments disclosed herein. [Figure 14] An exemplary node capable of implementing the embodiments disclosed herein is disclosed. [Figure 15A] An exemplary state estimation simulation flow is disclosed in which the embodiments disclosed herein can be implemented. [Figure 15B] An exemplary state estimation simulation flow is disclosed in which the embodiments disclosed herein can be implemented. [Figure 16] Exemplary node values ​​in different simulations are disclosed in which the embodiments disclosed herein can be implemented. [Figure 17] An exemplary flow chart is disclosed illustrating a sanity check to ensure that the initial state estimates are valid. [Figure 18] An exemplary simulation state estimation system is disclosed in which embodiments disclosed herein can be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Corresponding reference characters indicate corresponding components throughout the various views of the drawings. Those skilled in the art will appreciate that elements in the figures are illustrated for clarity and clarity and have not necessarily been drawn to scale. For example, some of the elements in the figures may be exaggerated in size relative to other elements to help facilitate a better understanding of the various embodiments. Also, common elements that are useful or necessary in commercially feasible embodiments, but are well understood, are often not shown in order to facilitate a less obstructed view of these various embodiments.

[0027] Disclosed below are representative embodiments of methods, computer readable media, and systems that have particular application to systems and methods for determining simulation validity. The described embodiments implement one or more of the described techniques.

[0028] In the following description, numerous specific details are set forth to obtain a thorough understanding of the present embodiment. However, it will be apparent to one skilled in the art that these specific details may not be employed to practice the present embodiment. Meanwhile, well-known materials or methods are not described in detail to avoid obscuring the present embodiment. "One embodiment," "an embodiment," "one example," or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present embodiment. That is, when the phrases "in one embodiment," "in an embodiment," "one example," or "an example" appear in various chapters throughout this specification, they do not necessarily all refer to the same embodiment or example. The systems, devices, and methods described herein may be modified, added, or omitted without departing from the scope of the present disclosure. For example, the components of the systems and devices may be integrated or separated. Furthermore, the operations of the systems and devices disclosed herein may be performed by more, fewer, or other components, and the methods described may include more, fewer, or other steps. In addition, the steps may be performed in any suitable order.

[0029] For convenience, relative terms may be used in describing this disclosure, including, for example, left, right, top, bottom, front, back, above, below, up, and down, among others, with it being understood that these terms are used for illustrative purposes only and are not meant to be limiting in any way.

[0030] In addition, it will be appreciated that the figures presented herewith are for explanatory purposes only to those skilled in the art, and that the drawings are not necessarily drawn to scale. In order to assist the Patent Office, and any reader of any patent issuing on this application, in interpreting the claims appended hereto, Applicant wishes to note that no appended claim or claim element is intended to invoke 35 U.S.C. 112(f) unless the words "means for" or "step for" are expressly used in a particular claim.

[0031] The present embodiments may be realized as an apparatus, a method, or a computer program product. Thus, the present embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be referred to as a "system." Additionally, the present embodiments may take the form of a computer program product of expression having computer usable program code embodied in any tangible medium embodied in the medium.

[0032] One or more computer usable or computer readable media, in any combination, may be utilized. For example, the computer readable media may include one or more of a portable computer diskette, a hard disk, a random access memory (RAM) device, a read-only memory (ROM) device, an erasable programmable read-only memory (EPROM or flash memory) device, a portable compact disk read-only memory (CDROM), an optical storage device, and a magnetic storage device. Computer program code for carrying out the operations of the present invention may be written in any combination of one or more programming languages.

[0033] The flowcharts and block diagrams in the flow diagrams illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code, and may include one or more executable instructions for implementing a specified logical function(s). It is also noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, may be implemented by a special purpose hardware-based system that performs the specified functions or acts, or a combination of special purpose hardware and computer instructions. These computer program instructions may also be stored on a computer-readable medium and may instruct a computer or other programmable data processing apparatus to function in a particular manner, such that storing the instructions on a computer-readable medium results in an article of manufacture that includes instruction means for implementing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0034] "Optimize" means to improve, but not necessarily completely. For example, a value or algorithm that has already been optimized may be capable of further refinement.

[0035] As used herein, the terms "comprises," "comprising," "include," "including," "has," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements, but can also include other elements not expressly listed or that are inherent to such process, article, or apparatus.

[0036] Furthermore, unless expressly stated otherwise, "or" refers to an inclusive or and not an exclusive or. For example, condition A or B is satisfied by any one of the following: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); A and B are true (or exist). "Program" is used broadly herein to include applications, kernels, drivers, interrupt handlers, firmware, state machines, libraries, and other code written by a programmer (also called a developer) and / or generated automatically. "Optimize" means to improve, but not necessarily completely. For example, an optimized program or algorithm may be capable of further improvements.

[0037] Additionally, any examples or illustrations provided herein should not be considered as restrictions, limitations, or explicit definitions of any term or terms used in conjunction therewith. Instead, these examples or illustrations are to be considered as described with respect to one particular embodiment and as merely illustrative. It will be appreciated by those skilled in the art that any term or terms used in conjunction with these examples or illustrations encompass other embodiments that may or may not be shown together or in other paragraphs herein, and that all such embodiments are intended to be included within the scope of the term or terms. Phrases indicating such non-limiting examples and illustrations include, but are not limited to, "for example," "for instance," "eg," and "in one embodiment."

[0038] A "cost function" is generally a function that determines how close the answer of a simulation model is to a desired answer, i.e., ground truth. That is, it quantifies the error between the predicted value and the desired value. The cost function returns a cost. The cost function can arrive at an answer using a least squares function, Mean Error (ME), Mean Squared Error (MSE), Mean Absolute Error (MAE), Categorical Cross Entropy Cost Function, Binary Cross Entropy Cost Function, etc. In some embodiments, the cost function is a loss function. In some embodiments, the cost function is a threshold. The threshold may be said to be a number that indicates that the simulated truth curve is close enough to the ground truth. In other embodiments, the cost function may be a slope. The slope may also indicate that the simulated truth curve and the ground truth are close enough. When a cost function is used, it may be time variant. It may also be linked to factors such as user preferences or changes in the physical model. The cost function applied to the simulation engine may include any one or more models of energy use, primary energy use, energy monetary cost, human comfort, safety of the building or contents, durability of the building or contents, microbial growth potential, system equipment durability, system equipment life, environmental impacts, and / or energy use CO2 potential. The cost function may also utilize a discount function based on discounted future value of costs. In one embodiment, the discount function may devalue future energy compared to current energy to account for future uncertainties to ensure optimized operation over time.A discount function can discount future cost functions of control regimes based on the accuracy or probability of forecast weather data and / or the value of an energy source on a utility rate schedule. A cost can be derived from the cost function. This cost can be a single number. A "goal function" can read in the cost (a value from the cost function) and determine if the cost meets a criterion to reach the goal so that the simulation iterations stop. Such criteria can be the cost reaching a particular value, being lower or higher than a particular value, being between two values, etc. The goal function can also look at the time spent running the entire simulation model and / or how many iterations have been performed to determine if the goal function has been met.

[0039] After running the simulation model, a "machine learning algorithm" or "optimization method" is used to determine the next set of inputs. These machine learning algorithms or optimization methods can include gradient descent, methods based on Newton's method, inversions of the Hessian using conjugate gradient techniques, evolutionary computing such as swarm intelligence, bee colony optimization, self-organizing migrating algorithm (SOMA), particle swarms, nonlinear optimization techniques, and other methods known to those skilled in the art. A "state," as used herein, can be air temperature, radiant temperature, atmospheric pressure, sound pressure, occupancy amount, indoor air quality, CO2 concentration, light intensity, or any other state that can be measured and controlled.

[0040] A deep physics network, as used herein, is a type of structured neural network similar to a neural network. However, unlike the homogeneous activation function of neural nets, each neuron contains a unique physics equation (for the equipment model) or resistance / capacitance values ​​(for the building model). Once constructed, known sensors are fed to their corresponding nodes in the network. Once the network is trained, any location in the thermodynamic system can be asked to extract data about the model at that point. The figure "Possible Equipment Model Implementation" shows a portion of a database structure that may hold queryable data for an equipment model. Querying the model can also be called introspecting. Similar data structures exist for building the model. This process allows powerful and extensive data fusion, data synthesis, and quality assessment for any thermodynamic system, even without the presence of sensors, by inference. The same mechanism allows for optimization of the model, and the time series generated from the model can then be used for real-time sequence generation and fault detection. To automate the structure, a digital twin version of the structure (Structural Simulation) is created. Matching digital twin versions of the equipment in the building (Equipment Simulation) are created as well. The equipment model comprises nodes representing the individual material layers of the building, as well as their resistances and capacitances. These are formed into parallel and branchless neural network strings through which heat (or other state values) propagate. The equipment model comprises nodes representing the equipment, their connections, and external influences on the equipment, such as weather. The nodes have physical equations that describe the equipment state changes.An equipment node may also have state input(s) and state output(s), state parameters with values, allowable state parameter values, state input location data, and state output location data. The location data may be cross-referenced with the thermodynamic building model location. In an embodiment, the equipment nodes may also form a control loop. These node inputs and outputs, along with the connections between the equipment, form a heterogeneous neural network. State information flows throughout the model according to physics rules.

[0041] Conceptually, running a structural simulation model involves injecting a quantity of some measurable state into the building. This could be temperature, humidity, sound, vibration, magnetism, light, pressure, moisture, etc. This state then propagates through the different layers and affects the structure. This example uses temperature to describe aspects of the system and method of heating a room (blocks, insulation, drywall, etc.), as represented by the interior nodes. The output nodes are associated with a time heat curve T. The curve value at time T1 is injected into one or more exterior nodes. This temperature propagates throughout the building (by the values ​​of the nodes in the simulation neural net). As seen at the block nodes, the outside of the building (e.g., blocks) has its temperature changed by the exterior nodes and also by the known manner of how heat transfers through the blocks. The blocks then heat the next layer, perhaps insulation. This continues throughout the building until the interior nodes are reached. At the interior nodes, other functions can be applied, such as functions representing lighting and people (warmth) in a zone. The state information continues to propagate until it reaches another exterior layer, where each node parameter value represents the heat generation occurring in the building at time T1. In one embodiment, each exterior surface has its own time-temperature curve. In one embodiment, the building is decomposed into smaller subsystems (or zones), so that rather than propagating the temperature throughout the entire structure, only a portion of the structure is acted upon by a given input. In one embodiment, a digital twin model associated with the controlled building is built on the controller. In one instance, the controller is embedded in the controlled building and used to automate the building. The controller may comprise a simulation engine, which itself constitutes a model of the controlled system in which the controller is embedded, or a model of the equipment in the controlled system in which the controller is embedded. This model may also be referred to as a "physical model." This physical model may itself comprise a past regression and a cost function.Past regressions are instances of these models that have been run in the past, and the results. The controlled system has at least one sensor whose value can be used to calibrate the physical model(s) by checking how close the model value at the sensor location is to the simulated sensor value equivalent in the physical model. A cost function can be used to determine the distance between the sensor value and the simulated sensor value equivalent. This information can then be used to refine the physical model. This controller-controlled system loop can be implemented without the use of the Internet. The controller can control and / or run a local area network (LAN) over which it communicates to the sensors and other resources. The controller can be hardwired and embedded within the sensors and other resources, or it can be a composite system where some resources are hardwired and others connect to the LAN. The "simulation model" can be a resource model or a building model.

[0042] The technical features of the embodiments described herein will be apparent to those skilled in the art and will become apparent to the broader attentive reader in a variety of ways. In one embodiment, the technical activity is based on computing techniques, such as determining how to run a simulation more efficiently. These simulations can simulate energy flows in a building or other structure. Energy flow simulations can generate control sequences that allow a building to be operated in a much more energy efficient manner. The simulation itself can run very stably because the appropriate time step is known. It is also fair to say that the simulation can run faster, with less computational cost and without worrying about the entire simulation becoming unstable. Using the methods and systems disclosed herein to determine the time step can make a significant speed difference and can make the difference between when a system can run in a reasonable amount of time and when it cannot. For example, a simulation with too small a timestamp can require a lot of memory and the entire program can be shown down exponentially. A program with too large a timestamp can become unstable. The systems and methods disclosed herein can arguably provide timestamps in the "right size" so that they can run as fast as possible without destabilizing the system. Other advantages based on the technical features of the present teachings will also be apparent to those skilled in the art from the description provided.

[0043] 1.Big picture To run a useful building simulation that starts at t=0 and goes into the future, the simulation of the building itself cannot be put into a random state. To see why, consider the following example: Imagine starting the simulation at -100 degrees Celsius at every node in the building. Unlike a simulation of such a building that predicts future reality, any simulation of reasonable length would spend the entire simulation time heating up to ambient temperature. Furthermore, the output of the node representing the zone temperature is a useless line that starts at -100 degrees Celsius and then gradually heats up. A real building would consist of accurate temperature oscillations from time t=0 onwards based on weather inputs, inter-building dynamics, and human interactions.

[0044] Also, setting all nodes to the ambient temperature at t=0 fails because the dynamics of a building are a complex set of relationships involving material resistances, mass capacitance, solar absorption, etc. The various zones in a building are never the same and the relationships between them are complex. Starting all nodes in the model at the same temperature is an oversimplification and will not give any meaningful simulator output after t=0. Again, this risks the entire simulation finishing before the building dynamics have time to stabilize and reflect reality.

[0045] Even if historical temperature readings exist, the area of ​​the building represented by the temperature sensors is only a small subset of the nodes. Empirical historical data may exist for only 2 or 3 percent of the locations in the building represented by the nodes. Thus, deriving estimates from such sparse data may lead to most nodes having values ​​that are very far from what is believed to be their corresponding actual values. Again, the entire simulation may end before the building dynamics have time to override the estimation error and reflect reality. The longer the simulation runs, the more time and resource intensive it becomes. By their very nature, such models use a large amount of computer resources and time. Using the techniques disclosed herein, only a small amount of empirical data is used to infer the correct temperature at t=0 for unknown nodes in a building (or similar structure), reducing the use of valuable computer resources. The state estimation simulation method and system taught herein runs much more quickly to reach a valid starting point than running a simulation over a long period of time.

[0046] Learning models are very powerful when it comes to solving difficult problems. However, such models must be trained on a large set of training data before they can provide valid solutions. Obtaining sufficient training examples is often difficult and sometimes insurmountable. Developing a training set involves obtaining and / or generating many examples of input data that can generate meaningful output data. One way to do this is to use synthetic or real data examples from the problem the learning model is designed to solve as training data. It makes sense for learning models to preferentially train on real-world data, if possible, and thus receive a representative training set. However, when modeling a building, data can only be collected in real time, and it takes a year to generate a one-year data set. Providing synthetic data creates its own problems. When using synthetic data, learning models have a hard time providing accurate answers to real-world data, because the real world contains examples that were not generated in the synthetic data generation procedure, and the data is either overfitted (noise is misinterpreted as parameters) or underfitted (losing parameters). The problem is so demanding that "it is common to invest days or months on hundreds of machines to solve even a single instance of the neural network training problem" Goodfellow et al., "Deep Learning (Adaptive Computation and Machine Learning series)" (MIT Press, Nov. 18, 2016, p. 274). Another challenge in overcoming the problem is the large number of training sets required to generate a usable solution. As the number of parameters in the model increases, the amount of training data required grows exponentially until a state solution is obtained. Also, the training data must represent the data the model will actually encounter, or the results will deviate in ways that are difficult to verify.Also, these models are computationally intensive; the models may require very expensive equipment to run, require very large amounts of time, etc. This is made more difficult by the nature of real-world data collection; for example, as mentioned above, it may take a year to compile a year's worth of data for a building.

[0047] To train a learning model using the optimizer, we start by repeatedly determining legal node starting values ​​using the optimizer for a building simulation. During the optimization process, thousands of simulations may be run, one for each input and output set. The learning model is then trained using each of the optimizer's input and output sets. Since the simulations discussed here use a thorough understanding of the physics and the modeled structure to determine their results, the inputs and outputs from the optimizer simulations can be used as real-world data sets. After training, it would be ideal if the learning model could produce similar outputs as achieved by the optimizer-simulator optimization, even given a unique problem. Even though the learning model can be considered trained, the optimizer-simulator is still used, at a minimum, to perform sanity checks on the learning model's results to ensure that the learning model continues to give accurate answers. This, among other things, prevents over- and under-fitting caused by, for example, anomalous scenarios. This also significantly reduces the computational power required to run any given model, since the training set is generated automatically rather than using extra computational power, the trained learning model requires much less computational power and time to run than an optimizer, etc.

[0048] More specifically, in one embodiment, a state that can be measured (or at least determined) in the structure being modeled is selected. In a building being modeled for HVAC control, this may be room temperature, which is used in this embodiment. Often rooms have thermometers installed inside, or it is easy enough to measure temperature in a room over time. This selected measurement (e.g., temperature) is measured in the room over a period of time to obtain a state-time curve. These temperature-state-time curves are then used as the "ground truth" for the initial value simulation. That is, the optimizer modifies the starting values ​​in the simulator to try to match the "ground truth" (the temperature-time curve in the building's interior) throughout the simulation. Before proceeding with this example, we will discuss the assembly of a simulation model. Digital twin simulation can break down the structure being modeled into nodes (more specifically described with reference to Figures 12 and 16). These nodes can be parts of a building that separately carry state values ​​(temperature, humidity, etc.), such as windows, ceilings, floors, studs, etc. Nodes can even represent smaller basic units, such as layers of building sections. For example, an interior wall can be described by a node representing a layer of gypsum board with a particular insulation value, an insulation layer with its own value, a tack, and another layer of gypsum board, etc. A representation of an exterior wall can have nodes representing exterior materials, such as blocks, then insulation, then gypsum board, etc. A digital twin simulator is made up of these connected nodes, each with a state value (e.g. temperature). The point of the optimizer simulation and state estimation simulation is to give justified values ​​to these state values ​​in the nodes.

[0049] To determine how a building naturally heats up, ideally the temperature of multiple rooms is measured at the same time that the weather outside is being measured. This measured weather can then be used as input to the optimizer simulation model. The optimizer selects temperature values ​​(in this embodiment) for each node in the simulator. The simulation is then run. Running the simulation consists of the simulation applying weather condition inputs to the outside and then allowing the weather conditions (e.g., temperature) to permeate the entire structure over the simulation time. The first time, the optimizer can randomly select its node values, it can also use previous model node values, etc. At the end of the simulation, the nodes representing room temperature are checked against the desired room temperature, and the optimizer then selects new values ​​for the nodes in the simulation. The simulation is run multiple times, each time the optimizer selects new node values ​​until an optimal output is reached. This optimal output provides the initial node values ​​for a state estimation simulation run using the same simulator. At the end of the state estimation simulation, the simulation is deemed to have a legitimate temperature inside it, and thus is ready to run.

[0050] At the same time this is happening, the learning model is also being trained. You can think of the learning model as running a lower version of the optimizer simulation. Once the desired output (the desired temperature time curve for the structure) and external state (weather) information is defined, the initial values ​​must be restored to the node values ​​of the digital twin simulation. That is, when you select the node values ​​for the learning model and run the simulation (for the desired time), the simulation model is considered to have a legitimate state, i.e., warmed up. How does this happen? The learning model uses the inverse output of the optimizer simulation as input and then produces the inverse node state values ​​as output. Specifically, the learning model uses as input the ground truth (in the current example, the room temperature time value curve) and the weather time value curve that the optimizer uses to run its simulation. However, the learning model reverses these time curves. If the curves originally ran from time -300 to 0, they now run from 0 to -300, producing an output from 0 to -300.

[0051] More specifically, the weather condition time series (inverse) and the desired node value subset time series (inverse) are fed as inputs to the learning model. The learning model is configured to have the same number of output nodes as the simulator has. These node values ​​at the end of the learning model run are then checked against the start values ​​of the optimizer simulation run, e.g., the desired values. The cost (the difference between the two values) is then used to update the learning model using backpropagation or a similar learning method. This creates, for example, an inverse building simulator given the desired end values, generating start values ​​that reach these end values. The learning model then becomes a little more accurate with each training set. Since thousands of training sets can be generated in one optimization cycle, it is safe to say that the learning model can be trained relatively quickly. Furthermore, since the data sets use not only the actual data to be solved but also the actual solutions, under-learning and over-learning of the training data is ameliorated.

[0052] This method is very counter-intuitive with respect to current uses of learning models. At an intuitive level, cause and effect are reversed here. The simulator tries to represent the evolution of cause-and-effect dynamics going forward. Since causality cannot be relied upon, going backwards is not possible (depending on the situation). As an example, if billiard balls are arranged in a starting triangle and a white ball is moving towards them, a simulator can be written to tell with a reasonable amount of precision how the collisions and movements will unfold. However, if the balls start in a situation where they are in random positions, it is not possible to write a simulator that tells you the position of the ball prior to the last shot. This is because there is no way to know which position the ball was in, although it could have been in any number of positions, including the starting triangle information. It is easy to simulate forward, but not backward in our lifetime. One of the reasons for this is the second law of thermodynamics. Entropy in a system always increases until equilibrium is reached. In some situations, once equilibrium is reached, all information content about previous system states is erased. If this is true, even the most intelligent simulator cannot tell us anything about how the system looked in the past. For example, imagine two rods immersed in a tank of room temperature water. The left rod is hot and the right rod is room temperature. The left rod sheds heat into the water (and to the other rod). After a long enough wait, the two rods will be at the same temperature. If someone now walks into the room and asks which rod was hot in the initial state, there is no way of knowing. All information about previous states has been erased. It is impossible to write an inverse simulator that will tell us which rod was hot. As another example, imagine a pot of room temperature water on a stove. Run the burner at randomly increasing heat settings for 10 minutes. During that time, keep a record of the burner's operation. After 10 minutes, the water will be at a certain temperature.At this point, it is not possible to recover the entries in the log. It is only possible to determine the total amount of energy that was injected into the water. The exact sequence and strength of the "signals" in the log cannot be reproduced. That information is lost, as if the thermodynamic evolution of the water acts as a lossy low pass filter.

[0053] Now, despite all the difficulties, somehow a backward run is performed. The learning model repeatedly monitors the behavior of the structure being simulated. That is, it learns and uses what information is available much more effectively than, for example, a pure backwards-physics simulator, but is significantly hampered by the information theory problems detailed above. In the absence of the ability to simulate deterministically, the output from an optimization simulator can be used as a training model to learn from past experience. In short, the optimizer chooses values ​​for a problem and then runs the simulator over and over again using these values. With each simulator run, it gets closer and closer to the optimized result. Each of these simulation runs is used as a training run for the learning model.

[0054] FIG. 1 discloses at 100 an overview of the training process for training a learning model. A learning model is a model that needs to be trained to obtain an optimized result, typically but not always a supervised training model. Examples include neural networks, naive Bayes classifiers, decision trees, support vector machines, etc. To create a training model for the learning model 125 using the simulator 110, the simulator 110 first runs the digital twin for a period of time, for example from a point in time prior to the simulation, such as t(-300) (representing 300 time units from when the actual simulation takes place) to t(0) (when the simulation begins). The digital twin may be a heterogeneous node system. This heterogeneous node system may be a heterogeneous neural network. The input 105 may represent external conditions. The external conditions act on the building over time, such as weather, humidity, vibration, magnetism, light, pressure, moisture, etc., and are communicated to the digital twin and used by the optimizer. This state can then be propagated through the different layers, affecting the structure over the time of the simulation. The input may be a state curve that runs for the same amount of time as the simulation itself, so in this example, from t(-300) to t(0). The simulator 110 itself may model a digital twin of a building described as nodes. These nodes may represent various chunks of a building. The constitution of the chunks may vary depending on the particular implementation. In some cases, the chunks represent large structures in a building, such as rooms. In some implementations, the chunks represent building parts, such as walls, windows, floors, and ceilings. In some representations, the chunks represent individual components of a building, such as layers in a wall. That is, a wall may be modeled as a particular type of drywall, then a particular type of insulation, then a particular layer of drywall, and so on. In some implementations, both chunks representing large structures and individual components, etc. may also be used. The nodal values ​​may then be used to determine the thermo-mechanical behavior of the building.Since individual nodes may represent different building units, the nodes are heterogeneous and may therefore have different properties. An attribute may be said to be a state value. A "state," as used herein, may be air temperature, radiant temperature, atmospheric pressure, sound pressure, occupancy amount, indoor air quality, CO2 concentration, light intensity, or other condition that can be measured and controlled. An attribute may also be, for example, a value associated with the building unit that the node represents in some embodiments. Attributes associated with equipment may also be used in nodes, etc., as appropriate. For example, a node may have attributes such as specific heat ratio, efficiency, specific enthalpy, resistance, etc. These attributes may be used in equations to determine how a state changes over time. An example of a node is shown with reference to FIG. 14.

[0055] The inputs 105 can be passed within the heterogeneous node model to locations that correspond to the areas they act on in the building. Thus, for example, a weather input can be passed to a node representing an exterior wall, a solar input can be passed to a node representing a window facing the sun, and so on. The optimizer generates start value node states as inputs 105 for the initial simulation start time. After running a simulation with the optimizer start values, an output 115 is generated. This output is the state of the nodes in the simulation versus the simulation time-state time curve. The values ​​of selected nodes, such as the node representing indoor temperature, can be measured against a set of ground truth state curves. The ground truth state curves can be actual temperatures measured over time in the space represented by the simulator. For example, the indoor temperature in a building can be measured over time, and the temperature and other state values ​​at that location can also be measured simultaneously. At the end of the simulation, the values ​​of selected nodes can be measured against the ground truth vectors. These selected node values ​​can be used in a cost function along with the ground truth values. The cost function can measure the output (e.g., historical values ​​of room temperature) against ground truth to determine the cost. The optimizer can then use the cost to refine the initial values. At that point, the optimizer is rerun with new initial values ​​for the nodes in the simulator. Selected node values ​​from the output, along with state dates, are passed through the learning model as new training examples. This refinement can continue until a stopping state is reached. Once a stopping state is reached, i.e., the simulator 110 (e.g., a heterogeneous node system) has an optimal solution, a particular number of cycles have run, the model has run for a maximum time, etc., the solution can be used as a starting value to estimate a starting state for the final simulation 215A. This is described in more detail with reference to FIG. 2A.

[0056] When using the simulator output 115 as input for the learning model, the values ​​of selected nodes from the simulator can be from simulation time (e.g., from time t(-300) to t(0)) and are inverted so that they go backwards from time t(0) to t(-300). This inverted output 130 is then used as input 135 for the training model 125. Similarly, some of the simulation inputs become the ground truth against which the learning model output 140 is compared. The output of the learning model and the inputs of the simulator, the learning model can have a node parallel to each node (or most of the nodes) in the simulation 110. The simulation can be a heterogeneous node simulation. The inputs used by the simulator model are similarly inverted (120) and can also be used as inputs to the learning model 125. The learning model can be configured to output 140 states in reverse time order, such as a node representing the internal temperature in a room in a building. The original output from the simulation model run at the start of the model run (e.g., t(-300) in this example) is then used as the ground truth. This ground truth is compared to the trained model output in a cost function to determine a cost. This cost can then be used to backpropagate through the trained model to improve the model's output. According to the Wikipedia entry "backpropagation", "The backpropagation algorithm works by computing the gradient of the loss function with respect to each weight in the chain rule, computing the gradient one layer at a time, and iterating backwards from the last layer to avoid redundant computation of intermediate terms in the chain rule." Backpropagation techniques are known to those skilled in the art. When the trained model is run with the inputs and outputs from the optimizer simulation, the trained model simulation gets better and better, and the results it gets get closer to the results it gets with the optimizer it was trained on.

[0057] FIG. 2 discloses at 200 the interaction between the learning model and the simulator when the learning model is fully trained. The desired output values ​​205 of a subset of the nodes from time t(0) to time t(-n) (e.g., historical room temperature values ​​for the same time period as the simulator run) are used as inputs to the learning model 210, in addition to other state information, e.g., weather, also reversed from t(0) to t(-n). The learning model then outputs values ​​215 for all (or almost all) of the nodes at time t(-n), i.e., the starting state for the initial value simulation 205A. These values ​​can also be used as a sanity check 220 to ensure that the learning model is generating valid outputs, as will be described with reference to FIG. 17. In brief, the outputs 215 from the learning model 210 are reversed and used as inputs 225 to the simulator 230. Selected outputs 235 from the simulator 230 are checked against the ground truth, i.e., desired outputs, to see how well the trained model is performing.

[0058] FIG. 2A discloses at 200A a set of simulations that can be run to accurately determine starting values ​​for warming up the digital twin. This provides greater confidence that the simulation will run with accurate starting values. The system represented by the digital twin can then be simulated over time to determine how it will perform in the future with various inputs, different values, etc. This simulation can then be used to determine how to operate the equipment, etc., in the system that the digital twin is simulating. First, an initial value simulation is run as described with reference to FIGS. 1 and 2. This generates initial response values ​​for the nodes in the simulation model. The model outputs 115, 215 correspond to the nodes in the simulation model, such as the simulator 110, and are then used as initial values ​​for the nodes in the state estimation simulation 210A. The state estimation simulation 210A runs from a time t(-n) prior to the start of the simulation to a time t(0) when the simulation begins. This state estimation simulation is run using the same node system that is run for the state simulation 215A. That is, the state estimation simulation modifies the nodes in the state simulation so that they have valid starting values. At the start of the simulation at time t(0), the nodes in the simulation model must have reasonable starting values ​​so that a state simulation 215A (e.g., a state simulation of a structure) can be run. Once a simulation is run, the output from the simulation can be used downstream to change the simulated system 220A. For example, a digital twin simulation may include a floor of a building. A state simulation can be run to optimize energy usage within that floor of the building.Once energy usage is optimized, device simulations can be performed to determine optimal (or moderately better than current) HVAC conditions for a time period. These optimal HVAC conditions determined by the digital twin simulation can then be used to operate equipment within a floor of a building to effect system changes 220A. In an embodiment, the digital twin state simulation 215A can provide control sequences for specific equipment. These control sequences can then be used to operate the specific equipment for a simulation period, i.e., from t(0) to t(n). For example, the digital twin simulation can provide a control sequence for a heater, specifying when to turn it on and off, resulting in specific system changes 220A. In an embodiment, the simulation can also provide control sequences for periods shorter or longer than the simulation period, and the simulation output can be used along with the control sequences. Disclosed herein are methods and systems for initial value simulation 205A to determine initial values ​​for state estimation simulation 210A.

[0059] FIG. 2B discloses at 200B the relationship between optimizer 205B and simulator 210B. Optimizer 205B determines values ​​for some of the nodes in simulator 210B that, when optimized, will produce desired values ​​for at least a subset of the nodes at the end of the simulator's simulation. The optimizer may do this by an iterative machine learning system, as discussed in other chapters. Optimizer 205B may use the desired values ​​for a selected set of nodes as ground truth to determine optimal node values. This is described in more detail with reference to 525 in FIG. 5. The optimizer may run the simulation multiple times using different starting values ​​to determine a set of optimized starting values ​​for use in state estimation simulation 210A. Each (or most or some) of the simulation runs using the initial values ​​determined by the optimizer may be used as training data 215B for a learning model, e.g., 125.

[0060] II. Exemplary Computing Environment 3 illustrates a generalized example of a suitable computing environment 300 in which the described embodiments may be implemented. The computing environment 300 is not intended to suggest any limitation as to the scope of use or functionality of the present disclosure, as the present disclosure may be implemented in a variety of general-purpose or special-purpose computing environments.

[0061] Referring to FIG. 3, the core processing is illustrated by the core processing 330 box. The computing environment 300 includes at least one central processing unit 310 and memory 320. The central processing unit 310 executes computer executable instructions and may be a real or virtual processor. It may also include a vector processor 312 that allows for fast processing of node strings of the same length. In a multi-processing system, to increase processing power, multiple processing units execute computer executable instructions and thus the vector processor 312, GPU 315, and CPU 310 may run simultaneously. The memory 320 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of the two. The memory 320 stores software 385. The software 385 implements the method and system for estimating state simulation starting states described using optimizers and / or learning models.

[0062] A computing environment may also have additional features. For example, computing environment 300 includes storage 340, one or more input devices 350, one or more output devices 355, one or more network connections (e.g., wired, wireless, etc.) 360, and other communication connections 370. An interconnection mechanism (not shown), such as a bus, controller, or network, interconnects the components of computing environment 300. Typically, operating system software (not shown) provides an operating environment for other software executing in computing environment 300 and coordinates the activity of the components of computing environment 300. A computing system may also be distributed, with multiple portions of software running on different CPUs.

[0063] Storage 340 may be removable or non-removable and includes magnetic disks, magnetic tapes or cassettes, CD-ROMs, CD-RWs, DVDs, flash memory, or any other medium that can be used to store information and that can be accessed within computing environment 300. Storage 340 stores software instructions, such as software 385 that implements a system and method for warming up a simulation model that relies on states being at legal values.

[0064] The input device(s) 350 may be any device that allows a user or other device to communicate with the computing environment 300, such as a touch input device like a keyboard, a video camera, a microphone, a mouse, a pen, or a trackball, a digital camera, a scanning device like a digital camera with a scanner, a touch screen, a joystick controller, a wii remote, or other device that provides input to the computing environment 300. For audio, the input device(s) 350 may be a sound card or similar device that accepts audio input in analog or digital form, or a CD-ROM reader that provides audio samples to the computing environment. The output device(s) 355 may be a display, a hardcopy generating output device like a printer or plotter, a text-to speech voice-reader, a speaker, a CD-writer, or other device that provides output from the computing environment 300.

[0065] The communication connection(s) 370 enable communication with other computing entities over a communication medium. The communication medium conveys information such as computer-executable instructions, compressed graphics information, or other data in a modulated data signal. The communication connection(s) 370 may include input devices 350, output devices 355, and input / output devices that allow the client device to communicate with other devices over the network 360. The communication devices may include one or more wireless transceivers to perform wireless communications, and / or one or more communication ports to perform wired communications. These connections may include network connections, which may be wired or wireless networks, such as the Internet, an intranet, a LAN, a WAN, a cellular network, or other types of networks. It will be appreciated that the network 360 may be a combination of multiple different types of wired or wireless networks. The network 360 may be a distributed network having multiple computers and building controllers working in conjunction. The communication connection 370 may be a portable communication device, such as a wireless handheld device, a personal electronic device, or the like.

[0066] A computer readable medium is any available non-transitory tangible medium that can be accessed within a computing environment. By way of example, and not limitation, in computing environment 300, computer readable media includes memory 320, storage 340, communication media, and any combination of the above. Computer readable storage media 365 that can be used to store computer readable media includes instructions 375 and data 380. The data source may be a computing device, such as a general hardware platform server configured to receive and transmit information via a communication connection 370. Computing environment 300 may also be an electrical controller directly connected to various resources, such as HVAC resources, having CPU 310, GPU 315, memory 320, input devices 350, communication connection 370, and / or other features shown in computing environment 300. Computing environment 300 may be a series of distributed computers. These distributed computers may also be comprised of a series of connected electrical controllers.

[0067] For convenience of presentation, the operations of some of the disclosed methods may be described in a particular sequential order, but it should be understood that this style of description encompasses rearrangements, unless a particular order is required by specific language set forth below. For example, operations described sequentially may also be rearranged or performed simultaneously. Moreover, for simplicity, the accompanying drawings may not show the various ways in which the disclosed methods, apparatus, and systems can be used with other methods, apparatus, and systems. In addition, the description may use terms such as "determine," "build," and "identify" to describe the disclosed technology. These terms are high-level abstractions of the actual operations that are performed. The actual operations that correspond to these terms may vary depending on the particular implementation, but are readily identifiable by one of ordinary skill in the art.

[0068] Additionally, data generated from any of the disclosed methods can be created, updated, or stored on a tangible computer readable medium (e.g., one or more CDs, a volatile memory component (such as DRAM or SRAM), or a non-volatile memory component (such as a hard drive)) using a variety of different data structures or formats. Such data can be created or updated locally on a computer or across a network (e.g., by a server computer), or can be stored and accessed in a cloud computing environment.

[0069] FIG. 4 illustrates a distributed computing system 400 in which embodiments disclosed herein can be implemented. Two or more computerized controllers 405 can incorporate all or part of the computing environment 300, 410. These computerized controllers 405 can be connected 415 to each other using wired or wireless connections. The controllers can also be placed in a controlled space 420. The controlled space 420 can be a space that has resources, sensors, or other equipment that can change or determine one or more states of the space, such as sensors (to determine space state), heaters, air conditioners (to change temperature), speakers (to modify noise), locks, lights, etc. The controlled space can be divided into zones that may have separate constraint state curves. The controlled space can be, for example, an automated building, a process control system, an HVAC system, an energy system, an irrigation system, a building-irrigation system, etc. These computerized controllers 405 can also constitute a distributed system that can operate without a connection (such as an Internet connection) outside of the computing system 400 itself. This allows the system to run with low latency and other benefits of edge computing systems. The computerized controller can also run using the internal network without an external network connection. This allows for a much harder and much more secure system against external attacks like viruses, ransomware, and in general against computer security being compromised in many other ways.

[0070] III. DISCLOSURE OF AN EXEMPLARY SYSTEM FOR TRAINING A SIMULATOR FIG. 5 discloses an overview 500 of a training process for training a learning model by using an optimizer. The optimizer does not need to train itself to solve the problem, i.e., it is not trained itself. The optimizer may be a machine learning algorithm, as known to those skilled in the art. The machine learning algorithm selects initial state values ​​for the node inputs. This initial selection may be random. These values ​​are then used as initial state values ​​in a simulation 110, 205A that runs from time t(0) to t(n). The desired values ​​for the state values ​​at time t(n) are known, i.e., the ground truth. The values ​​generated by the simulation at t(n) are compared to the ground truth using a cost function that determines the cost. The cost is used by the machine learning algorithm to select a new set of values ​​that should produce values ​​that are closer to the ground truth at the end of the simulation. When the simulator 110 is run, the output is used to train the learning model 125.

[0071] The simulator 110 may be a heterogeneous neural network. The neural network may have activation functions at the nodes that perform separate calculations to determine the weight values ​​that flow into and out of the nodes. These separate calculations may be thermodynamic calculations that determine how state flows through the nodes. State may be characterized as weights that enter and exit the nodes. An example of such a heterogeneous neural network is described in U.S. Patent Application No. 17 / 009,713, “Neural Network Methods For Describing System Topologies,” filed September 1, 2020, which is incorporated herein by reference in its entirety. A typical neural network includes inputs, outputs, and hidden layers connected by edges, where the edges have weights associated with them. The neural net sums the weights of all the incoming edges, applies biases, and then uses activation functions to introduce nonlinear effects, essentially squashing or expanding the weight / bias values ​​into a useful range. In many cases, a node is determined to be inherently firing or not. This new value then becomes the weight used to connect to the next hidden layer of the network. Activation functions are not separate calculations for some nodes, they do not have physics equations associated with them, etc.

[0072] Some heterogeneous neural networks that can be used with the embodiments described herein utilize the fundamentals of physics to model components or devices one-to-one with neural net nodes. When multiple components are linked together in a schematic diagram, a neural net is created that models these components as nodes. Values ​​between objects flow between the nodes as weights of the connected edges. These neural nets can model not only the actual complexity of systems, but also their emergent behavior and system semantics. Thus, they can bypass two major steps of traditional AI modeling techniques: determining the shape of the neural net and training the neural net from scratch. The nodes are arranged in the order of the actual system (or set of equations), and the nodes themselves contain an equation or set of equations that describe the functions of their associated objects, so that the specific relationships between them are determined by their position in the neural net. Thus, most of the training is no longer necessary. Because the neural net itself contains location information, behavior information, and interaction information between the different objects represented by the nodes. Moreover, the values ​​held by the nodes in the neural net at a given time represent the real-world behavior of the objects so represented. The neural net is no longer a black box, but houses important information in itself. This neural net structure also provides much deeper information about the system and objects it describes. Unlike traditional AI structures, because the neural network is based on physics and location, it is not limited to a specific model, and multiple models of the system it represents can be run without the need for separate creation and training.

[0073] In some embodiments, the heterogeneous neural network shapes the node positions to tell something about the physics of the system. Also, the actual equations can be put into the activation functions. The weights moving between the nodes can be variables in the equations. Different nodes can have unrelated activation functions depending on the nature of the model being represented. In an exemplary embodiment, each activation function in the neural network can be different. For example, a pump can be represented in a neural network as a series of network nodes, some of which represent efficiency, energy consumption, pressure, etc. A node has a set of weights (variables) that feed into the next node (e.g., which has an equation as its activation function), and these weights (variables) are used by the next node. Here, the two steps previously required, namely, shaping the neural net and training the model, may already have been performed, at least to some extent. Using the embodiments discussed herein, it is not necessary to train the neural net model on information already known. It still needs to be trained on other information, as detailed herein.

[0074] In one embodiment, each node represents a physical representation of a basic unit of building material within a structure, equipment, etc. These individual nodes can hold parameter values ​​that help define the physical representation. Thus, when running the neural net, the parameters that help define the physical representation can be tweaked to more accurately represent a given physical representation. This has the effect of pre-training the model with a qualitative set of guarantees, because the physical equations that describe the objects being modeled are true, and it avoids having to discover a training set and then using a large amount of computation time to run the training set through the model and train them. The model does not need to be trained with information about the world that is already known. Just like objects connected to a neural net, they are connected in the real world, and emergent behaviors arise in the model that map to the real world. This revealed model behavior is computationally too complex to determine in any other way. Furthermore, the nodes represent real objects, not just black boxes. The behavior of the node itself can be examined to determine the behavior of the object and can also be used to further understand the object behavior.

[0075] Conceptually, optimizing a structural simulation model involves inserting a quantity of some measurable state into the structure that can be modeled by the heterogeneous network. This can be temperature, humidity, acoustics, vibration, magnetism, light, pressure, moisture, etc. This state then propagates through the different layers and acts on the structure. The input data 505 may be such states represented by time curves. As an example, an external node in the heterogeneous network may be associated with a time-thermal curve T. The curve value at time t(-n) is injected into one or more external nodes. By modifying the values ​​of the nodes of the simulation neural net using a physical representation, this temperature propagates into the building, for example through the represented layers (blocks, insulation, drywall, etc.). The outside of the building (e.g., blocks) has its temperature modified by the external node and the manner of heat transfer through the blocks, as seen at the block nodes. The blocks then heat the next layer, perhaps the insulation. This continues throughout the building until an interior node is reached. This could also represent an interior space of a room. At the interior nodes, other functions can be applied, such as functions representing lighting and people (heat) within that node's zone. The state information continues to propagate until other exterior layers are reached, with the individual node parameter values ​​representing the heating occurring within the building at time T0. In one embodiment, each exterior surface has its own time-temperature curve. In one embodiment, the structure is decomposed into smaller subsystems (or zones), so that rather than propagating temperatures throughout the entire structure, only parts of the structure are affected by a given input.

[0076] The optimizer 510 selects initial values ​​for the states and then passes these to the simulation as optimizer output 511. To select the initial values, the optimizer uses a machine learning algorithm that does not require training or has already been trained. The optimizer runs iteratively and ideally selects better node initial node values ​​with each iteration. The simulator runs with the starting values ​​selected by the optimizer for each iteration (or every few iterations). As output 513, the nodes in the simulator generate node values ​​over the length of the simulation, as described in FIG. 6. Of these nodes, it outputs a subset as selected node values ​​515. These are state / time curve(s) that run from the start to the end of the simulation. In one embodiment, these selected node values ​​are the temperature of rooms in the structure. Other states can be used as well. The ground truth for the machine learning algorithm can be the state / time curves of desired node values ​​520. These desired node values ​​can be taken from previous simulations, can be approximated, can be historical values, etc. The cost function of the machine learning algorithm uses the selected node value 515 and the desired node value 520 to run a cost function 527 to determine a cost 525. The cost 525 is the difference between the predicted selected node value 515 from the simulator 512 and the expected value (ground truth) 520.

[0077] A "cost function" generally compares the output of a simulation model to the ground truth, i.e., a time curve that represents the answer that the model is trying to match. This gives the cost-the difference between the simulated truth curve value and the expected value (ground truth). The cost function may use a least squares function, Mean Error (ME), Mean Squared Error (MSE), Mean Absolute Error (MAE), Categorical Cross Entropy Cost Function, Binary Cross Entropy Cost Function, etc. to arrive at the answer. In some embodiments, the cost function is a loss function. In some embodiments, the cost function may also use a threshold. The threshold may be a single number that indicates the simulated truth curve is close enough to the ground truth, may be a range of numbers where values ​​within the range indicate that the simulated truth curve is close enough to the ground truth, etc. The threshold may be predetermined, determined by an internal value, etc. In other embodiments, the cost function may be a gradient. The slope may also indicate that the simulated truth curve and the ground truth are sufficiently close. When a cost function is used, it may be time-varying. It may also be linked to factors such as user preferences or changes in the physical model. The cost function applied to the optimizer may include any one or more models from energy use, primary energy use, energy monetary cost, human comfort, safety of the building or contents, durability of the building or contents, microbial growth potential, system equipment durability, system equipment life, environmental impact, and / or energy use CO2 potential, or any other. The cost function may also utilize a discount function based on the discounted future value of costs.

[0078] In some implementations, the objective function is used to determine if the cost is such that the operation may be stopped. Some implementations also include secondary states, such as a stop state, other quit options, such as quitting when the optimizer 510 or simulator 512 has run for a certain amount of time, or quitting when it has run for a certain number of cycles. If indicated (by a stop state, objective function, or cost function), the machine learning algorithm continues. In some implementations, the optimizer 510 uses information from the output of the simulation, such as the simulation predicted selected node values, and the cost function, to update the initial state values. This value update may also be implemented, at least in part, by backpropagation 530. After the update, the optimizer 510 then selects new state node values ​​for the next round of simulation. The optimizer 510-simulation 512 cycles may run until a stopping condition (defined by some combination of one or more of the cost function, the time the optimizer has run, the time the optimizer and simulator have run, the time the entire program has run, the number of cycles the program has run, the number of training runs performed by the learning model, etc.) is reached.

[0079] FIG. 6 discloses at 600 some output of the simulation 512, 605 with starting values ​​generated by the optimizer 510. The simulator 512, 605 outputs (625) time-state curves 615 for all (or some selected quantity) of the nodes 610 in the simulator 605 as simulator output. These time-state curves continue for the length of the simulation or for a period of time within the simulation. The initial simulator node values, i.e., the node values ​​at the start of the simulation (t=(-n)), are then saved for use as training examples for the learning model. In some cases, these node values ​​at the start of the simulation are also used as input to the state estimation simulation 210A to estimate the state of the model and generate the valid state simulation 215A.

[0080] FIG. 7 discloses at 700 the use of simulator model output as learning model input. The selected node values ​​output 515 from simulator 512 are inverted 705 so that the state / time curve is reversed back and forth. That is, instead of t(-n) to t(0), it now goes from t(0) to t(-n). This results in the reversed selected node values. Conceptually, one can think of running the simulation as going backwards in time. Similarly, the input data 505 to the simulator is also inverted 710 to get the inverse simulator input. This is then used as input to the learning model 715. The learning model 715 may be any type of neural network that requires training, including, but not limited to, recurrent neural networks, convolutional neural networks, feed-forward neural networks, multi-layer perceptrons, long / short term memory (LTSM), gated recurrent units (GRUs), autoencoders (AEs), variational autoencoders (VAEs), denoising AEs (DAEs), deep convolutional networks (DCNs), etc. The learning model must be configured to have output nodes that are the same as all (or essentially all, or a portion of) the nodes in the node system 210B used in the simulator. Thus, the learning model outputs state / time curves for these nodes 725. The ground truth for the simulation is the starting node values ​​620, 720 for a given optimizer-simulator run. The learning model reverses the order of the state / time curves and outputs the state / time curve for the node. The learning model cost function 737 then compares the last value for the node 730 (e.g., at time t(-n)) with the starting value 620, 720 of the node from the run of the simulator 600 to arrive at a cost 735.The cost values ​​are then used in backpropagation 740 to refine the internal node values ​​of the learned model 715. Backpropagation propagates the costs throughout the neural network and modifies the weights so that nodes with higher error rates have smaller weights, while nodes with lower error rates have larger weights.

[0081] Turning now to Figures 8 and 9, Figure 8 shows an example 800 of a heterogeneous neural network model optimizer 810. The optimizer 810 is used both to generate starting values ​​used to estimate initial states for a state estimation state estimation run of a building simulator, and to train an RNN to obtain the same or similar starting results. RNN stands for Recurrent Neural Network. Recurrent neural networks were developed in the 1980s and are well known to those skilled in the art. In simple terms, an RNN has loops from one node to itself, between nodes, etc., allowing the RNN to essentially remember information and therefore make decisions based on previous inputs. Figure 9 shows an example of training an RNN learning model 915 using input values ​​taken from the optimizer's input and output values.

[0082] First, the temperature inside the rooms of the building to be modeled over a time period and the weather data (weather at the building location for the same time period) are determined. The optimizer 810 determines the starting temperature values ​​as optimizer output 811 for the nodes in the heterogeneous node model 812. The heterogeneous node model 812 is then run, using the weather data 805 (which may be temperature data) as input at the actual building location equivalent to the location in the heterogeneous node model that inputs the weather values, such as the node representing the exterior wall. The heat values ​​from the weather data 805 are propagated throughout the building, for example, the heterogeneous node model 812, during the simulation run time, in this case from t(-300) to t(0). The heterogeneous node model 812 returns the interior node value temperature 815 (at the simulation time) of the room as represented by the simulation. The internal node room temperature values ​​815 are then compared to the historical internal temperature state / time curve 820 over the length of the simulation using a cost function 827 to determine a cost 825 at the end of the simulation 835. Typically, the entire time curve (or a large portion of it) is used because if only the temperature for the last time value was used, for example, the optimizer solution may not represent a reasonable temperature state estimate. For example, the model may represent a temperature that rises sharply at the last time value, which would not provide the desired accurate initial temperature. In one embodiment, a portion of the time curve is used for the internal node values ​​815. The cost function uses the selected internal node values ​​815 and measures the historical node temperatures 820 against the ground truth using a cost function 827 to determine the cost 825. The cost 825 is then used by the optimizer 810 to modify the starting node temperature value, and the modified temperature value is then used as the starting node value within the heterogeneous node system. This set of actions is repeated 830 until a stopping condition is reached, producing an optimized set of internal temperature node values ​​for all (or most) of the nodes in the heterogeneous node model 812.When the heterogeneous node model 812 runs, it reports temperature values ​​of its nodes 845 (which can have many state values, such as temperature, humidity, etc.) over the simulation time, for example, from time t(0) to time t(-300). In one embodiment, the heterogeneous node system can also report node values ​​throughout the model run, rather than at the end. This is described in more detail with reference to Figures 12 and 13. The initial values ​​of the heterogeneous node model's nodes 840 at the end of run 835, i.e., at time t(-300), are used as ground truth for training the RNN. The optimizer 810-heterogeneous node model 812 cycle continues until a stop state is reached. Thus, when a time starting value for the state estimation simulation is determined, many training examples for the RNN are generated.

[0083] FIG. 9 shows an example 900 of training an RNN learning model 915 using the results of the optimizer 810-heterogeneous node model 812. The RNN must have nodes as outputs that correspond to the nodes in the optimizer model. This is discussed in more detail with reference to FIG. 12 and FIG. 13. Thus, the RNN can generate starting node values ​​that can be used for state estimation simulations, e.g., 210A. The internal node values ​​815 (temperature inside the room) from the heterogeneous node model 812 are rewritten and reversed 905, e.g., running from t(0) to t(-300). The weather data 805 is also rewritten 910 in reverse. These reversed internal node values ​​905 and reversed weather data 910 are then fed as inputs to the RNN, which outputs values ​​925 for each (or most) of the nodes in the heterogeneous node model. The RNN 915 returns a last value 930 (in this example, at time (t-300)) to the nodes in the RNN. This last value 930 can be understood as the first value, for example at t(-300), when the temperature curve is reversed forward in time again.

[0084] These output values ​​930 are then used in a cost function 945 along with the heterogeneous node model node temperature values ​​840, 920. The heterogeneous node model node temperature values ​​840, 920 are sampled at the beginning of the warm-up period (e.g., t(-300)) to determine a cost 940. This cost 940 is then used in backpropagation 935 to train the network. In this process, the RNN is trained using the inputs and outputs from the optimizer until the RNN is considered trained. Backpropagation is short for "backpropagation" and is widely used to train neural networks sequentially. According to the Wikipedia entry "backpropagation", "The backpropagation algorithm works by computing the gradient of the loss function with respect to each weight through the chain rule, computing the gradient one layer at a time, and repeating backwards from the last layer to avoid redundant computation of intermediate terms in the chain rule."

[0085] 10 discloses the execution 1000 of the RNN after it has been trained. The desired reversed backward internal node values ​​1005, represented as a state-time curve over the length of the warm-up state estimation simulation, in this case from t(0) to t(-300), where "t" stands for "time in time," and the reversed weather data 1010, are used as input to the RNN 1015. The RNN then outputs a final value 1020 of the RNN (e.g., at t(-300)). This final value becomes the initial value input to the warm-up state estimation simulation 210A to estimate the initial starting value of the normal simulation 215A.

[0086] IV. EXEMPLARY SIMULATOR TRAINING METHODS 1100A in FIG. 11A and 1100B in FIG. 11B are flow diagrams describing how a simulator can be trained and how the trained output can be used. The operations of methods 1100A, 1100B described below are intended to be exemplary. In an embodiment, methods 1100A, 1100B can also be performed with one or more additional operations not described and / or without one or more of the operations discussed. Additionally, the order in which operations of methods 1100A, 1100B are shown in FIG. 11A and FIG. 11B and described below is not intended to be limiting.

[0087] In an embodiment, the computer-implemented methods 1100A, 1100B may be implemented in one or more processing devices (e.g., digital processors, analog processors, digital circuits designed to process information, analog circuits designed to process information, state machines, and / or other mechanisms for electronically processing information, multi-processing systems, etc.). The one or more processing devices may include one or more devices that perform some or all of the operations of the methods 1100A, 1100B in response to instructions stored electronically on an electronic storage medium. The one or more processing devices may include one or more devices configured with hardware, firmware, and / or software specifically designed to perform one or more of the operations of the methods 1100A, 1100B.

[0088] In operation 1105, an optimizer is run to determine (some) initial node values ​​for a simulator that simulates a structure, process, etc. These node values ​​may be initial state values, such as temperature, for multiple nodes. These nodes in the simulator may have physics equations associated with them, as described, for example, with reference to FIG. 14 and the surrounding text. The optimizer is described with reference to 110 in FIG. 1, 510 in FIG. 2B, and 810 in FIG. 8. These physics equations may model how states (such as input time series) move through the nodes over time. If the optimizer is run after a learning model has finished running, the optimizer may start from values ​​previously generated from the learning model. An example of such a previous execution of the learning model may be seen with reference to FIG. 17, more specifically at 1705 and 1730. In operation 1108, the output of the optimizer (e.g., 811) is used as the starting node values ​​for some or all of the simulator initial node values. In operation 1110, an input time series is used as an input to the simulator. In some embodiments, many time series may be used as input. 505 in FIG. 5 and 805 in FIG. 8 show examples of input time series. In operation 1112, the simulator is run. At the end of the simulation (e.g., at the end of the input time series), an optimizer output is generated. The optimizer output may consist, in part, of selected node output values. 515 in FIG. 5, 610 in FIG. 6, and 820 in FIG. 8 describe aspects of the simulator output. FIG. 12 describes further details. In operation 1115, a cost function is used to compare the selected node output values ​​to the ground truth to determine the optimization cost. In operation 1120, the output of the optimizer simulation is inverted. Examples are shown with reference to 705 in FIG. 7 and 905 in FIG. Also in operation 1120, the optimizer input is inverted. An example of this is shown with reference to 710 in FIG. 7 and 910 in FIG.In operation 1125, the inverted optimizer simulation output and at least a portion of the inverted optimizer simulation input are used as inputs to a learning model, examples of which are shown with reference to 705 and 710 in FIG. 7 and 905 and 910 in FIG.

[0089] In operation 1130, the learning model is run to generate a reverse time series from t(0) to t(-n) as the learning model output. This learning model output is mapped to some (or all) of the nodes used in the simulation. In operation 1135, the learning model output value at time t(-n) is used as an input to a cost function, e.g., 737, 945, to determine a cost 735, 940. The optimizer simulation output start node value (620, 720) is used as the ground truth for the cost function 737, as described in more detail with reference to FIG. 7 and the surrounding text. To determine the cost 735, i.e. the difference between the ideal output (ground truth) and the actual learning model value, the learning model value for that node at its end (actual start) is compared to the physical model output at the start. An example of this is shown with reference to 720, 730, 735, and 737 in FIG. At operation 1140, the calculated cost is then used to perform backpropagation of the learning model to further train the training model. At decision point 1145, it is determined whether the optimizer, i.e., the simulation, has reached a stopping state. This stopping state may be when the cost from the cost function of the optimization simulation output (see 1115) is small enough to indicate that the initial warm-up state estimates for the simulation model are optimized. Other stopping states may be used, such as the number of simulator iterations, the total amount of time the model has run, etc. If not, the process repeats at operation 1105. If it is determined to be in a stopping state, then at operation 1147, a (warm-up) state estimation simulation is run using the optimizer output (the starting state values ​​for the simulation). As shown with reference to FIG. 6 and FIG. 8, the optimizer output is the starting node state value for the warm-up state estimation simulation. The warm-up state estimation simulation may be run from the time the initial value simulation was first run, for example, from a time prior to the simulation (t=-n) to a time when the simulation begins (t=0).

[0090] In act 1149, the optimizer output is saved for the next iteration of the optimizer, if necessary. The saved output can be used, for example, as a starting value for the optimizer in act 1105. In act 1150, a state simulation is run using the warmed up simulation. Specifically, the state simulation is run for an appropriate amount of time, such as from time t=0 forward to t(m), where m is equal to a positive number. The state simulation can be an extension of running the warm up state estimation simulation. In act 1155, after running the state simulation, the results of the state simulation can be used to modify the behavior of the structure that the simulation is modeling. One way that the behavior of the structure can be modified is by using the simulation output to ultimately dictate the behavior of the device. The structure can contain various devices that can be controlled. These devices can be scientific instruments, HVAC equipment, etc. These devices can be modeled by separate device simulations 1160. The results of this device simulation can be control sequences for the modeled devices. A simulation can optimize the behavior of devices to reach a particular outcome. For example, a simulation of a building may seek to optimize energy usage. Thus, devices such as conditioners, movable ventilators, heaters, boilers, etc. can be modeled within the building structure. In operation 1165, the state simulation can generate outputs, such as control sequences, that can modify the behavior of the structure being modeled by the simulator. These control sequences can then be used to operate at least one device in operation 1170. For example, the control sequences generated by the state simulation can be used to operate an air conditioner.

[0091] FIG. 12 discloses at 1200 an example simulation model 1235 having example nodes in which embodiments disclosed herein can be implemented. When the simulation model is run, input data 505, 1205 is provided to particular nodes. Input data for the optimizer run of state simulation is described in more detail with reference to FIG. 5. In some implementations, the nodes may be arranged according to levels. For example, nodes in the first level 1220 may be provided with input data 505, 1205 from N1 to N a As shown by the ovals, not all nodes may be shown. Below this level, nodes are numbered P1 through P a The other levels 1230 are nodes Q1 to Q C In some implementations, the nodes do not have to be arranged according to levels. Input data 1205 (e.g., condition data 505, such as weather data) can be provided to nodes in the first level 1220 and to nodes in different levels 1210. Nodes at one level can provide information to nodes other than those in the next level. For example, node N a (at level N) skips level P and sends information 1225 to node Q C (at level Q). There may also be other node levels, such as R, but for clarity of the drawing, the inputs and outputs are not shown. As shown with reference to P2, 1215, nodes at one level do not necessarily send information to other nodes at the next lower level. The entire set of nodes for the optimizer in the current example is N1 through N a , P1~P b , Q1~Q C , R1~R d It can be written as:

[0092] The model output, e.g., selected node value 515, may not be a particular layer, but rather a collection of nodes 1240 deep with the model. In the illustrated example, N3 (the third node from row N on the left), P10 , P 21 , Q7, and Q 15 may all be considered as selected node values. Other output nodes may not be shown. There may be reasons for considering this particular set of nodes as output nodes, for example that they represent states that have actual measurements in a corresponding real world structure. For example, the selected node may represent the interior of a room, and the temperature of the room represented by this node may be the temperature that the node value represents. Thus, the selected node may be a node that represents the interior of a room in a building, and the selected node value is the temperature of the selected node.

[0093] FIG. 13 discloses at 1300 an example learning model and output that can implement embodiments disclosed herein. The inputs 1305 can be the outputs of a selected set of nodes 1240 from the simulation. The state outputs (such as temperature) of these nodes are then inverted 1305 and then used as inputs 1310 to the learning model. In this example, the learning model has an output 1315, which is then applied to nodes N1-N in the simulation model 1235. a , P1~P a , Q1~Q a , R1~R a That is, the nodes of the simulation model 1235 have corresponding outputs 1315 of the learning model. In one embodiment, some of the nodes in the output model have corresponding outputs in the learning model. Nodes can have multiple attributes associated with them. In such cases, one or more values ​​of a particular attribute information within the node can be recorded over the length of the simulation (t(0) to t(n)) and then used as the output 1240.

[0094] FIG. 14 discloses an exemplary node at 1400. The attributes 1410 are attributes of the object represented by the node, i.e., the building unit. For example, node 1405 may have temperature 1415 as an attribute 1410 that changes throughout the simulation. Other attributes, such as surface area 1420, may remain static. In one embodiment, the node may also include equations 1425 to be solved in the system. These equations may have both variables that are represented as weighted edges in the neural net and variables that are attributes 1410 of the node itself. Referring to FIG. 15A, a work flow 1500A is shown that can be used to estimate starting state values ​​for a state simulation. An optimizer 1505A is run to generate values ​​that, after optimization, are used as initial node values ​​within a state estimation simulation 1510A. The state estimation simulation 1510A is used to estimate suitable starting states for an actual simulation (e.g., 215A). There may be a one-to-one correspondence between the values ​​in the optimizer 1505A and the values ​​at the nodes in the simulation 1510A that are used for the state estimation function. This state estimation simulation 1510A can be run from a time prior to the simulation (-t) to the start of the simulation (0). When the simulation is at time (0), the same simulation model (with legal starting states within the nodes) can be run (1515A) for some time in the future.

[0095] Referring to FIG. 15B, a workflow 1500B is shown that can be used to estimate starting values ​​in a start state simulation estimation model using a trained learning model. Running the learning model 1505B produces values ​​that are used as initial node values ​​within the simulation 1510B. The simulation 1510B is used for state estimation. There may be a one-to-one correspondence between values ​​in the learning model output and nodes in the simulation that are used for the state estimation function. This start state simulation estimation can be run from a time (-t) prior to the simulation to the beginning of the simulation (0). When the start state simulation estimation is at time (0), the same simulation model 1515B can be run for a time in the future using the estimated node values ​​provided by the start state simulation estimation 1510B.

[0096] FIG. 16 discloses at 1600 node values ​​in an initial value simulation 1602, a state estimation run 1606 of a simulation model, and a simulation run 1608 of the simulation model. In the abbreviated model shown (for simplicity), 1602 has four nodes, as does the simulation model. These four nodes 1605, 1610, 1615, and 1620 generate state values ​​at the end of the initial value simulation, whether it is an optimizer or a learning model. These state values ​​within the nodes are used as initial values ​​for the corresponding state estimation runs 1604, 1606 of the simulation model used to simulate structures, processes, etc. in the digital twin. For example, at the end of the initial value simulation 1605, e.g., at time t(-n), node 1 equivalent has a temperature state value of 72°. This temperature state value is then forwarded to the simulation being prepared for the state estimation run 1604, with node 1 receiving the temperature state of 72°. This is true for the remaining nodes in this example. For example, node 2 1630 receives a temperature of 69° from node 2 1610 of the initial values ​​simulation 1602, node 3 1635 receives a node equal value of 54° from node 3 equal value 1615, and node 4 1640 receives a temperature of 74° from node 4 equal value 1620. In one embodiment, the state (e.g., temperature in this model) is transferred without change. In one embodiment, the state from the node equal value of the initial values ​​simulation 1602 undergoes a transformation before being transferred to the start of the state estimation run 1604. Once the state estimation run 1604 of the simulation has been seeded with the node values ​​from the initial values ​​simulation run 1602, it is run for a period of time, changing the state values. This is similar to what is seen for the node values ​​at the end of the starting state simulation estimation 1606.At the end of the start states simulation estimate, there may be a check to determine whether the simulation is sufficiently warmed up, i.e., whether the internal node values ​​are in a legal state (or states) to begin an actual simulation. If there is such a check and it is determined that the model is sufficiently warmed up, or if there is no check, then the main simulation run 1608 is started immediately after the state estimation run 1604. At the start of the main simulation run 1608 of the simulation model, the simulation models 1665, 1670, 1675, 1680 may have the same nodes 1645, 1650, 1655, 1660 as the nodes of the start states simulation estimate 1606. In that case, they will automatically have the same state values ​​that were generated at the end of the state estimation run 1606. For example, node 1 1665 at time t(0) has a value of 74°, which is the same value as node 1 1645 at the end of the state estimation run. In an embodiment, the state estimation end values ​​may also be transferred to a different simulation. In general, a node value at the start of a main simulation run 1608 of a simulation model may have the same value as a corresponding node in a state estimation run of the simulation at the end 1606 of the starting state simulation estimation.

[0097] FIG. 17 discloses, at 1700, the execution of a trained learning model using a sanity check. The validity of the learning model's output can be determined by passing the learning model's output through a simulation model and checking how close the simulation model's output is to previously selected desired node values, i.e., ground truth. At operation 1705, the learning model is executed using the desired selected node values ​​in the reverse direction and the state values ​​in the reverse direction as input, as discussed in FIG. 9. At operation 1710, a start state simulation estimate, e.g., 1510B, is executed using the node values ​​provided by the learning model as initial values ​​for the nodes in the start state simulation estimate. The values ​​at the end of the learning model simulation (e.g., t=-n, as shown in FIG. 7 at 730) are used as the initial node values ​​for the start state simulation estimate. An example of a simulation receiving input values ​​can be seen with reference to FIG. 14. The node 1405 may have a temperature 1415 as an attribute 1410. The temperature attribute 1415 may have an input value, which is the appropriate initial node value from the value at the end of the learning model simulation. At operation 1715, the selected node value from the simulation and the ground truth (e.g., historical node values, etc.) are used in a cost function to determine a cost that measures the distance between the two sets of values. At decision point 1720, it is determined whether the cost is close enough to a threshold. This threshold may be predetermined. The threshold may be set such that if the cost is lower than the threshold, the result from the starting state estimate is close enough to the desired value and is considered optimized. Optimization indicates that the value is close enough for a given use, not the best possible value. If the cost is low enough, at decision point 1725, the learning model result is used as the starting value for the node in the starting state simulation estimate 1510B.If it is determined that the two sets of values ​​are too far apart, e.g., if a threshold has not been reached, then at decision point 1730, the results of the learning model node values ​​(which may be considered as already optimized results) are used as starting values ​​for the optimizer simulation, as described with reference to FIG. 5 and the surrounding text. These starting values ​​are considered to be closer to the final optimal value than random values. At operation 1735, the optimization-simulation model is repeatedly run until a stopping state is reached. At operation 1740, the results of the repeated runs of the optimization-simulation model are used to train the learning model. FIGS. 11A and 11B provide further details on how operations 1735 and 1740 are performed. Either every time, or every other time, or according to some predetermined cycle, the results of the optimization-simulation run (and the inputs used) are used as training examples for the learning model. This allows the start-state simulation estimation using the optimizer to perform multiple training runs on the learning model. Thus, the learning model is trained at the same time as the actual start-state simulation estimation is performed. By performing the start-state simulation estimation, the learning model is automatically trained. Once the learning model is trained, it may be possible to generate initial state estimates much faster than using an optimizer. In operation 1745, once the optimization-simulation model reaches an optimum, the node values ​​generated at the start of the last iteration are used as the starting node values ​​for the starting state simulation estimation, and this starting state simulation estimation is run. See 804 in FIG. 8 for an example of the node values ​​generated at the start of the iteration. In operation 1750, the simulation model is run, and the results of the simulation model are used to modify some aspect of the modeled structure. This may involve the results of the simulation model being used to run other models, such as device simulation models.The device simulation model can then generate control sequences for one or more devices modeled in the device simulation model. The generated control sequences can then be used to operate the devices. The generated control sequences can operate the devices in a manner that minimizes some aspect of the devices, such as energy usage. For example, energy usage can be minimized by as much as 30%.

[0098] FIG. 18 discloses at 1800 a learning model training system that can be used to warm up a simulation. The system can include a processor that can be all or part of the core processing system 330. The processor can also be one or more processors in the computing environment 300, 410 that are themselves part of multiple computerized controllers 405 that are networked together in a shared computing system. The shared computing system can share processing power between the computing environments. The system also includes memory in operative communication with the processor and computing code associated with the processor configured to create a simulator trainer. The memory and / or computing code can also be shared between the computing environments in the shared computing environment.

[0099] Block 1805 discloses an optimizer that determines initial node values ​​for a simulator. The simulator comprises nodes having values. With reference to FIG. 1, a simulator is shown at 110. More particularly, with reference to, for example, FIG. 12 and the surrounding text, at least nodes having values ​​are shown. The relationship of the optimizer and the simulator is described, for example, with reference to 510, 511, and 512 in FIG. 5, and with reference to 810, 811, and 812 in FIG. 8.

[0100] Block 1810 opens a simulator. The simulator uses as input an input time series from time t(-n) to time t(0). The output of the simulator and the output of the node may be an output time series from time t(-n) to time t(0). For example, referring to 505 in FIG. 5, an input of the series t(-n) to t(0) is described. For example, an example of an input time series from time t(-n) to time t(0) may be input data 505, shown as running from t=-n to t=0. This input data may be, for example, weather or other condition data, as shown with reference to 805 in FIG. 8.

[0101] Block 1815 discloses a reverser that reverses the input time series from time t(0) to t(-n) to generate a reversed input time series. An example of this can be seen with reference to 505 in FIG. 5 showing the original time series, and 710 in FIG. 7 showing the reversed time series. The reverser can also reverse the output time series from time t(0) to t(-n). This output time series may be a selection value time series, as shown with reference to 515 in FIG. 5. 705 in FIG. 7 then shows the output (in this case, selection node value) reversed time series.

[0102] Block 1820 discloses a learning model that uses the inverted input time series as a training input and uses the selected value of the output time series at t(-n) as the ground truth for a cost function associated with the learning model. Referring to FIG. 7, the learning model is shown taking the input data 505 shown in FIG. 5 reversed (by reverser 815) and using this as input data to the learning model 715. Referring to FIG. 9, 910, another example is shown.

[0103] Block 1825 discloses a cost function determiner. Referring to FIG. 5, the simulator generates selected internal node values ​​515 after execution. These selected internal node values ​​515 are then measured against a set of desired node values ​​520. Using a cost function 527, these two sets of node values ​​are measured and then a cost 525 is determined. Similarly, the learning model 715 generates a set of output nodes. Its final values ​​(tn) are compared with the simulator node values ​​620, 720 by a cost function 737 to determine a cost 735. The selected node values ​​515 and the desired node values ​​520 are input to the cost function so that the learning model output at time t(-n) is compared with the initial simulator node values. The action of a similar cost function is shown with reference to FIG. 8 and FIG. 9.

[0104] Block 1850 discloses an iterator. An optimizer, such as the optimizers disclosed herein, learns through an iterative process. A set of possible values ​​is selected as input, the selected inputs are used by the simulation in a simulation run, and then the output is checked using a cost function to see how close it is to the desired output, as described in other sections. If the stop state is not reached, the optimizer then selects a new set of values ​​for the simulation, and this process continues until the stop state is reached. In one embodiment, the iterator controls these iterations. For example, the iterator can run the optimizer, pass the optimized node values ​​to the simulator, and run a cost function at the end of the simulation to determine whether the stop state has been reached. If not, the iterator can have the optimizer determine the next set of optimized node values, using the cost. This cycle continues until the stop state is reached. When the stop state is reached, the initial values ​​of the last iteration, e.g., 840 in FIG. 8, are used as input to the start state simulation estimate, e.g., 210A. In one embodiment, a starting state simulation estimate is run from time t(-n) to time t(0). After running the starting state simulation estimate, a state simulation (using the same simulator with warmed up node values) can be run in a forward direction from time t(0) to time t(m). The state simulation can generate outputs that can ultimately be used to generate control sequences for a device. For example, the results of the state simulation can be used to run other simulations, such as device simulations. The device simulation can generate control sequences that can then be used to drive a device modeled in the device simulation model. The generated control sequences can then be used to drive the device. This is also described with reference to FIG. 11B and the surrounding text.

[0105] V. Exemplary Computer-Readable Medium for Training a Simulator 3 and 4, an embodiment includes a configured computer-readable storage medium 365. The medium 365 can include disks (magnetic, optical, or other), RAM, EEPROM, other ROM, and / or other configurable memories. Other configurable memories include computer-readable media (not covering manufactured transitory phenomena such as electrical, optical, or acoustic signals). The configured storage medium may be a removable storage medium 365, such as a CD, DVD, or flash memory. A program on the storage medium can be executed on a processor (e.g., 310, 312, 315, etc.) coupled to the memory. Such memory may be general purpose memory (primary memory such as RAM, ROM, CMOS, or flash, or secondary memory such as a CD, hard drive, optical disk, or removable flash drive) and may be configured in an embodiment using the computing environment 300, the computerized controller 405, or any combination thereof, in the form of data 380 and instructions 375, which may be read from a source such as the output device 355 to form a configured medium. The instructions, when executed by the processor, perform a method of calculating travel comfort information. The configured medium 365 may cause the computer system to perform actions as related herein.

[0106] Some embodiments provide or utilize a computer readable storage medium 365 having software 385 configured thereon. The software 385, when executed by the at least one central processing unit 310, performs the methods and systems described herein.

[0107] In view of the many possible embodiments to which the principles of the disclosed invention may be applied, it should be recognized that the illustrated embodiments are merely examples of the invention and should not be taken as limiting the scope of the invention. On the contrary, the scope of the invention is defined by the following claims. We therefore claim as our invention all that comes within the scope and spirit of these claims.

Claims

1. 1. A computing system comprising one or more processors and one or more memories in operative communication with said one or more processors, said one or more memories comprising computer-executable instructions for causing said computing system to perform operations, said operations comprising: Executing a learning model using the input to obtain a learning model result; An operation of performing a simulation using the learning model result as an initial node value; after running the simulation, checking selected node values ​​of the simulation against desired node values ​​to determine cost; using the learned model results as starting values ​​for an optimizer when the cost is within a threshold; repeatedly running the optimizer such that, for each iteration, a result of running the optimizer is used as a training input to the learning model; 1. A computing system comprising:

2. 2. The computing system of claim 1, further comprising the operation of using the learned model results as starting values ​​for the simulation.

3. 3. The computing system of claim 2, comprising at least two processors in two computerized controllers.

4. 4. The computing system of claim 3, wherein the results of running the optimizer include initial node values ​​for the simulator.

5. 5. The computing system of claim 4, wherein the simulator is a heterogeneous neural network.

6. 6. The computing system of claim 5, wherein the operation of repeatedly executing the optimizer comprises: running the optimizer using previously optimized values ​​to determine initial simulator node values; running the simulator using the initial node values ​​to obtain simulator outputs; inverting the simulation input to generate an inverse simulation input; inverting the selected node value to obtain an inverse selected node value; using the inverse simulation inputs and the inverse selected node values ​​as training inputs to the learning model; 1. A computing system comprising:

7. 7. The computing system of claim 6, further comprising the operation of executing the learning model to generate a reverse time series for a learning model output node as a learning model output.

8. 8. The computing system of claim 7, further comprising an operation of comparing the learned model output to the desired node value to determine a cost.

9. 9. The computing system of claim 8, further comprising the operation of the learning model using the costs to back-propagate through the learning model to update values ​​in the learning model.

10. 2. The computing system of claim 1, further comprising the operations of checking an optimizer output for a stall condition, and further checking when said optimizer uses said optimizer output.

11. 1. A computer-implemented method for determining validity of a learned model output, comprising: Executing the learning model to obtain a learning model result; running a simulation using the learned model results as initial node values; after running the simulation, checking selected node values ​​of the simulation against desired node values ​​to determine cost; when the cost is at a threshold, using the learned model result as a starting value for an optimizer; running the optimizer repeatedly such that for each iteration, results of a simulation using optimizer output are used as training data for the learning model; 4. A computer-implemented method comprising:

12. 12. The computer-implemented method of claim 11, wherein the simulation is a neural network.

13. 13. The computer-implemented method of claim 12, wherein the neural network is an RNN.

14. 14. The computer-implemented method of claim 13, wherein the optimizer is a self-organizing migrating algorithm (SOMA).

15. 15. The computer-implemented method of claim 14, further comprising inverting the learned model results before using them as starting values ​​for the optimizer.

16. 16. The computer-implemented method of claim 15, wherein the learned model results are a time series.

17. 17. The computer-implemented method of claim 16, further comprising inverting the learned model results before being used as initial node values ​​for the simulation.

18. A computer-readable storage medium having instructions configured thereon that, when executed by one or more processors, perform a method for training a simulator, the method comprising: Executing the learning model to obtain a learning model result; running a simulation using the learned model results as initial node values; after running the simulation, checking selected node values ​​of the simulation against desired node values ​​to determine a cost; when the cost is at a threshold, using the learned model result as a starting value for an optimizer; running the optimizer repeatedly such that for each iteration, results of a simulation using optimizer output are used as training data for the learning model; 23. A computer readable storage medium comprising:

19. 20. The computer-readable storage medium of claim 18, wherein the step of repeatedly executing the optimizer comprises: running the optimizer using previously optimized values ​​to determine initial simulator node values; running the simulator using the initial node values ​​to obtain simulator outputs; inverting the simulation input to generate an inverse simulation input; inverting the selected node value to obtain an inverse selected node value; using the inverse simulation inputs and the inverse selected node values ​​as training inputs to the learning model; 23. A computer readable storage medium comprising:

20. 20. The computer readable storage medium of claim 19, wherein the simulation is a starting state simulation estimate.

Images