Device and method for controlling system using digital twin

Abstract

To disclose a device and a method for controlling behaviors of a system to execute a task.SOLUTION: A system is communicatively coupled to a digital twin which is configured to simultaneously simulate behaviors of the system. This method includes acquiring a set of control inputs for controlling the system so as to change a state of the system according to a task. This method further includes acquiring a set of outputs of the system which are generated by a corresponding set of control inputs. This method further includes estimating the current internal state of the digital twin by using a neural network trained to estimate a set of internal states of the digital twin that maps the set of control inputs to the set of outputs. This method further includes executing a task using the current internal state of the digital twin.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present disclosure relates generally to control systems, and more specifically to devices and methods for controlling systems using digital twins. [Background technology]

[0002] Various systems, such as vapor compression systems and robots, are controlled using state observers. Designing a state observer requires knowledge of the system's dynamics and a mathematical model that specifies the system's dynamics. In the absence of a mathematical model, operational data is collected from the system. The collected operational data is then utilized to learn the unmodeled dynamics, for example using neural networks or Bayesian optimization, and then the state estimator gains are calculated.

[0003] Advances in modeling and numerical computation have led to the emergence of digital twins. Digital twins are replicas of the behavior of a system. Digital twins can be used to design state observers. However, using digital twins to design state observers is challenging. For example, digital twins are typically composed of modules that are "black boxes" that are made opaque for reasons such as protecting proprietary information, reducing code complexity, or improving user interface and experience. Therefore, digital twins may not allow simple model structures that can be used to design standard model-based state observers. Furthermore, digital twin simulations produce the same output as a system that uses the digital twin's internal state. However, the internal state is not readily available online from the system.

[0004] Therefore, there remains a need to design state observers that can be integrated with black-box simulation environments (i.e., digital twins) and that can use such state observers to control systems. Summary of the Invention

[0005] It is an object of some embodiments to provide devices and methods for controlling a system based on the internal state of a digital twin of the system, and to estimate the internal state of the system using measurements of the system, as well as to train a neural network to estimate a set of internal states of the digital twin that maps a set of control inputs to a set of outputs.

[0006] Feedback occurs when the output of a system is fed back as an input that forms part of a chain of cause and effect forming a circuit or loop. Thus, the system is said to feed back onto itself. For example, state feedback control is a control technique used in the field of control systems engineering that involves using the state of the system (or a measurable and / or estimated subset) to compute control actions to regulate the system's behavior.

[0007] In a control system, "state" refers to the set of variables that describe the internal dynamics and behavior of a system at any one time. The set of variables includes position, velocity, acceleration, temperature, pressure, or any other measurable quantity that characterizes a system. State feedback control utilizes a feedback loop in which the state of the system is measured or estimated, compared to a desired state, and the difference (known as the error) is used to calculate a control signal to be applied to the system. The control signal is designed to steer the system to a desired state by influencing the dynamics of the system.

[0008] However, in many industrial systems and processes, the concept of the state of the system is too abstract and / or complex to be practical. For example, manufacturing processes, monitoring and control of industrial equipment, and optimization of energy consumption in buildings depend on too many state variables that are impractical to measure in real-time control applications. Examples of such unmeasured state variables are pressure or enthalpy in a vapor compression system. In other applications, there is not even a clear definition of the concept of a state variable. For example, it is reasonable to assume that couplings between independent processes in a complex manufacturing system are responsible for forming the state of the manufacturing system. However, it is often not clear how to measure and define such couplings and / or whether such couplings even exist in the first place. Thus, state feedback control is impractical or even impossible in many practical applications.

[0009] Some embodiments are based on the recognition that digital twins can be used to control systems. A digital twin is a digital representation of an intended or actual, real-world physical product, system, or process (physical twin) that serves as a virtually indistinguishable digital counterpart of the physical twin for practical purposes such as simulation, integration, testing, monitoring, and maintenance. Digital twins can be used to optimize manufacturing processes, monitor and control industrial equipment, analyze patient health data for personalized medicine, optimize energy consumption in buildings, and even model and simulate entire cities for urban planning.

[0010] For a set of inputs, the digital twin of a system produces a set of outputs identical to what the system is capable of outputting for this set of inputs. In other words, the dynamics of the digital twin provide a mapping between a set of inputs and a set of outputs. However, the digital twin has its own dynamics. As an example, the internal state of a digital twin is usually different from the internal state of its physical twin (i.e. the system). In addition, unlike a physical twin, the internal state of a digital twin may not have a physical meaning and may depend on the objectives of the digital twin's designer. Furthermore, a digital twin may have a complex internal dynamics model with unknown components, which means that the dynamics of the digital twin are unknown.

[0011] As such, an objective of some embodiments is to capture or learn the dynamics of a digital twin that provides a mapping between a set of inputs and a set of outputs. Some embodiments are based on the recognition that a neural network can be trained to infer a set of internal states of a digital twin that maps a set of inputs to a set of outputs, where the set of internal states represents the dynamics of the digital twin.

[0012] Such a trained neural network can be used online (i.e., during real-time operation) to estimate the current internal state of the digital twin. Furthermore, a system may be controlled using the current internal state of the digital twin to perform a task. As an example, a feedback controller associated with the system is configured to determine current control inputs for controlling the system based on the current internal state of the digital twin. Further, the feedback controller controls the system based on the determined control inputs to perform the task. Feedback control of the system based on the internal state of the digital twin is more accurate than feedback control based on the measurements of the system because the internal state is a more accurate representation of the system's behavior than measurements of the system.

[0013] Accordingly, one embodiment discloses a device for controlling the operation of a system to perform a task. The system is communicatively coupled to a digital twin configured to simultaneously simulate the operation of the system. The system and the digital twin operate simultaneously. The device comprises a processor and a memory having instructions stored thereon that, when executed by the processor, cause the device to: collect a set of control inputs for controlling the system to change a state of the system according to the task; collect a set of outputs of the system caused by the corresponding set of control inputs; estimate a current internal state of the digital twin using a neural network trained to estimate a set of internal states of the digital twin that maps the set of control inputs to a set of outputs; and perform the task using the current internal state of the digital twin.

[0014] Accordingly, another embodiment discloses a method for controlling operation of a system to perform a task. The system is communicatively coupled to a digital twin configured to simultaneously simulate operation of the system, and the system and the digital twin operate simultaneously. The method includes collecting a set of control inputs for controlling the system to change a state of the system in accordance with the task. The method further includes collecting a set of outputs of the system caused by a corresponding set of control inputs. The method further includes estimating a current internal state of the digital twin using a neural network trained to estimate a set of internal states of the digital twin that maps the set of control inputs to a set of outputs. The method further includes performing the task using the current internal state of the digital twin.

[0015] Accordingly, another embodiment discloses a non-transitory computer-readable storage medium including a program executable by a processor to execute a method for controlling operation of a system to perform a task. The system is communicatively coupled to a digital twin configured to simultaneously simulate operation of the system, where the system and the digital twin operate simultaneously. The method includes collecting a set of control inputs for controlling the system to change a state of the system according to the task. The method further includes collecting a set of outputs of the system caused by a corresponding set of control inputs. The method further includes estimating a current internal state of the digital twin using a neural network trained to estimate a set of internal states of the digital twin that map the set of control inputs to a set of outputs. The method further includes performing the task using the current internal state of the digital twin. [Brief description of the drawings]

[0016] [Figure 1A] FIG. 2 shows a block diagram of a device for controlling a system according to some embodiments of the present disclosure. [Figure 1B] FIG. 1 illustrates a block diagram for performing a task using the current internal state of a system's digital twin according to some embodiments of the present disclosure. [Figure 1C] FIG. 1 illustrates a block diagram of a device including a feedback controller according to some embodiments of the present disclosure. [Figure 1D] FIG. 1 illustrates a diagram showing the simultaneous control of a system and a digital twin of the system according to some embodiments of the present disclosure. [Figure 1E] FIG. 1 illustrates a block diagram for online learning of the internal state of a digital twin according to some embodiments of the present disclosure. [Figure 1F] FIG. 1 illustrates an implementation of a digital twin simulation module for a building energy system according to some embodiments of the present disclosure. [Diagram 2]FIG. 1 illustrates training a neural network to learn the internal state of a system's digital twin, according to some embodiments of the present disclosure. [Figure 3A] FIG. 2 illustrates an autoencoder according to some embodiments of the present disclosure. [Figure 3B] FIG. 2 illustrates the operation of an autoencoder according to some embodiments of the present disclosure. [Figure 3C] FIG. 2 illustrates an example implementation of an autoencoder according to some embodiments of the present disclosure. [Figure 4A] FIG. 1 illustrates an environment for controlling a vapor compression system according to some embodiments of the present disclosure. [Figure 4B] FIG. 1 illustrates a block diagram for controlling a vapor compression system according to some embodiments of the present disclosure. [Diagram 5] FIG. 1 illustrates control of a robotic manipulator using a device according to some embodiments of the present disclosure. [Figure 6] 1 shows a schematic diagram of a computing device that can be used to implement the devices of the present disclosure.

[0017] The embodiments disclosed below are further described with reference to the accompanying drawings, in which: The drawings shown are not necessarily to scale, but rather emphasis is generally placed on illustrating the principles of the embodiments disclosed below. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] For purposes of explanation, numerous specific details are set forth in the following to provide a thorough understanding of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be practiced without these specific details. In other instances, the apparatus and methods are shown in block diagram form solely to avoid obscuring the present disclosure.

[0019] As used in the specification and claims, the terms "for example," "for instance," and "such as," as well as the verbs "comprising," "having," "including," and other forms of these verbs, when used in conjunction with a list of one or more components or other items, should each be construed as open-ended, meaning that the list should not be considered to exclude further components or items. The term "based on" means based at least in part on. Furthermore, it should be understood that the style and terminology used herein are for purposes of illustration and should not be considered limiting. Any headings used herein are for convenience only and should not be considered to have any legal or limiting effect.

[0020] Feedback occurs when the output of a system is fed back as an input that forms part of a chain of cause and effect forming a circuit or loop. Thus, the system is said to feed back onto itself. For example, state feedback control is a control technique used in the field of control systems engineering that involves using the state of the system (or a measurable and / or estimated subset) to compute control actions to regulate the system's behavior.

[0021] In a control system, "state" refers to the set of variables that describe the internal dynamics and behavior of a system at any one time. The set of variables includes position, velocity, acceleration, temperature, pressure, or any other measurable quantity that characterizes a system. State feedback control utilizes a feedback loop in which the state of the system is measured or estimated, compared to a desired state, and the difference (known as the error) is used to calculate a control signal to be applied to the system. The control signal is designed to steer the system to a desired state by influencing the dynamics of the system.

[0022] However, in many industrial systems and processes, the concept of the state of the system is too abstract and / or complex to be practical. For example, manufacturing processes, monitoring and control of industrial equipment, and optimization of energy consumption in buildings depend on too many state variables that are impractical to measure in real-time control applications. Examples of such unmeasured state variables are pressure or enthalpy in a vapor compression system. In other applications, there is not even a clear definition of the concept of a state variable. For example, it is reasonable to assume that couplings between independent processes in a complex manufacturing system are responsible for forming the state of the manufacturing system. However, it is often not clear how to measure and define such couplings and / or whether such couplings even exist in the first place. Thus, state feedback control is impractical or even impossible in many practical applications.

[0023] Some embodiments are based on the recognition that digital twins can be used to control systems. A digital twin is a digital representation of an intended or actual, real-world physical product, system, or process (physical twin) that serves as a virtually indistinguishable digital counterpart of the physical twin for practical purposes such as simulation, integration, testing, monitoring, and maintenance. Digital twins can be used to optimize manufacturing processes, monitor and control industrial equipment, analyze patient health data for personalized medicine, optimize energy consumption in buildings, and even model and simulate entire cities for urban planning.

[0024] For a set of inputs, the digital twin of a system will generate a set of outputs identical to what the system is capable of outputting for this set of inputs. In other words, the dynamics of the digital twin provide a mapping between a set of inputs and a set of outputs. However, the digital twin has its own dynamics. For instance, the internal state of a digital twin is usually different from the internal state of its physical twin (i.e., the system). In addition, unlike a physical twin, the internal state of a digital twin may not have a physical meaning but may depend on the objectives of the digital twin's designer. Furthermore, a digital twin may have a complex internal dynamics model with unknown components, which means that the dynamics of the digital twin are unknown.

[0025] Thus, an objective of some embodiments is to capture or learn the dynamics of a digital twin that provides a mapping between a set of inputs and a set of outputs. The digital twin's internal state represents the current dynamical state of the digital twin. The set of internal states captures the dynamics of the digital twin, which are assumed to be identical for the digital twin and the physical twin even if the digital twin has different types of states, because the same dynamical system can have infinite state representations. Thus, a set of inputs and a set of outputs can be used to recover the set of internal states of the digital twin, including the current state.

[0026] Some embodiments are based on the recognition that a neural network can be trained to estimate a set of internal states of a digital twin that maps a set of inputs to a set of outputs, where the set of internal states represents the dynamics of the digital twin. Some embodiments are further based on the recognition that an initial internal state can be applied to the neural network to accurately estimate the set of internal states. Thus, in some embodiments, a state estimator (an optimization problem) is used to minimize the error between the outputs of the system and the outputs of the digital twin, which in turn determines the initial internal state of the neural network.

[0027] Based on this recognition, the present disclosure provides a device for controlling a system using the internal state of a digital twin, as described in FIG. 1A below.

[0028] 1A illustrates a block diagram of a device 101 for controlling a system 103 according to some embodiments of the present disclosure. The system 103 can be a vapor compression system, a robotic manipulator, etc. The system 103 is configured to perform a task. For example, the system 103 is a robotic manipulator configured to track a reference trajectory to perform a task of moving an object to a target location. In another example, the system 103 is a vapor compression system configured to maintain an environment at a target temperature.

[0029] The system 103 is communicatively coupled to the digital twin 105. The digital twin 105 is a digital representation of the system 103 that serves as a virtually indistinguishable digital counterpart of the physical twin, i.e., the system 103. The system 103 and the digital twin 105 operate simultaneously. The digital twin 105 is configured to simultaneously simulate the operation of the system 103. In other words, the digital twin 105 simultaneously simulates the real-time operation of the system 103. The system 103 and the digital twin are communicatively coupled to the device 101 via wired or wireless communication.

[0030] The device 101 includes a transceiver 107, a processor 109, and a memory 111. The processor 109 may be a single-core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory 111 may include a random access memory (RAM), a read-only memory (ROM), a flash memory, or any other suitable memory system. Additionally, in some embodiments, the memory 111 may be implemented using a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof. The memory 111 is configured to store a neural network 111a. The neural network 111a is trained to estimate a set of internal states of the digital twin 105 that maps a set of control inputs to a set of outputs. As described below with reference to FIG. 1B, the neural network 111a is used to estimate a current internal state of the digital twin and to perform a task using the current internal state of the digital twin 105.

[0031] FIG. 1B illustrates a block diagram for performing a task using a current internal state of the digital twin 105, according to some embodiments of the present disclosure. In block 110, the processor 109 is configured to collect a set of control inputs for controlling the system 103. For example, in one embodiment, the transceiver 107 is configured to send the set of control inputs to the system 103. If the system 103 is a vapor compression system, the set of control inputs includes one or a combination of a compressor speed, a fan speed, and an expansion valve position of the vapor compression system. If the system 103 is a robotic manipulator, the set of control inputs includes voltages and / or currents to actuators of the robotic manipulator. The set of control inputs changes the state of the system 103 according to a task. For example, the set of control inputs changes the position of the robotic manipulator toward a target position to perform a task. In another example, the set of control inputs changes the current temperature of the environment to a target temperature.

[0032] Further, in block 120, the processor 109 is configured to collect a set of outputs of the system 103 resulting from a corresponding set of control inputs. As an example, to collect the set of outputs of the system 103, the transceiver 107 is configured to receive a feedback signal including a corresponding set of measurements. Each measurement is indicative of an output of the system resulting from a corresponding control input. For example, the measurements include temperature, humidity, and / or velocity of air output from a vapor compression system. In another example, the measurements include a position of a robotic manipulator. The measurements are collected from one or more sensors associated with the system 103.

[0033] In block 130, the processor 109 is configured to use the neural network 111a to estimate a current internal state of the digital twin 105. In block 140, the processor 109 is configured to perform a task using the current internal state of the digital twin 105. In one embodiment, a feedback controller is used to perform the task using the current internal state of the digital twin 105.

[0034] 1C illustrates a block diagram of device 101 including a feedback controller 150, according to some embodiments of the present disclosure. Feedback controller 150 is configured to determine current control inputs for controlling system 103 based on a current internal state of digital twin 105. Further, feedback controller 150 controls system 103 to perform a task based on the determined control inputs.

[0035] Additionally, as described in FIG. 1D below, in some embodiments, the feedback controller 150 is configured to simultaneously control the system 103 and the digital twin 105 based on the determined control inputs.

[0036] 1D illustrates the simultaneous control of system 103 and digital twin 105, according to some embodiments of the present disclosure. Based on the determined control inputs, feedback controller 150 simultaneously controls system 103 to update the state of system 103 and controls digital twin 105 to update the internal state of digital twin 105.

[0037] Digital twin 105 produces the same outputs as system 103 through changes in their internal states, and the internal states of digital twin 105 can be used to determine control inputs that result in desired outputs. Furthermore, feedback control of system 103 based on the internal states of digital twin 105 is more accurate than feedback control based on measurements of system 103, because the internal states are a more accurate representation of the behavior of system 103 than measurements.

[0038] In one embodiment, feedback controller 150 is implemented by a Proportional Integral Derivative (PID) controller. In another embodiment, feedback controller 150 is implemented by an extremum-seeking control. In yet another embodiment, feedback controller 150 is implemented by a Model Predictive Control (MPC).

[0039] Some embodiments are based on the recognition that in some circumstances, the digital twin 105 is designed as a black box, making the internal state of the digital twin 105 not easily available online during the control of the system 103. However, in contrast to the system 103, the internal state of the digital twin is well-defined, even if the values ​​of the internal state are unknown. Therefore, it is possible to learn the internal state of the digital twin 105 online from one or a combination of the inputs and outputs of the digital twin 105, as described below in FIG. 1E.

[0040] FIG. 1E illustrates a block diagram for online learning of an internal state of a digital twin 105, according to some embodiments of the present disclosure. The digital twin 105 is communicatively coupled to an edge platform 113 and a cloud platform 115. The edge platform 113 is communicatively coupled to the system 103. The digital twin 105 includes a digital twin adaptation module 105a and a digital twin simulation module 105b. The edge platform 113 includes a sensor 113a, a decision-making module 113b, and an application module 113c. The cloud platform 115 includes a storage medium 115a, a data analysis module 115b, and an application module 115c. The application module 115c and the application module 113b may be configured for one or more applications, such as set point tracking, model prediction, and anomaly detection.

[0041] The digital twin 105 is a high fidelity simulation software in which expert-driven domain-informed modules are introduced in addition to a learning framework adapted to operational data to accurately reflect the dynamics of the system 103. In other words, the digital twin 105 is a digital object that reflects the fundamental properties of an observable physical object. The digital twin 105 can perform monitoring and control functions by connecting to the physical world and perform the fundamental functions of the observable physical object. The digital twin 105 can adapt to changes in the properties of the system 103 by learning from the measured outputs of the observable physical object, for example through the use of machine learning. As an example, the sensor 113a outputs measurement data including measurements of the outputs of the system 103, and the storage medium 115a stores data related to the intrinsic properties of the system 103 and its environment, for example exogenous inputs or disturbances. The intrinsic properties include the architecture of the building in which the vapor compression system is installed, the properties of the materials of the building, occupancy information, environmental information, and personalized profile information of the user, for example, how the occupants use the equipment / hot water / air conditioning / lighting in the building. The data analysis module 115b extracts features from the data stored in the storage medium 115a. The measured data and the extracted features are input to the digital twin adaptation module 105a. The digital twin adaptation module 105a comprises a machine learning mechanism that utilizes the extracted features and the measured data to update / learn the internal state of the digital twin 105.

[0042] The learned or updated internal state of the digital twin is used to initialize a simulation of the operation of the system 103 by the digital twin simulation module 105b. The digital twin simulation module 105b is configured to output simulated data including a prediction of the behavior of the system 103 (i.e., a prediction of the state variables of the system 103) considering the measurements, inputs, and initial state of the system 103. The simulated data is input to the decision-making module 113b. Based on the simulated data, the decision-making module 113b determines control inputs for performing simultaneous control of the system 103 and the digital twin 105. As an example, the system 103 is controlled based on the control inputs for updating the state of the system 103. Furthermore, the digital twin 105 is also controlled based on the control inputs for updating the internal state of the digital twin 105. Alternatively, in some embodiments, the decision-making module 113b analyzes the behavior of the system 103 under different simulated scenarios for predictive analysis and maintenance based on the simulated data. An example implementation of the digital twin simulation module 105b for a building energy system is described in FIG. 1F below.

[0043] FIG. 1F illustrates a software implementation of a digital twin simulation module 105b for a building energy system according to some embodiments of the present disclosure. 117a is a specific implementation of a digital twin of a building energy system from Modelica where sensible heat load 117b and latent heat load 117e can be injected along with sensible heat load split into convection heat load 117c and radiant heat load 117d. Cooling is provided by VRF systems 117g-j, and the building thermodynamics is represented by 117n, with dry and wet valve ambient temperatures fed into the digital twin 117l-o from weather data files and additional environmental inputs. 117k illustrates a software tool that generates the building thermodynamics 117n. As is evident from 117p, the software is generating its own realization of continuous-time digital twin states that may not have closed-form analytical expressions.

[0044] 2 illustrates training of a neural network 201 to estimate a sequence of internal states of a digital twin 105, according to some embodiments of the disclosure. Neural network 201 corresponds to neural network 111a stored in memory 111. Neural network 201 is trained using supervised learning based on training data 203. Training data 203 includes training sequences 205 of the digital twin's internal states and training sequences 207 of the digital twin's corresponding outputs. Neural network 201 is trained using supervised learning based on training data 203 to map outputs of the digital twin 105 to the digital twin's internal states that gave rise to the digital twin's outputs 105.

[0045] In some embodiments, the neural network 201 is offline, i.e., pre-trained, and the trained neural network 201 is executed online, i.e., during real-time operation, to estimate the current internal state of the digital twin 105. For example, the processor 109 is configured to execute the trained neural network 201 to estimate the current internal state of the digital twin 105 during real-time operation.

[0046] In some embodiments, the neural network 201 is a neural network with an autoencoder architecture. In other words, the neural network 201 is an autoencoder. Some embodiments are based on the realization that an autoencoder can be used to learn a transformation that "lifts" the dynamics of the system 103 to a latent space where a finite-dimensional linear state space model can provide an accurate predictive model. The advantage of an autoencoder is that the state observer design can exploit the linear system representation and standard observer gain adjustment methods can be applied. In addition, the effectiveness of learning lift transformations in identifying nonlinear systems can be linked to a Koopman operator-theoretic design, where the linear state transition operator learned by the autoencoder represents a finite approximation of the Koopman operator.

[0047] In this way, autoencoders can be advantageously used for learning the internal state of the digital twin 105. To that end, some embodiments use an autoencoder with a latent space adapted for dynamic propagation of an encoding of the internal state of the digital twin 105 within a prediction horizon to a decoding that represents a combination of the internal state of the digital twin 105 and the output (measurements) of the system 103. Such an autoencoder is described in detail in FIG.

[0048] 3A illustrates an autoencoder 300 according to some embodiments of the present disclosure. The autoencoder 300 includes an encoder sub-network 301, a latent sub-network 303, a first decoder sub-network 305, and a second decoder sub-network 307. The autoencoder 300 is input with a current input internal state. As described in FIG. 3B below, the autoencoder 300 outputs a next internal state of the digital twin 105 and a current output of the system 103 based on the current input internal state.

[0049] As shown in FIG. 3B, a current input internal state 309 is input to the encoder sub-network 301. The encoder sub-network 301 is configured to generate a current encoding of the internal state in the latent space by encoding the current input internal state 309 into the latent space. The current encoding of the internal state in the latent space is input to the latent sub-network. The latent sub-network 303 is configured to generate a next encoding of the internal state in the latent space by propagating the current encoding of the internal state in the latent space forward in time. Furthermore, the next encoding of the internal state in the latent space is input to the first decoder 305. The first decoder sub-network 305 is configured to decode a next internal state 311 of the digital twin 105 from the next encoding of the internal state in the latent space. The second decoder sub-network 307 is configured to decode a current output of the system 103 from the current encoding of the internal state in the latent space.

[0050] Some embodiments are based on the realization that it is advantageous to design the control law of the feedback controller 150 using the latent space of the autoencoder 300, since such a design allows to transform the nonlinear control of the system into a linear control in the latent space. The feedback controller 150 is therefore configured to control the system 103 using a linear control law that includes a control matrix formed by the values ​​of the parameters of the latent sub-network 303. The latent sub-network 303 is trained to perform a linear propagation of the encoding of the internal state.

[0051] In some embodiments, the processor 109 is configured to solve an optimization problem that generates a set of internal states within a prediction horizon that are transformed by the autoencoder 300 to match the set of outputs of the system 103. The autoencoder 300 and the optimization problem are mathematically described below.

number

[0052] Unlike standard input-output system identification, where the designer chooses the notion of state, in simulator-driven identification the state space is fixed by the digital twin 105 and the outputs are nonlinear functions of the states fixed by the digital twin 105. A state feedback controller designed for the digital twin 105 requires the state x online, but the state is typically not measured and therefore must be estimated only from the measured output y.

[0053]

number

[0054]

number

[0055] In one embodiment, the operator A319 and the decoder D y By choosing 325 as a linear layer with zero bias, we can force it to be the state transition matrix and the output matrix, respectively. The state transition operator A 319 can be viewed as a Koopman operator that allows for a linear description of the nonlinear dynamics (1) in a high-dimensional latent space.

[0056] According to some embodiments, the autoencoder 300 is trained to minimize a loss function of the combination of the internal states of the digital twin 105 and the outputs of the digital twin 105 over a prediction horizon. Training the autoencoder 300 is performed using the encoder E x 317, Two Decoders D x 321 and D y 325, as well as computing the weights of the state transition operator A 319. For this purpose, a multi-stage reconstruction loss with regularization is used.

number

[0057]

number

[0058] Since the encoder and decoder transforms may be nonlinear, the optimization problems (4a)-(4e) are nonlinear least-squares problems. The optimization problems (4a)-(4e) may be solved using limited-memory BFGS implemented with automatic differentiation utilizing the PyTorch library. Alternatively, the optimization problems (4a)-(4e) may be solved using standard nonlinear least-squares methods such as the Levenberg-Marquardt algorithm or a nonlinear moving horizon estimator using the identified nonlinear dynamics.

[0059] Use cases for device 101 are now described in Figures 4A, 4B, and 5. Device 101 can be used to control the operation of a vapor compression system. Additionally or alternatively, device 101 can be used to control the operation of a robotic manipulator. Control of a vapor compression system using device 101 is described in Figures 4A and 4B. Control of a robotic manipulator using device 101 is described in Figure 5.

[0060] 4A illustrates an environment 400 for controlling a vapor compression system 410 according to some embodiments of the present disclosure. The environment 400 includes the vapor compression system 410, a digital twin 420 of the vapor compression system 410, and the device 101. The vapor compression system 410 and the digital twin 420 are communicatively coupled to the device 101. The vapor compression system is configured to perform a task to maintain the environment at a target state. The target state may include a target temperature.

[0061] Vapor compression system 410 includes a compressor 401, a condensing heat exchanger 403, an expansion valve 405, and an evaporating heat exchanger 407 disposed in space 409. Heat transfer from condensing heat exchanger 403 is facilitated by fan 411, and heat transfer from evaporating heat exchanger 407 is facilitated by fan 413. Vapor compression system 410 may include variable actuators such as variable compressor speed, variable expansion valve position, and variable fan speed. There are numerous other alternative equipment architectures to which this disclosure pertains, having multiple heat exchangers, compressors, valves, and other components such as accumulators or reservoirs, pipes, etc., and the description of vapor compression system 410 is not intended to limit the scope or application of the disclosure to any system.

[0062] In the vapor compression system 410, the compressor 401 compresses a low pressure, low temperature vapor phase fluid (refrigerant) to a high pressure, high temperature vapor state, which then flows into the condensing heat exchanger 403. As the refrigerant passes through the condensing heat exchanger 403, the heat of the high temperature, high pressure refrigerant is transferred to the lower temperature ambient air by heat transfer facilitated by the fan 411. As the refrigerant transfers heat to the ambient air, the refrigerant gradually condenses until it is in a high pressure, low temperature liquid state. The refrigerant further exits the condensing heat exchanger 403 and passes through the expansion valve 405, expanding to a low pressure boiling state and entering the evaporative heat exchanger 407. Since the air passing through the evaporative heat exchanger 407 is warmer than the refrigerant itself, the refrigerant gradually evaporates as it passes through the evaporative heat exchanger 407. The refrigerant exiting the evaporative heat exchanger 407 is in a low pressure, low temperature state. The low pressure, low temperature refrigerant enters the compressor 401 again, and the same cycle is repeated.

[0063] 4B illustrates a block diagram of a method 415 for controlling a vapor compression system 410, according to some embodiments of the present disclosure. In block 417 of the method 415, the transceiver 107 of the device 101 sends a set of control inputs to the vapor compression system 410. The set of control inputs includes one or a combination of a speed of a compressor 401, a speed of a fan 413, and a position of an expansion valve 405 of the vapor compression system 410.

[0064] In block 419 of method 415, transceiver 107 of device 101 receives a collection of a series of outputs of vapor compression system 410 caused by a corresponding series of control inputs. The series of outputs of vapor compression system 410 may correspond to a series of measurements, with each measurement indicative of an output of vapor compression system 410 caused by a corresponding control input. For example, the measurements may include a temperature, humidity, and / or velocity of the air output by vapor compression system 410.

[0065] In block 421 of the method 415, the processor 109 of the device 101 estimates the current internal state of the digital twin 420 using the neural network 111a.

[0066] In block 423 of the method 415, the feedback controller 150 of the device 101 determines a current control input for controlling the vapor compression system 410 based on the current internal state of the digital twin 420. The current control input is sent to the vapor compression system 410. The current control input changes the current state to a target state. As an example, the current control input changes the current temperature to a target temperature to perform the task of maintaining the target temperature.

[0067] 5 illustrates control of a robotic manipulator 501 using a device 101, according to some embodiments of the present disclosure. The robotic manipulator 501 is configured to perform a manipulation task to move an object 503 to a goal state 505, e.g., to a goal position. The robotic manipulator 501 is communicatively coupled to a digital twin 507 of the device 101 and the robotic manipulator 501.

[0068] The transceiver 107 of the device 101 sends a set of control inputs to the robotic manipulator 501. The set of control inputs includes voltages and / or currents to actuators of the robotic manipulator 501. Additionally, the transceiver 107 of the device 101 collects a set of outputs of the robotic manipulator 501 that are caused by a corresponding set of control inputs.

[0069] Furthermore, the processor 109 of the device 101 estimates a current internal state of the digital twin 507 using the neural network 111a. Furthermore, the feedback controller 150 of the device 101 determines a current control input for controlling the robot manipulator 501 based on the current internal state of the digital twin 507. The feedback controller 150 controls the actuator of the robot manipulator 501 according to the current control input to push the object 503 by the end effector 509 from the current state to the target state 505.

[0070] FIG. 6 shows a schematic diagram of a computing device 600 that can be used to realize the device 101 of the present disclosure. The computing device 600 includes a power supply 601, a processor 603, a memory 605, and a storage device 607, all connected to a bus 609. In addition, a high-speed interface 611, a low-speed interface 613, a high-speed expansion port 615, and a low-speed connection port 617 can be connected to the bus 609. In addition, a low-speed expansion port 619 is connected to the bus 609. In addition, an input interface 621 can be connected to an external receiver 623 and an output interface 625 via the bus 609. The receiver 627 can be connected to an external transmitter 629 and a transmitter 631 via the bus 609. An external memory 633, an external sensor 635, a machine 637, and an environment 639 can also be connected to the bus 609. In addition, one or more external input / output devices 641 can be connected to the bus 609. A network interface controller (NIC) 643 may be adapted to connect to a network 645 via bus 609 to, among other things, enable data or other data to be rendered on a third party display device, a third party imaging device, and / or a third party printing device external to computing device 600.

[0071] The memory 605 can store instructions executable by the computing device 600, as well as any data that can be utilized by the methods and systems of the present disclosure. The memory 605 can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory system. The memory 605 can be one or more volatile memory units and / or one or more non-volatile memory units. The memory 605 can also be another form of computer-readable medium, such as a magnetic disk or an optical disk.

[0072] The storage device 607 may be adapted to store supplemental data and / or software modules used by the computer device 600. The storage device 607 may include a hard drive, an optical drive, a thumb drive, an array of drives, or any combination thereof. Additionally, the storage device 607 may include a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or an array of devices, including a tape device, a flash memory or other similar solid-state memory device, or a storage area network or other configuration of devices. The instructions may be stored on an information carrier. The instructions, when executed by one or more processing devices (e.g., the processor 603), perform one or more methods, such as those described above.

[0073] Computing device 600 can optionally be linked through bus 609 to a display interface or user interface (HMI) 647 adapted to connect computing device 600 to a display device 649 and keyboard 651, which may include a computer monitor, a camera, a television, a projector, or a mobile device, among others. In some implementations, computing device 600 may include a printer interface for connecting to a printing device, which may include a liquid inkjet printer, a solid ink printer, a large scale commercial printer, a thermal printer, a UV printer, or a dye sublimation printer, among others.

[0074] The high-speed interface 611 manages bandwidth-intensive operations of the computing device 600, and the low-speed interface 613 manages low-bandwidth-intensive operations. Such an allocation of functionality is merely one example. In some implementations, the high-speed interface 611 can be coupled to memory 605, a user interface (HMI) 647, a keyboard 651, and a display 649 (e.g., through a graphics processor or accelerator), and can be coupled to a high-speed expansion port 66 via the bus 609 that can accept various expansion cards.

[0075] In one implementation, the low speed interface 613 is coupled to storage 607 and low speed expansion port 617 via bus 609. The low speed expansion port 617, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet), may be connected to one or more input / output devices 641. The computing device 600 may be coupled to a server 653 and a rack server 655. The computing device 600 may be realized in a number of different forms. For example, the computing device 600 may be realized as part of a rack server 655.

[0076] The present disclosure provides a device 101 for controlling a system 103 based on the internal state of a digital twin 105 of the system 103. The digital twin 105 produces the same outputs as the system 103 through changes in their internal states, and the internal state of the digital twin 105 can be used to determine control inputs that result in the desired outputs. Furthermore, feedback control of the system 103 based on the internal state of the digital twin 105 is more accurate than feedback control based on measurements of the system 103, because the internal state is a more accurate representation of the behavior of the system 103 than measurements.

[0077] This specification provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. It is contemplated that various changes may be made to the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter as set forth in the appended claims.

[0078] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments can be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. Additionally, like reference numbers and names in the various drawings refer to like elements.

[0079] Also, the individual embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any process specifically described may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, or the like. When a process corresponds to a function, the end of the function may correspond to a return of the function to the calling function or to the main function.

[0080] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automated implementation may be performed or at least assisted through machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor(s) may perform the necessary tasks.

[0081] The various methods or processes outlined herein may be coded as software executable on one or more processors employing any one of a variety of operating systems or platforms. In addition, such software may be written using any of a number of suitable programming languages ​​and / or programming or scripting tools, and compiled as executable machine language code or intermediate code that runs on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

[0082] The embodiments of the present disclosure may be implemented as a method, an example of which is provided. The order of operations performed as part of the method may be determined in any suitable manner. Thus, the embodiments may be configured to perform operations in an order different from that illustrated, which may include performing some operations simultaneously, although in the illustrated embodiment they are shown as a sequence of operations.

[0083] Furthermore, the embodiments of the present disclosure and the functional operations described herein can be realized in digital electronic circuitry, in tangibly implemented computer software or firmware, in computer hardware including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. Furthermore, some embodiments of the present disclosure can be realized as one or more computer programs, i.e., as one or more modules of computer program instructions encoded on a tangible non-transitory program carrier for execution by or for controlling the operation of a data processing apparatus. Still further, the program instructions can be encoded on a human-generated propagated signal, for example, on a machine-generated electrical, optical, or electromagnetic signal. The propagated signal is generated to encode information that is transmitted to a suitable receiving device for execution by a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random access memory device or a serial access memory device, or one or more combinations thereof.

[0084] According to an embodiment of the present disclosure, the term "data processing apparatus" may encompass all kinds of apparatus, devices, and machines that process data, including, by way of example, a programmable processor, computer, or multiple processors or computers. The apparatus may include special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program, such as code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0085] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and can be deployed in any form, such as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in part of a file that holds other programs or data, for example in one or more scripts stored in a markup language document, in a single file dedicated to the program, or in multiple coordinated files, for example in a file that stores one or more modules, subprograms, or portions of code.

[0086] A computer program can be deployed to be executed on one computer or on multiple computers located at one site or distributed at multiple sites and interconnected by a communication network. A computer suitable for executing a computer program may be based, by way of example, on a general-purpose or special-purpose microprocessor or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a central processing unit for carrying out or executing instructions, and one or more memory devices for storing instructions and data.

[0087] Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic, magneto-optical, or optical disks, or is operatively coupled to said disks to receive data therefrom or transfer data thereto, or both. However, a computer need not have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.

[0088] To provide for user interaction, embodiments of the subject matter described herein may be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse or trackball, for allowing the user to provide input to the computer. Other types of devices may be used to provide user interaction. For example, feedback provided to the user may be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic input, speech input, or tactile input. Additionally, the computer may provide for user interaction by sending documents to and receiving documents from a device used by the user, e.g., by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0089] Embodiments of the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., as a data server, or includes a middleware component, e.g., an application server, or includes a front-end component, e.g., a client computer having a graphical user interface or web browser that allows a user to interact with an implementation of the subject matter described herein, or includes any combination of one or more of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), e.g., the Internet.

[0090] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises from computer programs running on the respective computers and having a mutual relationship of client and server.

[0091] Although the present disclosure has been described with respect to certain preferred embodiments, it should be understood that various other adaptations and modifications can be made within the spirit and scope of the disclosure. It is therefore the object of the following claims to cover all such variations and modifications that come within the true spirit and scope of the disclosure.

Claims

1. 1. A device for controlling operation of a system to perform a task, the system being communicatively coupled to a digital twin configured to simultaneously simulate operation of the system, the system and the digital twin operating simultaneously, the device comprising a processor and a memory having instructions stored thereon, the instructions, when executed by the processor, causing the device to: collecting a set of control inputs for controlling the system to change a state of the system in accordance with the task; collecting a set of outputs of the system caused by the corresponding set of control inputs; estimating a current internal state of the digital twin using a neural network trained to estimate a set of internal states of the digital twin that maps the set of control inputs to the set of outputs; and performing the task using the current internal state of the digital twin.

2. Further comprising a feedback controller, the feedback controller comprising: determining a current control input of the set of control inputs for controlling the system based on the current internal state of the digital twin; The device of claim 1 , configured to control the system based on the determined control inputs to perform the task.

3. 3. The device of claim 2, wherein the feedback controller is further configured to simultaneously control the system to update a state of the system and control the digital twin to update an internal state of the digital twin based on the current control input.

4. The device of claim 2 , wherein the feedback controller comprises one or a combination of a proportional-integral-derivative (PID) controller, an extremum-seeking control, and a model predictive control (MPC).

5. 10. The device of claim 1, wherein the processor is further configured to estimate an internal state of the digital twin online based on the measured data of the system and one or more inherent characteristics of the system.

6. 2. The device of claim 1, wherein the neural network is trained using supervised learning based on training data including training sequences of internal states of the digital twin and training sequences of corresponding outputs of the digital twin.

7. 2. The device of claim 1, wherein the neural network is an autoencoder having a latent space adapted for dynamic propagation of an encoding of the digital twin's internal state to a decoding that represents a combination of the digital twin's internal state and an output of the system.

8. 8. The device of claim 7, wherein the processor is further configured to solve an optimization problem that generates a set of internal states within a prediction horizon that are transformed by the autoencoder to match the set of outputs of the system.

9. The autoencoder an encoder sub-network configured to encode a current input internal state into a latent space to generate a current encoding of the current input internal state in the latent space; a latent sub-network configured to propagate the current encoding of the internal state in the latent space forward in time to generate a next encoding of the internal state in the latent space; a first decoder sub-network configured to decode a next internal state of the digital twin from the next encoding of the internal state in the latent space; and a second decoder sub-network configured to decode a current output of the system from the current encoding of the internal state in the latent space.

10. The device of claim 9 , wherein the processor is further configured to control the system using a linear control law that includes a control matrix formed by values ​​of parameters of the latent sub-networks.

11. The device of claim 9 , wherein the latent sub-network is trained to perform linear propagation of an encoding of the internal state.

12. 10. The device of claim 9, wherein the autoencoder is trained to minimize a loss function of combined differences between the digital twin's internal states and the digital twin's outputs over a prediction horizon.

13. 2. The device of claim 1, wherein the system is a vapor compression system, and the feedback controller is further configured to determine a current control input to the vapor compression system that changes a current state of the vapor compression system to a target state based on the current internal state of the digital twin.

14. 2. The device of claim 1 , wherein the system is a robotic manipulator, and the feedback controller is further configured to determine, based on the current internal state of the digital twin, a current control input to the robotic manipulator that changes a current state of the robotic manipulator to a goal state.

15. 1. A method for controlling operation of a system to perform a task, the system being communicatively coupled to a digital twin configured to simultaneously simulate operation of the system, the system and the digital twin operating simultaneously, the method comprising: collecting a set of control inputs for controlling the system to change a state of the system in accordance with the task; collecting a set of outputs of the system caused by the corresponding set of control inputs; estimating a current internal state of the digital twin using a neural network trained to estimate a set of internal states of the digital twin that maps the set of control inputs to the set of outputs; and performing the task using the current internal state of the digital twin.

16. 16. The method of claim 15, wherein the neural network is trained using supervised learning based on training data including training sequences of internal states of the digital twin and training sequences of corresponding outputs of the digital twin.

17. 16. The method of claim 15, wherein the neural network is an autoencoder with a latent space adapted for dynamic propagation of an encoding of the digital twin's internal state to a decoding that represents a combination of the digital twin's internal state and outputs of the system.

18. The autoencoder an encoder sub-network configured to encode a current input internal state into a latent space to generate a current encoding of the current input internal state in the latent space; a latent sub-network configured to propagate the current encoding of the internal state in the latent space forward in time to generate a next encoding of the internal state in the latent space; a first decoder sub-network configured to decode a next internal state of the digital twin from the next encoding of the internal state in the latent space; and a second decoder sub-network configured to decode a current output of the system from the current encoding of the internal state in the latent space.

19. 20. The method of claim 17, wherein the autoencoder is trained to minimize a loss function of combined differences between the digital twin's internal states and the digital twin's outputs over a prediction horizon.

20. 1. A non-transitory computer readable storage medium comprising a program executable by a processor to execute a method for controlling operation of a system to perform a task, the system being communicatively coupled to a digital twin configured to simultaneously simulate operation of the system, the system and the digital twin operating simultaneously, the method comprising: collecting a set of control inputs for controlling the system to change a state of the system in accordance with the task; collecting a set of outputs of the system caused by the corresponding set of control inputs; estimating a current internal state of the digital twin using a neural network trained to estimate a set of internal states of the digital twin that maps the set of control inputs to the set of outputs; and performing the task using the current internal state of the digital twin.

Images