Hybrid quantum-classical information processing system with variational quantum circuit and associated controller

Abstract

A system comprises a quantum computing subsystem that includes at least one variational quantum circuit, and a classical computing subsystem coupled to the quantum computing subsystem. The quantum computing subsystem utilizes the at least one variational quantum circuit to generate, for each of a plurality of iterations, a different set of parameters for an optimization problem to be solved in the classical computing subsystem. The classical computing subsystem receives from the quantum computing subsystem a given one of the sets of parameters generated for a given one of the iterations, formulates a corresponding instance of the optimization problem based at least in part on the given set of parameters, and solves the corresponding instance of the optimization problem to generate one or more controls. The one or more controls generated by the classical computing subsystem are applied to a dynamic environment which provides state information to the quantum computing subsystem.

Description

[0001] 10949-03-PC HYBRID QUANTUM-CLASSICAL INFORMATION PROCESSING SYSTEM WITH VARIATIONAL QUANTUM CIRCUIT AND ASSOCIATED CONTROLLER

[0002] Related Applications

[0003] The present application claims priority to U. S. Provisional Patent Application Serial No.

[0004] 63 / 616,966, filed January 2, 2024, and U. S. Provisional Patent Application Serial No. 63 / 643,534, filed May 7, 2024, each entitled “Hybrid Quantum-Classical Information Processing System with Variational Quantum Circuit and Associated Controller” and incorporated by reference herein in its entirety.

[0005] Field

[0006] The field relates generally to quantum computing and machine learning, and more particularly to hybrid quantum-classical information processing systems.

[0007] Background

[0008] Quantum computing has the potential to impact problems at various scales, including demand response in buildings and a wide variety of other optimal control problems, by addressing the limitations of conventional demand response techniques and other techniques implemented with classical computers.

[0009] Summary

[0010] Illustrative embodiments disclosed herein provide techniques for implementing hybrid quantum-classical information processing systems utilizing variational quantum circuits (VQCs) and associated controllers.

[0011] In one embodiment, a system comprises a quantum computing subsystem that includes at least one VQC, and a classical computing subsystem coupled to the quantum computing subsystem. The quantum computing subsystem is configured to utilize the at least one VQC to generate, for each of a plurality of iterations, a different set of parameters for an optimization 10949-03-PC problem to be solved in the classical computing subsystem. The classical computing subsystem is configured to receive from the quantum computing subsystem a given one of the sets of parameters generated for a given one of the iterations, to formulate a corresponding instance of the optimization problem based at least in part on the given set of parameters, and to solve the corresponding instance of the optimization problem to generate one or more controls. The one or more controls generated by the classical computing subsystem are applied to a dynamic environment for utilization therein to perform one or more sequential decision-making operations. The quantum computing subsystem receives state information from the dynamic environment for use in determining via the VQC another one of the sets of parameters for a subsequent one of the iterations.

[0012] Some embodiments disclosed herein are advantageously configured to counter the significant contribution of buildings to global energy consumption and greenhouse gas emissions, by facilitating the operation of demand response programs that incentivize grid-interactive buildings to curtail their load demand and promote energy efficiency with environmental sustainability.

[0013] For example, in some embodiments, a VQC-based hybrid control strategy is implemented for demand response that leverages the complementary strengths of quantum and classical computing paradigms. The VQC-based hybrid control strategy for demand response exploits the expressive power of parameterized quantum circuits trained under a reinforcement learning setting, while a classical optimization solver computes controls for a demand response problem formulated as a sequential decision-making problem.

[0014] The VQC-based hybrid control strategy in this example use case exhibits improved load reduction and carbon emissions reduction capabilities as compared to classical baselines like deep deterministic policy gradient and other control approaches, while ensuring scalability as the number of buildings increase.

[0015] As another example, some embodiments disclosed herein are configured to address excessive energy consumption and greenhouse gas emissions in artificial intelligence (Al) data 10949-03-PC centers. As the demand for Al models and applications continues to grow, data centers that handle Al workloads are experiencing a rise in energy consumption and associated carbon footprint. Some of the disclosed embodiments address this issue, illustratively by providing a VQC-based robust optimization (VQC-RO) framework for control and energy management in large-scale data centers. For example, such embodiments illustratively provide a quantum computing-based robust control framework for energy-efficient and sustainable data center operation, achieving substantial reductions in energy consumption and carbon emissions in large-scale data centers handling Al workloads.

[0016] As yet another example, some embodiments provide techniques for adaptive quantum computing-based model predictive control (MPC), and are advantageously configured for decarbonization of building operations, but can be adapted for numerous other use cases and control applications. Tn one or more such embodiments, a learning-based parameter transfer approach is implemented to realize adaptive quantum optimization that leverages Bayesian optimization to predict initial VQC parameters in the context of a quantum approximate optimization algorithm (QAOA). When applied to an MPC problem formulated as a quadratic unconstrained binary optimization (QUBO) problem, this approach computes optimal controls to minimize the net energy consumption levels in buildings and promotes decarbonization while reducing the computational efforts required for the QAOA as the building energy system trajectory progresses.

[0017] It is to be appreciated that the foregoing arrangements are only examples, including examples of illustrative applications of the disclosed techniques, and numerous alternative arrangements are possible.

[0018] These and other illustrative embodiments include but are not limited to systems, methods, apparatus, processing devices, integrated circuits, and computer program products comprising processor-readable storage media having software program code embodied therein. 10949-03-PC

[0019] Brief Description of the Figures

[0020] FIG. 1 is a block diagram of a hybrid quantum-classical information processing system configured for solving optimal control problems in an illustrative embodiment.

[0021] FIG. 2 shows an example of a VQC implemented in a quantum computing subsystem of the hybrid quantum-classical information processing system of FIG. 1.

[0022] FIG. 3 shows an example machine learning algorithm for training a VQC-based controller for demand response using reinforcement learning in an illustrative embodiment.

[0023] FIG. 4 shows graphical plots comparing performance of a VQC-based controller for demand response using reinforcement learning in an illustrative embodiment to alternative approaches.

[0024] FIG. 5 shows an example digital twin of an Al data center in an illustrative embodiment. FIG. 6 shows an example energy control configuration for an Al data center utilizing a hybrid quantum-classical information processing system in an illustrative embodiment.

[0025] FIG. 7 shows an example of a VQC implemented in a quantum computing subsystem of the hybrid quantum-classical information processing system of FIG. 6.

[0026] FIG. 8 shows an example machine learning algorithm for training a VQC-based controller for Al data center energy control optimization using reinforcement learning in an illustrative embodiment.

[0027] FIG. 9 shows graphical plots comparing performance of a VQC-based controller for Al data center energy control optimization using reinforcement learning in an illustrative embodiment to alternative approaches.

[0028] FIG. 10 is a block diagram of a hybrid quantum-classical information processing system configured for solving optimal control problems utilizing adaptive quantum computing-based MPC in an illustrative embodiment.

[0029] FIG. 11 shows components of an example modeled building energy management system subject to adaptive quantum computing-based MPC in an illustrative embodiment. 10949-03-PC FIG. 12 shows an example adaptive quantum computing-based MPC arrangement in an illustrative embodiment.

[0030] FIG. 13 shows an example of a VQC implemented in a quantum computing subsystem of the hybrid quantum-classical information processing system of FIG. 10.

[0031] FIG. 14 shows graphical plots comparing performance of an adaptive quantum computingbased MPC arrangement in an illustrative embodiment to alternative approaches.

[0032] Detailed Description

[0033] Illustrative embodiments can be implemented, for example, in the form of information processing systems comprising one or more processing platforms each having at least one computer, such as a quantum computer, a classical computer, and combinations of one or more quantum computers and one or more classical computers, each considered a “processing device” as that term is broadly used herein. A number of examples of such systems will be described in detail herein. It should be understood, however, that embodiments disclosed herein are more generally applicable to a wide variety of other types of information processing systems and associated computers and other components. Accordingly, the term “information processing system” as used herein is intended to be broadly construed so as to encompass these and other arrangements.

[0034] Some embodiments disclosed herein can utilize one or more of the techniques described in U. S. Patent No. 11,769,070, entitled “Quantum Computing Based Hybrid Solution Strategies for Large-Scale Discrete-Continuous Optimization Problems,” which is incorporated by reference herein in its entirety.

[0035] To counter the significant contribution of buildings to global energy consumption and greenhouse gas emissions, participation in demand response programs incentivizes grid-interactive buildings to curtail their load demand and promote energy efficiency with environmental sustainability. In some embodiments, quantum computing is used to impact such problems at various scales, including demand response in buildings, by addressing the limitations of 10949-03-PC conventional demand response techniques implemented with classical computers. For example, some embodiments provide a VQC-based hybrid control strategy for demand response that leverages the complementary strengths of quantum and classical computing paradigms. The hybrid VQC-based demand response technique in some embodiments exploits the expressive power of parameterized quantum circuits trained under a reinforcement learning setting, while a classical optimization solver computes controls for the demand response problem formulated as a sequential decision-making problem. The applicability and efficiency of the disclosed hybrid quantum-classical control strategy are demonstrated through computational experiments involving energy management in grid-interactive buildings equipped with various energy storage devices. The hybrid VQC-based strategy in some embodiments exhibits improvement in load reduction and carbon emissions reduction of over 13.6% compared to classical baselines like deep deterministic policy gradient and conventional MPC while ensuring scalability as the size of the building microgrids increases.

[0036] Buildings are a significant contributor to global energy consumption and energy-related emissions, accounting for 30% and 26%, respectively. As over 70% of electricity in the U. S. is consumed by buildings, initiatives that leverage grid-interactive efficient buildings have been proposed to improve demand flexibility, energy efficiency, and reliability of the power grid. Such initiatives aim to lower the reliance of residential and commercial buildings on the electric power grid by leveraging on-site renewable energy sources and energy storage devices, which directly translates to reduced carbon emissions, economic efficiency, and relieved stress on the grid. Grid-interactive buildings equipped with such resources present an opportunity to participate in demand response programs for net load reduction through sophisticated energy management strategies. It is important to address the uncertainties associated with renewable energy sources to increase the adoption of buildings into demand response programs by preventing instability and ensuring resource availability. In addition to the renewable energy generation uncertainties, demand responsive energy management in buildings can be further complicated by the load shifting requirement as an adaptive response to grid signals. Developing demand response strategies for 10949-03-PC grid-interactive buildings capable of addressing these issues is crucial to supporting sustainable energy development by achieving energy efficiency and grid stability. Embodiments disclosed herein meet this critical need.

[0037] In some embodiments, quantum computing is used to address problems associated with renewable and sustainable energy systems at various scales, exploiting recent advancements in the development of quantum hardware and algorithms. Armed with a higher computational power than classical computers, quantum-enhanced optimization and quantum machine learning offer effective avenues to address the demand response problem in building microgrids. Quantum-engineered power grids can also be enabled by exploiting the inherent characteristics of power grid problems and mechanisms to develop tailored quantum computing-based solutions. Applications of quantum computing for modernizing power grids have been previously explored and have unveiled potential benefits for enhancing security, reliability assessment, and optimizing power flow. However, limitations of the current quantum computers, illustratively referred to as noisy intermediate-scale quantum (NISQ) devices in terms of their error rates and size, can induce complexities with the adoption of quantum computing techniques for real-world problems. Quantum algorithms for optimization and machine learning are typically tailored for specific problem classes to derive a computational advantage over their classical counterparts and may lose their utility for problems relevant to energy systems. To this end, illustrative embodiments disclosed herein consider the problem formulation and associated intricacies of the demand response problem along with the hardware limitations to realize the advantages offered by quantum computing techniques implemented on NISQ devices.

[0038] For example, in order to overcome various shortcomings associated with standalone classical and quantum approaches, illustrative embodiments disclosed herein leverage hybrid quantum-classical strategies for demand response in grid-interactive buildings that can exploit the strengths of VQCs and classical optimization in a complementary manner.

[0039] There are several challenges associated with developing a hybrid quantum computingbased control framework for demand response. A first challenge lies in constructing a quantum 10949-03-PC circuit architecture that balances the tradeoff between the expressivity and trainability of the VQC. As the performance of variational quantum algorithms is heavily influenced by the circuit architecture, it is important to provide a robust quantum circuit to balance the effects of applying noisy quantum gates on NISQ devices. Another challenge lies in enabling a hybrid mode of operation between the quantum circuit execution and classical subroutines by integrating information from quantum operations to construct an optimization problem for control computation. The lack of provisions for strictly enforcing constraints on the applied controls in VQC-based reinforcement learning approaches is an additional challenge, especially in energy management, and can be overcome by an optimization problem formulation of the type disclosed herein that can be tractably solved with a classical solver. A final challenge lies in training a hybrid VQC-based controller that facilitates circuit learning while avoiding training pitfalls like sample inefficiency and adapting to time-varying uncertainties associated with the demand response problem.

[0040] In illustrative embodiments disclosed herein, we provide a VQC-based controller for demand response in building microgrids that utilizes quantum circuit execution capabilities and a classical optimization problem solution approach in a complementary manner to exploit the strengths offered by both quantum and classical computing. Embedding the classical information comprising information such as building states, renewable energy generation, and environmental disturbances onto the quantum device is conducted with amplitude encoding to enforce controller scalability. The VQC serves as a parametric function approximator for estimating the value function for a state and action pair, illustratively representing the discounted negative sum of net electricity consumption or carbon emissions. A robust VQC is constructed to transform the encoded quantum state representing the multidimensional classical information vector with parametric gate operations. An observable incorporating the controls is further used to compute the expectation value and translate the quantum information into classical information. Training the VQC for efficient value estimation without high resource utilization is performed in a reinforcement learning setting. Furthermore, an optimization problem formulated as a linear 10949-03-PC program to maximize the value estimate for a fixed building microgrid state serves as the classical component of the hybrid quantum-classical strategy for demand response in some embodiments. Multiple computational experiments involving demand response in building microgrids of varying sizes were conducted as described herein in order to validate the performance efficiency of illustrative embodiments of the disclosed hybrid VQC-based demand response strategy, along with classical demand response techniques as baselines. We also consider different objectives, like net load reduction and net carbon emissions reduction, to evaluate the adaptive capabilities of the disclosed hybrid controller towards varying sources of uncertainties.

[0041] Accordingly, some embodiments disclosed herein provide a hybrid quantum-classical framework for optimal demand response in grid-interactive buildings enabled by gate-model quantum computing and classical optimization.

[0042] Some embodiments additionally or alternatively provide a controller design with an integration of VQC for estimating net electric consumption equipped with scalable quantum encodings and optimization problem solution with a classical solver for computing controls associated with energy storage devices.

[0043] Computational experiments as described herein were performed to validate the net load reduction and net carbon emission reduction capabilities along with the scalability of illustrative embodiments of the disclosed hybrid VQC-based controller, as compared to classical baseline strategies.

[0044] Some preliminary details associated with some embodiments will now be described prior to further description of illustrative embodiments.

[0045] Quantum computers operate on quantum bits, known as qubits, in contrast to classical computers manipulating bits to perform operations. Unlike bits, each qubit exists in a two-dimensional vector or Hilbert space and can assume values beyond the discrete 0 and 1. An n-qubit system can then be represented by a 2ndimensional Hilbert space. Unlike classical bits, a general qubit state

[0046]

[0047] can be in a coherent superposition of its basis states, |0) and |1). Qubits 10949-03-PC also exhibit quantum entanglement, which enables them to establish co-relations between their individually random behaviors.

[0048] Measuring a quantum state results in a random collapse into one of its basis states and is an irreversible operation. A series of operations performed on a set of qubits or a quantum register within a quantum computer to accomplish a specific goal is referred to as a quantum algorithm. These operations are guided by different models of quantum computation like adiabatic quantum computing and the circuit model. In adiabatic quantum computing, computations are performed by evolving a quantum state from a known initial state to final state under adiabatic conditions. On the other hand, circuit or gate-model quantum computers use quantum logical gates in a quantum circuit to manipulate qubit states in a sequential manner. Some embodiments herein use methodologies associated with the circuit-based model of quantum computation owing to its universal nature of computation.

[0049] FIG. 1 shows a hybrid quantum-classical information processing system 100 configured for solving optimal control problems in an illustrative embodiment. The system 100 comprises a quantum computing subsystem 102, a classical computing subsystem 104 coupled to the quantum computing subsystem 102, and a dynamic environment 105 coupled to both the quantum computing subsystem 102 and the classical computing subsystem 104, as illustrated in the figure. The quantum computing subsystem 102 comprises at least one variational quantum circuit (VQC) that includes a plurality of stages including an initial quantum state preparation stage 110, a parametric gate operations stage 112 and a measurement stage 114. An additional component 115 of the quantum computing subsystem 102 is utilized to update VQC parameters in conjunction with training of the VQC.

[0050] The quantum computing subsystem 102 illustratively comprises one or more quantum computers, and the classical computing subsystem 104 illustratively comprises one or more classical computers, and each may comprise additional components. A given “quantum computer” as that term is broadly used herein illustratively comprises at least one quantum processor and associated memory, and a given “classical computer” as that term is broadly used herein 10949-03-PC illustratively comprises at least one classical processor and associated memory. Additional components can be included in one or both of the quantum computing subsystem 102 and the classical computing subsystem 104, and those terms as used herein are therefore also intended to be broadly construed.

[0051] As illustrated in the figure, the system 100 further comprises a number of databases, including an experience replay database 120 and a sampled data database 122, which may be implemented using one or more storage devices, including by way of example electronic memories and / or cloud-based storage.

[0052] The quantum computing subsystem 102 is illustratively configured to utilize at least one VQC to generate, for each of a plurality of iterations, a different set of parameters for an optimization problem to be solved in the classical computing subsystem 104. A given set of parameters in some embodiments comprises, for example, a plurality of rotation parameters of the VQC and / or a plurality of pooling parameters of the VQC, but may include additional or alternative parameters in other embodiments.

[0053] The classical computing subsystem 104 is illustratively configured to receive from the quantum computing subsystem 102 a given one of the sets of parameters generated for a given one of the iterations. The classical computing subsystem 104 is further configured to formulate a corresponding instance of the optimization problem based at least in part on the given set of parameters, and to solve the corresponding instance of the optimization problem to generate one or more controls.

[0054] It is to be appreciated that phrases such as “formulating a corresponding instance of an optimization problem” and “solving the corresponding instance of the optimization problem” as used herein are intended to be broadly construed. For example, in some embodiments disclosed herein, such as those to be described in conjunction with FIGS. 10 through 14, such phrases can involve computing linear combinations for rescaling to generate one or more controls. Numerous other types and arrangements of optimization problem formulation and solution are intended to be encompassed by these phrases as broadly used herein. 10949-03-PC The one or more controls generated by the classical computing subsystem 104 are applied to the dynamic environment 105 for utilization therein to perform one or more sequential decisionmaking operations. The dynamic environment 105 illustratively comprises one or more controlled components of at least one system that is driven by the one or more controls generated by the classical computing subsystem 104.

[0055] The quantum computing subsystem 102 receives state information from the dynamic environment 105 for use in determining via the VQC another one of the sets of parameters for a subsequent one of the iterations. Such iterations in some embodiments herein are more particularly referred to as “timesteps.”

[0056] In some embodiments, the optimization problem comprises an optimal control problem formulated at least in part as a sequential decision-making problem, although it is to be appreciated that the disclosed techniques can be adapted for use with a wide variety of other types of optimization problems, and are not limited to optimal control problems and / or sequential decisionmaking problems.

[0057] The optimal control problem in some embodiments may comprise, for example, a demand response problem. As a more particular example, the dynamic environment 105 may comprise at least one energy system of at least one building coupled to a power grid, and the optimization problem may comprise a demand response problem for optimizing operation of the at least one energy system. This may illustratively involve managing load of the at least one energy system, charging and / or discharging the at least one energy system, or performing other types of operations. It is to be appreciated that the disclosed techniques are applicable in a wide variety of other use cases. For example, the at least one energy system may additionally or alternatively be part of or otherwise associated with at least one of a facility, various types of equipment, a factory, a greenhouse, a manufacturing plant, or an energy consuming unit, or combinations thereof. Also, terms such as “optimization” and “optimal” as used herein are intended to be broadly construed, and should not be viewed as requiring any particular absolute maximum or minimum result in a given context. 10949-03-PC In some embodiments, the at least one VQC of the quantum computing subsystem 102 is trained at least in part utilizing a reinforcement learning algorithm. Other types of algorithms that may be utilized in training the at least one VQC of the quantum computing subsystem 102 in some embodiments include, for example, a mathematical optimization algorithm, a deep learning algorithm, a generative learning algorithm, or a metaheuristic algorithm, or combinations of these and possibly other algorithms. Such algorithms can be used in addition to or in place of the reinforcement learning algorithm.

[0058] Additionally or alternatively, the at least one VQC may be configured as a parametric function approximator for estimating a value function for a state-action pair, where the state comprises a state of the dynamic environment and the action comprises a particular instance of the one or more controls.

[0059] In some embodiments, the reinforcement learning algorithm utilized in training the VQC implements an epsilon-greedy strategy to determine one or more controls to be applied to the dynamic environment 105 for a given state of the dynamic environment 105. Additional or alternative strategies can be used in other embodiments.

[0060] The above-noted controls may comprise, for example, one or more control signals for controlling various operations performed by one or more controlled components in the dynamic environment 105. The term “controls” as used herein is therefore intended to be broadly construed, so as to encompass, for example, control signals and / or other types of information that may be utilized to initiate, trigger or otherwise control the performance of one or more operations in the dynamic environment 105.

[0061] Such operations performed in the dynamic environment 105 can include a wide variety of different types of automated actions associated with at least one controllable component of the dynamic environment 105. The at least one controllable component may comprise, for example, an energy system or another type of system deployed in the dynamic environment 105. The particular automated action or actions will tend to vary depending upon the particular application in which the system 100 is deployed. 10949-03-PC Additional examples of applications are provided elsewhere herein. It is to be appreciated that various types of actions can be automatically initiated, triggered or otherwise performed based at least in part on controls generated by a hybrid quantum-classical information processing system as disclosed herein.

[0062] Additionally or alternatively, in some embodiments, one or more automated actions or other types of actions can themselves be considered examples of “controls,” as that latter term is broadly used herein. For example, a given action of a state-action pair in some embodiments illustratively comprises a particular instance of one or more controls.

[0063] The quantum computing subsystem 102 and the classical computing subsystem 104 are illustratively implemented using one or more processing platforms each comprising at least one processing device, where a given such processing device may comprise a quantum computer and / or a classical computer. Such computers or other processing devices each illustratively comprise at least one processor coupled to a memory, where the processor in the case of the quantum computer comprises a quantum processor, and the processor in the case of the classical computer comprises a classical processor.

[0064] For example, the quantum computing subsystem 102 and the classical computing subsystem 104 may be implemented on a single processing platform that includes both quantum and classical processors. Alternatively, the quantum computing subsystem 102 and the classical computing subsystem 104 may be implemented on respective different processing platforms, comprising respective quantum processors and classical processors.

[0065] A given one of the quantum computing subsystem 102 and the classical computing subsystem 104 may also be distributed over multiple processing platforms, each comprising one or more quantum and / or classical processors. Terms such as “processing platform” as used herein are therefore intended to be broadly construed.

[0066] The above-noted processing platforms can include each multiple processing devices. Examples of such processing devices include quantum and / or classical computers comprising respective quantum and / or classical processors and associated memory, or other types of 10949-03-PC processing devices. Storage devices such as storage arrays or cloud-based storage systems used for implementing databases herein are also considered “processing devices” as that term is broadly used herein.

[0067] The processing devices can be configured to communicate over one or more networks. The one or more networks can comprise, for example, a global computer network such as the Internet, a wide area network (WAN), a local area network (LAN), a satellite network, a telephone or cable network, a cellular network such as a 4G, 5G or 6G network, a wireless network implemented using a wireless protocol such as Bluetooth, WiFi or WiMAX, or various portions or combinations of these and other types of communication networks.

[0068] A processing platform in some embodiments comprises at least one processor, at least one memory and at least one network interface. The processor is assumed to be operatively coupled to the memory and to the network interface. The processor may comprise a quantum processor and / or a classical processor. The term “processor” as used herein is therefore intended to be broadly construed.

[0069] The processor in some embodiments may comprise, for example, a microprocessor, an application-specific integrated circuit (ASIC), a system-on-chip (SOC), a field-programmable gate array (FPGA), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a data processing unit (DPU), a tensor processing unit (TPU), an arithmetic logic unit (ALU), a digital signal processor (DSP), and / or other similar processing device components, as well as other types and arrangements of processing circuitry, in any combination. At least a portion of the functionality of at least one machine learning system and its associated machine learning algorithms provided by one or more processing devices as disclosed herein can be implemented using such circuitry.

[0070] In addition to various types of classical processors, a wide variety of different types of gatebased quantum processors and / or other circuit-based quantum processors can be used in a given embodiment, for example, within a quantum subsystem to implement at least one VQC as disclosed herein. 10949-03-PC At least a portion of the functionality of at least one of the quantum computing subsystem 102 and the classical computing subsystem 104 and their respective associated processing algorithms provided by one or more processing devices as disclosed herein can be implemented using such circuitry.

[0071] In some embodiments, the processor comprises one or more graphics processor integrated circuits. Such graphics processor integrated circuits are illustratively implemented in the form of one or more GPUs. Accordingly, in some embodiments, system 100 is configured to include a GPU-based processing platform. Such a GPU-based processing platform can be at least partially cloud-based and is illustratively configured to implement one or more machine learning systems as part of the hybrid quantum-classical functionality disclosed herein. Similar arrangements can be implemented using NPUs, DPUs, TPUs and / or other processing devices, in addition to or in place of GPUs. A wide variety of quantum processors, illustratively comprising various types of quantum hardware and supporting different numbers and arrangements of quantum bits (“qubits”), can additionally or alternatively be used in illustrative embodiments.

[0072] A memory illustratively stores software program code for execution by a corresponding processor in implementing portions of the functionality of the processing platform. For example, at least portions of the functionality of at least one of quantum computing subsystem 102 and classical computing subsystem 104 can be implemented using program code stored in one or more memories of one or more processing devices.

[0073] A given such memory that stores such program code for execution by a corresponding processor is an example of what is more generally referred to herein as a processor-readable storage medium having program code embodied therein, and may comprise, for example, electronic memory such as SRAM, DRAM or other types of random access memory, flash memory, read-only memory (ROM), magnetic memory, optical memory, or other types of storage devices in any combination. 10949-03-PC Articles of manufacture comprising such processor-readable storage media are considered embodiments of the present disclosure. The term “article of manufacture” as used herein should be understood to exclude transitory, propagating signals.

[0074] Other types of computer program products comprising processor-readable storage media can be implemented in other embodiments.

[0075] In addition, illustrative embodiments may be implemented in the form of integrated circuits comprising processing circuitry configured to implement processing operations associated with one or both of the quantum computing subsystem 102 and the classical computing subsystem 104 as well as other related functionality. For example, at least a portion of a given such subsystem is illustratively implemented in at least one integrated circuit of a processing device of the processing platform.

[0076] The network interface is configured to allow the processing platform to communicate over one or more networks with other system elements, and may comprise one or more conventional transceivers.

[0077] In some embodiments, the classical computing subsystem 104 is further configured to generate at least one of a sequence of decisions and / or an associated control profile that is illustratively a result of at least a portion of the sequence of decisions, after a designated number of iterations or a designated computation time limit, with the sequence of decisions or the associated control profile being configured to instruct one or more physical devices and / or one or more physical components to operate, manipulate and / or adjust one or more energy systems. Such physical devices, physical components and energy systems may be viewed as examples of “controlled components” as that term is broadly used herein. The one or more physical devices and / or the one or more physical components in some embodiments comprise one or more of an actuator, a heater, a cooler, a heat exchanger, a dehumidifier, a humidifier, an energy supplier, an energy consuming unit, a valve, a pump, or any combination thereof.

[0078] Additionally or alternatively, the system 100 in some embodiments is further configured to minimize energy consumption of one or more energy systems subject to one or more dynamic 10949-03-PC constraints within a desired control time limit, with the desired control time limit being less than an amount of time used by the classical computing subsystem to solve at least one instance of the optimization problem.

[0079] Further examples of illustrative quantum and / or classical processing devices and associated processing platforms utilized to implement quantum and / or classical computing subsystems are described elsewhere herein.

[0080] It is to be appreciated that the particular arrangement of components and other system elements shown in FIG. 1 is presented by way of illustrative example only, and numerous alternative embodiments are possible. For example, other embodiments of information processing systems can be configured to implement hybrid quantum-classical functionality of the type disclosed herein.

[0081] Additional illustrative embodiments will now be further described with reference to FIGS.

[0082] 2 and 3. These embodiments and others herein are described in the context of particular hybrid quantum-classical system configurations, but it is to be appreciated that the disclosed techniques can be adapted in a straightforward manner to a wide variety of other hybrid quantum-classical system configurations.

[0083] FIG. 2 shows an example of a VQC 200 implemented in the quantum computing subsystem 102 of the hybrid quantum-classical information processing system 100 of FIG. 1. It is to be appreciated that other types of VQCs, as well as various combinations of multiple VQCs, can be used in other embodiments.

[0084] In this embodiment, the VQC 200 is assumed to be trained at least in part utilizing a reinforcement learning algorithm. The VQC 200 is illustratively configured as a parametric function approximator for estimating a value function for a state-action pair wherein the state comprises a state of the dynamic environment 105 and the action comprises a particular instance of the one or more controls.

[0085] As shown in the figure, the VQC 200 comprises a plurality of stages including at least an initial quantum state preparation stage 210, a parametric gate operations stage 212 and a 10949-03-PC measurement stage 214. This arrangement is an example only, and additional or alternative stages can be used in other embodiments.

[0086] The initial quantum state preparation stage 210 is illustratively configured to encode an input state vector into a quantum state with multiple qubits.

[0087] The parametric gate operations stage 212 illustratively comprises an entanglement block configured to entangle quantum states of respective qubits, a parametric gate operations block configured to perform sequential rotation operations in respective parametric gates utilizing respective ones of a plurality of rotation angles, and a pooling operations block configured to perform pooling operations between respective pairs of consecutive qubits.

[0088] The measurement stage 214 is illustratively configured to generate an expectation value of an observable.

[0089] In some embodiments, the expectation value of the observable is scaled by a trainable parameter to provide an estimate of the value function for the state-action pair.

[0090] Additional details regarding the operation of the example VQC 200 can be found elsewhere herein.

[0091] The previously-mentioned corresponding instance of the optimization problem that is formulated based at least in part on the given set of parameters and solved by the classical computing subsystem 104 to generate one or more controls illustratively comprises maximizing the value function for a given input state over a specified control space subject to one or more actuator constraints of the dynamic environment 105.

[0092] FIG. 3 shows an example machine learning algorithm, denoted Algorithm 1, for training a VQC-based controller for demand response using reinforcement learning in an illustrative embodiment. The controller in this embodiment and others herein illustratively comprises at least the quantum computing subsystem 102 and the classical computing subsystem 104, which collaborate with one another as disclosed herein to generate one or more control signals or other optimal controls for application to the dynamic environment 105. Other types and arrangements of components can be utilized to implement a given “controller” as that term is broadly used herein. 10949-03-PC Algorithm 1 illustratively utilizes an epsilon-greedy strategy to determine one or more controls to be applied to the dynamic environment for a given state of the dynamic environment, as described in more detail elsewhere herein.

[0093] Additional details regarding the embodiments of FIGS. 1 through 3 and the associated hybrid VQC-based control framework for demand response will now be described.

[0094] Quantum algorithms with the circuit model are typically presented as a sequence of quantum circuits that are executed in a sequential manner. These circuits involve various steps such as initial state preparation, applying unitary gate operations, and measurement operations to obtain interpretable classical information. Circuit-based quantum devices allow construction of parameterized models with VQCs. VQCs have been extensively utilized in quantum machine learning algorithms due to their ability to retain advantages high expressive power despite implementation on NISQ devices.

[0095] In some embodiments, primary components of a VQC-based algorithm include an encoder circuit, a variational circuit or parametric gate operations, and circuit learning. The encoder circuit translates a classical datapoint x into a quantum state \i[)x) referred to as quantum encoding or embedding. Data encoding is an important component of many quantum machine learning techniques as it directly affects their computational performance. The variational circuit applies parametric gate operations represented by 1 / (0) to the input quantum state \ix), where 0 denotes a parameter vector. The circuit learning component involves measurement of the resulting quantum state with an observable M to estimate the target function

[0096]

[0097] This allows for converting the quantum information produced by the variational circuit represented by the quantum state \ipg) into classical information to approximate the target function value as f (x; 0) = {ipg\M\ipg). Measurements with Pauli operators ox, oy, ozare performed in some embodiments and can be expressed as shown in Equation (1):

[0098]

[0099] 10949-03-PC Learning the function / (%) from data can be formulated in some embodiments as the minimization of a loss function L(0) as the objective function. A gradient descent algorithm can then be employed to update the parameters 0 associated with the variational circuit.

[0100] As indicated previously, some embodiments provide a hybrid VQC -based control framework for demand response. For example, in some embodiments, a VQC-based controller for demand response in grid-interactive buildings is configured to utilize a VQC for value function estimation, to extract controls from a trained VQC with a classical optimization problem, and to integrate the quantum and classical subroutines to enable circuit learning and optimal control computation. Some embodiments more particularly employ a VQC to approximate the state-action value for a state and control (x, u) pair. Within a reinforcement learning setting, the state-action value or the Q-value function represents the cumulative sum of discounted rewards over an infinite horizon.

[0101] Referring again to FIG. 2, the example VQC 200 is illustratively configured to estimate the value function for a state and action pair. More particularly, the VQC 200 estimates the value of a state-action pair through the initial quantum state preparation stage 210, the parametric gate operations stage 212, and the measurement stage 214 which computes the expectation value of an observable.

[0102] In some embodiments, the operation of a building microgrid comprising grid-responsive buildings is modeled as a finite Markov decision process (MDP). An MDP is illustratively defined by a state space S, an action space A, a reward function r, a discount factor y, and transition dynamics. At any time t, a building consumes energy to satisfy the heating and cooling loads along with its non-shiftable appliance load. As grid-responsive buildings are equipped with energy storage devices to promote grid flexibility, an energy storage subsystem is considered to offset the building’s load demand. This subsystem comprises battery energy storage as well as chilled water and domestic hot water (DHW) tanks. Additionally, the building’s electric demand can be supplemented by renewable energy generation facilitated with photovoltaic (PV) arrays. The hybrid VQC-based controller assigned to grid-responsive buildings illustratively operates without 10949-03-PC prior knowledge of energy model attributes or system transition dynamics. While historical data informs simulations during the training phase, we safeguard against data leakage to validate the generalization capabilities of the disclosed controller. At each timestep, a VQC-based reinforcement learning agent observes the current environment state and produces actions to control all energy components within the building microgrid. These actions influence corresponding building operations, with reward signals received in response. The system then advances to the next state, with the agent learning by updating value functions and policies to maximize cumulative rewards. The transition dynamics of the underlying storage subsystems are modeled based on existing energy models but are treated as unknown by the reinforcement learning agent. The collective effort by all buildings within the microgrid enforced by the centralized VQC-based agent promotes grid stability with an efficient demand response strategy, simultaneously considering various factors affecting the system dynamics.

[0103] At any time t, the observable state xt encompasses each building’s state as well as shared information such as, for example, weather data. The shared information illustratively comprises outdoor ambient air temperature, relative humidity, and solar radiation. Carbon intensity levels associated with electricity generation are also included within the shared information. The state xt also includes building-specific data like indoor temperature and relative humidity, along with its current cooling / heating and non-shiftable appliance loads. As PV generation directly affects the net electricity consumption levels within each building, the renewable generation level is also considered as a state variable. In addition, the current state-of-charge of energy storage devices specific to buildings in the microgrid, illustratively including the DWH and chilled water storage along with the battery energy storage, also constitutes the observable state. The control actions associated with each storage device are scaled between [-1,1] based on the charging / discharging limits posed by the equipped device characteristics. The reward function rt dictates the objective for the reinforcement learning agent and can be set to -Enet for net electricity consumption reduction. 10949-03-PC As the information pertaining to the building microgrid, including the observable state xt, is classical, converting a classical data vector into a quantum state is the first step towards leveraging a VQC for function approximation. It is important to ensure the scalability of the VQC-based controller as the building microgrid size increases. So, an amplitude encoding technique is used to encode the classical information into an n-qubit system. The logarithmic scaling offered by amplitude encoding allows for embedding multidimensional classical information onto the quantum device with only a few qubits. In one example, the net electric consumption and the outdoor weather conditions for a building microgrid recorded over a year are embedded using a single qubit. In some embodiments, t-distributed stochastic neighbor embeddings (t-SNE) for such an example can be visualized for the recorded data as well as the quantum amplitudes of encoded states to highlight the similarity between the original data and their quantum embeddings achieved with only one qubit.

[0104] A normalized vector such that

[0105]

[0106] = 1canbe represented by the amplitudes of a quantum state \ipp) as shown in Equation (2) below. For a state vector with dim(x) < 2n, the redundant quantum amplitudes can be set to zero as xt= 0, Vi = dim(x) + 1,..., 2n. Amplitude encoding is restricted to normalized classical vectors, leading to data representation with one less degree of freedom. To prevent such loss of information, we utilize an additional qubit to encode the norm of the state vector ||x||2with a parametric quantum gate, as illustrated in the initial quantum state preparation stage 210 of the VQC 200. The Rx gate acts on a single qubit to perform a rotation about the x-axis by a specific amount. As the amount of rotation 0xis bounded, establishing bounds for ||x||2allows for mapping the state vector norm to 6X. The generated input state for a classical state x can then be described as shown in Equation (3) below. It should be noted that the parameter Qxin the data encoding step is illustratively fixed for each classical state and action pair.

[0107] M = S2=ixil0 (2) I A) = | >P)® OO> (3) 10949-03-PC Q

[0108]

[0109] vqc(x,u) = Qo• (4)

[0110] Following the initial quantum state preparation stage 210, the parametric gate operations stage 212 of the VQC 200 is used to process the input n + 1 qubit quantum state. As shown in FIG. 2, we first entangle the quantum states from each qubit by applying a set of controlled NOT gates to consecutive qubits. A controlled NOT operation entangles two initially separable systems which plays an important role in the efficiency of a variational quantum algorithm. A parametric gate operation represented by G'wis applied as a single qubit unitary operation as shown in Equation (5) below:

[0111] GR i'Pi'Yi) = Rz a^Ry^^Rz yi) (5)

[0112] The parameters aL, pi and

[0113]

[0114] form the rotation angles for the three consequent gates dictating the unitary operation for the / -th qubit state following the entanglement block. The Rx, Ry, and Rz gates perform a single qubit rotation along the corresponding axis and are given by Equation (6) where I ando- indicate an identity and Pauli operator, respectively:

[0115] / ?c(A) = cos Q) I — i sin Q) ac, Vc 6 {%, y, z} (6)

[0116]

[0117] Also as shown in FIG. 2, pooling operators between consecutive qubits are also applied in the VQC 200. The pooling operator P <pi, Pi,8i) comprises a sequence of operations between a source and sink qubit pair, namely, a fixed inversion of the sink qubit along the z-axis, an inverted controlled NOT operation, and rotation of the source and sink qubits along z-axis and y-axis by; and pi amounts, respectively. Lastly, a controlled NOT rotation followed by a rotation of the sink qubit state by parameter 8talong the y-axis are applied in the pooling operator. These operators applied to each qubit pair promote the entangling capability of the VQC 200. 10949-03-PC Performing parameterized gate operations within the VQC 200 with an initial quantum state \ px) yields the quantum state

[0118]

[0119] An expectation value of an observable Muwith the resulting quantum state \ipf) is further computed in the measurement stage 214 to translate quantum information into classical information. More particularly, in some embodiments, a Hermitian observable Mu= diag(u^, —u,...,um, —um) where m = 2n, is used to compute the expectation value of the observable represented by

[0120]

[0121] (ipf | MuFinally, the value of an input state-action pair denoted by Qvqc(x, u) is estimated by scaling the expectation value by a trainable parameter Qoas shown in Equation (4) above. The estimated state-action value Qvqc(x, u) for the input observable building microgrid state and the corresponding controls represents the discounted negative sum of net electricity consumption over an infinite horizon for the load reduction objective.

[0122] An example optimization problem in some embodiments can be formulated in the following manner. Optimal controls for an observable building microgrid state x can be extracted from the policy zr(x) that maximizes the expected discounted rewards. Here, we leverage a deterministic policy represented by u = n(x) to compute the controls that optimize the value function as zr(x) = argmaxuQvqc(x, u). The encoded n + 1 qubit quantum state |i / r embedding the classical information x can be written as | ix) =

[0123]

[0124] S; |i) where s, indicates the quantum amplitude for the / -th basis state. The VQC 200 includes a fixed set of parameters 0 = (a, [3, Y> <p, p, 8) as unitary operations applied to the initial state |i >x). The quantum amplitudes for the resulting quantum state | / ^) are then linear combinations of the initial amplitudes s = (s1(s2,..., sn+1)r. The linear combinations can be represented as shown in Equation (7) below, where 2j(6)Tindicates the z-th row of the parametric unitary posed by the VQC 200 while sf (x) denotes the z-th quantum amplitude for the obtained quantum state |i / y):

[0125] M io = sxTsf io (7)

[0126]

[0127] 10949-03-PC 2n

[0128] DR(x, O^) max (x) |2 —|4O) |2) wi

[0129]

[0130] i=i

[0131] s.t. ATu < b (8)

[0132] As the scaled expectation computed over the observable Muyields the value of a stateaction pair, the policy TT(X) which maximizes the value function can be formulated as a math programming problem as shown in Equation (8) above. It should be noted that sf is a complex number with its magnitude represented by |s (x)|. Given a trained VQC, the optimization problem DR(x, 6) comprises maximizing the value estimate for an input building microgrid state over the control space encompassing the charging / discharging of energy storage devices in buildings subject to actuator constraints. For energy management in buildings, the bounds on charging and discharging levels [umin,umax] can be established at each timestep as they are dependent on the current state of charge of the storage devices comprising the classical state x. Additionally, constraints ATu < b can encompass measures to improve demand flexibility in grid-interactive buildings. The optimization problem formulated at each timestep comprises m = 2nvariables and can be solved with a classical linear solver in a tractable manner.

[0133] Referring again to FIG. 3, an overview of an example training algorithm for VQC learning is shown. The constructed circuit is initialized with parameters 0 = (a, ft, y, <p, p, <5) sampled from a uniform distribution denoted by ( / (— n, it). A replay buffer D is used to store transitions and serves as experience replay information to help increase sample inefficiency and prevent training instabilities. Each transition at timestep t comprises of observed state t, applied controls ur, the obtained reward rt, and the following state xt+1. To balance the exploration and exploitation while training the hybrid VQC-based controller, an epsilon-greedy strategy is used wherein the probability of choosing controls that maximize the value estimate is 1 — s as shown in Equation (9) below. Instead of randomly sampling controls during the exploration phase, a rule-based controller (RBC) is used to ensure reasonable charging and discharging levels of energy storage 10949-03-PC devices in grid-interactive buildings. It should also be noted that the probability E is decayed by a factor of Edecay with each timestep.

[0134] Pr(ut= argmaxuD R(xt, 0)) = 1 — E (9) yt= rt+ y - max D R(xt+1, 0) (10) = 2QO• {pvqc (^ + - Pvqc (0 - 7)} (11)

[0135]

[0136] PvqcW = (12)

[0137] During the learning phase, a batch of transitions of size fee is randomly sampled from the experience replay buffer. In Q-learning, the value function estimator is trained to approximate the Bellman equation, as described in more detail elsewhere herein. The targets for all state and action pairs are computed by maximizing the value function over the set of next states as shown in Equation (10) above. Here, y represents the discount factor. Computing the target values illustratively involves solving an optimization problem DR(xt+1,0) with a classical solver for each xt+1in the sampled batch of transitions. Although this can be performed separately for each transition, the computation overhead on the classical end can be further reduced by formulating a joint optimization problem over the entire batch. The joint maximization problem comprises m ■ BSize variables for which the solution obtained with a classical solver maximizes value estimate for each transition owing to the disjoint nature of state dependent actuator constraints.

[0138] With the computed targets, the VQC-based value function estimator can be updated in a supervisory manner to minimize the mean squared error objective L(0) = P x,u,r,xf) [(Qvqc(x’u) ~ y) ] Gradients of the objective with respect to individual parameters 9 E 0 forming the VQC 200 are determined to perform a gradient descent step. Here, we use a parameter shift rule to estimate the gradients of the VQC-based value function approximator Qvqcgiven by Equations (11) and (12) above. Prgc(0) in Equation (12) represents the classical information obtained for an individual data point following preparation of the quantum state with 10949-03-PC the VQC 200 and the measurement operation. Gradients of the objective function with respect to each VQC parameter are estimated with the above strategy. Lastly, the scaling factor (Lis updated with a gradient descent step. These steps are repeated over multiple training episodes for circuit learning in the hybrid VQC-based controller.

[0139] Computational results based on experiments performed on illustrative embodiments will now be described. It is to be appreciated that the particular features of these embodiments as described below may not be included in other embodiments disclosed herein.

[0140] To evaluate the performance of illustrative embodiments of the disclosed hybrid VQC-based controller for demand response in building microgrids, we formulate the demand response problem in grid-interactive buildings as a finite MDP using CityLeam, which allows for controlling energy storage devices in each building to manage net electricity consumption and associated carbon emissions. With pre-simulated heating and cooling demands and non-shiftable load, CityLearn facilitates the construction of a grid-interactive environment wherein thermal and electric storage devices in buildings can be managed for efficient net load reduction. Here, we use historically recorded data for heating, cooling, and non-shiftable loads in Cornell University’s Ithaca campus. Simulated operations of several microgrids with varying number of buildings are constructed to validate the performance efficiency and scalability of the disclosed VQC-based demand response strategy. The applicability of the disclosed strategy is further demonstrated by evaluation with a ten-building microgrid. Buildings in the microgrid are also equipped with a solar photovoltaic array, which supplements the building’s load demand. The thermal energy storage and battery storage devices are sized such that they can satisfy the building’s energy requirements.

[0141] We conducted two computational experiments with different objectives, namely, reducing the net electricity consumption of buildings in the microgrid and lowering the net carbon emissions associated with the electric power generation. The reward functions for both experiments are varied as rt= —Etand rt= — ctEt, where Li and Ct indicate the net electricity consumption in buildings and the carbon intensity levels associated with electricity generation at time t. For the disclosed hybrid VQC-based control strategy, quantum circuits executed on NISQ hardware are 10949-03-PC susceptible to noise and decoherence effects, which can lead to errors in the estimation of the value function for the observable microgrid state and control pair. During the training of the VQC in a reinforcement learning setting, errors in estimation can further lead to biased gradient updates for the parameters of the quantum circuit. To evaluate the effectiveness of the VQC-based controller in the presence of noise associated with NISQ hardware, we conducted experiments with building microgrids of varying sizes using a noise model incorporating the quantum gate errors and readout errors. The IBM Brisbane quantum device equipped with the Eagle processor comprising 127 qubits with a gate error rate of 1.9% was used as the noise model for the computational experiments associated with all building microgrids. Model -based approaches like conventional MPC and deep reinforcement learning approaches like deep deterministic policy gradient (DDPG) are also implemented as baselines against the VQC-based hybrid demand response strategy.

[0142] As VQC offers high expressivity with a smaller number of parameters, we also implement DDPG using neural networks with fewer parameters, denoted herein as Lo-DDPG. Lo-DDPG serves as a fair classical baseline against the hybrid VQC-based controller. The MPC strategy uses a state-space model for the building microgrid operation, which is extracted from a simulation performed over historical data by taking random actions, in addition to using a certainty equivalence strategy for optimization problem formulation. The certainty equivalent formulation assumes nominal values for uncertainty sources like PV generation, outdoor weather conditions, and carbon intensity levels. These nominal values are computed from historical data and used for conducting simulations with microgrids of varying sizes across different time periods. The DDPG algorithm employs a combination of deep neural networks to approximate both the actor and critic functions, allowing it to learn a policy for energy management that maximizes long-term rewards. Specifically, the actor-network outputs continuous control, while the critic network estimates the state-action value of the input state-action pair. The hyperparameters used for training the DDPG algorithms for load reduction and carbon emission reduction are consistent with those used for training the VQC-based controller. 10949-03-PC FIG. 4 shows graphical plots comparing performance of a VQC-based controller for demand response in an illustrative embodiment to alternative approaches including DDPG, Lo-DDPG and MPC.

[0143] The number of parameters used by the DDPG policy and value function estimator network are provided in part (a) of FIG. 4. Lo-DDPG technique is a variant of the DDPG algorithm with the exception of the use of function approximators with fewer parameters, as also shown in part (a) of FIG. 4. The hyperparameters used during the training of the parametric policy and value function estimator networks are the same as those of DDPG and the disclosed VQC-based controller to ensure a fair comparison.

[0144] With regard to energy management, the VQC-based demand response strategy is evaluated over multiple time periods for experiments with varying number of buildings, their attributes, and objectives. It was found that the net electricity consumption by a single building achieved with the VQC-based controller is significantly reduced as compared to its overall energy requirements or net load demand, illustratively over a period of one week in the month of January. Along with its efficient net load reduction capabilities, the VQC-based hybrid technique also exhibits a significant reduction in energy required to operate DHW storage and heating devices to meet the building’s heating energy requirements in winter.

[0145] To validate the performance of the disclosed controller subject to varying uncertainty sources like solar radiation, we compute the total PV generation and the net electricity consumption for the single building over months in different seasons. It was found that the VQC-based controller is capable of adjusting to uncertain renewable energy generation, resulting in reduced electricity consumption in response to increased PV generation in summer and vice-versa in colder months. Also, with regard to the electricity consumption for each building in the four-building case, along with the aggregate loads and PV generation over a week in January, it was found that an imbalance of increased load in some buildings along with decreased consumption in remaining buildings resulted in net overall load reduction capabilities. A decrease in electricity consumption in all buildings during periods of high PV generation is also observed. For the load 10949-03-PC reduction objective, the performance of the MPC-based demand response technique is significantly lower than the hybrid VQC-based controller. This can be attributed to the inability of the MPC controller to address various uncertainties associated with the building microgrid.

[0146] On the other hand, DDPG, which uses feedforward neural networks with nonlinear activations to parameterize policy and value functions, exhibits better load reduction capabilities than the MPC controller. Specifically, a slight improvement in performance over the hybrid VQC-based demand responsive strategy is observed with DDPG for the one-building microgrid. For example, as the temperature drops over the months of July and October, it was found that DDPG adapts better to such declines, resulting in improved load reduction performance for the smaller building microgrids but remains within 2% of the observed consumption levels with the VQC-based approach.

[0147] As the hybrid VQC-based demand response strategy with fewer parameters performs competitively against the DDPG technique implemented on a classical computer, it is important to validate their performance against a fair classical counterpart. The Lo-DDPG uses affine policies combined with an activation function and comprises significantly fewer trainable parameters than DDPG. Despite this affine approximation, the number of policy parameters in Lo-DDPG scales linearly with the state space size in contrast to logarithmic scaling with the VQC-based strategy. Lo-DDPG based demand response technique performs poorly in terms of net load reduction over all problem sizes with few exceptions for the one-building microgrid. A tradeoff between the Lo-DDPG performance and the sample inefficiency with an increasing number of parameters was observed by comparing the difference in net loads with that of the hybrid VQC-based approach. As the problem size increases, performance degradation observed with Lo-DDPG over the hybrid VQC-based controller also increases accompanied by an increased computational effort required to train the parametric networks.

[0148] Carbon dioxide emissions were also analyzed. As varying levels of carbon intensity, which represent the carbon dioxide (CO2) emission rate accompanied by power generation, serve as an additional source of uncertainty to the building microgrid, we evaluate the VQC-based controller 10949-03-PC for reducing net CO2emissions. The load demand for a single building over a week in January was analyzed, along with its observed net electricity consumption achieved with VQC-based energy management. It was found that despite the emissions reduction objective, there is a substantial decrease in net electricity consumption to satisfy the load requirements. The carbon intensity levels associated with power generation on the grid side were also determined. The net CO2emissions over this period corresponding to the single building’s electricity needs were determined, along with the building’s PV generation levels. It was found that, despite minimal correlations between electricity consumption and carbon intensity levels, the emitted CO2levels remain consistently low during periods of high renewable generation. The net carbon emissions recorded for building microgrids of varying sizes with different controllers were also determined. Aligning with the load reduction objective, MPC exhibits poor carbon emission reduction capabilities as varying carbon intensity serves as an additional source of uncertainty in the MPC problem formulation. DDPG demonstrates competitive emissions reduction performance for the smaller microgrids. However, deep reinforcement learning approaches like DDPG typically exhibit sample inefficiencies with an increase in problem size which was evident from the net emissions for seven and ten building microgrids.

[0149] Problem size and qubit count metrics for the building microgrids of varying sizes addressed with the VQC-based hybrid strategy are listed in Table 1 below. As evident from these metrics, both the seven and ten-building microgrid use only eight qubits. This means that a significant number of quantum amplitudes during the embedding stage of the input state for the sevenbuilding microgrid are set as zero. It was found that the improvements obtained with the hybrid VQC-based controllers over the Lo-DDPG technique for both load reduction and carbon emissions reduction objectives vary as the building microgrid size increases. These trends also indicate a correlation between the number of nonzero quantum amplitudes during the embedding stage and the controller performance. The logarithmic scaling for qubit counts with that of the state dimensionality contributes to the sparsity of quantum states and, in turn, can affect the overall demand response performance. 10949-03-PC Table 1. Problem size metrics for experiments with multiple buildings along with computational time for computing controls at each timestep.

[0150] Buildings State size # Qubits Time (ms)

[0151] 1 17 6 3.1 ± 0.11

[0152] 4 41 7 5.7 ± 0.39

[0153] 7 65 8 8.2 ± 0.19

[0154] 10 89 8 10.8 ± 0.71

[0155]

[0156] In addition to the efficient load reduction capabilities of the hybrid VQC-based demand response strategy, it also exhibits scalability in terms of computational resource utilization. As amplitude encoding is used to encode classical state x as quantum amplitudes of the initial quantum state, the required number of qubits n scales logarithmically with the dimensionality of input state and can be represented by n = log2|x|] + 1. For the VQC, a similar scaling follows with an O(n) number of circuit parameters. We have presented metrics associated with problem size and qubit count in Table 1 for the computational experiments comprising building microgrids of varying sizes.

[0157] The number of parameters used to approximate policy and value function networks in Lo-DDPG and DDPG are also compared with those utilized by the corresponding VQC-based controller, as shown in part (a) of FIG. 4 for building microgrids of different sizes. In contrast to the hybrid VQC-based controller, the number of parameters increase linearly with the size of the state for classical deep reinforcement learning algorithms like DDPG. Increasing dimensionality of the problem size typically results in training pitfalls associated with classical reinforcement learning techniques like sample inefficiency. On the other hand, the VQC-based hybrid strategy offers a parameter-efficient approach without affecting the demand responsive control performance. 10949-03-PC Both MPC and the hybrid VQC-based strategy for demand response involve solving an optimization problem for computing controls. The computational time utilized by the solvers to solve the corresponding optimization problems for building microgrids of different sizes are presented in part (b) of FIG. 4.

[0158] It can be seen that the computational resources required to solve the optimization problems for control computation within the hybrid VQC-based strategy also remain low on the classical end. At any time, the size of the formulated optimization problem increases linearly with the dimensionality of the control space |u| or the number of buildings. For the MPC-based demand response strategy, the size of the optimization problem scales as O H |x| • |u|) where His, the size of the receding horizon. Linear programming solvers can solve such optimization problems in polynomial time. Table 1 shows the computational time required to compute controls at multiple timesteps for each computational experiment. These recorded computational times are also compared to the control computation times utilized by the MPC strategy in part (b) of FIG. 4. It is clear that the computation time for MPC-based techniques grows quickly as the size of building microgrids increases, owing to the increase in state dimensionality. As evident from these metrics, the VQC-based hybrid technique for demand response not only provides a parameter-efficient value estimation strategy with a quantum circuit but also ensures computational efficiency offered by classical optimization solvers.

[0159] Although the hybrid VQC-based strategy for demand response in grid-interactive buildings exhibits performance and computational efficiency, certain factors may pose challenges to this approach as the building microgrid size increases. Due to the logarithmic scaling for the number of qubits obtained with the data encoding utilized in some embodiments, the number of parameters in the VQC leveraged for value function estimation can remain constant for problems of varying sizes. This can be observed in Table 1, where eight qubits are utilized for problems of varying dimensionality, potentially leading to inconsistencies in performance improvement against classical demand response techniques. An additional potential bottleneck that can be encountered with the adoption of the disclosed techniques in some embodiments is the connectivity of quantum 10949-03-PC hardware. Constructing the quantum ansatz used for the disclosed VQC-based controller may require high-depth circuits, which can lead to qubit decoherence owing to error-prone quantum hardware. On the classical end, the computational time required by the classical solver for control computation is trivial, as shown in Table 1, and can exhibit polynomial scaling with the number of control variables for linear optimization problems. Despite this, as the control computation with a classical solver relies on the quality of value estimation with the VQC, it is important to consider factors like quantum hardware connectivity and gate fidelities for scaling up the disclosed approach for large-scale building microgrids.

[0160] Some embodiments described above provide a hybrid VQC-based control framework for demand response in grid-responsive buildings. The controller in some embodiments leverages a hybrid quantum-classical strategy by integrating quantum circuit operations with classical optimization to enhance the control performance. A robust VQC was constructed by balancing its expressivity and trainability properties for value function estimation. Quantum unitary operations executed in the parametric circuit were exploited to construct an optimization problem for value function maximization over the control space. Solving the maximization problem with a classical solver yielded controls at each timestep. A Q-learning reinforcement learning strategy was employed to perform circuit learning and value function estimation.

[0161] The applicability and performance efficiency of the hybrid VQC-based controller were evaluated through several computational experiments with building microgrids of varying sizes. The scalability of the hybrid VQC-based controller was demonstrated with a microgrid comprising ten buildings. Compared to the baseline demand response techniques implemented on a classical computer, the VQC-based demand response approach demonstrated adaptivity and scalability in terms of computational resource utilization as well as efficient net load reduction and carbon emissions reduction capabilities. The hybrid VQC-based controller maintains logarithmic scaling with respect to the dimensionality of the state space of the underlying MDP.

[0162] Although the presented computational results demonstrate the efficiency and scalability of the disclosed hybrid VQC-based technique for demand response in building microgrids under the 10949-03-PC NISQ setting, this control framework is extensible and can be adapted for control problems that can be cast as a sequential decision-making problem. Owing to the amplitude encoding in VQC for value function estimation, computing controls for HVAC systems in buildings and wind turbines illustratively utilizes O(log N) qubits with N representing the number of buildings and wind turbines, respectively.

[0163] Illustrative embodiments can also be extended or otherwise modified in a wide variety of different ways, including by way of example through utilization of alternative quantum ansatz, such as a hardware-efficient ansatz or a tailored ansatz inspired by a specific decision-making problem. Additionally or alternatively, advanced quantum circuit optimization techniques can be applied to reduce the number of quantum gates for the VQC to improve scalability and performance. As another example, the impact of decoherence and noise on the VQC-based controller can be mitigated and the accuracy of value function estimation enhanced using robust training approaches.

[0164] Again, these are only examples, and the disclosed techniques can be applied in numerous other applications and use cases.

[0165] Illustrative embodiments provide significant advantages over conventional hybrid quantum-classical approaches to solving optimization problems.

[0166] For example, in some embodiments, VQC is used as a tool to parameterize the state-action value function in reinforcement learning, and classical optimization problem solving is used to derive optimal controls.

[0167] Additionally or alternatively, illustrative embodiments provide a data-driven methodology wherein historical trajectories are used to train the VQC in a reinforcement learning setting.

[0168] By way of example, some embodiments deal with energy management in complex energy systems with hybrid quantum-classical control and real-time optimization. At each of a plurality of timesteps, a decision-making problem is solved in real-time using the techniques disclosed herein. This illustratively involves computing optimal decisions or controls associated with 10949-03-PC individual components of the energy system and executing them accordingly. Again, numerous other use cases in other contexts are possible using the disclosed techniques.

[0169] As more particular examples, some embodiments disclosed herein focus on the optimal control problem with a hybrid quantum-classical approach leveraging VQCs of gate-model quantum computers and classical optimization solvers. Optimal control problems are optimization problems and typically take the form of mixed-integer linear programming (MILP) and mixed-integer quadratic programming (MIQP) problems. Many commonly arising optimal control problems are formulated over an infinite horizon and need rigorous approximations for obtaining solutions in the classical paradigm. In the VQC-based approach to solving control problems in some embodiments herein, a VQC is employed to approximate the optimization objective over an infinite horizon, followed by using a classical solver to obtain the optimal controls. This approach illustratively operates under a hybrid quantum-classical paradigm as disclosed herein, where a classical solver produces solutions in the form of controls. The controls are then applied via a controller to a dynamic environment, the state of which is used to update the VQC, followed by using the updated VQC to extract an updated optimization problem. Such an approach overcomes limitations that might otherwise arise for optimal control problems formulated as MILP / MIQP problems over an infinite horizon. Numerous other optimal control problems and other types of optimization problems can be efficiently solved using the techniques disclosed herein.

[0170] In some embodiments, the demand response problem is utilized as an example use case to demonstrate the applicability of the VQC-based hybrid control framework. The objective in this example use case is illustratively to control energy systems deployed in buildings such that the overall load on the power grid is reduced. Another way of casting the demand response problem in some embodiments is as a scheduling problem that optimizes the charging / discharging schedules of the energy systems.

[0171] In some embodiments, a classical component solves an optimization problem to produce controls which are then used to update the VQC. The updated VQC in turn affects the classical optimization problem. 10949-03-PC Illustrative embodiments can be viewed as solving a “new” optimization problem of a similar structure at each of a plurality of timesteps, corresponding to respective iterations, rather than an optimization problem that remains static over time. Such an approach is particularly advantageous for problems that need fast solutions, including control problems in a broad array of applications from planes and spacecraft to agriculture and building HVAC.

[0172] Unlike conventional approaches, illustrative embodiments incorporate a classical solver which solves an optimization problem, to get better inputs for updating a VQC, which in turn improves the optimization problem. Such embodiments provide an improved integration of quantum and classical components refining each other at each step, using tools such as classical solvers and VQCs.

[0173] Some embodiments provide a hybrid quantum-classical control strategy, showcasing the application of VQCs for optimal control problems using reinforcement learning and classical optimization. These embodiments illustratively apply a VQC-based hybrid control strategy for optimal control problems spanning various domains.

[0174] Such embodiments leverage a VQC trained under a reinforcement learning setting, integrating classical optimization for decision-making. These embodiments merge quantum and classical computing paradigms for sequential decision-making problems, and are not restricted to domain-specific use cases.

[0175] Moreover, illustrative embodiments are not restricted by the types of variables, such as binary or continuous variables. In some embodiments, any problem that can be formulated as a sequential decision-making problem can be solved using the disclosed hybrid quantum-classical approach, which advantageously combines VQC with classical optimization for improved sequential decision-making. Other embodiments can be configured to solve other types of optimization problems, and are not limited to sequential deci si on -making problems.

[0176] Accordingly, information processing systems with hybrid quantum-classical functionality as disclosed herein can be configured to support a wide variety of distinct applications, in numerous diverse contexts. 10949-03-PC References herein to particular applications, such as hybrid quantum-classical approaches to solving demand response control problems in the grid-interactive buildings context, are therefore presented by way of illustrative example only, and the disclosed techniques can be adapted for use in any of a wide variety of other contexts.

[0177] Additional illustrative embodiments will now be described in further detail with reference to FIGS. 5 through 9. These illustrative embodiments are advantageously configured to provide VQC learning-enabled robust optimization for Al data center energy control and decarbonization, but can be adapted for numerous other use cases and control applications.

[0178] As indicated previously, as the demand for Al models and applications continues to grow, data centers that handle Al workloads are experiencing a rise in energy consumption and associated carbon footprint. Illustrative embodiments disclosed herein provide a VQC-based robust optimization (VQC-RO) framework for control and energy management in large-scale data centers to address the computational challenges and overcome limitations of conventional model-based and model-free strategies. The VQC-RO framework integrates VQCs with classical optimization to enable efficient and uncertainty-aware control of energy systems in Al data centers. Quantum algorithms illustratively executed on NISQ devices are used for value function estimation trained with Q-learning, leading to the formulation of a robust optimization problem with uncertain coefficients. The quantum computing-based robust control strategy is designed to address uncertainties associated with weather conditions and renewable energy generation while optimizing energy consumption in Al data centers. These embodiments provide a quantum computing-based robust control framework for energy-efficient and sustainable data center operation, achieving substantial reductions in carbon emissions and energy consumption in large-scale data centers handling Al workloads.

[0179] Data centers currently account for about 1% to 1.5% of global electricity consumption and are responsible for one percent of energy-related greenhouse gas emissions. As the demand for Al models for chatbots and generative applications grows, cyberinfrastructure equipped with vast amounts of computational resources designed for dedicated Al workloads like training and 10949-03-PC deployment has been leveraged by Al data centers. These illustratively comprise high-performance hardware accelerators like GPUs, storage systems, servers, and networking architectures. Advances in Al hardware accelerators have enabled energy-efficient computing without compromising processing speed, resulting in a 6% increase in global data center energy consumption over the last decade despite an explosive growth in computing capacity. Even with sophisticated hardware and software designs to handle Al workloads, electricity consumption associated with Al data centers may rise with the increasing popularity of Al-powered applications, as exemplified by the growing energy usage of large-scale data centers by 20-40% annually. Integration of renewable energy sources and battery energy storage to power Al workloads in data centers is a promising strategy to lower reliance on electricity grids while mitigating the carbon footprint and has been adopted by several major providers. Apart from the computing hardware, infrastructure components, like cooling and power delivery systems, also contribute significantly to the overall energy consumption.

[0180] Resource management systems employed to regulate infrastructure operation present an opportunity to establish efficient energy management strategies for Al data centers that incorporate renewable generation and energy storage devices to minimize electricity consumption and accompanying carbon emissions. Although the development of optimization-based strategies has been a promising approach for energy management in data centers, complicated formulations derived by physics-based modeling of energy systems operations and real-world phenomena like dynamic electricity tariffs, can lead to intractability, especially for data centers that handle large-scale Al workloads.

[0181] There are several challenges associated with developing a quantum computing-based robust control framework for energy management in Al data centers. Today’ s quantum computers, illustratively including NISQ devices, exhibit limitations in terms of their error rates and size, which can induce complexities with the application of quantum computing techniques for energyefficient control. A first challenge lies in adopting quantum algorithms capable of deriving practical utility for real-world problems. Another challenge is to enable a mode of operation that 10949-03-PC integrates quantum algorithms with classical subroutines, using information from quantum operations to create an optimization problem for control computation in a model-free reinforcement learning setting. The inability to strictly enforce constraints on the various control actuators associated with energy components in the Al data center with a reinforcement learningbased approach can form an additional challenge. A final challenge lies in ensuring that the quantum computing-based reinforcement learning strategy can effectively deal with different uncertainties associated with varying weather conditions and renewable energy generation. Controls obtained with conventional robust optimization strategies can produce conservative solutions, resulting in increased data center power consumption. Thus, it is important to establish the robustness of the quantum computing-based control strategy without compromising its energyefficiency capabilities.

[0182] To address these and other challenges, illustrative embodiments disclosed herein provide a control framework that leverages VQCs to enable robust optimization in a reinforcementlearning setting for energy management in Al data centers. As described previously herein, VQCs are parameterized models executed on circuit-based or gate-based quantum computers and have been extensively used in quantum machine learning algorithms owing to their expressive power on NISQ devices. The control framework referred to herein as VQC-RO leverages a VQC for value function estimation of a state-action pair comprising data center observations, future uncertainty information, and control actions. With efficient value function approximation achieved with a Q-learning based approach, a robust optimization problem can be formulated with a trained VQC, which allows to hedge against future uncertainties like weather conditions or renewable generation and yield robust controls. Classical state-action information is encoded onto the quantum circuits in a way that enables extraction of a linear programming problem with uncertain coefficients in the objective function, which results in an efficient control computation strategy facilitated by a classical optimization solver.

[0183] We investigate the power reduction performance and impact of integrating solar generation into Al data centers by conducting multiple computational experiments involving illustrative 10949-03-PC embodiments and various locations in the United States, including California, Texas, and Virginia. Carbon emission levels associated with the data center operations are also analyzed to quantify the impact of the disclosed quantum computing-based control framework for efficient energy management in Al data centers. In addition, the computational efficiency and scalability of the disclosed energy management framework are also established with comparisons against classical counterparts.

[0184] Accordingly, some embodiments disclosed herein provide a VQC-RO control framework for energy management in Al data centers that exploits the computational efficiency offered by quantum algorithms executed on NISQ devices along with uncertainty-tackling capabilities of robust optimization.

[0185] Additionally or alternatively, some embodiments provide controller design and learning established with a mode of operation integrating a variational quantum algorithm for value function estimation with classical optimization to produce robust controls.

[0186] Illustrative embodiments of the disclosed control framework in terms of its power consumption and carbon emissions reduction capabilities are analyzed through computational experiments performed with reference to Al data centers at various locations. A comparative study, comparing the performance of illustrative embodiments with a rule-based control method and a deep reinforcement learning approach, is also presented.

[0187] Referring initially to FIG. 5, an example digital twin 500 of an Al data center in an illustrative embodiment is shown. The Al data center is assumed to be capable of handling intensive computational workloads like Al. The digital twin 500 is a model of the corresponding Al data center, and is configured to enable accurate simulation and visualization of the data center’s energy usage, allowing for energy efficiency improvements and optimization through high-performance control strategies as disclosed herein. The digital twin 500 models an Al data center that includes components corresponding to IT infrastructure, illustratively one or more rows of racks 501 comprising servers and other IT components, as well as a cooling system comprising a computer room air handler 505 coupled to a chiller 506 and a cooling tower 507, battery storage 10949-03-PC 508, and a PV generation module 510, among other components. The figure depicts the power flow as well as thermal energy flow for the Al data center. In this example, a two-zone data center equipped with HVAC is considered for energy modeling. The IT infrastructure of the Al data center is more particularly modeled to include the one or more rows of racks 501, which are standardized enclosures used to mount various computing equipment modules such as servers, switches, storage devices, and networking equipment. The Al data center is also coupled to an electric power grid 512 as illustrated in the figure.

[0188] FIG. 6 shows a hybrid quantum-classical information processing system 600 that includes a quantum computing subsystem 602, a classical computing subsystem 604 and an Al data center digital twin 605. The Al data center digital twin 605 may be more particularly configured in the manner illustrated by the digital twin 500 of FIG. 5, although numerous other types and configurations of data centers or other dynamic environments can be modeled using digital twins as disclosed herein. In this embodiment, energy management in an Al data center is achieved by computing controls in accordance with the above-noted VQC-RO framework.

[0189] In the FIG. 6 embodiment, the quantum computing subsystem 602 comprises at least one VQC. As illustratively shown, an example VQC of the quantum computing subsystem 602 comprises a plurality of stages including at least a quantum encoding stage 610 and additional stages 612 for quantum gate operations, including parametric gate operations, and measurement. The quantum encoding stage 610 is an example of what is more generally referred to herein as an initial quantum state preparation stage, and the stages 612 are examples of what are more generally referred to herein as a parametric gate operations stage and a measurement stage. It is to be appreciated that numerous other arrangements of additional or alternative VQCs and associated stages can be used in other embodiments.

[0190] The quantum computing subsystem 602 is configured to utilize the at least one VQC to generate, for each of a plurality of iterations, a different set of parameters for an optimization problem to be solved in the classical computing subsystem 604. The classical computing subsystem 604 is configured to receive from the quantum computing subsystem 602 a given one 10949-03-PC of the sets of parameters generated for a given one of the iterations, to formulate a corresponding instance of the optimization problem based at least in part on the given set of parameters, and to solve the corresponding instance of the optimization problem to generate one or more controls. The one or more controls generated by the classical computing subsystem 604 are applied to the Al data center digital twin 605 for utilization therein to perform one or more sequential decisionmaking operations. The Al data center digital twin 605 and other digital twins referred to herein are considered examples of a “dynamic environment” as that term is broadly used herein. The quantum computing subsystem 602 receives state information from the Al data center digital twin 605 for use in determining, via the VQC comprising stages 610 and 612, another one of the sets of parameters for a subsequent one of the iterations.

[0191] It should be noted that, although the dynamic environment in this embodiment illustratively comprises a digital twin of an Al data center, in other embodiments the dynamic environment can comprise at least a portion of the Al data center itself, such as one or more energy systems of the Al data center, in place of or in addition to the digital twin. Thus, the use of a digital twin in this embodiment should be viewed as an example only, and not limiting in any way on the scope of embodiments of the present disclosure. For example, one or more controls computed using the digital twin 500 of the Al data center can be applied to control one or more corresponding real-world energy systems of the Al data center itself.

[0192] FIG. 7 shows an example of a VQC 700 that may be implemented in the quantum computing subsystem 602 of the hybrid quantum-classical information processing system 600 of FIG. 6. In this example, the VQC 700 receives as its inputs an observation x of the dynamic environment comprising the Al data center digital twin 605, an action a, and a disturbance w. The disturbance w illustratively represents future uncertainty information of the dynamic environment comprising the Al data center digital twin 605, such as outdoor weather conditions and solar energy generation, which in this embodiment is directly encoded into the quantum state of the VQC. The state s of a given state-action pair (s, a) in some embodiments comprises at least one observation 10949-03-PC of the dynamic environment and at least one corresponding disturbance of the dynamic environment, illustratively denoted as (x, w).

[0193] The VQC 700 of the quantum computing subsystem 602 therefore illustratively receives state information of the dynamic environment for a given state of a state-action pair, with the state information comprising one or more observations of the dynamic environment and one or more corresponding disturbances representing future uncertainty information of the dynamic environment. As indicated above, the future uncertainty information illustratively comprises information relating to at least one of outdoor weather conditions and solar energy generation associated with the dynamic environment, although additional or alternative types of future uncertainty information can be used in other embodiments.

[0194] The VQC 700 may be viewed as an example of an arrangement that utilizes a first type of quantum encoding to encode the one or more observations of the dynamic environment, and a second type of quantum encoding, different than the first type of quantum encoding, to encode the one or more corresponding disturbances representing future uncertainty information of the dynamic environment.

[0195] The first type of quantum encoding illustratively utilizes a first set of one or more parametric gate operations, and the second type of quantum encoding illustratively utilizes a second set of one or more parametric gate operations different than the first set of one or more parametric gate operations. For example, the first and second types of quantum encoding may differ at least in terms of their respective orders of applied parametric gates, but may differ in additional or alternative ways.

[0196] The VQC 700 of the quantum computing subsystem 602 also illustratively receives action information for the action of the state-action pair, the action information comprising one or more controls. Another type of quantum encoding, different than at least one of the above-noted first and second types of quantum encoding, possibly a third distinct type of quantum encoding, may be used to encode the one or more controls of the action information. For example, each of the different types of quantum encoding may utilize a corresponding different set of one or more 10949-03-PC parametric gate operations. Each different set of one or more parametric gate operations can involve, for example, different orders of applied parametric gates and / or other different types and arrangements of parametric gates.

[0197] Numerous alternative arrangements are possible. For example, in some embodiments, only the one or more observations of the state information and the one or more controls of the action information are encoded in the VQC, such that the one or more disturbances are not encoded in the VQC. In such an embodiment, the quantum encoding of the one or more observations and the quantum encoding of the one or more controls can utilize respective first and second different types of quantum encoding, each using respective distinct sets of one or more parametric gate operations.

[0198] FIG. 8 shows an example machine learning algorithm for training a VQC-based controller for Al data center energy control optimization using reinforcement learning in an illustrative embodiment. In this embodiment, an alternative version of Algorithm 1 of FIG. 3 is shown, adapted for the above-described VQC-RO framework. The operation of the example machine learning algorithm of FIG. 8 is generally similar to that of the FIG. 3 version as previously described, but adapted for use in the Al data center context rather than the microgrid context. Other versions of Algorithm 1 can be adapted in a similar manner for a wide variety of alternative contexts.

[0199] Additional details regarding the embodiments of FIGS. 5 through 9 and the associated VQC-RO framework will now be described.

[0200] In the following, an example digital twin of the Al data center comprising computing infrastructure, battery energy storage, and PV generation is described, along with its MDP formulation. The disclosed VQC-RO control framework is then described, followed by description of computational experiments that were performed to evaluate the performance efficiency of illustrative embodiments.

[0201] In some embodiments, we construct a digital twin of an example data center capable of handling intensive computational workloads like Al to enable accurate simulation and 10949-03-PC visualization of the data center’s energy usage, allowing for energy efficiency improvements and optimization through high-performance control strategies, as illustrated by the digital twin 500 of FIG. 5.

[0202] As one possible example, a two-zone data center equipped with HVAC is considered for energy modeling. The Al data center is modeled to include multiple rows of racks, which are standardized enclosures used to mount various computing equipment modules such as servers, switches, storage devices, and networking equipment. We define each rack as a collection of computing units and information technology (IT) equipment fans that draw power to handle corresponding workloads. The instantaneous power consumption associated with the IT equipment in the racks is dependent on the current computing workload, the inlet temperature, and the corresponding power curves. Energy models are leveraged to prototype the power consumption of rack-mounted servers denoted by Erack. The instantaneous power consumption is computed with the power curve for IT equipment in each rack, which is a piecewise function of the workload ratio. The number of computing units within each rack is chosen such that the power density of a single rack is between 20-60kW to mimic the operation of high-performance computing clusters commonly used for Al training and inference workloads. The IT fans perform active cooling using forced convection followed by dissipation of the heat generated by the computing units into the surrounding environment. The constructed model considers the dynamic interactions between the IT equipment, airflow, and cooling systems in the Al data center to predict energy usage and thermal conditions under different operational scenarios.

[0203] Another major component of a typical Al data center is the computer room air handling (CRAH) unit, which removes heat with a chiller system. Accordingly, we equip the server room in the Al data center digital twin with a floor-mounted CRAH unit to facilitate cold air flow through the racks. With the cooling setpoint temperature as input, the power consumption of the cooling system Ecoolingcan be calculated by considering power drawn by cooling components like the cooling tower and the chilled water pump in response to the current room temperature and the ambient air temperature. 10949-03-PC Referring again to FIG. 5, the example Al digital twin 500 is shown, and models the Al data center comprising IT infrastructure with the one or more rows of racks 501, the cooling system comprising cooling components 505, 506 and 507, the battery storage 508, and the PV generation module 510, depicting the power flow as well as thermal energy flow as described above.

[0204] Data centers equipped with battery energy storage systems can contribute to the overall resilience of the electrical grid by not only mitigating the impact of short-term power outages but also providing peak reduction and load balancing capabilities. The energy storage system in the digital twin is defined by its capacity C, nominal capacity Cnom, charging / discharging efficiency factor and the capacity loss coefficient TJ. The nominal capacity determines the amount of power that can be delivered by a fully charged battery under specific conditions, while the capacity loss coefficient measures the degradation of the battery over each charging / discharging cycle. For a rechargeable battery, the nominal capacity and charging efficiency typically vary based on its current state of charge (SOC) and can be represented by piecewise linear functions given by the power capacity curve Cnom(SOCf) and efficiency curve ("(SOC), respectively. The dynamics associated with the battery energy storage system comprising charging and discharging along with the battery capacity degradation can be modeled with Equations (13) and (14) below, respectively. Specifically, Equation (14) represents the reduction in battery capacity over time, where iSOCf — SCCf- indicates the amount of energy that has been charged or discharged in the previous cycle.

[0205] ( min{Ct, SOCt+ Ebat■ 7?} if Ebat> 0

[0206] (13) |max |o, SOCt+ Ebat• -y= } if Ebat< 0

[0207] r_r r\SOCt- SOCt-r\ct+i—UG) ’ (14)

[0208] ~ ApyTJpvPf ’ G (15) Enet max{0, + Ecooung + EbatEpV(16)

[0209]

[0210] 10949-03-PC

[0211] In Equations (13) and (16) above, Ebat represents the amount of power drawn by the battery storage for charging and is bounded by its current nominal capacity described by [—^nom^OCf), CnomSOCt)].

[0212] In addition to contributing towards efficient energy management, battery energy storage systems also allow easier renewable energy integration. To analyze the impact of adopting renewable energy sources for energy management in data centers, we incorporate PV generation to supplement the energy needs of the Al data center and reduce reliance on the electric power grid. In some embodiments, we simulate the installation of PV panels for data center operations, allowing the generated power to be used to minimize energy consumption or charge the data center’s battery energy storage system. The amount of energy produced by the PV panels is determined by the area exposed to solar radiation APV, the efficiency coefficient TJPVand the packing factor Pf. The packing factor, or the density of solar cells in a PV module, influences the module’s output power and is dependent on the shape of the solar cells used. Using a total solar irradiation of G, the power generated by the PV panels denoted by Epi can be modeled with Equation (15) above. Controlling the cooling setpoint for the CRAH unit in the data center and the charging / discharging levels for the battery energy storage system dictate the electricity consumption incurred by the Al data center. Additionally, power consumption in the data center can be supplemented by PV generation and battery discharging due to the provisions for equipped battery energy storage and renewable generation. The net electric power demand requirement for the Al data center can then be modeled as Enetthat is drawn from the electric power grid, as shown in Equation (16).

[0213] In some embodiments, an MDP is used to enable the development of a model-free approach for efficient energy management in an Al data center. The MDP illustratively characterizes operations encompassing the data center components of computing and cooling as well as energy storage. As indicated previously, an MDP is defined by a state space S, an action spaced, a reward function r, a discount factor y, and transition dynamics. At time t, we consider the state vector 10949-03-PC st= (xt, wt) to include the measurable states of the Al data center xt and future uncertainty information wt. Environmental parameters like the air temperature of the server room and ambient air temperature of the external surroundings, along with the state of charge of the energy storage device, constitute the data center state xt. As the computational workload directly contributes to the power consumption levels in the Al data center, the workload ratio is also considered as a state variable.

[0214] Enabling time awareness while addressing sequential decision-making problems involving state variables with seasonal variations is important to improve the learning capabilities of the reinforcement learning agent. To accommodate time awareness in the reinforcement learning agent, cyclical feature encodings using sine and cosine transformation for the day of the week and month are included in the data center state xt spanning the state space stG IE8. Uncertainties like future forecasts for ambient temperature, carbon intensity levels, or PV generation can be included in the uncertainty vector wt. The state variables can be adjusted based on specific objectives. For a sequential decision-making problem tasked with minimizing net carbon emissions, it is important to consider the current carbon intensity as a state variable in xt and forecast carbon intensity as wt to form the state space st6 Ik9.

[0215] As the constructed digital twin for the Al data center allows provisions for controlling the cooling setpoints and energy storage charging and discharging levels, the action space can be described as atG R2. The action space can be limited to prevent instabilities during training and mitigate extensive computational resource utilization. The action variable corresponding to the cooling setpoint is then reformulated as the change of the previous cooling setpoint constrained in [— 6Tmax, <5T]. Similarly, the charging / discharging actions for the battery energy storage system are restricted within [—CnomSOCt), Cnom^SOCt')]. Using the known bounds on the control actions, we further define the action space as atE [—El]2. The transition dynamics for the MDP are deterministic and align with the energy model constructed for the Al data center digital twin. The reward structure for the MDP can be modified depending on the reinforcement learning task. For power consumption reduction and carbon emission reduction, the reward function at any time 10949-03-PC t can be defined as rt= —Enet tand rt= — ctEnet t, respectively, where Ct and Enet.t indicate the carbon intensity and net power drawn by the Al data center in response to its current state and chosen control actions. As the objective of reinforcement learning agents is to identify a policy TT: S - A which maximizes the expected discounted rewards over an infinite horizon Ea~π[Σ∞t=0γtrt] the reward function is scaled as (Rmin— 'rt) / Rminto bound the individual and total rewards.

[0216] Referring again to FIG. 6, this illustrative embodiment shows an overview of energy management in an Al data center in which controls are computed in the manner described herein, using the disclosed hybrid VQC-RO framework comprising quantum computing subsystem 602, classical computing subsystem 604 and Al data center digital twin 605.

[0217] The disclosed methodology in some embodiments employs a Q-learning approach to estimate the state-action value function using a VQC, which represents the cumulative sum of discounted rewards over an infinite horizon. Encoding the input data onto the quantum circuit, applying parametric gate operations, and measuring quantum states involved in the VQC are the key steps comprising value function estimation. Extracting the robust optimization problem that incorporates future uncertainties from the trained VQC is solved for computing controls at each timestep. The training process for the VQC involves using a Q-learning based approach to update the gate parameters and estimate the target function, while the implementation phase involves using the trained VQC to obtain control actions for the Al data center by solving a robust optimization problem.

[0218] An example Q-leaming approach aims to obtain a value function Q s, a) that satisfies the Bellman equation in Equation (17) below, wherein 7r(s) represents a deterministic policy function. Here, we leverage the example VQC 700 of FIG. 7 to parameterize the state-action value or the Q-value function capable of estimating the cumulative sum of discounted rewards over an infinite horizon with the control trajectory initialized with the (s, a) pair. Executing the VQC 700 illustratively includes encoding the input data, applying the parametric gate operations or the variational circuit, and measuring the quantum states. 10949-03-PC Translating a classical vector into a quantum state is referred to as quantum embedding or encoding. To encode the state-action pair (s,d) comprising s = (x, w), separate quantum embedding techniques are employed to ease formulation of the optimization problem for control computation in the subsequent steps. The observation vector x E IR8is encoded with the amplitude encoding technique with a four-qubit quantum state. A quantum state |i / i) = EF=i7rrr 10canbe

[0219]

[0220] |ki|2prepared by normalizing the data center observations with the amplitudes of the three-qubit computational basis states. Loss of information by using one fewer degree of freedom can be prevented by encoding the norm of the observation with a parametric quantum gate. The Rx quantum gate performs a specified rotation about the x-axis of a single-qubit state. By establishing bounds for the amount of rotation, the state vector norm can be mapped to 0X. The input state |i / r generated for the observation x can then be described by Equation (18) below. A similar transformation is applied to encode the control variable a E [—1, 1] and the uncertainty w by rotation about the x-axis with Rx gates and specific angles 0aand 0Wobtained from known bounds. Two single-qubit quantum states |u) = Rx(0a~) with 0a= n • a and |w) = 7?x(0w) are prepared by encoding the classical information. It should be noted that for state-action value estimation, the parameters in the data encoding step (0X, 0a, 0W) remain fixed for each classical state and action pair.

[0221] Q(s,a) = £>[r + y(?(s',7r(s'))] (17)

[0222] / \ / cos— \ |^x) = |^ 0 / ?x(0x)|O) = |i / i) 0 1 (18)

[0223] I “x I \ — isin — / \ 2 / At = (aUiicos9u / 2+ Pu,i sin0u / 2) (aw,t cos0w / 2+ fiw lsin0w / 2) (19)

[0224] Q(s, a;0) = Qw- m + Qb(20)

[0225]

[0226] 10949-03-PC After the initial quantum state following data encoding is prepared, adjustable parameters of the VQC 700 are used to process the input quantum state. As shown in FIG. 7, pooling gate operations are also applied between consecutive qubits comprising the quantum state \ipx) which represents the data center observations. The pooling operator

[0227]

[0228] pt, S^) involves a series of gate operations between the z-th source and sink qubit pair, including a fixed inversion of the sink qubit along the z-axis and an inverted controlled NOT operation. These are followed by rotation of the source and sink qubits along the z-axis and y-axis by <pt and angles with the Rz and Ry gates, respectively. Finally, a controlled NOT operation followed by a rotation of the sink qubit state by a parameter along the y-axis is applied. These pooling operators are applied to each qubit pair comprising the four-qubit quantum state \ipx) sequentially, to enhance the entangling capability of the VQC and distill the information within the quantum state into a single-qubit sink qubit. This qubit state is further entangled with the single-qubit states |iz) and |w) using controlled NOT operations, which directly contribute to the efficiency of variational quantum algorithms. Following the parameterized unitary gate operations in the VQC 700, the resulting zz-qubit quantum state can be represented by 2" quantum amplitudes corresponding to the individual basis states. The z-th quantum amplitude can be represented by Atas shown in Equation (19) above, where a, E (C that satisfy the complementary condition a • ft = 0. Measuring this quantum state yields probabilities of the quantum state collapsing into the z-th computational basis state as m, = |J42. The measurement vector m is scaled by parameters Qwand Ob as shown in Equation (20) above to estimate the state-action value function for the input (s, a).

[0229] Using a trained VQC with parameters 0 to estimate the value for a given state-action pair, a robust optimization problem can be formulated to extract the controls with a deterministic policy zr(%), as illustrated by an example min-max problem in Equations (21)-(24):

[0230] 2n

[0231] R0 x,&) max min " c2• f(w~)u (21) u w

[0232] i=l

[0233] s.t. Ct = Qw■ (x, 0) Vi 6 [l,2n] (22) 10949-03-PC 0 < u < 1 (23) (w) E U (24)

[0234] The VQC in this embodiment is trained to estimate the cumulative discounted rewards with the current data center observations and actions along with perfect information of future uncertainty. At any timestep / , the max-min problem in Equations (21)-(24) above with the current set of VQC parameters and data center observations x is solved to compute auxiliary variables u. The robust optimization formulation hedges against future uncertainties w by incorporating uncertainty sets for the function f(w) = cos212 inEquation (21). The uncertainty sets for variables w can be further transformed using the same transformation process and known bounds to derive uncertainty sets U in Equation (24). It should also be noted that the auxiliary variable u is chosen such that the resulting optimization problem is linear for faster solution times. The objective function in Equation (21) is to maximize the worst-case realization of the scaled measurement values which represent the state-action value. The coefficients of this linear objective comprise the transformed uncertain variables as well as squares of linear combinations of encoded observations represented by j(x, 0). These linear transformations are a direct result of the unitary gate operations applied to the input quantum state that encodes the data center observations xt. Constraints like Equation (23) can also be leveraged in the robust optimization problem formulation to enforce solution search within the feasible action space. Following the solution procedure, the optimal values of the auxiliary variables can be further used to compute optimal control actions with 7r(xt) = G(u* = argmax^RO^Xf, 0)) where the mapping G is obtained by inverting the transformation obtained from Equations (19) and (20). This mapping is used to compute control actions for the MDP at any time t and can be described as at— G(u) — TT COS-1( U).

[0235] Additional details regarding controller learning and implementation in illustrative hybrid VQC-RO embodiments will now be described, with reference to the machine learning algorithm of FIG. 8. Learning in a VQC illustratively involves measurement of the quantum state obtained 10949-03-PC after parametric gate operations to estimate the target function, and updating the gate parameters by adhering to the minimization of a specified loss function. Here, as the VQC approximates the state-action value function, the VQC is trained with a Q-learning based approach, as mentioned previously. The VQC is initialized with parameters 0 sampled from a uniform distribution U —n, if). A replay buffer D is used to store transitions and serves as experience replay to help increase sample efficiency and prevent training instabilities. During the training process, each transition at timestep t includes the current state st comprising data center observations xt and perfect knowledge of the future uncertainty wt, applied control actions at, the obtained reward rt, and the following state st+i. To balance exploration and exploitation while training the hybrid VQC-RO controller, an epsilon-greedy strategy is used. This strategy determines the probability of choosing controls that maximize the value estimate, as shown in Equation (25) below. During the exploration phase, instead of randomly sampling controls, an RBC is illustratively used in this embodiment to ensure reasonable charging and discharging levels of energy storage devices equipped in the Al data center. Although typical Q-learning techniques rely on random exploration, the use of a rule-based control can help to avoid unnecessary exploration and subsequent computation. The probability is decayed by a factor of edecay with each timestep as the training progresses. It should also be noted that during the training phase of the controller, RO(xt, 0) is a deterministic optimization problem due to perfect information availability of the future uncertainties and can be denoted by

[0236]

[0237] DO (xt, 0) =ci • subject to constraints in Equations (22) and (23).

[0238] A batch of transitions of size B_size is sampled from the replay buffer / ). As the objective of Q-learning is to approximate the Bellman equation, the targets for all state-action pairs are computed by maximizing the value function over the set of next states with yt= rt+ y • maxsDO(xt+1, 0). It should be noted that computing the targets ytillustratively involves solving the deterministic optimization problem instead of the robust counterpart owing to the perfect knowledge of the future states within recorded trajectories. Computing these target values 10949-03-PC illustratively involves solving an optimization problem with a classical solver for each transition in the sampled batch.

[0239] P at— G(argmaxaRO(xt, 0)) = 1 — e (25)

[0240] - 2QW• {(? (st, at; 0 + - Q (st, at; 0 - ^)} (26)

[0241]

[0242] Using the computed targets, the VQC-based value function estimator can be updated in a supervised manner to minimize the mean squared error objective. Gradients of the objective with respect to individual parameters of the VQC are determined for a gradient descent step. We use a parameter shift rule to estimate the gradients of the VQC-based value function approximator, as given by Equations (20) and (26). Equation (26) involves the computation of value function estimates with two sets of gate parameters for obtaining gradients of the loss function. Finally, the scaling parameters Qwand Qbare updated with a gradient descent step. These steps are repeated over multiple training episodes for circuit learning during the training phase of the hybrid VQC-RO controller. During the implementation or the evaluation phase, only data center observations xt are available at any timestep. To hedge against future uncertainties wt, the robust optimization problem R0(xt, 0) is solved with formulated uncertainty sets and a current set of gate parameters 0 to obtain control actions at. During the implementation stage, the VQC parameters can also be updated at larger intervals with the training procedure described above to ensure the adaptability of the controller to previously unseen data center trajectories. As indicated previously, an overview of the above-described controller learning procedure is shown in FIG. 8, as an example algorithm for training and implementing the disclosed VQC-RO controller for energy management in an Al data center.

[0243] A number of computational experiments were performed with the Al data center digital twin to evaluate the performance of the disclosed VQC-RO control framework. For these experiments, the data center digital twin is set up to mimic large-scale Al workloads by adjusting 10949-03-PC the power density levels of each rack in the data center. Data center operations at three locations in the United States are simulated to analyze the energy management performance of the VQC-RO control strategy in varying weather conditions. The battery energy storage system in the Al data center is sized such that it can supplement its net power consumption requirements. Locationspecific simulations are performed with historical weather and carbon intensity data recorded in 2021-2022 at three locations in the United States, namely, in Los Angeles (California), Dallas (Texas), and Ashburn (Virginia). The carbon intensity levels indicate the amount of carbon emissions released by the grid for generating one unit of electricity. The higher concentration of data centers at these locations in the United States as compared to other locations forms the basis for evaluation in our experiments. We use real-world computational workload data from Alibaba for each Al data center under consideration.

[0244] Apart from the VQC-RO control framework, an RBC strategy and a DDPG strategy are applied as respective alternative approaches for energy management in these Al data centers, for benchmarking purposes. The RBC strategy relies solely on a proportional-integral-derivative (PID) loop to adjust controls based on the errors associated with the measured computing room temperature and the cooling setpoint. The DDPG strategy is a model-free deep reinforcement learning method, and is chosen to benchmark the VQC-RO control framework, which adopts a reinforcement learning approach to train the VQC-based value function estimator. As the disclosed control framework does not rely on a model derived from first principles, model-based robust optimization techniques are not selected as baselines to ensure a fair comparison. The model-free approaches can also aid in evaluating the adaptability of the controllers under varying Al data center characteristics.

[0245] For the reinforcement learning-based approaches, a reward function to minimize the net power consumption of the data center is utilized along with a discount factor y = 0.995. The hyperparameters for both these approaches are also kept consistent for a fair comparison. It should also be noted that the training process for the learning-based controllers is conducted with historical data in 2021, while their performance is evaluated with recorded weather conditions in 10949-03-PC 2022. Control computation for the VQC-RO control framework leverages box uncertainty sets during the evaluation process. Initially, we apply the energy management strategies to the Al data centers at various locations without considering the PV integration to analyze their impact on existing data centers. The adaptability capabilities of the control strategies are also further evaluated by introducing PV generation capabilities at each Al data center to measure the power consumption reduction performance in response to cyclical uncertainties of solar radiation levels. Finally, the net carbon emissions levels associated with electricity consumption are also analyzed to estimate the reduction in carbon dioxide emissions with advanced energy management strategies in the Al data centers.

[0246] Data center operations at all three locations were simulated over the year 2022 with different energy management strategies. The annual net power consumption by each Al data center without considering PV generation was analyzed. The highest power consumption levels are observed in Texas, which can be attributed to the large amounts of power drawn for cooling the data center with relatively higher ambient air temperatures. On the other hand, the least power consumption is incurred for the Al data center located in Virginia as a result of colder temperatures during the winter months. The annual power consumption obtained with the disclosed VQC-RO control strategy is significantly lower than the DDPG controller for Al data centers at all three locations, with the percentage reduction ranging between 7.3 to 12.5%. For the states of California and Virginia, VQC-RO demonstrates a reduction of approximately 5% in its power consumption levels as compared to the RBC strategy, while only a 0.4% variation is observed in the state of Texas.

[0247] To analyze the seasonal variations in power consumption, we also measure the net monthly consumption for months representative of the seasons of winter, spring, summer, and fall. It was found that the VQC-RO consistently produces lower power consumption across all seasons in California and Virginia. For the colder months in Texas, the RBC strategy exhibits improved performance over both learning-based energy management techniques. The lower energy consumption in January achieved with RBC also aligns with the annual power consumption in 10949-03-PC Texas. Additionally, the DDPG energy management strategy performs worse than the RBC or VQC-RO in each season at all Al data center locations, with only two exceptions and limited variability. We also analyzed the weekly trend for each month and location obtained with the various control techniques. From these trend lines, sudden drops or rises in power consumption levels are observed in the Texas data center in the months of January and April. These abrupt changes could be attributed to extreme weather events like the North Texas snowstorm of 2022 and are responsible for varying power consumption reduction performance in contrast to remaining seasons where only slight fluctuations in trends are observed.

[0248] To further validate the robustness of the applied controls, we conducted experiments by integrating PV generation into the data center operations as described herein.

[0249] We analyzed the effect of PV integration into the Al data center in Virginia demonstrated by the net incurred load demands and the reduction in net carbon emissions in the months of winter (January) and summer (July).

[0250] The learning-based control techniques are also updated to include PV generation levels as a state variable. It should be noted that for the disclosed VQC-RO strategy, the qubit count remains fixed despite an increase in dimensionality owing to the logarithmic scaling of the amplitude encoding technique used to encode data center observations. However, the uncertainty variable is updated to consider solar radiation levels instead of ambient temperature. To visualize the effectiveness of power consumption reduction by considering solar radiation uncertainties during the day, we plot various consumption levels for the Al data center situated in Virginia. For this Al data center, power consumed by computing infrastructure and cooling systems, along with the net power drawn from the electric grid, are determined for one-week periods of January and July. These periods are chosen due to the distinction in temperature and solar radiation levels in Virginia. From the determinations, it was found that the key contributor to the varying power consumption levels is the cooling load.

[0251] Apart from the reduction in power consumption, the integration of PV into the Al data center and consideration of solar radiation uncertainties with the disclosed VQC-RO control 10949-03-PC framework resulted in significant carbon dioxide emission reduction. Renewable integration combined with applying robust controls obtained with the VQC-RO strategy results in an annual carbon emission reduction of 6.9%, with only 3.49% in winter and as high as 9.85% in the summer season. Summers in Virginia exhibit a significant drop in net power consumption of the Al data center owing to the consistent PV generation. As the net electricity consumption can be supplanted with PV, small reductions in carbon emissions are observed in winter owing to the limited PV generation in the winter in contrast to the summer season. It should also be noted that the average carbon emission level attained in summer is 1819 tonnes of CO2 and is much higher than that of the emission levels in winter with 1227 tonnes of CO2, where “tonne” denotes a metric ton. This is mainly because energy requirements associated with cooling operations in Al data centers remain high in summer, accompanied by significant carbon emissions despite the high PV generation in the season. On the other hand, the RBC and the DDPG controllers yield a 4.6% and 3.5% increase over the VQC-RO controller, respectively, in terms of yearly net carbon emission levels for the Al data center situated in Virginia.

[0252] In addition to demonstrating the energy efficiency and adaptability of the disclosed VQC-RO control framework for energy management in Al data centers, we also quantify the utility of the VQC used within the framework.

[0253] FIG. 9 illustrates a comparison of computational efficiency between an example VQC within the disclosed VQC-RO control framework and corresponding classical counterparts, in terms of the number of trainable parameters and computational resource utilization for control computation.

[0254] For comparison with the classical counterparts, a neural network is used as the state-action value function approximator. To accommodate the robust optimization problem formulation in the subsequent steps of the disclosed control framework, a feedforward neural network with rectified linear units as the nonlinear activation is used. We conduct multiple computational experiments with various numbers of hidden layers in the feedforward neural network to analyze its effect on the control computation task. Similar experiments are conducted by repeating the 10949-03-PC selected quantum ansatz described herein. The number of parameters comprising each combination are presented in part (a) of FIG. 9. As evident from this figure, the number of parameters in the classical neural network scale exhibits a significant growth with layer repetitions compared to the quantum circuit. The number of training parameters in a classical feedforward neural network used as a value function approximator scales as O(r ■ h ■ (|S| + |ri|)) where h represents the number of hidden units with the state space and action space of the underlying MDP denoted with S and A, respectively. In contrast, the number of training parameters comprising the VQC increases linearly with the number of ansatz layer repetitions and can be denoted by O(r). Here, r represents the number of layer repetitions for both VQC and the feedforward neural network.

[0255] As demonstrated by the energy management performance of the VQC-RO control framework described previously, the VQC is able to efficiently approximate the state-action value function with fewer parameters. As the number of parameters influences the learning capabilities and sample complexity associated with parametric models, classical neural networks can be expected to utilize an increased amount of computational resources during training as compared to their quantum counterparts, owing to the high expressivity of the VQC. Furthermore, the choice of value function estimator directly affects the computational resource utilization required for computing controls with the classical optimizer. A feedforward neural network results in a mixed-integer linear program as the surrogate model, while the VQC as a function approximator, yields a linear program. The number of variables and constraints comprising the robust optimization problem formulation in the disclosed control framework grows quickly with the number of layer repetitions and hidden units in the feedforward neural network. On the other hand, the number of variables and constraints in the optimization problem formulated with the VQC remain fixed and are not dependent on the ansatz repetitions. This can be substantiated by the solution times observed with classical neural networks and the quantum circuits, as shown in part (b) of FIG. 9. Even though the ansatz repetitions are accompanied by an increase in gate parameters of the VQC, the time required to compute controls with the resulting robust optimization problem is consistent 10949-03-PC over all experiments. The use of a quantum circuit within the disclosed VQC-RO control framework not only exhibits scalability in terms of its learning capabilities but also demonstrates computational efficiency for control computation with the subsequent optimization problem solving by a classical optimizer.

[0256] As is apparent from the description above, the illustrative embodiments of FIG. 5 through 9 provide a control strategy leveraging quantum computers and robust optimization for energy management in data centers capable of handling Al workloads.

[0257] Such embodiments provide significant advantages relative to alternative approaches. For example, by integrating VQCs with robust optimization, the disclosed control framework addresses the challenges of producing robust controls by hedging against weather and renewable generation uncertainties while ensuring computational efficiencies offered by quantum algorithms. The computational experiments conducted at various locations in the United States have demonstrated the framework’s ability to significantly reduce power consumption and carbon emissions associated with data center operations. Additionally, the comparative study with other methods has confirmed the superiority of the VQC-RO framework. The disclosed approach for energy management in Al data centers in some embodiments derived practical utility from NISQ devices while producing competitive results with a power consumption reduction of over 12.5% and carbon emissions reduction of 9.8% compared to conventional control techniques and classical computing counterparts. The use of VQC as a value function approximator in the disclosed control framework for energy management in Al data centers benefits from its expressivity while facilitating a computationally efficient and scalable approach for control computation in contrast to its classical counterparts.

[0258] These and other embodiments are advantageously configured to hedge against uncertainties like outdoor weather conditions and solar energy generation. In some embodiments, encoding known future uncertainties into the input quantum state during training allows the extraction of a robust optimization formulation for control computation in contrast to a deterministic optimization problem. 10949-03-PC As another example, some embodiments encode the controls into the input quantum state and perform measurements onto the computational basis which may be easier in some cases than computing expectation of a custom Hermitian observable.

[0259] It is to be appreciated, however, that these and other embodiments can alternatively use the same tailored Hermitian strategy described elsewhere herein without losing the ability to formulate robust optimization problems.

[0260] Some embodiments can be implemented using a VQC with a reduced parameter set, providing advantages in terms of reducing computational costs associated with executing quantum circuits and preventing instances of barren plateau problems.

[0261] Additional illustrative embodiments will now be described in further detail with reference to FIGS. 10 through 14. These illustrative embodiments provide techniques for adaptive quantum computing-based MPC, which provide significant advantages over conventional MPC approaches referred to elsewhere herein. Although some embodiments are advantageously configured for decarbonization of building operations, the disclosed techniques can be adapted for numerous other use cases and control applications.

[0262] In some embodiments, an adaptive quantum computing-based MPC strategy is provided for energy management in buildings equipped with battery energy storage and renewable energy generation systems. A learning-based parameter transfer process to realize adaptive quantum optimization leverages Bayesian optimization to predict initial VQC parameters in the context of a quantum approximate optimization algorithm (QAOA). When applied to the MPC problems formulated as quadratic unconstrained binary optimization (QUBO) problems, this approach computes optimal controls to minimize the net energy consumption levels in buildings and promotes decarbonization while reducing the computational efforts required for the QAOA as the building energy system trajectory progresses. The energy efficiency and the decarbonization benefits of the adaptive quantum computing-based MPC strategy are demonstrated in experiments performed using buildings at the Cornell University campus, demonstrating energy-efficient and low-carbon building operation in illustrative embodiments, with a 6.8% improvement over 10949-03-PC deterministic MPC, while also supporting scaling to larger control problems with a significant reduction in utilized quantum computing resources. A reduction of 41.2% in carbon emissions is also achieved in some embodiments with illustrative embodiments of the disclosed control strategy facilitated by efficiently managing battery energy storage and renewable generation sources to promote a push toward carbon-neutral building operations.

[0263] Buildings are responsible for a significant portion of global energy use, primarily through heating, cooling, lighting, and the operation of appliances and systems. Residential and commercial buildings consume approximately 30% of global energy annually, while emissions from the building sector also contribute to 21% of global greenhouse gas emissions. Several directives have been implemented to mitigate these environmental impacts and promote a push towards carbon neutrality. Initiatives like the Paris Agreement impact building regulations by setting national carbon emission reduction targets that can be translated into adopting building energy codes and minimum performance standards to enhance energy efficiency. Promoting energy-efficient technologies and energy storage facilities accompanied by integrating renewable energy in the building sector to encourage on-site renewable energy generation is another key strategy to enable sustainable building operations. Building energy management systems present a significant opportunity to realize carbon-neutral building operations by optimizing performance and cost-efficiency, effectively reducing the environmental impact. Owing to advanced technologies like smart meters and smart sensors, energy management systems in buildings can manage renewable energy sources, energy storage devices, and building loads to facilitate energyefficient operations. Factors like varying sources of uncertainty, such as climate and weather conditions and unpredictable occupant behavior, can significantly influence energy consumption patterns in buildings. Additionally, diverse appliance loads from electrical equipment and HVAC systems, along with variable hot water demand, add complexity to the energy management systems. Developing high-performance and energy-efficient building operation strategies is crucial to addressing the complexities inherent to pushing building energy systems toward carbon neutrality. Illustrative embodiments disclosed herein meet this important need. 10949-03-PC There are numerous challenges associated with developing a quantum computing-based solution strategy to address the MPC problem in building energy management systems for improving energy efficiency and enhancing decarbonization efforts. Quantum computers can address specific classes of optimization problems like QUBO problems and quadratic programming problems with continuous variables. A first challenge lies in casting the MPC problem as an optimization problem that can be solved with quantum computing-based strategies. As the MPC problem structure remains consistent over a time horizon while the system uncertainties vary over time, it is important to exploit knowledge of problem structure to reduce computational efforts at subsequent steps. Developing an optimization strategy enabled by learning from prior experiences to improve the performance of the quantum optimization algorithm is another challenge. A further challenge lies in handling the system uncertainties in an adaptive manner to mitigate the gap in control performance guided by discrepancies between model prediction and actual realizations. In some embodiments, we provide an adaptive quantum computing-based MPC strategy to address these research challenges. A building energy system equipped with an HVAC system and provisions for electric load incurred by DHW and appliances is considered in some embodiments. Battery energy storage systems and PV generation as renewable resource integration are also modeled to promote flexibility. The optimization problem constructed for the MPC strategy is further reformulated into a QUBO problem for solving with the QAOA. We further disclose a learning-based parameter transfer process for the QAOA to enhance computational efficiency by providing a good starting point for each problem instance. This approach not only reduces the need for repeated quantum computational but also implicitly handles uncertainties and adapts to varying system states and external disturbances. To substantiate the energy-efficient behavior of the disclosed strategy and measure its impact on decarbonization, the adaptive quantum optimization-based control for building energy management is further evaluated with various computational experiments using real-world building data along with benchmarking computational performance against classical and quantum counterparts. 10949-03-PC Some embodiments provide an adaptive learning-based parameter transfer process for quantum optimization to enable energy-efficient and decarbonized building operations with MPC.

[0264] Additionally or alternatively, some embodiments provide an improved formulation for a building energy management problem that significantly reduces the number of binary variables, thereby enhancing computational efficiency and scalability, and ensuring optimal controls that minimize net energy consumption in buildings.

[0265] Computational experiments were performed with real-world buildings on the Cornell University campus, as will be described in more detail below, using relevant operational data to validate the energy efficiency of the adaptive quantum computing-based MPC strategy and to measure its impact on carbon emissions, along with analyzing the associated computational efforts compared to quantum and classical baselines.

[0266] Some embodiments described below consider smart buildings that utilize a comprehensive suite of modern smart technologies to meet their load demand. These embodiments implement an adaptive quantum computing-based MPC strategy designed for energy management in smart buildings to promote decarbonization.

[0267] FIG. 10 shows another example of a hybrid quantum-classical information processing system 1000 configured for solving optimal control problems in an illustrative embodiment. The system 1000 comprises a quantum computing subsystem 1002, a classical computing subsystem 1004 coupled to the quantum computing subsystem 1002, and a state-space model 1005 within an MPC formulation that dictates state transitions. The state-space model 1005 may be viewed as an example of what is more generally referred to herein as a “dynamic environment” coupled to both the quantum computing subsystem 1002 and the classical computing subsystem 1004, as illustrated in the figure. The quantum computing subsystem 1002 comprises at least one VQC that includes a plurality of stages including an initial quantum state preparation stage 1010, a parametric gate operations stage 1012 and a measurement stage 1014. An additional component 1015 of the quantum computing subsystem 1002 is utilized to update VQC parameters in conjunction with training of the VQC. 10949-03-PC The quantum computing subsystem 1002 illustratively comprises one or more quantum computers, and the classical computing subsystem 1004 illustratively comprises one or more classical computers, and each may comprise additional components. As indicated previously, a given “quantum computer” as that term is broadly used herein illustratively comprises at least one quantum processor and associated memory, and a given “classical computer” as that term is broadly used herein illustratively comprises at least one classical processor and associated memory. Additional components can be included in one or both of the quantum computing subsystem 1002 and the classical computing subsystem 1004, and those terms as used herein are therefore also intended to be broadly construed.

[0268] As illustrated in the figure, the system 1000 further comprises a number of databases, including a system operating parameters database 1020 and a sampled data database 1022, which may be implemented using one or more storage devices, including by way of example electronic memories and / or cloud-based storage.

[0269] The operation of the system 1000 is similar in some respects to that of the example system 100 of FIG. 1, but in this embodiment implements a QAOA-based workflow to solve an MPC problem reformulated as a QUBO problem.

[0270] An MPC problem may be viewed as a special case of the sequential decision making problem as previously described. MPC implementations typically operate with known transition dynamics dictated by the state-space model 1005, in contrast with an unknown dynamic environment. A given MPC approach illustratively solves an optimization problem over a fixed horizon, and therefore may be viewed as a subset of the reinforcement learning methods described elsewhere herein that in some embodiments attempt to approximately solve a sequential decision making problem over an infinite horizon. For example, some reinforcement learning embodiments disclosed herein are configured to maximize rewards, which may be viewed as analogous to minimizing costs in the case of MPC.

[0271] The system operating parameters database 1020 is illustratively configured to store information such as, for example, current observations, disturbances, optimal VQC parameters, 10949-03-PC and a target K that dictates proximity to optimal solutions. Portions of this information are illustratively captured as a data sample in sampled data database 1022 and utilized for further downstream learning tasks.

[0272] The quantum computing subsystem 1002 illustratively implements a QAOA-based process for an MPC optimization problem, as will be described in more detail below. The sampled data from sampled data database 1022 is used to perform a Bayesian optimization step in the initial quantum state preparation stage 1010 to predict the initial VQC parameters. It should be noted in this regard that QAOA typically utilizes a fixed VQC structure and embeds the MPC optimization problem reformulated as a QUBO problem. The remainder of the process for updating the VQC in quantum computing subsystem 1002 is generally similar to that previously described in conjunction with system 100 of FIG. 1. The addition of the Bayesian optimization step in the initial quantum state preparation stage 1010 to predict the initial VQC parameters introduces a data-driven component and can be viewed as a special case of using the sampled data to train the VQC directly as in system 100 of FIG. 1.

[0273] The classical computing subsystem 1004 illustratively implements a simplified form of the optimization process utilized in system 100 of FIG. 1. The example operations shown in FIG. 10 as being performed within the classical computing subsystem 1004 can therefore be generally viewed as an illustrative example of solving a corresponding instance of an optimization problem to generate one or more controls, as that phrase is intended to be broadly construed herein. In this embodiment, this illustratively involves postprocessing of information provided by the quantum computing subsystem 1002 by selecting a particular one of a plurality of bitstrings as a highest-probability bitstring, and then utilizing the selected bitstring to compute controls that are applied to the state-space model 1005 and / or its corresponding physical environment. Accordingly, in the case of this example QAOA-based method for MPC, the classical computing subsystem 1004 selects the solution with the highest probability and rescales the solution to obtain optimal controls. This rescaling procedure may be viewed as an exact inverse of the reformulation step used in casting the MPC optimization problem as a QUBO problem. 10949-03-PC It should be understood that the particular arrangement of system components and processing operations illustrated in FIG. 10 is presented by way of example only, and can be varied in other embodiments. For example, additional or alternative components and processing operations can be used to implement adaptive quantum computing-based MPC in other embodiments.

[0274] Additional details of illustrative embodiments implementing adaptive quantum computingbased MPC utilizing arrangements of the type shown in FIG. 10 will now be presented in conjunction with FIGS. 11 through 14.

[0275] FIG. 11 shows an example of a modeled building energy management system 1100 for a building 1102 coupled to a PV generation system and a power grid as shown. The model of the building 1102 includes an HVAC system for heating and cooling, DHW system load, appliance load and a battery energy storage system.

[0276] The building’s net load constitutes energy consumed by the HVAC system, DHW system, and appliances that pose a time-shiftable load demand. The building is also equipped with a battery energy storage system as shown, to promote grid stability and flexibility. Additionally, we consider a setup for integrating renewable energy using solar panels of the PV generation system to further reflect the real-world challenges of a building energy management system. An overview of the these and other components comprising the building energy management system is shown in FIG. 11. The smart building is considered to be electrified so that its overall load demand can be supplied by electricity from the power grid.

[0277] FIG. 12 illustrates an example system 1200 implementing the disclosed quantum computing-based MPC strategy for energy management in a building 1202 that includes an HVAC system for heating and cooling, DHW system load, appliance load and a battery energy storage system, with connections to a PV generation system and a power grid as shown. Aspects of the quantum computing-based MPC strategy are illustrated in the figure with reference to particular corresponding equations from the following description. 10949-03-PC As shown in FIG. 12, the energy management system in this illustrative embodiment can receive information about the current state from the building 1202 and its equipped energy devices, as well as provide relevant control signals that help to lower carbon emission levels through an energy-efficient behavior. The MPC problem constructed for the building energy system to optimize net energy consumption can be tackled with quantum computing-based optimization strategies followed by reformulation into QUBO problems. The QAOA is capable of solving combinatorial optimization problems and is suited for NISQ devices. Although QAOA has been widely explored for its potential applications in solving optimization problems on quantum computers, it exhibits scalability issues in a practical setting. It is also unclear whether QAOA provides any computational advantages over classical metaheuristics and deterministic solvers. To address these and other challenges, we develop an adaptive quantum-enhanced optimization strategy that in some embodiments exploits the related optimization problems posed by the MPC problem formulation to reduce the computational cost associated with the classical optimization of the QAOA parameters.

[0278] With regard to modeling building operations, we first consider the building’s thermal dynamics to manage the thermal demand for space heating and cooling accurately. The thermal dynamics are influenced by several key environmental and operational factors, which include the ambient air temperature, heating or cooling power provided by the HVAC system, and solar irradiation, denoted by Ta, Uhc, and < >s, respectively. The amount of heat added to the system is given by Uhcand can be used as a control variable for the HVAC system. Positive and negative values of this control variable indicate the addition and removal of heat from the system, respectively. A lumped capacitance modeling approach is leveraged here, which simplifies the complex interactions of heat within the building into manageable calculations such that the building can be treated as a few aggregated heat storage units or capacitances. For the building’s thermal dynamics, we consider two primary states comprising the temperature of the internal heataccumulating medium Tmapart from the indoor air temperature Tt. The internal heat-accumulating medium typically includes materials within the building that have substantial heat capacity, such 10949-03-PC as furniture, floors, and ceilings. By aggregating these components into a single medium, the model can effectively simulate how heat is stored and released over time, influencing the overall thermal environment inside the building. We then model the thermal dynamics of the building with the first-order differential equations in Equations (27) and (28) below. The parameters C£and Cmdenote the heat capacities of the indoor air and the medium, respectively. Rimdenotes the thermal resistance between indoor air and the heat accumulating medium, while Riais the thermal resistance between the indoor air and the outdoor ambient air. Abrepresents the windowed area of the building, while pbis the solar transmittance of the windows.

[0279] / 1 1 \ ( 1 \ ( 1 \ Ti = - - 1“rp ) • Tt+ ( — — — ) ■ Tm+ r / h• Uhc+ ( J ■ Ta+ Ab(l — pb) ■ (f)s(27)

[0280] Tm = (rn )Ti\C P )T™+ AbPb • 0s (28)

[0281] SOCt+l= SOCt+ ■ Uch— ■ Udch(29)

[0282]

[0283] We equip the building with a battery energy storage system and PV generation system. The battery storage system’s dynamics are modeled with Equation (29) above. The battery energy storage system is characterized by its capacity, nominal capacity, and the charging / discharging efficiency. The nominal capacity Cnom(SOCt) and efficiency of the battery (SOCt) as functions of the battery’s state of charge can be represented by piecewise linear functions. These curves allow for a more granular understanding of how battery performance metrics change with varying levels of charge. The charging and discharging energy levels represented by Ucfland Udc}lserve as controls for the energy storage system. Furthermore, the maximum power drawn for charging the battery and power discharged from the battery at any given time is constrained by the battery’s current nominal capacity, which is depicted by the range [0, CnomJ. This highlights the variable capacity of the battery to charge or provide energy, depending on its charge level. The energy generated by the PV panels can be used to directly supplement the building’s load demand or to 10949-03-PC charge the equipped battery energy storage. At any time t, the PV energy is generated as a function of solar irradiation (j)s tand can be modeled as shown in Equation (30) below, where Apvis the surface area of the PV panels that receive sunlight, r]pvis the efficiency coefficient, and the packing factor pf refers to the density of solar cells within a PV module. We treat the load incurred by DHW demand and the appliances as uncertainties, denoted by <pdhw and 0a, respectively. Simulating the system trajectories involves utilizing historically realized data for the DHW and appliance load. Since we provision the additional or removal of heat from the system to control the internal temperature, the amount of energy incurred by the actuator is considered to be proportional to | Uhc 11. The net energy consumption of the modeled building energy system can then be modeled as Enet, which is drawn from the power grid and characterized as shown in Equation (31) below.

[0284] Epv,t ^pv pvPf ’ 0s, t (30) E

[0285]

[0286] net,t | Uhc,t | "b ^ch,t ^dch,t 3” (jPdhw,t 3” 0a,t) (31)

[0287] We then extract a state-space formulation to represent the dynamics of building energy system followed by the formulation of an optimization problem for the MPC control strategy, as shown in Equations (32)-(39) below. A discrete-time state-space model represented by Equation (33) is obtained by discretizing the continuous differential equations in Equations (27) and (28), along with the battery dynamics in Equation (29). At any time t, the state variable vector xt= (T^t'Tmt. SOCf comprises the temperatures associated with the building environment and the battery storage state, while ut=

[0288]

[0289] Uch t, Udch t) form the control variables. wt=

[0290]

[0291] (Ta t, <ps t, < Pa,t)Tis defined as the disturbance vector and includes the weather uncertainties as well as the varying load demand associated with DHW and building appliances. Equation (34) ensures that the thermal comfort of the occupants is maintained at all times wherein Tmin,tan(3 Tmax,t constitute the acceptable temperature range at time t. Charging of the battery 10949-03-PC energy storage system beyond its capacity Cmaxis restricted in Equation (35). Control actuator constraints are enforced by Equation (36) and include provisions for heating, cooling, charging, and discharging levels as dictated by the building HVAC system and nominal capacity of the energy storage. Furthermore, additional constraints are considered to prevent simultaneous charging and discharging at any given time, as shown in Equation (37) and (38). The binary variable ytindicates whether the storage is being charged over the time horizon. The objective function in Equation (32) for the constructed optimization problem reflects minimizing the building’s net load demand. As the net energy drawn from the power grid can translate to carbon emissions, minimizing the building’s net load can help to promote decarbonization efforts which can also be accompanied by economic benefits. Directly incorporating time-varying carbon intensity into the objective function would introduce additional uncertainties and computational complexities during the QUBO formulation step, as described elsewhere herein. In some embodiments, the MPC control strategy is configured to solve a new optimization problem formulated over a prediction horizon H at each timestep k where the set Hk= k,k + 1,..., k + H} represents a set of discrete time intervals. At the timestep k, the optimal controls ufeare applied to the building energy system.

[0292] k+H min E-net,t (32)

[0293] s.t. xt+1= Axt+ But+ Cwt(33)

[0294] (34) SOCt< Cmnx, Vt E Hk(35)

[0295] < um(tx, Vt E Hk(36) Uch,t — Cnom ‘ yt’^t ^k (J2) Udch,t — Qiom (l - yt), Yt E Hk(38) xt> 0,ytE {0,1}, Vt E Hk(39)

[0296]

[0297] 10949-03-PC An example QUBO formulation will now be described in more detail. The optimization problem in Equations (32)-(39) above is a mixed-integer linear programming problem that requires solving at every timestep and scales linearly with respect to a number of variables and constraints over the prediction horizon. Due to hardware constraints and native support offered by specific quantum optimization algorithms, we reformulate this optimization problem into a QUBO problem. This reformulation also facilitates the utilization of qubit interconnections specific to quantum hardware, enabling faster solutions compared to classical solution approaches in some cases. We consider a deterministic MPC problem to minimize the building’s net energy consumption for reformulation into a tailored QUBO problem wherein future uncertain disturbances are treated as fixed value realizations over the optimization horizon. At time t = k, we have information about the system states xkand the realized disturbances wk, while the future uncertainties are treated as fixed in the certainty equivalence formulation and denoted by <( / c= {wt, Vt — k + 1,...,k + H}. It is important to note that uncertainties are fixed only to simplify the QUBO formulation and are implicitly handled by the adaptive quantum optimization approach described below.

[0298] During the QUBO reformulation step, various considerations are utilized to discretize the control variables ut. The heating or cooling power Uhcis subj ect to the building’ s HVAC actuator constraints and restricted to [— Hmax, Hmax], To discretize the heating or cooling power at timestep t, a binary variable zhcis introduced. The binary variable zflcindicates whether heating or cooling power is being supplied over the time horizon, while Zjtrepresents the j-th bit in the bitstring representation. Combining this with a binary discretization technique, the control variable can be reformulated with Equation (40) below, where 2nindicates the number of intervals into which the continuous domain [0, Hmax] is divided. Similarly, we consider charging or discharging of the battery energy storage system by a fixed amount Ebatat each timestep to limit the binary variables in the QUBO problem. A binary variable zb tis further introduced to indicate whether the battery energy storage system is charging or discharging at time / . Substituting this in Equation (29), the battery storage dynamics can be represented with Equation (41). 10949-03-PC

[0299] H v- 1,1uhc,t= ■ (2zhc- 1) V 27• zjit(40)

[0300] 2 2—i j=QUch,t Ebat ' %b,t

[0301]

[0302] Udch,t Ebat• (1 Zb,t) (41)

[0303] The mixed-integer optimization problem can be cast as a QUBO problem represented by Q(u|xk, wk, <"k) in Equation (42) below. The QUBO problem at each timestep k comprises three components, Qo, Qh, and Qb. The QUBO subproblem Qoassociated with the objective function in Equation (32) can be written with Equation (43) below and can be obtained by substituting Uhc t, Uch tand Udch twith Equations (40) and (41), respectively. As the term 2z)lc— 1 dictates the positive or negative nature of the variable Uhc t, value of | U}lc 11 in Equation (31) can be obtained by eliminating this multiplier as shown in Equation (43) below.

[0304] Q(u|xk,wk,<k) = Qo(u|xk,wk,<k) + Ah■ QZl(u|xk,wk,k) +b■ Qb(u|xk,wk) (42) k+Wr z xQ0(u|xk,wk,<k) = ■ Zj,tj + Ebat■ (2zbit- 1) t=k 'J~ ' k+H A~(jPdhw,k 4” < Pa,k 4” j (4>dhw,t 4” < Pa,t Epv,t) (43)

[0305]

[0306] t=k + l

[0307] Qbis the QUBO subproblem that enforces battery charging and discharging dynamics while preventing charging beyond acceptable limits. This is given by Qb= Sm=iQb(u |xk,wk) where Q,mis described in Equation (44) below and corresponds to the maximum battery SOC limit at the timestep k + m over the MPC horizon m = 1,

[0308]

[0309] Here, Qb,m represents the QUBO for the constraint SOCk+m< Cmax, where SOCk+mis written as a function of the initial battery’s state of charge SOCkand the control variables {Uch k+m, Udch k+m] substituted with their discrete reformulations in Equation (41) above. Similarly ensuring the 10949-03-PC building indoor air temperature lies within the temperature range [Tsp— 8, Tsp+ <5] is achieved with the QUBO subproblem Qh= m=i Qh,m(ulxk'wk)- This subproblem ensures the indoor climate conditions are maintained over the entire time horizon and is given in Equation (45) below. Here, [M]1;notation is used to describe the first row of the matrix M. The subproblem Qh,m represents the constraint Tmin>k+m< Ti k+m< TmaX)k+mwherein Ti>k+mcan be written in terms of the initial indoor temperature Ti kand the heat-accumulating medium temperature Tm kalong with the control inputs Uhc t. This is followed by substituting expressions for Ti k+mobtained after replacing state variables with the discretized control variable reformulation using xt+1= Axt+ But+ Cwt. The resulting inequality constraints for the state variables SOCt+mand Ti k+m are dependent on the binary variables zb tand Zjt, and the model parameters for the MPC optimization problem. Inequality constraints with binary variables can be modeled as QUBO problems.

[0310] k+m-1 k+m-1 ^?b,7n(^|Xfcz Wjj) ' (1 Fk,rn) ' %b,t + ' 2zb b' ^b,i (44) t=k t=k,i>t p, _ V ' ^max V ’ SOCk+ TH • / T / ,at (k'm~ Ebat• (1 + if) k+m— 1 \ H ’ v~ ui \ TsP- Gk,m~ ^ 2. ^(fc+7n_1-t)5]1:X -_027•(45)t=kJ~ / k+m-1 Gfc,m= [^m]1:(ri,fc,rm,fc)7’ + £ [>1^-1-^]^,

[0311]

[0312] t=k

[0313] Within the QUBO problem at timestep k, Ahand Abare fixed parameters and are commonly set as large values to ensure constraint satisfaction. These parameters are chosen such that they follow the criteria A E {IR+| A » Hmax, A » 2Ebat} and can be fixed throughout all experiments for a given system. Although it is straightforward to incorporate the decarbonization objective 10949-03-PC into the QUBO formulation step, the selection of these scaling parameters with varying binary coefficients in the QUBO associated with the objective can lead to additional computational steps that can affect the controller performance and result in suboptimal solutions for the MPC optimization problem. It should be noted that the constructed QUBO problem comprises O H • n) binary variables where n denotes the number of bits used to represent the continuous variable domain and H is the MPC horizon. Performance of the quantum optimization solvers is directly correlated with the number of variables within the QUBO problem.

[0314] Additional details regarding example QAOAs will now be described. The QAOA in some embodiments utilizes a VQC to parameterize the value of the QUBO problem. Within the quantum circuit model of computing, quantum algorithms are generally depicted as an ordered set of quantum circuits executed sequentially. These circuits encompass steps such as the preparation of the initial state, the application of unitary gates, and measurement operations to derive classical information that can be interpreted. These VQCs are commonly employed in many quantum machine learning algorithms because they maintain a high expressive power, even when implemented on NISQ devices. The variational circuit applies parametric gate operations to the initial quantum state, and the resultant quantum state is measured against an observable M to estimate the target objective function.

[0315] Q = hiZi + JtjZiZj (46) i i,j>i \

[0316]

[0317] iP) = U(aM... U(a2,p2) ■ (47)

[0318] The first step towards solving a QUBO problem with the QAOA is casting the QUBO problem as an Ising Hamiltonian, as shown in Equation (46) above. For the MPC optimization problem reformulated as a QUBO problem, the linear coefficients h and quadratic coefficients J can be determined after casting {0,1} binary variables as {-1,1} variables. 10949-03-PC FIG. 13 shows an example VQC 1300 implemented in a quantum computing subsystem 1002 of the hybrid quantum-classical information processing system 1000 of FIG. 10, and configured for use in the disclosed adaptive quantum computing-based MPC strategy. The figure more particularly illustrates an example workflow for the QAOA implemented by the VQC 1300 to iteratively minimize QUBO problems as disclosed herein.

[0319] As shown in FIG. 13, the QAOA commences with the preparation of an initial state which is the superposition of all basis states denoted by | / >0) or |+)w. It operates on an A-qubit system, where N corresponds to the number of binary variables in the QUBO problem. The problem Hamiltonian Hpis designed to have its ground state encode the solution to the QUBO problem. Additionally, a mixer Hamiltonian Hmis defined to represent an X-rotation on each qubit. These Hamiltonians facilitate the construction of VQC 1300 with parameters a and ft denoted by the parametric gate operation U a,ft) = U(Hm,a) ■ U(Hp, ). The phase separation operator is designed to apply a phase to each computational basis state that is proportional to its cost by effectively embedding the problem’s landscape into the quantum state’s phases and is represented by U(Hp,ft) = e~il3Hp. The mixing operator given by U Hm, a) = eiaHmenhances the exploration of the solution space. The quantum state |i ) emerges after applying the parameterized gate operations repeated L times as shown in Equation (47) above. An expectation value M = (

[0320]

[0321] ip \Hp|I / I) is computed through measurements along the Z-basis. The objective of QAOA is to fine-tune the parameters a = (c, a2, aL~) and ft = ( / ?x, ft2, ■■■ < / L) t° minimize M using a classical optimization algorithm such as gradient descent which is denoted by the return arrow in FIG. 13. The parameter selection strategy for QAOA is crucial for its performance, as it can affect the computational effort required by QAOA to optimize the VQC parameters. After the optimal gate parameters are obtained, the constructed VQC is reapplied with these optimal parameters and the probabilities for each possible combination of the binary variables are measured. The bitstring with the highest probability represents an approximate solution that minimizes the QUBO problem. 10949-03-PC Learning-based parameter transfer is implemented in some embodiments, as follows. QAOA for MPC problems within the building energy management system can be applied with random initialization of the gate parameters (a, () for each QUBO problem instance Q(xk, wfc,k) at timestep k. However, this can pose challenges in terms of computational resources required for solving individual problem instances over the system trajectory. To mitigate this, we employ a learning-based parameter transfer process for QAOA to efficiently optimize the gate parameters when solving a series of related optimization problem instances. An important aspect of a parameter transfer process for QAOA is that the optimal parameters for one QUBO problem instance may provide a good starting point for another problem instance that shares structural similarities. In addition to improving the computational performance of QAOA applied to the optimization problem at each timestep, it is important to ensure that the controller performance is maintained by adapting to varying building states and weather disturbances while addressing future uncertainties. We leverage a learning-based parameter transfer process to predict QAOA parameters as well as implicitly handle future uncertainties as the building energy system states evolve.

[0322] The adaptive quantum computing-based MPC strategy can be realized with the use of Gaussian process to serve as a surrogate for the control performance as a function of the system states, current weather conditions, and the optimal QAOA parameters. At each timestep fc, the system states xkand the disturbances wkform partial inputs to the Gaussian process denoted by Xk. The optimal QAOA parameters ak,p ) obtained by solving the corresponding QUBO problem Q(u|xk, wk, <^.) further constitutes the inputs Xk, and can be represented by Xk= (xk, wk, ak, pk~). The targets for the Gaussian process Ykare calculated with Equations (48) and (49) below. In Equation (48) the first term measures the value of knowing future uncertainties before making the decisions, while QVkrepresents the expected energy consumption incurred by solving a deterministic optimization problem with fixed uncertainty values computed as their expectationsk= E[k], The Gaussian process can then be represented as 10949-03-PC Yk~GP m Xkc(Xk, Xk' ) specified by its mean function ^(X ) and covariance function c(Xk, Xk'

[0323] Yk= (E<k[min Q(u|xk, wk,k)] - QVk)2(48) QVk= E<fc[2(u*|xk,wk;<k)] (49)

[0324]

[0325] u* = argminuQ(u|xk,wk,<k) i / CB(Xt) = / z(Xt) + K - (Xt) (50)

[0326] The controller can then be implemented using a Bayesian optimization setting to identify initial QAOA parameters for the corresponding QUBO problem at each timestep. This is realized primarily due to the expensive evaluation of the target Ykwhich requires multiple evaluations of min Q(u|xk,wk,k) over sampled uncertainty realizationsk. An initial set of observations are (Xk, Yk'^Q. T G D are collected to fit a Gaussian process prior. It should be noted that collecting this initial set requires solving QUBO problem instances at timesteps k = 1,..., T with initial QAOA parameters sampled randomly. Furthermore, computing min Q(u|xk, wk,k) over multiple realizations of <j). can be performed with a classical solver to limit the use of quantum computing resources. Following the learning stage, the QAOA parameters can be determined by minimizing the upper confidence bound given by Equation (49) above as the acquisition function. Optimizing the confidence bound UCB Xt') in Equation (50) can be conducted with gradient descent, however, it is important to note that the (xt, wt) G Xtremain fixed while the gate parameters are optimized. For a given observation vector (xt, wt) G Xt, the gate parameters obtained by minimizing UCB Xtserve as initial parameters for the QAOA. We control the exploration of the solutions (at, / ?£) by using a small K value for the confidence bound which ensures smaller values for the mean function. Using the predicted gate parameters, the last step within the QAOA comprising application of the constructed VQC to represent the QUBO problem Q(u|xk, w,k) is executed to obtain a binary bitstring solution. The binary variable values are then utilized to compute the optimal controls for building energy management. In a 10949-03-PC typical Bayesian optimization setting, the posterior is updated with newly sampled datapoints, which would require computing targets Yt. The computational resource utilization can be further limited by updating the posterior for the Gaussian process at fixed intervals while deploying the disclosed controller for building energy management to improve energy consumption and carbon emission levels.

[0327] We conducted several computational experiments to demonstrate the energy efficiency and decarbonization benefits of the disclosed adaptive quantum computing-based MPC strategy for building energy management. The experimental setup includes two buildings on the Cornell University campus with varying system parameters. For the uncertainties associated with each building’s load demand, historical data collected for Carpenter Hall and Baker laboratory situated at Cornell University’s Ithaca campus is utilized. A simulation for building energy management case studies utilizes heat capacity of indoor air Ci= 1.183kWh / °C and heat capacity of the heat accumulating medium as Cm= 4.005kWh / oC. Thermal resistances between the indoor air and the heat-accumulating medium Rim= 0.4789 C / kW and between indoor and outdoor air Ria= 29.25 C / kW are considered. The first building’s windowed area is 2.866m2with a solar transmittance of 0.101, while the other building’s windowed area is 3.213m2. The simulated energy management system also includes battery energy storage systems with maximum capacities of 0.9MWh and 1.1MWh, along with solar panels with an area of 1.4m2and 0.8m2for Carpenter Hall and Baker laboratory, respectively.

[0328] Weather data comprising ambient air temperature and solar radiation levels recorded in Ithaca also serve as the uncertainties present in the optimization problem posed by the MPC strategy for building energy management. Historical data recorded in the year 2021 is used to compute expectation values during the initial exploration phase of the learning-based approach to predict QAOA parameters. On the other hand, historical data from 2022 is utilized to conduct empirical evaluations of the disclosed adaptive quantum computing-based MPC strategy. The system parameters for the battery energy storage device and PV generation module for each 10949-03-PC building are chosen so that the energy devices can supplement a significant portion of the building’s load demand.

[0329] During the initial computational experiments, we consider an optimization problem over a horizon H = 5 with two bits to discretize the heating or cooling power control variable. The time interval for the discrete state-space model derived for the MPC strategy is set to one hour. Simulations over four months of January, April, July, and October are conducted to analyze the energy efficiency of the disclosed adaptive quantum computing-based MPC strategy, measure its impact on corresponding carbon emissions, and study the adaptability of the designed controller. Several key assumptions are made to simplify the building energy management system modeling and the experimental setup. The model assumes that PV-generated power is instantly available for either direct building consumption or battery storage, with no transmission losses or delays. The thermal modeling assumes uniform temperature distribution within spaces and constant thermal properties for resistances and capacitances. Additionally, the control system is assumed to have perfect communication between components with no delays or sensor errors.

[0330] Apart from the illustrative embodiments of the adaptive quantum computing-based MPC strategy disclosed herein, we conduct benchmarking of the controller performance against a baseline of deterministic MPC and quantum annealing. The deterministic MPC leverages a certainty equivalence formulation (CEMPC) wherein expectation values for the uncertainties are considered to solve the optimization problem over a fixed time horizon. As both the QAOA and quantum annealing address the QUBO problem, we do not compare their controller performance. Quantum annealing for addressing QUBO reformulations of the optimization problems within MPC requires resolving them at each timestep. Hence, we benchmark the solution times associated with quantum annealing against the adaptive computational efforts required with the learning-enhanced QAOA. Apart from the CEMPC and quantum annealing, we also compute the net energy consumption achieved with perfect knowledge about the future uncertainties to establish lower bounds on the incurred energy consumption with each control strategy. During 10949-03-PC the simulations, individual heating and cooling power required by the building HVAC control system to maintain indoor temperature are also recorded.

[0331] The learning-based QAOA technique is executed with noisy simulations for the IBM Brisbane quantum device equipped with the Eagle processor comprising 127 qubits. The D-Wave Advantage system is utilized to conduct benchmarking with quantum annealing. The experiments involving classical components are conducted with a Dell Optiplex system with Intel Core i7-6700 3.40 GHz CPU and 32 GB RAM. These components involve the CEMPC implementation as well as the parameter update step and Gaussian process-based Bayesian optimization associated with the disclosed quantum computing-based MPC strategy.

[0332] Load demand and net energy consumption incurred in two buildings were determined over a period of one week in January with the adaptive quantum computing-based MPC strategy for energy management.

[0333] For each building energy system, we conduct simulations with the various controllers, including the adaptive quantum computing-based MPC strategy over different time periods. Each building’s net load demand and the energy consumption facilitated by the quantum computingbased adaptive optimization for building energy management in the first week of January were determined. The resulting net load curves indicate that the load demand exhibits significant variability and peaks at different times. This aligns with the typical behavior expected in building energy management, where usage can fluctuate based on various factors like occupancy, appliance use, DHW load, and the HVAC system. The net consumption tends to follow the load demand patterns but with notable deviation indicating the impact of factors such as energy storage and PV generation. This alignment of net consumption with load demand but with mitigated peaks suggests the effective learning and adaptability of the disclosed adaptive quantum computingbased MPC strategy for building energy management. The presence of higher net consumption over load demands can be attributed to the initial phase of the quantum-enhanced MPC strategy in a Bayesian optimization setting. During these time periods, the controller solves certainty equivalent formulations of the MPC problem to collect initial samples for fitting the Gaussian 10949-03-PC process prior. However, the following time periods where net consumption dips below load demand indicate periods of energy-efficient behavior using stored energy and reduced load with optimal controls achieved with the adaptive strategy.

[0334] We also determined monthly net energy consumption and associated carbon emissions incurred with different control strategies for both buildings, along with the percentage constraint violation measured during the corresponding simulations. The above-noted CEMPC refers to the deterministic MPC strategy, while QC-MPC is the adaptive quantum computing-based MPC strategy.

[0335] The monthly net consumption incurred by the control strategies for each building was determined, along with lower bounds established by perfect knowledge of future uncertainties. The carbon emissions from the power grid associated with the building’ s energy consumption were also calculated. Apart from the energy consumption and carbon emission levels, we also measure the percentage of constraint violations incurred by both CEMPC and the adaptive quantum computing-based MPC strategy. Net consumption is generally highest under the CEMPC approach due to its deterministic approach using fixed expectation values for uncertainties. The adaptive quantum computing-based MPC strategy exhibits energy consumption closely matching the lower bounds set by perfect information. This is evident in both buildings and reflects the capability of the disclosed adaptive quantum computing-based MPC strategy to optimize with varying system states and uncertainties adaptively. A similar trend follows for the net carbon emission levels incurred with the corresponding control strategies. In the months of January and October, the adaptive quantum computing-based strategy incurs significantly lower carbon emissions and energy consumption than CEMPC, highlighting its superior energy efficiency and decarbonization capabilities. Furthermore, the disclosed adaptive quantum computing-based MPC strategy also displays a consistent pattern of lower constraint violations than CEMPC. This can be directly attributed to the uncertainty handling capabilities of the QAOA-based optimization. Even with the uncertainty handling capabilities, the adaptive quantum computing-based MPC strategy exhibits violations ranging from 4.12% to 6.67% across different months, indicating 10949-03-PC challenges in completely adhering to indoor temperature constraints. With lower violations than deterministic MPC, the observed patterns underscore the potential of the disclosed adaptive quantum computing-based MPC control strategy in managing system constraints even in the presence of various uncertainties. We also determined the reduction in carbon emissions achieved with the learning-based quantum optimization approach for the first building over a period of one week in January. The trend followed by the amount of reduction is consistent with the initial exploration steps in the Bayesian optimization setting, followed by the exploitation that results in reduced energy consumption and, consequently, lower carbon emissions.

[0336] In addition, reduction in carbon emission levels obtained with the quantum computingbased MPC strategy over a period of one week in January, as well as monthly emissions, were determined, along with a control trajectory for the building’s indoor temperature over a period of one week in winter obtained with the same control strategy.

[0337] The monthly carbon emission levels for different months also substantiate the ability of the disclosed adaptive quantum computing-based MPC strategy for building energy management to advance decarbonization efforts through sustainable operations in buildings. An annual 41.2% reduction in carbon emission levels is also measured, which further highlights the significant impact of decarbonization and a strong push towards carbon neutrality in buildings. The control trajectory for the indoor building temperature obtained with the adaptive quantum computingbased MPC strategy along with the temperature setpoints illustrates the proportion of constraint violations as the building energy system states evolve.

[0338] To further analyze the adaptability of the adaptive quantum computing-based MPC strategy, we determined the incurred heating and cooling loads for the first building over the winter and summer seasons. The outdoor ambient air temperatures are also determined, to serve as a reference. During colder weather conditions in January, the heating load generally increases as the outdoor temperature decreases. This correlation is evident from the inverse relationship between the two quantities when the temperature dips, often going below 0°C, and the energy used for heating surges. A highly sensitive heating response is realized with the adaptive control 10949-03-PC strategy with changes in outdoor temperature, reflecting an efficient and dynamic control system that adjusts heating output to compensate for heat loss as temperatures drop to maintain a comfortable indoor environment. The cooling load in summer also shows a direct correlation with increasing outdoor temperature. The peaks in energy usage correspond closely with the future high temperatures, illustrating the HVAC system’s reactive increase in cooling efforts to counteract the incoming heat. The results indicate that the building’s HVAC is actively managed by the adaptive QAOA-based MPC controller to respond to external temperature changes. This dynamic adjustment helps maintain indoor comfort regardless of extreme external temperatures and highlights the critical role of the disclosed control approach in realizing a responsive building energy management system.

[0339] Computational efficiency realized by adaptive quantum optimization to promote building decarbonization will now be described.

[0340] The disclosed adaptive quantum computing-based MPC strategy is an adaptive approach that predicts optimal QAOA parameters to solve corresponding QUBO problems at each timestep. Unlike CEMPC and quantum annealing, the computational efforts required by the learning-based QAOA reduce as the system trajectory evolves. To analyze this, we measure the number of iterations taken by the QAOA to reach its optimum after predicting with QAOA parameters by minimizing the upper confidence bound for the underlying Gaussian process. Similarly, we also measure the solution time required to solve QUBO problem instances with quantum annealing as applied to building energy systems. As quantum annealing and QAOA are supported by inherently different quantum hardware, a direct comparison cannot be made with respect to their computational performance.

[0341] FIG. 14 illustrates performance of illustrative embodiments in terms of computational efforts required by the learning-enhanced QAOA compared against quantum annealing, along with the constraint violation levels measured for QUBO problems of varying sizes and varying layer repetitions settings within the QAOA. 10949-03-PC As shown in part (a) of the figure, both the quantum annealing solution time and QAOA iterations are depicted over a period of one week. It can be seen that quantum annealing time exhibits consistent fluctuations throughout the period. This reflects the consistent effort required to solve the problem instances anew each time without the benefit of learning or adapting from previous solutions. Apart from tackling the MPC optimization problem by reformulation into QUBO problems and treatment with quantum optimization methods, the MPC optimization problem can also be addressed with quantum algorithms like HHL. Although HHL offers an exponential speedup over classical algorithms in solving a linear system of equations, these quantum advantages cannot be realized with NISQ devices as their implementation requires fault-tolerant quantum devices with error-corrected qubits. In contrast to the disclosed adaptive quantum computing-based MPC strategy, HHL-based algorithms commonly used to solve small-scale optimization problems cannot be directly leveraged to solve the MPC optimization problem relevant to building energy systems. On the other hand, the disclosed adaptive quantum computing-based MPC strategy in some embodiments utilizes a significant number of iterations during the initial exploration phase. However, this number drastically reduces after this phase, as shown in part (a) of the figure. This suggests that the adaptive mechanism in the learning-based QAOA approach quickly finds efficient parameters that allow for fewer iterations as time progresses. This rapid optimization and stability imply a high efficiency in adapting to the problem’s dynamics without needing extensive computation beyond the initial phase.

[0342] It should be noted that the variables of the QUBO problem representing the optimization problem at each timestep scales as O H • ri) where n is the number of bits used to represent the continuous variable domains, and H is the prediction horizon for MPC. A solution to the QUBO problem solved with QAOA strongly depends on the number of variational layer repetitions. We conducted an ablation analysis between the varying number of variables representing the same optimization problem and the number of layer repetitions for QAOA using constraint violation percentage as a performance metric, as illustrated in part (b) of FIG. 14 along with constraint violation levels under varying conditions. As shown in this part of the figure, the constraint 10949-03-PC violation levels tend to increase as the number of variables increases, even though this trend is not uniform. It can be clearly seen that for a fixed problem size, the number of layer repetitions induces variability in the constraint violations. The notable inconsistencies across the metrics observed with varying numbers of layers indicate that other factors, like the QUBO problem parameters, can significantly influence performance despite the increased number of layer repetitions within the QAOA.

[0343] While the building energy system model presented in the above-described experiments does not consider sophisticated MPC formulations, it serves as a representative example of common MPC problems in building energy management. The disclosed adaptive quantum computing-based optimization method is designed to be extensible to more complex system configurations. In sophisticated MPC formulations considering HVAC with weather-dependent system performance, the increased complexity would primarily affect the matrices A, B, and C. This would influence the parameters in the constructed QUBO problem. However, the fundamental method for QUBO construction would remain the same. The use of quantum computing in this context is justified by its potential for improved scalability, enhanced handling of uncertainty, and the development of future-proof algorithms that can leverage advancements in quantum hardware. Furthermore, the hybrid quantum-classical approach disclosed in illustrative embodiments herein demonstrates how quantum computing can complement existing classical methods, leading to improved solutions for increasingly complex building energy systems. It is important to note that computing the targets for Gaussian process fitting requires considering various scenarios of future uncertainty realizations. This approach allows our model to capture a range of possible outcomes to enhance its robustness in different future scenarios. By incorporating multiple realizations, we can better account for the stochastic nature of factors like weather patterns leading to more reliable predictions and control decisions. While our learningbased QAOA approach implicitly handles these uncertainties, limitations in predictive capabilities may arise in the case of longer horizons. Even though the disclosed adaptive quantum computingbased MPC approach allows the system to adapt to changing conditions without relying solely on 10949-03-PC potentially inaccurate long-term predictions, incorporating uncertainty quantification methods could provide valuable insights into the reliability of the system’s decisions under various scenarios.

[0344] Illustrative embodiments described above demonstrate the effectiveness of QAOA for building energy management leveraging a learning-based parameter transfer process. QAOA offers several advantages like its ability to run on near-term quantum devices and its flexibility in handling various problem structures, but in some cases may also face challenges like difficulty of finding optimal parameters for deep circuits. Quantum annealing implemented on specialized hardware like D-Wave systems can handle larger problem sizes but may struggle with certain problem structures that QAOA can address more effectively. The integration of quantum annealing with a learning-based approach can also be a hindrance to developing controllers for energy management that are capable of exploiting historical data. For building energy management specifically, QAOA’s ability to handle time-dependent constraints and its adaptability through the disclosed learning-based parameter transfer process makes it particularly suitable. However, for larger buildings with more complex energy systems, the increased number of variables might challenge QAOA’s current capabilities. The tradeoff between these algorithms can evolve as quantum hardware continues to improve. Development of quantum ansatz that are tailored to the underlying energy system can further enhance the performance of the adaptive QAOA approach while mitigating its limitations.

[0345] Some embodiments provide an adaptive quantum computing-based MPC strategy for building energy management to improve energy efficiency and promote decarbonization in buildings. The disclosed strategy in some embodiments utilizes a learning-based parameter transfer process for QAOA that enables the optimizer to learn from previous computations, significantly reducing the need for repeated recalibrations and intensive computations required to solve optimization problems within the MPC strategy applied to minimize net energy consumption in buildings. The parameter transfer process for quantum optimization in some embodiments leverages a Gaussian process in a Bayesian optimization setting to further enhance the efficiency 10949-03-PC and scalability of QAOA for MPC in building energy management systems. Various computational experiments illustrated a 6.8% improvement in energy efficiency over deterministic MPC approaches and also demonstrated computational efficiency over quantum annealing-based approaches as the building energy system evolved. Furthermore, the disclosed strategy yielded an annual carbon emissions reduction of 41.2% by optimally managing building energy systems equipped with HVAC, battery storage, and PV generation. The quantum computing-based MPC strategy demonstrated significant improvements in energy efficiency and decarbonization while adapting to changing environmental conditions and system states while leveraging renewable energy sources effectively.

[0346] Illustrative embodiments integrate quantum algorithms with learning-based strategies to develop robust and adaptable control techniques for realizing the full potential of quantum computing for practical applications like building energy management. By leveraging learningbased quantum algorithms, such embodiments can significantly speed up the solution of large-scale optimization problems inherent in MPC, where real-time control computation is crucial. As demonstrated by the decrease in computational efforts required by quantum optimization techniques as disclosed herein, the quantum computing-based MPC strategy can significantly improve the control computation speed in response to dynamic environmental and system changes in various domains in power systems, industrial process control, and energy systems at various scales.

[0347] Other embodiments can integrate real-time carbon intensity metrics and dynamic pricing signals into the adaptive quantum optimization framework to enhance the environmental and economic impact of the control strategy.

[0348] The disclosed techniques can be extended to diverse building environments including commercial complexes, industrial facilities, and residential communities.

[0349] In addition, the disclosed techniques can be extended to more complex control scenarios like demand response mechanisms in smart grid applications. 10949-03-PC It should also be noted that architecture-specific optimization of the underlying quantum algorithms through advanced error mitigation techniques can be used in some embodiments to accelerate the practical adoption of this approach in real-world building energy management systems.

[0350] As mentioned previously, it is to be appreciated that these embodiments and others disclosed herein are presented by way of illustrative example only, and should not be viewed as limiting in any way.

[0351] Moreover, the various features and advantages of illustrative embodiments need not be present in other embodiments.

[0352] As noted above, various types of automated actions can be taken based at least in part on outputs generated by a hybrid quantum -cl as si cal system of the type disclosed herein, such as particular actions involving interaction between a processing platform implementing at least a portion of the hybrid quantum -cl as si cal system and other related equipment utilized in one or more of the applications described herein. For example, outputs generated by a hybrid quantum-classical system can control one or more components of a related system in a dynamic environment. In some embodiments, the hybrid quantum-classical system and the related equipment are implemented on the same processing platform, and in other embodiments are implemented on separate processing platforms.

[0353] The hybrid quantum-classical information processing systems, processing platforms, processing devices, networks, quantum computing subsystems, classical computing subsystems, and other systems and components disclosed herein in conjunction with FIGS. 1 through 14 and other embodiments are illustratively configured in some embodiments to implement the example algorithms as described above.

[0354] It should also be understood that the particular arrangements shown and described in conjunction with FIGS. 1 through 14 are presented by way of illustrative example only, and numerous alternative embodiments are possible. The various embodiments disclosed herein should therefore not be construed as limiting in any way. Numerous alternative arrangements of 10949-03-PC one or more VQCs and hybrid quantum-classical computing can be utilized in other embodiments. Those skilled in the art will also recognize that a wide variety of alternative processing operations and associated system configurations can be used in other embodiments.

[0355] It is therefore possible that other embodiments may include additional or alternative system elements, relative to the entities of the illustrative embodiments. Accordingly, the particular system configurations and associated algorithm implementations can be varied in other embodiments.

[0356] A given processing device or other component of an information processing system as described herein is illustratively configured utilizing a corresponding processing device comprising a processor coupled to a memory. The processor executes software program code stored in the memory in order to control the performance of processing operations and other functionality. The processing device also comprises a network interface that supports communication over one or more networks.

[0357] The processor may comprise, for example, a microprocessor, an ASIC, an SOC, an FPGA, a CPU, a GPU, an NPU, a DPU, a TPU, an ALU, a DSP, and / or other similar processing device components, as well as other types and arrangements of processing circuitry, in any combination. For example, at least a portion of the functionality of at least one machine learning system, and its machine learning algorithms for use in hybrid quantum-classical computing as disclosed herein, can be implemented using such circuitry. Moreover, as indicated previously, in addition to various types of classical processors, a wide variety of different types of gate-based quantum processors and / or other circuit-based quantum processors can be used in a given embodiment, for example, within a quantum subsystem to implement at least one VQC as disclosed herein.

[0358] The memory stores software program code for execution by the processor in implementing portions of the functionality of the processing device. A given such memory that stores such program code for execution by a corresponding processor is an example of what is more generally referred to herein as a processor-readable storage medium having program code embodied therein, and may comprise, for example, electronic memory such as SRAM, DRAM or other types of 10949-03-PC random access memory, ROM, flash memory, magnetic memory, optical memory, or other types of storage devices in any combination.

[0359] As mentioned previously, articles of manufacture comprising such processor-readable storage media are considered embodiments of the present disclosure. The term “article of manufacture” as used herein should be understood to exclude transitory, propagating signals. Other types of computer program products comprising processor-readable storage media can be implemented in other embodiments.

[0360] In addition, embodiments of the present disclosure may be implemented in the form of integrated circuits comprising processing circuitry configured to implement processing operations associated with implementation of a machine learning system within a hybrid quantum-classical system.

[0361] An information processing system as disclosed herein may be implemented using one or more processing platforms, or portions thereof.

[0362] For example, one illustrative embodiment of a processing platform that may be used to implement at least a portion of an information processing system comprises cloud infrastructure including virtual machines implemented using a hypervisor that runs on physical infrastructure. Such virtual machines may comprise respective processing devices that communicate with one another over one or more networks.

[0363] The cloud infrastructure in such an embodiment may further comprise one or more sets of applications running on respective ones of the virtual machines under the control of the hypervisor. It is also possible to use multiple hypervisors each providing a set of virtual machines using at least one underlying physical machine. Different sets of virtual machines provided by one or more hypervisors may be utilized in configuring multiple instances of various components of the information processing system.

[0364] Another illustrative embodiment of a processing platform that may be used to implement at least a portion of an information processing system as disclosed herein comprises a plurality of processing devices which communicate with one another over at least one network. Each 10949-03-PC processing device of the processing platform is assumed to comprise a processor coupled to a memory. A given such network can illustratively include, for example, a global computer network such as the Internet, a WAN, a LAN, a satellite network, a telephone or cable network, a cellular network such as a 4G, 5G or 6G network, a wireless network implemented using a wireless protocol such as Bluetooth, WiFi or WiMAX, or various portions or combinations of these and other types of communication networks.

[0365] Again, these particular processing platforms are presented by way of example only, and an information processing system may include additional or alternative processing platforms, as well as numerous distinct processing platforms in any combination, with each such platform comprising one or more computers, storage devices or other processing devices.

[0366] A given processing platform implementing at least a portion of a hybrid quantum-classical system as disclosed herein can run on or be otherwise supported by cloud infrastructure or other types of virtualization infrastructure.

[0367] It should therefore be understood that in other embodiments different arrangements of additional or alternative elements may be used. At least a subset of these elements may be collectively implemented on a common processing platform, or each such element may be implemented on a separate processing platform.

[0368] Also, numerous other arrangements of computers, storage devices or other components are possible in an information processing system. Such components can communicate with other elements of the information processing system over any type of network or other communication media.

[0369] As indicated previously, components of the system as disclosed herein can be implemented at least in part in the form of one or more software programs stored in memory and executed by a processor of a processing device. For example, certain functionality disclosed herein can be implemented at least in part in the form of software.

[0370] The particular configurations of information processing systems described herein are exemplary only, and a given such system in other embodiments may include other elements in 10949-03-PC addition to or in place of those specifically shown, including one or more elements of a type commonly found in a conventional implementation of such a system.

[0371] For example, in some embodiments, an information processing system may be configured to utilize the disclosed techniques to provide additional or alternative functionality in other contexts.

[0372] It should again be emphasized that the embodiments of the present disclosure as described herein are intended to be illustrative only. Other embodiments of the present disclosure can be implemented utilizing a wide variety of different types and arrangements of information processing systems, quantum computing subsystems, classical computing subsystems, and associated quantum and classical processing devices than those utilized in the particular illustrative embodiments described herein, and in numerous alternative processing contexts. Also, the particular types of VQCs, parametric gate operations and types and arrangements of quantum encoding used in illustrative embodiments can be varied in other embodiments. In addition, the particular assumptions made herein in the context of describing certain embodiments need not apply in other embodiments. These and numerous other alternative embodiments will be readily apparent to those skilled in the art.

Claims

10949-03-PC ClaimsWhat is claimed is:

1. A system comprising:a quantum computing subsystem comprising at least one variational quantum circuit; anda classical computing subsystem coupled to the quantum computing subsystem; the quantum computing subsystem being configured to utilize the at least one variational quantum circuit to generate, for each of a plurality of iterations, a different set of parameters for an optimization problem to be solved in the classical computing subsystem;the classical computing subsystem being configured to receive from the quantum computing subsystem a given one of the sets of parameters generated for a given one of the iterations, to formulate a corresponding instance of the optimization problem based at least in part on the given set of parameters, and to solve the corresponding instance of the optimization problem to generate one or more controls;wherein one or more controls generated by the classical computing subsystem are applied to a dynamic environment for utilization therein to perform one or more sequential decision-making operations; andwherein the quantum computing subsystem receives state information from the dynamic environment for use in determining, via the at least one variational quantum circuit, another one of the sets of parameters for a subsequent one of the iterations.

2. The system of claim 1 wherein the optimization problem comprises an optimal control problem formulated at least in part as a sequential decision-making problem.

3. The system of claim 2 wherein the optimal control problem comprises at least one of a demand response problem and an energy control problem.10949-03-PC 4. The system of claim 1 wherein the dynamic environment comprises at least one energy system of at least one building, facility, equipment, factory, greenhouse, manufacturing plant, or energy consuming unit, and the optimization problem is configured to optimize operation of the at least one energy system.

5. The system of claim 1 wherein the at least one variational quantum circuit is trained at least in part utilizing at least one of a reinforcement learning algorithm, a mathematical optimization algorithm, a deep learning algorithm, a generative learning algorithm, or a metaheuristic algorithm.

6. The system of claim 5 wherein the at least one variational quantum circuit is configured as a parametric function approximator for estimating a value function for a state-action pair wherein the state comprises a state of the dynamic environment and the action comprises a particular instance of the one or more controls.

7. The system of claim 1 wherein the at least one variational quantum circuit comprises a plurality of stages including at least an initial quantum state preparation stage, a parametric gate operations stage and a measurement stage.

8. The system of claim 7 wherein the initial quantum state preparation stage is configured to encode an input state vector into a quantum state with multiple qubits.

9. The system of claim 7 wherein the parametric gate operations stage comprises: an entanglement block configured to entangle quantum states of respective qubits;a parametric gate operations block configured to perform sequential rotation operations in respective parametric gates utilizing respective ones of a plurality of rotation angles; anda pooling operations block configured to perform pooling operations between respective pairs of consecutive qubits.10949-03-PC10. The system of claim 7 wherein the measurement stage is configured to generate an expectation value of an observable.

11. The system of claim 10 wherein the expectation value of the observable is scaled by a trainable parameter to provide an estimate of a value function for a state-action pair.

12. The system of claim 11 wherein the corresponding instance of the optimization problem that is formulated based at least in part on the given set of parameters and solved by the classical computing subsystem to generate one or more controls comprises maximizing the value function for a given input state over a specified control space subject to one or more actuator constraints of the dynamic environment.

13. The system of claim 1 wherein the given set of parameters comprises at least one of (i) a plurality of rotation parameters of the at least one variational quantum circuit and (ii) a plurality of pooling parameters of the at least one variational quantum circuit.

14. The system of claim 5 wherein the reinforcement learning algorithm utilized in training the at least one variational quantum circuit implements an epsilon-greedy strategy to determine one or more controls to be applied to the dynamic environment for a given state of the dynamic environment.

1. The system of claim 1 wherein the classical computing subsystem is further configured to generate a sequence of decisions or an associated control profile, after a designated number of iterations or a designated computation time limit, the sequence of decisions or the associated control profile being configured to instruct one or more physical devices and / or one or more physical components to operate, manipulate and / or adjust one or more energy systems.10949-03-PC16. The system of claim 15 wherein the one or more physical devices and / or the one or more physical components comprise one or more of an actuator, a heater, a cooler, a heat exchanger, a dehumidifier, a humidifier, an energy supplier, an energy consuming unit, a valve, a pump, or any combination thereof.

17. The system of claim 1 wherein the system is further configured to minimize energy consumption of one or more energy systems subject to one or more dynamic constraints within a desired control time limit, wherein the desired control time limit is less than an amount of time used by the classical computing subsystem to solve at least one instance of the optimization problem.

18. The system of claim 1 wherein the dynamic environment comprises at least one of a data center energy system and a digital twin of the data center energy system.

19. The system of claim 1 wherein the at least one variational quantum circuit of the quantum computing subsystem receives state information of the dynamic environment for a state of a state-action pair, the state information comprising one or more observations of the dynamic environment and one or more corresponding disturbances representing future uncertainty information of the dynamic environment.

20. The system of claim 19 wherein the future uncertainty information comprises information relating to at least one of outdoor weather conditions and solar energy generation associated with the dynamic environment.

21. The system of claim 19 wherein the at least one variational quantum circuit utilizes a first type of quantum encoding to encode the one or more observations of the dynamic10949-03-PC environment, and a second type of quantum encoding, different than the first type of quantum encoding, to encode the one or more corresponding disturbances representing future uncertainty information of the dynamic environment.

22. The system of claim 21 wherein the first type of quantum encoding utilizes a first set of one or more parametric gate operations, and the second type of quantum encoding utilizes a second set of one or more parametric gate operations different than the first set of one or more parametric gate operations.

23. The system of claim 1 wherein the at least one variational quantum circuit of the quantum computing subsystem receives state information of the dynamic environment for a state of a state-action pair, the state information comprising one or more observations of the dynamic environment, and wherein the at least one variational quantum circuit further receives action information for the action of the state-action pair, the action information comprising one or more controls.

24. The system of claim 23 wherein the at least one variational quantum circuit utilizes a first type of quantum encoding to encode the one or more observations of the dynamic environment, and a second type of quantum encoding, different than the first type of quantum encoding, to encode the one or more controls of the action information.

25. The system of claim 24 wherein the first type of quantum encoding utilizes a first set of one or more parametric gate operations, and the second type of quantum encoding utilizes a second set of one or more parametric gate operations different than the first set of one or more parametric gate operations.10949-03-PC 26. The system of claim 1 wherein the at least one variational quantum circuit is configured to implement a quantum approximate optimization algorithm (QAOA).

27. The system of claim 1 wherein the optimization problem comprises a model predictive control (MPC) problem based at least in part on a state-space model of the dynamic environment.

28. The system of claim 27 wherein the MPC problem is reformulated at least in part as a quadratic unconstrained binary optimization (QUBO) problem for processing in the quantum computing subsystem.

29. The system of claim 1 wherein an initial set of parameters is determined for the at least one variational quantum circuit utilizing a Bayesian optimization process.

30. The system of claim 1 wherein the classical computing subsystem is configured to receive from the quantum computing subsystem a plurality of bitstrings, and to solve a corresponding instance of the optimization problem at least in part by selecting a particular one of the bitstrings.

31. The system of claim 30 wherein the classical computing subsystem is configured to select a highest probability bitstring from the plurality of bitstrings and to generate the one or more controls from the selected bitstring.

32. The system of claim 30 wherein the plurality of bitstrings are generated at least in part by a quantum approximate optimization algorithm (QAOA) implemented in the quantum computing subsystem.10949-03-PC 33. The system of claim 30 wherein the one or more controls are generated in the classical computing subsystem at least in part by rescaling of the selected bitstring.

34. The system of claim 33 wherein the rescaling of the selected bitstring in the classical computing subsystem comprises inverting a reformulation process previously utilized to reformulate a model predictive control (MPC) problem as a quadratic unconstrained binary optimization (QUBO) problem for processing in the quantum computing subsystem.

35. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code, when executed by at least one processing platform, causes the at least one processing platform:to configure a quantum computing subsystem comprising at least one variational quantum circuit to utilize the at least one variational quantum circuit to generate, for each of a plurality of iterations, a different set of parameters for an optimization problem to be solved in a classical computing subsystem coupled to the quantum computing subsystem; andto configure the classical computing subsystem (i) to receive from the quantum computing subsystem a given one of the sets of parameters generated for a given one of the iterations, (ii) to formulate a corresponding instance of the optimization problem based at least in part on the given set of parameters, and (iii) to solve the corresponding instance of the optimization problem to generate one or more controls;wherein the one or more controls generated by the classical computing subsystem are applied to a dynamic environment for utilization therein to perform one or more sequential decision-making operations; andwherein the quantum computing subsystem receives state information from the dynamic environment for use in determining, via the at least one variational quantum circuit, another one of the sets of parameters for a subsequent one of the iterations.10949-03-PC36. The computer program product of claim 35 wherein the at least one variational quantum circuit is trained at least in part utilizing a reinforcement learning algorithm and is configured as a parametric function approximator for estimating a value function for a state-action pair wherein the state comprises a state of the dynamic environment and the action comprises a particular instance of the one or more controls.

37. The computer program product of claim 35 wherein the at least one variational quantum circuit is configured to implement a quantum approximate optimization algorithm (QAOA), and further wherein the optimization problem comprises a model predictive control (MPC) problem that is reformulated at least in part as a quadratic unconstrained binary optimization (QUBO) problem for processing in the quantum computing subsystem utilizing the QAOA.

38. A method comprising:in at least one variational quantum circuit of a quantum computing subsystem, generating, for each of a plurality of iterations, a different set of parameters for an optimization problem to be solved in a classical computing subsystem coupled to the quantum computing subsystem; andin the classical computing subsystem, receiving from the quantum computing subsystem a given one of the sets of parameters generated for a given one of the iterations, formulating a corresponding instance of the optimization problem based at least in part on the given set of parameters, and solving the corresponding instance of the optimization problem to generate one or more controls;wherein the one or more controls generated by the classical computing subsystem are applied to a dynamic environment for utilization therein to perform one or more sequential decision-making operations; and10949-03-PC wherein the quantum computing subsystem receives state information from the dynamic environment for use in determining, via the at least one variational quantum circuit, another one of the sets of parameters for a subsequent one of the iterations.

39. The method of claim 38 wherein the at least one variational quantum circuit is trained at least in part utilizing a reinforcement learning algorithm and is configured as a parametric function approximator for estimating a value function for a state-action pair wherein the state comprises a state of the dynamic environment and the action comprises a particular instance of the one or more controls.

40. The method of claim 38 wherein the at least one variational quantum circuit is configured to implement a quantum approximate optimization algorithm (QAOA), and further wherein the optimization problem comprises a model predictive control (MFC) problem that is reformulated at least in part as a quadratic unconstrained binary optimization (QUBO) problem for processing in the quantum computing subsystem utilizing the QAOA.

Images