System for setting demand response of energy grid assets

Abstract

A method for controlling assets connected to an electric grid is disclosed. A trained neural network model is provided for each of a plurality of configured assets, the trained neural network model taking as input an asset configuration, the asset configuration defining the asset's response to variations in one or more operating conditions detected at the asset, and the neural network model uses the asset configuration to output one or more performance indicators related to the asset's operation during operation. The output of the neural network model is provided as input to an optimization function. A search process optimizes the optimization function by varying the asset configuration for the asset. The final asset configuration is then used to control the asset and provide a coordinated demand response service.

Description

Technical Field

[0001] The present invention relates to a system and method for controlling assets that supply energy to a power distribution network or consume energy from a power distribution network to regulate the energy flow into and out of an energy grid, for example, in response to grid frequency variations.

Background Art

[0002] Electricity suppliers and grid operators implement various energy management techniques in industrial sites or residences and in power distribution networks and transmission systems. There is an increasing problem for grid operators to manage each aspect of their respective energy grids, such as balancing power supply and demand and responding to frequency deviations of the power distribution network.

[0003] Generally, a grid operator can command (or provide a financial incentive for) the actions of energy producers or energy consumers to ensure a reliably responsive power distribution network. For example, a grid operator can purchase regulation capacity from industrial consumers and / or producers of electricity. Consumers or producers providing such services receive commands to reduce or increase their power consumption when required by the grid operator to maintain grid stability and quality. There may be specific requirements such as the reduction or increase in power consumption needing to be stable for a relatively long period or either such reduction or increase occurring rapidly. Importantly, the grid operator desires to manage assets (such as electrical loads or generators) that consume or supply electrical energy at the portfolio level rather than at the individual asset level.

[0004] High response times can be particularly important for electric grid operators. Grid operators are expected to maintain a stable grid-provided power frequency (usually 60 Hz in the United States and 50 Hz in Europe), but grid operators may face challenges in keeping the grid frequency within an acceptable margin. For example, when a power plant shuts down unexpectedly, a large amount of power suddenly becomes unavailable (demand exceeds supply), and the grid frequency decreases. Similarly, when a large industrial load starts operating and the supply slows down to meet that demand, the grid frequency decreases. When the grid frequency decreases, the frequency can be brought back to its reference level by reducing the grid's power consumption or by increasing the supply (or by a combination of both). However, it can be challenging to order a reduction in power consumption from various groups of industrial or residential consumers. Perhaps more importantly, it can also be very difficult for grid operators to achieve a reduction in power consumption as quickly as possible, typically aiming to achieve it in a matter of seconds (or even faster) rather than a matter of minutes. A centralized management system may not be able to detect the deviation, schedule the power reduction, and reliably deliver the schedule to industrial loads in that short time. Similarly, reverse power flow can also occur. When the supply is greater than the demand (e.g., when the production of renewable energy falls short of expectations), the frequency may exceed its reference level (50 Hz or 60 Hz). This can be offset either by reducing power production or by increasing power consumption (or a combination of both).

[0005] In the prior art, it includes using a simple binary switch for a load that turns off the entire load when a switch detects that the power frequency has dropped to a certain level (for example, when the frequency drops to 49.9 Hz, the load turns off). However, this is a static technique where the switch is a separate hardware device where the load is always fixed off at a specific frequency. Such a device can strictly turn off the load in such a manner for months or years without considering other information. For this technology, in the sense that it is impossible for a local operation manager to reject a power-on request based on operation constraints or business constraints, this technology may operate unidirectionally. Further, this technology is performed at the load level and does not result in benefits from any portfolio optimization.

[0006] Recently, technologies have been developed that use a combination of energy consumption assets and / or energy production assets to implement combined demand response and utilize portfolio optimization to improve the response.

[0007] International Publication No. WO 2015 / 059544 (the disclosure of which is incorporated herein by reference) describes an energy management system that enables a grid operator to manage a portfolio of energy loads at the aggregated portfolio level while responding rapidly and reliably to changes in grid characteristics. Based on instructions from a grid operator to decrease (or increase) power according to a frequency deviation within a frequency band, the central side uses a hybrid approach to determine an optimal frequency portfolio trigger at which each load within the configuration needs to decrease (or increase) power. Symbolically, to optimize the global loop response of the portfolio, the loads are "stacked" within this frequency band such that, in the presence of a grid frequency deviation, reliable optimal power is delivered to the portfolio. These triggers (and corresponding individual load power reductions) are sent periodically and immediately from the central side (responsive to changes in load and grid behavior) to each individual load. When a frequency deviation occurs, each load can independently (i.e., without interaction with the real world outside the industrial site) quickly reduce its power consumption and the corresponding power reduction received in advance according to the trigger. Each trigger sends a signal to the load to decrease or increase power according to the locally measured state of the grid. This approach reduces the reliance on the central side, detects frequency deviations, and then immediately communicates power reductions in real time.

[0008] However, when demand response is implemented at the aggregated portfolio level potentially using a large pool of assets, it can be difficult to determine the appropriate configuration of individual assets that fully takes into account the impact of their configuration on the grid and the assets themselves, as well as the potentially complex interactions between individual asset responses, and at the same time, it can also be difficult to ensure that the overall requirements of the total demand response service are met.

Prior Art Documents

Patent Documents

[0009]

Patent Document 1

SUMMARY OF THE INVENTION

PROBLEM TO BE SOLVED BY THE INVENTION

[0010] Embodiments of the present invention aim to address or at least mitigate this problem and to provide an improved technique for configuring a demand response service across an asset pool.

MEANS FOR SOLVING THE PROBLEM

[0011] Accordingly, an aspect of the present invention is a computer-implemented method for controlling assets connected to a power distribution network, the assets including assets configured to supply electrical energy to the grid and / or consume electrical energy from the grid, at least a portion of the assets being configurable to adjust the energy flow into or out of the grid in response to a change in the operating state of the grid. The method comprises accessing, for each of a plurality of configured assets, preferably in the form of a trained neural network model, a mapping, the neural network model or other mapping receiving an asset configuration as input, the asset configuration defining the response of the asset to one or more detected variations in the operating state of the asset, using the asset configuration to output one or more performance metrics relating to the operation of the asset during operation, and providing the input to an optimization function, the input based on the performance metrics being output by the neural network model or other mapping, the optimization function mapping the input to an optimization measure Performing a search process configured to change the optimization metric output by an optimization function by varying the configuration of one or more of a plurality of assets, the search process continuing until an end criterion is met, Transmitting the asset configuration determined during the search process to one or more asset control devices associated with the asset, wherein, according to the asset configuration, the control device is configured to control the energy flow between the asset and the grid,

[0012] This approach enables overall aggregate demand response for a pool of assets effectively configured to meet specific requirements of an aggregate demand response service (e.g., encoded by an optimization function), while at the same time enabling real-time control to occur directly at the asset controllers rather than at a centralized control server.

[0013] The term "asset" or "energy asset" preferably refers to a device, machine, or other equipment (or a collection of such entities) configured to provide energy from or consume energy to the grid. Energy flow includes both the flow from the asset to the grid (energy supply to the grid) and the flow from the grid to the asset (energy consumption from the grid). Typically, (electrical) energy flow can be measured, processed, and / or controlled with respect to power (e.g., expressed in appropriate units such as watts). Thus, energy flow regulation can include power regulation (e.g., increasing / decreasing power consumption or power supply). An asset controller can be part of the asset it controls or a separate device connected to the asset.

[0014] The search process preferably includes varying one or more asset configurations to change a performance metric output by a neural network (or other mapping), and thereby changing the value of an optimization metric, and preferably the variation steps are repeated until an end criterion is met. The output of the search process is preferably a set of asset configurations found during the final iteration when the optimization metric is satisfied, and / or a set of asset configurations corresponding to the optimal value (e.g., highest / lowest value) of the optimization metric found in the search. The search process preferably includes optimizing an optimization function with respect to the optimization metric (e.g., increasing or decreasing the optimization metric), using for example a gradient descent algorithm or other suitable optimization algorithm.

[0015] Preferably, the asset configurations of a plurality of assets define a set of dimensions of the search space, and the search process includes performing a gradient descent search of the search space to optimize the output value of the optimization function.

[0016] The method may include identifying one or more constraints for optimization and adding one or more penalty terms to the optimization function of the constraints. Weights may be associated with the penalty terms, and the method may include evaluating the solution found by the search process regarding the constraints after the search process has ended, increasing the weights of one or more violated constraints in response to identifying that one or more constraints are violated by the solution, and repeating the optimization using the increased weights.

[0017] The optimization function may be a cost function, and the search process includes minimizing the cost function by changing the asset configuration. The cost may refer to a financial cost or some other cost measure (e.g., performance regarding resource usage, expected (e.g., contracted) demand response service level, etc.). In certain embodiments, the cost (more specifically, the cost function value and / or individual cost terms) is CO2 (carbon dioxide) and / or NO xEmissions of, for example, nitric oxide or nitrogen dioxide and / or emissions of other substances and materials may refer to environmental impacts expressed, for example, as one or more environmental impact metrics.

[0018] The termination criteria may include one or more of a predetermined threshold or an optimization metric that achieves a local or global optimum value (e.g., a minimum or maximum value), a maximum number of iterations being performed, and a maximum computation time reached. For example, the search process may search for the best "solution" (e.g., an asset configuration that produces the highest / lowest value of an optimization function) within the scope of applicable processing time / resource constraints.

[0019] The search process preferably begins with an initial set of asset configurations, where the initial configuration includes one or more of the current asset configuration of the asset, the default asset configuration of the asset, and a randomly generated asset configuration.

[0020] In certain embodiments, the mapping is in the form of a neural network, but this is not essential and other types of mappings (e.g., lookup tables, decision trees, mathematically defined mapping functions, other machine learning models, etc.) may be used. Preferably, the mapping is a learned mapping that uses a machine learning model learned based on the history of the asset or simulated performance. Accordingly, the features related to the simulations described below may be applied to the derivation of other types of mappings. In the following, the mapping is generally described in the form of a neural network, but it should be understood that these may be substituted for other forms of mapping.

[0021] This method preferably includes training a neural network model (or other mapping) by a process for one or more of a plurality of assets, the process including generating a plurality of asset configurations, and for each asset configuration, simulating the operation of the asset according to the asset configuration to determine one or more performance metrics of the asset configuration based on the simulation, and training a neural network model (or other mapping) using a plurality of training samples, each training sample being based on an asset configuration and the corresponding performance metric being determined for the asset configuration by simulation. The simulation can be performed based on a model of the asset. Generating an asset configuration can include randomly selecting an asset configuration.

[0022] This method can include repeating the training of one or both of the neural network model (or other mapping) and the search process periodically or in response to changes in a plurality of assets, the changes optionally including the addition, removal, or change of the operating characteristics of one or more assets. On the one hand, the simulation / training of the neural network model (or other mapping) and on the other hand the search / optimization process can be performed at different times and / or repeated at different frequencies.

[0023] The asset configuration preferably includes configuration data that defines how the energy flow between the asset and the grid should change in response to changes in one or more measured operating states (s) of the asset. Preferably, the configuration data defines one or more response curves, each response curve defining the required power flow level or power flow change as a function of a given operating state parameter.

[0024] Optionally, the configuration data can define a plurality of response curves each defined for different operating state parameters and / or a plurality of response curves each defined for different value ranges of the same operating state parameter (e.g., for different frequency bands of a grid frequency parameter). This can enable more complex demand response requirements to be configured.

[0025] One or more operating states preferably include one or more parameters regarding a (preferably, local) grid frequency measured at the asset. The frequency parameter(s) can include at least one (optionally, both) of the local grid frequency measured at the asset and data derived from the local grid frequency (e.g., temporally filtered grid frequency values).

[0026] The configuration data preferably defines, for each of a plurality of individual grid frequency values, a required power input value or power output value or adjustment value of the asset.

[0027] Reference herein to a local grid frequency (or other condition) associated with an asset (or other system element) can refer to a grid frequency (or other condition) measured by or at the asset or at a grid location near (geographically or phase-wise) the asset. For example, the local grid frequency can be the frequency of the grid at or below the grid connection point of the asset (the location where the asset is connected to the grid). For example, the local grid frequency can be any grid frequency measured within 1 km of the asset, preferably within 100 m or even within 10 m of the asset (with respect to either straight-line geographical distance or connection length). The asset (or, more specifically, the asset controller that controls the asset) can include or be connected to respective frequency sensors to detect the local grid frequency at or near the asset (usually at (or below) the grid connection point of the site where the asset is located).

[0028] The grid frequency can be measured, for example, as a single measurement value at a particular time or as a representative value obtained from frequency data over a time window, such as an average over the time window (e.g., the average value of the last N seconds with respect to the measurement time). Thus, the term "grid frequency" and other references to frequency measurement values include any characteristic value extracted from the frequency data representing the grid frequency at a particular location / time.

[0029] The grid is typically configured to operate at a given grid frequency, which can be, for example, the standard expected operating AC frequency of the power transmitted on the grid, and this frequency is also referred to herein as the nominal frequency or reference frequency of the grid. However, it is understood that the actual grid frequency can vary instantaneously and across different locations on the grid from its given nominal frequency. Further, the given frequency itself can vary by the grid operator based on operating requirements (from nominal reference values (e.g., in Europe (50 Hz), in the United States (60 Hz))). The frequency values of the asset configuration can be expressed in absolute terms or in relation to the given grid frequency or reference grid frequency.

[0030] Alternatively or in addition, one or more operating states can include operating states other than frequency, including, for example, the operating state of an asset connected to the grid or another asset.

[0031] The search process can include varying one or more power flow values or power flow regulation values, and / or one or more frequency thresholds or other operating state thresholds of the power flow regulation values.

[0032] Preferably, the method includes, at a given control device, receiving one or more signals indicative of an operating state related to the operation of the grid or an asset of the grid, determining a power flow level of an asset controlled by the control device based on the one or more signals and the asset configuration of the asset, and controlling the asset according to the determined power flow level.

[0033] Determining the power flow level optionally includes calculating the power flow level based on a response curve defined by the asset configuration by interpolating the values of the response curve of the operating state parameters from a set of data points of a curve defined by the asset configuration. One or more signals may include local grid frequency measurements and / or signals derived from local grid frequency measurements (e.g., temporally filtered grid frequency measurements).

[0034] The method may include, in a control device, receiving a plurality of signals indicative of respective operating state parameters, and based on the signals, determining a plurality of power flow adjustment values, each power flow adjustment value being derived using a respective response curve defined by the asset configuration, the response curve mapping each operating state parameter to a power flow adjustment value, determining, based on the plurality of power flow adjustment values, a total power flow adjustment value, and controlling the asset according to the determined total power flow adjustment value. The total power flow adjustment value may be based on, for example, the sum of the plurality of power flow adjustment values.

[0035] Preferably, the performance metric output by a neural network (or other mapping) includes one or more measurements of the performance of the asset with respect to the required demand response service defined by the asset configuration. The performance metric may define one or more of the availability of the asset providing demand response, the amount of energy supplied or consumed over a period of time when providing demand response, the response time of the asset to achieve a desired energy flow adjustment value, the number of operating cycles or charge / discharge cycles of the asset over a period of time, a measure of success for delivering the configured demand response, and a measure of the cost of providing the demand response service (e.g., environmental impact or financial cost).

[0036] In a further aspect of the present invention, there is provided a computer system or apparatus optionally having means including one or more processors with associated memory for performing any of the methods described herein.

[0037] In a further aspect, the present invention provides a (tangible) computer-readable medium including software code adapted to perform any of the methods described herein when executed on one or more data processing devices.

[0038] Any of the features of one aspect of the present invention may be applied to other aspects of the present invention in any suitable combination. In particular, aspects of the method may be applied to aspects of the apparatus and computer program, and vice versa.

[0039] Furthermore, functions implemented in hardware may generally be implemented in software, and vice versa. Any reference herein to software and hardware functions should be construed according to the circumstances.

[0040] Reference will now be made to the accompanying drawings, which illustrate by way of example only, the preferred features of the present invention.

Brief Description of the Drawings

[0041]

Figure 1A

Figure 1B

Figure 1C

Figure 2A

Figure 2B

Figure 3

Figure 4A

Figure 4B

Figure 5

Figure 6A

Figure 6B

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0042] The adjustment of the power flow (supply or consumption) between the assets and the transmission grid or the distribution network is called "demand response". When performing demand response, for example, in order to affect the grid frequency and weaken it in response to fluctuations in the transmission frequency of the grid, this is also called frequency control response in this specification.

[0043] Note that the measure of operating the reserve to weaken the frequency fluctuation is also known as frequency containment reserve (FCR). In contrast, the term "frequency control response" used here refers to a control operation that utilizes that reserve by modifying the power supply / consumption of the assets.

[0044] Fluctuations in the AC (alternating current) transmission frequency of a power distribution / transmission system can be mitigated by adjusting power consumption from the grid or power supply to the grid. Specifically, in this specification, an increase in the grid frequency above a standard expected value, also referred to herein as a reference frequency value or a nominal frequency value (e.g., 50 Hz in Europe), can be mitigated by increasing consumption or reducing supply, while a decrease in the grid frequency can be mitigated by reducing consumption or increasing supply.

[0045] In some cases, a provider having suitable energy consumption assets or energy supply assets connected to the grid can contract with a grid operator (e.g., a transmission system operator or TSO) that provides demand response by adjusting the supply or consumption of one or more assets to mitigate frequency fluctuations. Energy consumption assets can be, for example, industrial loads such as factories or machinery, electric vehicle charging points, domestic power supply points or domestic loads, and partially or fully discharged energy storage devices that require recharging. Energy supply assets can be, for example, generators (e.g., gasoline generators, wind turbines, solar panels, etc.) or energy storage devices such as batteries. Multiple individual consumption devices and / or supply devices can operate together as a single asset (e.g., turbines in a wind farm, various machines and systems in a factory). In some cases, an asset may be capable of both supplying energy to the grid and consuming energy from the grid (e.g., a factory with a generator that consumes excess demand from the grid and supplies excess generation capacity to the grid at different times, or a battery that can store power obtained from the grid and then return and supply the stored power to the grid).

[0046] Frequency control generally requires the provider to provide power output that linearly responds to frequency deviations one after another. This can be done at the asset level or the pool level using aggregation techniques. The frequency control response service needs to be provided within the constraints of the assets participating in the service.

[0047] When providing services at the pool level, the system can utilize the fact that a combination of assets can provide a good response to the sum of all individual responses. Examples include the following. - Assets are each configured to respond to deviations in different parts of the frequency spectrum. For example, when the frequency drops in a first frequency band, the first asset can respond by adjusting its output, while the second asset can respond in a second frequency band. Assets can operate cumulatively. For example, when the frequency deviates further from the nominal value, additional assets activate their frequency control response. - Fast assets compensate for slow assets that cannot meet the associated ramp constraints. For example, a battery may be able to respond quickly (but its output capacity may be limited), while a generator takes a long time to reach the required output level after starting but can then provide a higher output capacity. In such situations, the fast asset can be configured to respond initially, while the second slow asset increases its output. The response speed of the decelerating asset can be characterized by the response delay and / or the deceleration ramp rate. - Continuous assets compensate for the separate behavior of separate assets (e.g., binary assets). A binary asset is an asset that is either on or off. That is, a binary asset supplies (or consumes) at a fixed level when active (on) and does not supply (or consume) when not active (off). Other separate assets may provide multiple individual power levels (in addition to the off state). A continuous asset is an asset that can vary its supply output or consumption gradually (e.g., linearly) in response to a control signal and / or in response to a required frequency adjustment or a locally measured frequency deviation. Thus, in one example, a continuous asset can be controlled to compensate for any deficiency in the response (e.g., power output) delivered by a separate asset. - When one or the other is partially or fully unavailable, those assets compensate for each other. For example, some assets may only be available for demand response during a specific time window (e.g., a solar generator is available only during the day and depending on the weather. An industrial generator is available only when not in use to support industrial facilities).

[0048] In a specific example of the total frequency control response, both a high-speed energy device such as a battery (usually with constrained energy) and a slow-speed energy load such as an industrial oven (usually with unconstrained energy) are used to respond to a frequency deviation. Usually, such a slow industrial load cannot adequately provide a frequency control service that complies with the requirements specified by the TSO, and it requires a partner (such as a battery) that can compensate for its low ramp rate. During the exchange, the slow load compensates for the throughput (number of cycles) and the limiting amount of the battery's energy content.

[0049] A typical goal when aggregating assets is generally to provide a response that is as linear as possible. Thus, an efficient aggregation can be viewed as a combination of optimization problems.

[0050] The way in which a set of assets is realized depends on the approach used to control and regulate the assets of the grid. Several control approaches can be considered, and some of the control approaches are shown in FIGS. 1A - 1C.

[0051] FIG. 1A shows a real - time central control approach using an external set - point control. In this approach, the power set - point is calculated at a remote location away from the assets (e.g., outside the geographical boundary). In one example, a battery can provide a frequency control response where the set - point is calculated at a remote server or a different asset. In the example shown in FIG. 1A, the controller 102 performs centralized control by sending control signals from a central controller (e.g., a server) to all or a portion of the assets connected to the power distribution network 100. In this example, two assets “A” 104 and “B” 106 within the grid 100 are shown receiving control data from the central controller 102. The control data determines the power input / output level required from the assets (e.g., the “set - point” measured in MW) and can be defined as an absolute or relative term as an increase or decrease in the current power level. As a result, the set - point of the asset (e.g., the battery) is not defined locally to the asset (where “locally” can mean, for example, below the grid connection point of the asset).

[0052] The controller performs real - time control (e.g., updates the set - point at a specific update frequency and / or in response to measured changes in the grid frequency). The control data can be based on a single centrally measured grid frequency (shown as “f” in the figure), measured locally to the controller or at some other central location, under the assumption that the frequency is substantially the same across the grid. Alternatively, the asset can send its local frequency measurement upstream towards the controller that calculates the set - point, and the controller calculates the set - point and returns the set - point to the asset. Although a central controller is shown, in reality, the controller can be located anywhere. For example, a particular asset can host a computing node for performing control functions and provide control data to other assets.

[0053] The controller can control only all assets or a subset in response to a request. The total set of available assets considered is generally what is available for performing demand response and / or frequency control response (referred to as an asset pool). These can include, for example, assets provided by a provider with an agreement to implement demand control by assets provided by the grid operator and / or the grid operator itself. The grid usually includes more assets that are either not available for demand control or are available.

[0054] As described above, the central controller can be applied to the asset set as needed. For example, based on the measured grid frequency, fast assets and slow assets are configured in real time to provide a complementary frequency control response.

[0055] Figure 1B shows a hybrid control approach as further described in International Publication No. WO 2015 / 059544 of the aforementioned patent publication.

[0056] In this approach, the central controller 102 receives contractual conditions from a grid operator that defines a specific portfolio response function that defines an amount of adaptable power provided as a function of signals that can be locally measured in the grid (e.g., when the frequency drops to the lowest point of its band, the amount of power by which the entire portfolio of frequency band, response time, and load should decrease). Based on those conditions, the central unit calculates a control configuration in the form of a set of local control parameters for each asset of the portfolio used to configure the processing unit of each asset. These control parameters describe the local response function of the asset that defines the amount of power provided (or, similarly, consumed) by each of the individual assets in response to a locally measured change in the grid frequency. The configured response can indicate, for example, the response provided by the asset and / or the frequency range for which the level of the response needs to be provided.

[0057] The configuration (control parameter) is sent to each asset, and then each asset can manage its power in real time based on the frequency deviation detected locally by the asset. Thus, the grid operator does not control the load in real time as in the above example of centralized control. A typical example of such a local response function is one that describes the linear relationship between the grid frequency deviation and the provided adaptable power (however, non-linear response functions can also be simple binary sets that provide a simple on / off response based on, for example, a frequency threshold).

[0058] In the example of FIG. 1B, asset A and asset B are pre-configured by their respective response functions 108, 110. Next, each asset implements a locally pre-set response based on the locally measured grid frequencies (f1, f2) and their respective response functions. For example, each asset detects the current grid frequency and its own power consumption. When the frequency of the asset reaches the trigger point of the asset (or when the frequency of the asset is within the range of its configured frequency sub-band), the frequency control response becomes active and the asset immediately and independently uses its response function to change its power consumption / supply.

[0059] The configured response function can be updated from time to time by the central controller as needed.

[0060] As described above, the central controller can adjust the responses by different assets and apply them to the set of assets as needed. For example, the assets can be configured based on their respective abilities to respond within different ranges of frequency bands (individually or cumulatively), such that the combined response results in a desired (almost as expected) linear response to the frequency deviation. Further, complementary assets (e.g., fast assets and slow assets) can be configured to use complementary response functions so that the required collective response is achieved even if the individual assets do not necessarily recognize the contribution of each other's frequency control responses.

[0061] Figure 1C shows a distributed self-organizing control approach. In this approach, asset communications communicate directly with each other over the network and exchange information about their respective capabilities (e.g., output / consumption capabilities, ramp rates, etc., regardless of whether the asset is a binary asset or a continuous asset) in the form of an operating model that describes the response characteristics of the asset. The exchange model enables an asset to inform how another asset will respond to a frequency change (e.g., whether the asset will respond slowly).

[0062] Next, based on the exchanged information, the assets autonomously determine and decide on an appropriate set group (e.g., pair high-speed assets and low-speed assets with complementary demand response capabilities within the set group). Next, each asset can perform real-time control based on its measured local grid frequency, taking into account the expected responses of the other set assets (if any) of the group to the frequency deviation based on the exchange model.

[0063] By eliminating the need for a central entity to directly control the loads and sources and the need to exchange real-time data between the loads / sources through model exchange between assets, resilience to communication failures or information loss is provided.

[0064] A set group of assets (loads / energy sources) is formed by peer-to-peer interactions and based on specific demand response services provided. The assets exchange models with each other, and after executing the models, determine partner candidates that can be offered for a specific demand response service, and the receiving asset decides whether to form a set group with the partner candidates. The set group can also be formed based on historical data learned over time by other loads and sources. Next, the assets of the set group may rely on the models of the partner assets of the group to calculate their own control policies.

[0065] The process includes a negotiation phase in which assets communicate among multiple assets to form one or more aggregation groups. In this step, an asset generally seeks another asset or other assets with which it can cooperate to provide an improved frequency control response. Advantageously, communication between devices is peer-to-peer and does not need to go through a central entity, providing resilience to communication failures.

[0066] Accordingly, an asset's model can be used in the decision-making and control policies of other assets. During operation, each asset follows its own local control policy that takes into account models or multiple models received from other devices, whereby other devices agree to form an aggregation group and the combination of responses to grid frequency variations is adjusted.

[0067] Communication and Control In the control configuration described above, assets can communicate with each other via any suitable communication means, including wired networks and / or wireless networks, using any suitable communication technology and media, including, for example, narrowband IoT (Internet of Things), ADSL, fiber networks, 4G, or other cellular systems, or Ethernet. The network can include private networks as well as dedicated connections and public networks such as the Internet. Communication with energy assets and communication between energy assets can be done under the control of and / or under the control of a central entity such as a central controller, or directly using peer-to-peer communication between assets.

[0068] In the various approaches described above, the asset is described as providing a processing function (e.g., implementing a frequency control response), (e.g., implementing a frequency control response in response to real-time centralized control as in FIG. 1A, monitoring a local frequency, and implementing a frequency control response using a pre-set response function stored in the asset as in FIG. 1B, or implementing peer-to-peer negotiation and model-based control as in FIG. 1C). Such a processing function can be implemented using a device agent that is software running on local computing hardware in close proximity to the asset being controlled. In one example, this local controller can include a built-in controller integrated into the asset. In another example, it can provide a locally connected controller. Also, such a controller can control multiple assets independently, for example, by a separate device agent that is launched by the controller. For example, a set of batteries can be controlled by a single network-connected control computer and maintain separate control parameters (e.g., response functions) and / or respective device agents. In the scenario of FIG. 1C, the device agent implements peer-to-peer negotiation, frequency data broadcast, etc. The device agent can be executed on a general-purpose computer or dedicated computing hardware, for example, using the "FlexTract" controller available from Centrica Business Solutions Belgium N.V. (Antwerp, Belgium). Thus, when an asset is referred to herein, this can generally be interpreted to include the energy supply device(s) and / or energy consumption device(s) themselves, along with any integrated or associated control hardware.

[0069] An external grid sensor is connected to the power distribution network, detects variables such as grid frequency, voltage, or power quality, and transmits this information to a device agent on local computing hardware. The local controller is also connected to a communication network to communicate with other assets and / or a central controller (if used).

[0070] Automated learning techniques for adjusting asset configuration Embodiments of the present invention further result in an improvement of the above control system that enables more effective adjustment of asset responses across an asset pool. The described embodiments can be viewed as an expansion of the control scheme of FIG. 1B, but the principles described can be adapted to other control schemes (e.g., by performing the calculations described in the distributed scheme of FIG. 1C rather than a centralized scheme).

[0071] The approach utilizes machine learning and optimization techniques to identify a control configuration for an asset pool that satisfies demand response criteria encoded by a cost function.

[0072] Hereinafter, an asset demand response control configuration (referred to as an "asset configuration") is assumed to define a response function that describes an asset response to some operating characteristics or parameters of the grid or assets connected to the grid (collectively referred to as operating states or parameters). Typically (but not exclusively), the operating characteristic considered is the grid frequency. Specifically, the control configuration typically defines a change in the power input / output of the asset (collectively, "power flow") in response to a change in the local grid frequency at the asset. Alternatively, the control configuration can specify a response to another asset state, another measured parameter of the power transmission system (e.g., by converting any of the aforementioned signals), another signal such as data derived from the grid frequency or another operating parameter, or any suitable combination of signals.

[0073] The process is shown in the overview of Figure 2A. Broadly speaking, the process involves obtaining sensor data of a pool of assets 202 indicating the local grid frequency in the asset over a historical period (optionally, other asset characteristics or grid characteristics such as energy consumed / supplied by the asset, current, voltage, temperature, buffer level, pressure measurements, price information, storage level indicators, etc.). The data is analyzed and the optimization process is performed in the central processing system 200 based on the overall demand response service required provided by the asset pool. Through optimization, a set of asset configurations to be transmitted to the asset controller 204 local to (or integrated with) the asset is identified. Next, the asset configuration is used by a controller that controls the asset to perform the demand response service by adjusting the power setpoint of the asset. Local execution of the calculation of the setpoint (near the controlled asset) can reduce the risk that demand response delivery is affected by communication problems.

[0074] Subsequently, the analysis and optimization are periodically repeated based on newly collected sensor data, enabling iterative improvement of the asset configuration and response to changes in the asset and the grid.

[0075] The analysis and optimization process is shown in Figure 2B.

[0076] In step 206, individual assets are simulated based on the asset model. For each asset, historical data of the grid frequency (or simulation data) is used to perform simulations of multiple asset configurations to determine the performance characteristics of the asset when responding to frequency changes using a specific asset configuration. A set of asset configurations used as inputs to the simulation is randomly generated (alternatively, these can be configured manually or generated algorithmically). The simulation can be based on the theoretical model of the asset, or the model can further be based on historical data.

[0077] In one embodiment, the simulation starts with a set of frequency history data and applies a random asset configuration to that data. When controlling the asset according to the selected random asset configuration, the system simulates what can occur with this asset. Usually, a simple simulation model is used that takes into account factors such as lamp rate, power level, and energy buffer.

[0078] Based on the simulation of the asset response, a set of performance metrics is calculated. The performance metrics can include, for example, any of the following. - Asset availability, - Total energy / power supplied / consumed, - Asset response time to achieve the desired power regulation, - Number of charge / discharge cycles (e.g., for batteries and other energy storage assets) over a given period (e.g., per year / month / week / day), - Number of activations over a given period (e.g., per year / month / week / day), - Total active time over a given period (e.g., per year / month / week / day), - Deepest activation, - Percentage of time using an amount greater than a threshold amount of power (e.g., X MW), - Percentage of time the energy buffer contains a value less than / greater than a specific threshold, - Measure of delivery achievement for delivering demand response (e.g., percentage of required increase in delivered power), - Environmental impact or cost expressed, for example, based on emissions of CO2, NO x , and / or other harmful substances, - Cost of providing demand response services (e.g., financial cost).

[0079] The exact performance metrics used typically depend on the type of asset and the goals and priorities of the demand response service. Depending on the nature of the performance metrics, they can be expressed over a given period (e.g., cycles per day, cost per day, etc.).

[0080] The simulation results in a set of training samples <C, P> for a given asset. Here, C is a specific asset configuration and P is the set of performance metrics determined for that configuration based on the simulation. The simulation is repeated for a set of distinct assets (the distinct assets can be specific existing assets of the grid or assets representing a specific asset type (e.g., a specific battery model)).

[0081] Note that in this example simulation is used, but for certain use cases and certain types of performance metrics, instead of obtaining information by simulation, the history of the measured performance metrics (from the deployed assets) can be used.

[0082] In step 208, for each simulated asset, the training samples are used to train a neural network to map the configuration C to the set of performance metrics P. Thus, once trained, the neural network can generate a set of performance metrics P for any input configuration C of that asset, including input configurations that were not simulated. In this step, for each simulated asset, a respective trained neural network results. The performance metrics output by a given neural network enable the evaluation of any asset configuration against various criteria, and determine whether the operation of the asset according to the asset configuration meets the requirements such as battery cycle limit values, output limit values, energy requirements, etc. demanded by the grid operator (e.g., TSO).

[0083] In step 210, the cost function is configured to have, for each of the assets, the performance metric output by the neural network as an input, and the cost function provides, as an output, a cost metric indicative of the overall performance of the assets in the asset pool during operation according to a given set of asset configurations. The cost metric provides a measure of how effectively the pool implements the desired demand response during operation according to a particular set of asset configurations. For example, the cost function may be based on, for example, a weighted sum of the performance metrics of the individual assets or some other numerical metrics indicative of the compliance level of the expected (or contracted) performance requirements. Alternatively, the cost amount may be expressed as, for example, an environmental impact (e.g., a value based on emissions) or a financial cost (e.g., a cost based on the agreement to pay a bonus or penalty for meeting / failing to meet the contracted demand response criteria).

[0084] For purely illustrative purposes, consider the following example. - The asset configuration and resulting control behavior are energy-neutral over the long term (the energy flowing from the grid is the same amount as the energy flowing towards the grid), - The supply contract for this site is asymmetric (the contract is expensive to obtain power from the grid and thus not beneficial to supply power to the grid).

[0085] In such a case, the performance metric may be the total energy throughput (energy obtained from the grid + energy fed into the grid), and the cost function (in financial terms) may be based on the energy throughput multiplied by the spread under the supply contract.

[0086] In step 212, an iterative optimization process such as gradient descent search is performed. The optimization process aims to reduce the value of the cost function by changing the asset configuration provided as input to the neural network for each asset (here, it is referred to as reducing / minimizing the value of the cost function, but any form of optimization may be used. For example, it should be noted that the embodiments may similarly aim to increase / maximize the value of the corresponding optimization function).

[0087] The initial configuration (before starting the optimization search) can be generated in any suitable manner by using the current actual configuration of the asset, the default configuration, or a randomly generated configuration (or a combination, e.g., the current configuration of the existing asset together with the default or random configuration of the newly added asset). In a preferred embodiment, optimization is performed frequently and generally starts from the latest information and state from the asset and the grid (including the current asset configuration).

[0088] The search is performed until the termination criterion is met. For example, the search is performed until the cost function value reaches the minimum value (local or global) or the required threshold, or until the change in value stops, or until the longest iteration time / calculation time is reached. Therefore, the term "optimization" as used herein means performing a search for an improved solution, but the final solution reached by the algorithm does not have to be "optimal" in any sense.

[0089] The final asset configuration applicable at the end of the search process usually corresponds to the lowest value of the cost function found in the search and is then used as the actual asset configuration of the asset. In step 214, those configurations are sent to the asset controller, and the asset controller stores and implements those configurations by performing a power flow change based on the variation of the local grid frequency according to the configurations (specifically, according to the response function defined by those configurations).

[0090] In a preferred embodiment, the simulation and training of the neural network (steps 206-210) are typically not as frequent as the configuration updates but are performed offline. For example, when a new type of asset (e.g., a battery model) is added or when there is a change in the asset (e.g., when the battery is degraded or the capacity of the generator is upgraded), the simulation and training can be performed. In some cases, the asset model is designed to handle small changes (e.g., expected degradation), so in such cases, retraining may not be necessary. The simulation and training can be repeated for individual assets as needed and when necessary. The simulation / training process can occur locally near the asset or at a central server.

[0091] The configuration updates (steps 212-214) are performed based on the currently trained neural network, which will typically be executed by a central server. Since these steps do not require regenerating the underlying neural network model, these steps can be repeated more frequently. The configuration updates can be repeated periodically (e.g., daily or more frequently) or in response to some triggers (e.g., addition / removal to / from the asset pool). In one example, the configuration updates can be performed at a high frequency, such as once an hour, multiple times an hour, or even more frequently, every few minutes or quasi-continuously. This can enable the system to quickly catch changes in the grid (e.g., when an asset slightly changes its power level, which can already change the amount of available power, causing a ripple effect by the complete asset pool, and such changes can be efficiently addressed by frequent re-optimization of the asset configuration).

[0092] Figure 3 schematically illustrates the approach being described. A grid 300 is shown with various asset types such as a battery, a renewable energy generator, an industrial load, etc. The database stores a dataset 302 that includes model data of the assets. Through the simulation of the assets and the training of the neural networks described above, a set of neural networks 304 corresponding to each respective asset or asset type is generated. The neural networks are shown to be combined into a meta-network 306 that combines the neural network outputs with the demand response performance for the assets that exhibit a combined output. In the embodiment being described, the meta-network 306 is in the form of a cost function (however, other approaches may be used to combine neural network outputs such as additional neural networks). The input to the neural networks 304 is the control configuration of the assets, and the neural networks provide various performance metric outputs that are used for the optimization of the cost function.

[0093] Asset configuration In a preferred embodiment, the asset configuration is in the form of one or more response curves. Broadly speaking, a response curve is a conversion of a portion of local information such as locally measured grid frequency (or locally modeled asset state) into a power difference value that indicates a power output adjustment (e.g., an increase or decrease in power output or power acquisition).

[0094] An example of a response curve is shown in FIG. 4A, showing a curve of power difference value ΔP (the amount by which the power flow into and out of the grid increases or decreases) as a function of grid frequency f (the frequency measured locally with respect to the asset). In some embodiments, a positive P value may indicate flow into the grid, a negative P value may indicate flow from the grid (or vice versa), and ΔP is represented with respect to its value. Alternatively, the response curve may represent the absolute value of the power input / output level as a function of frequency. In this example, the curve is defined by two inflection points p1 and p2, a first steady-state output difference level 402 in the low-frequency range (less than f1), a second steady-state output difference level 406 at high frequency (greater than f2) (in this case, a low output level), and a linear change region 404 between p1 and p2.

[0095] In some embodiments, the response curve is stored as a data structure that includes each data element that defines the number of such inflection points (points on the curve where the output changes). For example, the response curve of an asset may be represented by a set of settings {(f,p)} that indicate a set of tuples such as {(49.85 Hz, 10 MW), (49.9 Hz, 5 MW), (50 Hz, 0 MW)} that parameterize the response function of the asset.

[0096] In a simple embodiment, each response curve may be defined by such exact two points, and the asset controller interpolates the value for any frequency value f based on those points (e.g., ΔP is set for the value defined at p1 for f < f1 and the value defined at p2 for f ≧ p2, and linearly interpolated between the values of f1 and f2 for f1 ≦ f < f2). However, in a preferred embodiment, a fairly large number of points on the response curve may be defined to allow for more precise control. Further, instead of linear interpolation, any suitable curve fitting approach may be used. In one example, each control configuration may consist of a set of values of output ΔP (or P) for each of a predetermined set of frequency values (usually equally spaced) over a defined frequency range.

[0097] Response curves can be defined with respect to variables other than the local grid frequency. Further, multiple response curves can be defined for an asset based on different input variables (in addition to / instead of the locally measured grid frequency). The sum of the asset responses of an asset is in this case calculated based on the sum of the power differences obtained from each of the applicable response curves of the asset, and the final asset setpoint is determined based on the sum of the asset responses.

[0098] Next, the asset is configured to increase or decrease its power output (in the case of a power generation asset) or power acquisition (in the case of a consumption asset) to the required level defined by the setpoint, based on the final setpoint.

[0099] Figure 4B shows the control of an asset based on the response curve. Here, the asset controller 410 is associated with (e.g., incorporated in or connected to) an energy asset 414 such as a battery. The asset controller is a local processing device as described above and stores an asset configuration 412 that includes a set of one or more response curves. In a simple example, a single response curve based on the grid frequency is used. The asset configuration is received from the central control server 200, which performs the analysis, optimization, and generation of the asset configuration as described above.

[0100] The asset controller also receives the local grid frequency measurement from the grid sensor 416. The asset controller calculates the power adjustment value defined by the response curve(s) for the currently measured frequency (optionally, other input signals) and combines the results (in the case of multiple response curves), for example, by addition. Thus, the asset controller determines the final setpoint (power output level or consumption level) of the asset and transmits a control signal defining the setpoint (absolute power value or power adjustment value) to the energy asset 414. Next, the asset adjusts its power output / consumption based on the defined setpoint.

[0101] The use of multiple response curves can enable, for example, the following. - Dividing the frequency into multiple bands (e.g., different response curves are defined for different frequency bands), - Dividing the frequency into temporally filtered components, - Implementing energy management by adding response curves that affect the SoE (state of energy) of the battery. These battery SoEs can be modeled locally in each battery (SoE is usually estimated by a battery integrator and has the role of a battery management system).

[0102] Dividing the frequency into components can include applying a filter bank to the raw frequency signal to decompose the frequency signal into components that vary slowly and components that vary quickly. Components that vary slowly usually suit slower types of assets well, while components that vary quickly (components that are energy-neutral on a shorter time scale) usually suit the battery well. Next, different response curves can be defined for each component.

[0103] In addition to the frequency sensor, other types of sensors can be associated with / connected to the asset controller and / or the asset to obtain additional input signals (e.g., the voltage of the asset, the operating temperature of the asset, the ambient temperature, etc.). Further, the additional input signals can be derived from existing sensor signals (e.g., generating a temporally filtered frequency signal from the fundamental frequency signal measured by the grid sensor 416).

[0104] As a specific example, the asset configuration 412 can include a first response curve based on the measured grid frequency and a second response curve based on a temporally filtered version of the measured frequency, and the final setpoint is determined based on the outputs of both response curves.

[0105] Neural network The neural network is defined for each asset and maps the control configuration in the form of a response curve (or multiple such curves) as the neural network input to a set of performance metrics as output.

[0106] As described above, the response curve can be defined as continuous points on the curve. However, other representations can be used. For example, the curve can be defined with respect to a Taylor expansion or one or more basis functions. The data representation of the response curve defines a set of input dimensions of the neural network (e.g., the input dimensions can be a set of values of P or ΔP for each of a defined set of frequency increments). If there are multiple response curves, additional curve(s) can be provided as additional input dimensions of the neural network. The output dimension of the neural network is the numerical value of each of the selected performance metrics (which can be a single performance metric in a simple case or multiple performance metrics in a more complex case).

[0107] The neural network is differentiable with respect to each performance metric. The (partial) derivative coefficient with respect to a given performance metric indicates the function gradient encoded by the neural network, and the partial derivative coefficient can be used to determine how to change the input (asset configuration) to achieve a desired change in the output (performance metric), e.g., to increase / decrease the metric. This is utilized in the optimization process described in more detail below based on a gradient descent type search algorithm. For example, if a battery asset is required to operate for a fairly large number of cycles with a particular response curve, the response curve can change (e.g., to change the frequency threshold when the battery becomes active to provide demand response). Generally, changes to the response curve can include raising or lowering the power level of a particular portion of the response curve, moving the inflection point / frequency threshold, etc.

[0108] In this example, each asset is modeled by a neural network, and in some cases, a particular asset (e.g., an industrial asset) may be modeled by a different (e.g., simpler) model and may not use a neural network in some cases. For example, in some cases, a simple formula or rule that links an input (e.g., grid frequency) to an output (e.g., power difference) (such as a rule that requires the power response to be zero for all frequencies above a particular threshold frequency) may be used. Further, other types of machine learning models may be substituted for the neural network.

[0109] Figure 5 shows a neural network and its relationship to process inputs and process outputs. Specifically, two neural networks 502 and 504 are shown, each corresponding to a respective set of input features (or dimensions) 506 and 508. The input feature sets define a particular response curve, such as a set of power values (difference values). Each neural network uses a set of weights and an appropriate number and arrangement of neuron layers to combine the input features and generate an output feature set (i.e., performance metrics 510 and 512, respectively). Those performance metrics are input into a cost function 514 that generates a total cost value 516 as an output.

[0110] Embodiments may use multiple designs of neural networks (e.g., depending on the predicted performance metrics). As an example, neural networks typically have the following characteristics. - Have 1 to 10 hidden layers - All layers are fully connected - May include one or more convolutional layers - Typically have 10 to 100 input dimensions - Typically have 1 to 100 output dimensions - Use activation functions including softmax, leaky ReLU, and sigmoid.

[0111] As a specific example, the neural network may include a convolutional layer used to learn a frequency signal filter.

[0112] However, any suitable neural network design may be used and may be adapted to the specific needs of a given application context.

[0113] In FIG. 5, two neural networks are shown, each having eight input features and three output features. Note that the number of these elements is chosen for purely illustrative reasons. In practice, the exact configuration may be chosen based on factors such as the number of pooled assets, the required degrees of freedom for control of individual assets, the required performance measurement accuracy, and computational limitations.

[0114] Optimization The optimization process involves a search aimed at minimizing the value of a cost function. The result of the optimization is a set of asset configurations that yields the minimum cost value found by the search (within the scope of any applicable search constraints such as available computation time and other termination criteria).

[0115] As described with respect to the control method of FIG. 2B, in this approach, the control configuration of the assets is calculated on the premise that the assets operate together to realize an overall demand response service. Accordingly, the cost function is designed to provide an overall performance metric that balances the performance of individual assets against the overall goal (e.g., with respect to technical performance requirements, cost, etc.).

[0116] The cost function uses cost terms that represent the necessary constraints of the demand response service, is efficiently differentiable, and enables efficient calculation of the gradient of the optimization objective. As an example for illustration, if the number of cycles needs to be below some thresholds, the corresponding cost term can be the price value multiplied by the number of cycles exceeding the threshold, optionally with some smoothing at the edges (when referring to battery cycles in this specification, usually, the discharge cycles are considered (e.g., judged) as the energy extracted from the battery divided by the rated capacity, but note that the charge cycles and / or discharge cycles can be measured in any suitable manner). This approach leads to scalable optimization and enables efficient calculation of control configurations for large pools of assets.

[0117] In FIG. 6A, the optimization problem is visually represented.

[0118] As shown, the total cost 602 is a function (e.g., weighted sum or more complex function) of several cost terms 604. The cost terms (based on performance metrics) are themselves calculated by a trained neural network from a set of asset configurations 606. Optionally, parameterization 608 can be used to define the asset configuration.

[0119] These components of the optimization problem are further described in the following sections.

[0120] Cost Terms The cost terms are based on performance metrics generated by a neural network for each asset. The cost terms can simply correspond to the performance metrics, or the cost terms can be calculated from the performance metrics using any suitable processing step. Different performance metrics can be represented in different coordinate systems (e.g., different units of measurement) such as the number of cycles, response speed, environmental impact, financial cost, etc. In one approach, for example, all performance metrics are transformed to a common reference coordinate system by assigning an environmental impact metric or a financial cost or other common metric to each performance metric output.

[0121] The financial cost is merely one example of a metric, and when used herein, the term "cost" is not limited to financial cost. Rather, it should be noted that cost terms and functions can measure the effect of configuration in any suitable coordinate system. In some embodiments, the cost is expressed in terms of the production / emission amount of CO2 and / or NO x (and / or the production of other environmentally harmful substances), enabling optimization with respect to the environmental impact of the asset configuration. The cost metric can also combine multiple measurement criteria (e.g., combining environmental and monetary scales).

[0122] This system uses two types of cost terms (asset-specific cost and portfolio-wide cost). The portfolio-wide cost represents the quality of service delivery. The asset-specific cost represents the constraints and costs of different assets that are controlled in a specific way.

[0123] The cost terms can have a complex dependence on the behavior of the grid frequency. A neural network-based approach enables capturing that dependence. Regarding accuracy, it may be preferable to use a large amount of data (e.g., several years' worth of frequency data) in training the neural network. The neural network then provides an approximation of the dependence that can be efficiently differentiated.

[0124] A given cost item may be related to a single asset, but also some asset-specific cost items may depend on the configuration of other assets. This may occur when an asset responds to the state of another asset (e.g., the SoE of another asset). In that case, the SoE of the other asset is affected by the asset configuration of that asset. Therefore, in order to determine the cost of controlling the first asset, the cost item may need to consider the control configuration of other assets. To improve the robustness of the solution, the dependencies between assets are realized by the dependent asset evaluating a model of the state of the dependent asset.

[0125] Asset configuration The asset configuration includes information required to convert local frequency measurement values into power output setpoints. In a preferred embodiment, the asset configuration includes a set of one or more response curves that define the frequency control response of the asset described above. However, the asset configuration may include response curves based on inputs other than the grid frequency and / or control information and configuration information of other types of assets.

[0126] Asset configuration parameterization In some embodiments, it is possible to define the asset configuration in a (optionally simplified) parameterized way. The parameterization may vary depending on the different assets / asset types. Thus, although the format of the asset configuration 606 is the same for all assets, the system allows different parameterizations of this configuration for different assets. This allows the system to utilize certain strict constraints and thus reduce the complexity of the optimization problem.

[0127] Such simple examples of optimization occur for resources that can respond in only one direction (e.g., that only increase consumption). For these types of assets, a response curve (e.g., a response curve that defines decreasing consumption) may not make sense. In such cases, the system uses a simple parameterization that allows it to generate only response curves that are feasible. As a further example, a binary asset that results in only on / off states, such as a given fixed power output level when on and zero output when off, can be parameterized by a single frequency value at which switching occurs, reducing the search space for that asset during optimization. Parameterization 608 maps to the underlying asset configuration 606 in terms of the shape of the response curve, but reduces the search space during optimization by restricting the range of configurations (feasible response curves) that are possible.

[0128] Parameterization may be used for only specific assets or may be omitted entirely.

[0129] Cost function Cost terms 604 (e.g., performance metric values generated by a neural network) are combined by a final cost function that produces the total cost 602. The cost function may simply sum the individual cost terms or may calculate a weighted sum or some other function based on the individual cost terms.

[0130] Thus, the cost function is defined for the performance metrics generated by each neural network and, by extension, is also defined indirectly for the asset configurations provided as inputs to those neural networks. The cost function effectively "wires" the neural networks together in a meta-network as shown in FIG. 3.

[0131] Typically, it may be desirable for the sum of the response curves to be linear and to have a slope that matches the desired demand response service. The cost function can be used to provide a measure of how well those requirements are satisfied.

[0132] Thus, the input to the cost function at the reference level is the input to the neural network, specifically, the asset configuration (e.g., the response curve, although this is defined or parameterized), and the output is the cost value calculated by the performance metric generated by the neural network from the asset configuration. Purely by way of example, if each asset configuration is defined by a response curve defined by 20 data points (points on the curve) and the optimization is performed on a pool of 5 assets, the optimization problem is defined for 20 x 5 = 100 input variables or dimensions (note that, as explained elsewhere, a particular asset may be represented by a different / simpler model instead of a neural network and / or the response curve of a particular asset may be represented by a simplified parameterization).

[0133] The cost function is optimized using a gradient descent type algorithm. Gradient descent involves repeatedly changing each individual cost term 604 and then changing the asset configuration 606 that generates the cost term.

[0134] For example, since the power output of a given asset is quite high, the total cost 602 may be seen as quite high, resulting in a difference from the required linear response. The optimization may identify a new configuration of the low-output-level asset to reduce the total cost, or alternatively, identify a change to the configuration of another asset to compensate for the over-output of the original asset. These adjustments result in a more favorable configuration solution to reduce the value of the cost function.

[0135] Advantageously, the approach described can scale linearly with the number of assets in the pool.

[0136] In a typical embodiment, the optimization search is performed by a central server or server cluster, but alternatively, the optimization search can also be decomposed and computed in a distributed manner.

[0137] Constraints on Asset Configuration In addition to ensuring responsive delivery, the set points used to command different assets should preferably also satisfy asset-specific constraints (if any). Embodiments of the present invention enable consideration of various asset constraints and enable the incorporation of assets that cannot follow an arbitrary set point signal.

[0138] Typical constraints encountered include not exceeding thresholds related to power, energy, energy throughput (over a defined period), period of activation, number of activations (over a defined period), etc.

[0139] Asset constraints can be incorporated, for example, by providing a neural network (or other mapping function, simply a simple look-up table) that estimates the value of a given constrained parameter based on the asset configuration. The penalty cost term is then added to a cost function that is zero (or another low value) for constraint values below the constraint threshold and (usually strongly) increasing for values above the threshold. The penalty cost term can be weighted by a weight (e.g., a Lagrange multiplier) that may increase iteratively if the solution to the unconstrained optimization problem violates the constraints.

[0140] Exemplary embodiments of the optimization process Preferred embodiments are based on variations of gradient descent type algorithms. In each iteration, the gradient of the cost function at the current point (a point initially randomized at the start of the process) is determined, and then the gradient descent algorithm operates by moving the current point in the direction of the gradient. The current point is defined by a vector of input variables (i.e., the neural network inputs that define the asset configuration of the asset being optimized), and similarly, the gradient is a gradient vector defined for those input variables.

[0141] An exemplary embodiment is shown in FIG. 6B. First, in step 610, a cost function is obtained. The cost function can be, for example, the sum (or weighted sum) of cost terms defined using any suitable data representation (based on a performance metric determined by a neural network). The cost function can be fixed / pre-defined or can be configured by user input.

[0142] Also, in step 612, a set of constraints is obtained (again, these can be pre-defined or obtained by user input). The constraints define any additional constraints that need to be applied to the optimization (e.g., asset-specific constraints such as the maximum number of discharge cycles of a battery asset). In step 614, the constraints are combined with the cost function to generate a data representation of an extended cost function. The constraints are used to define additional penalty terms for the cost function (e.g., in the above example, the penalty term can result in a value of zero for any number of discharge cycles up to a maximum value, and otherwise, a non-zero penalty value that rapidly increases as the number of cycles exceeds that maximum value). Each penalty term is associated with weights initialized to some predetermined values.

[0143] In step 616, a random starting point is selected as the current point. This is a point randomly selected in the optimization space (i.e., the space defined by the input variables for each of the neural networks representing the assets). In the above-given example of a pool of 5 assets each represented by a neural network with 20 input variables defining the response curve, the optimization space has 100 dimensions, and thus, the starting point is a point of a randomly selected 100-dimensional vector in that space. Note that some other initialization approaches can be used instead of random selection (e.g., the approach can start from a starting point corresponding to the currently configured configuration or the default asset configuration).

[0144] In step 618, the cost function is evaluated at the current point. This includes the following sub-steps. - Determining the value of the extended cost function at the current point (step 620), - Evaluating the gradient of the extended cost function (step 622). - Optionally, one or more higher-order gradients of the extended cost function can be evaluated (step 624).

[0145] In step 626, the process determines whether the optimization has converged. If not, in step 628, the next point to be evaluated is selected based on the current point using the information determined between steps 618 - 624 (i.e., the value, gradient, and optionally higher-order gradients of the extended cost function). Next, the process returns to the evaluation step 618 based on the next point selected as the current point.

[0146] As shown above, the gradient of the extended cost function is a gradient vector defined with respect to the dimensions of the optimization problem. Each dimension represents an input variable (part of the asset configuration), and the corresponding component of the gradient vector representation represents the direction in which the total improvement of the cost terms for all input variables should change. Gradient descent uses the calculated gradient vector to search for the minimum value of the cost function.

[0147] The selection of the next point (step 628) can be based on one or more of the following. - The value of the (extended) cost function (optionally, both this iteration and past iterations), - The value of the gradient of the (extended) cost function (optionally, both this iteration and past iterations), - The value of the higher-order gradient of the (extended) cost function (optionally, both this iteration and past iterations).

[0148] As an example, the next point can be selected as follows. - Using the standard gradient descent approach, Calculating the next point = current point + gradient * step size, - Using an approach based on the Adam optimizer, for example, Calculate the next point = current point + gradient * normalized step size + inertia term, - Use an approach based on the Levenberg-Marquardt optimizer that locally approximates the cost function as a quadratic function that can be solved. Then use this solution as the next point.

[0149] Repeat the above loop (step 626) until the process converges. For example, based on a cost value below a threshold, or a cost value that does not change or changes by less than a threshold amount over subsequent iterations (which can be interpreted as indicating a cost value reaching a global or local minimum), based on the gradient (and / or higher-order gradient(s)), based on the maximum number of iterations to reach, or the computation time, etc., convergence can be determined in any suitable manner. More generally, multiple convergence criteria can be used to determine convergence, using any of the following information. - Optimization time, - Number of iterations, - Search space size, - Value of the (extended) cost function (optionally, both this iteration and past iterations), - Value of the gradient of the (extended) cost function (optionally, both this iteration and past iterations), - Value of the higher-order gradient of the (extended) cost function (optionally, both this iteration and past iterations).

[0150] Generally, convergence aims to identify the minimum value of the cost function, but other criteria can make it possible to limit the optimization (e.g., within time / resource), and ensure process completion at an appropriate time.

[0151] After convergence, next, the process checks whether the identified solution is feasible with respect to the constraints to which it is applied in the cost function (step 630). In one embodiment, for each constraint, a penalty term threshold defines whether a given constraint is satisfied. If any of the penalty terms falls below the applicable threshold, the solution is considered feasible with respect to that constraint. If the threshold is satisfied for all penalty terms / constraints, the solution is feasible; otherwise, the solution is considered infeasible because it violates one or more constraints.

[0152] If the solution is infeasible, the weight of one or more penalty cost terms is increased (step 632). Specifically, in the process, the weight of any penalty term violated by the identified solution is increased. This increases the contribution of that constraint to the total cost value and, in turn, biases the optimization to satisfy that constraint in future iterations. In step 614, the extended cost function is modified based on the adjusted weights. Next, in step 630, the optimization is repeated until a solution is identified as being feasible.

[0153] Next, in step 634, the process outputs the identified solution.

[0154] The solution is a point in the optimization space defined by the input variables to the neural network, and the input variables, in turn, define the asset configuration for each asset in the pool for which the optimization is being performed. Thus, the solution determined by the optimization results in an asset configuration for each asset in the pool, in the shape of the response curve. These configurations are transmitted to the assets, as appropriate, to configure the assets. Once configured, the assets perform a frequency response service (or other demand response) according to their configured response curve as described above. However, if necessary, some preprocessing is done on the optimization output, for example, converting the response curve defined by the optimal solution into a representation usable by each asset.

[0155] Therefore, in the above approach, the optimization problem is treated as an unconstrained optimization problem (in the inner loop), but it should be noted that there are constraints represented by the penalty terms. Next, the constraints are explicitly verified in the outer loop and, if necessary, the optimization is repeated.

[0156] In some embodiments, the above optimization is performed multiple times in parallel, and each optimization uses a slightly different configuration with respect to steps 616 - 628. For example, each optimization process may start from a different random starting point and / or use a different optimization approach. One or more of the following optimizers may be used in parallel. - Conventional gradient descent method, - Adam optimizer, - Levenberg - Marquardt algorithm - Broyden - Fletcher - Goldfarb - Shanno algorithm.

[0157] For the most recent known state of the actual asset, all the proposed solutions (the solutions output at step 634) of these parallel optimizers are compared and evaluated according to the benchmark. Based on this information, the most favorable asset configuration is then selected.

[0158] Instead of or in addition to the algorithms outlined above, other optimization approaches (e.g., branch algorithms and bound algorithms) that may include some binary variables and some continuous variables for performing gradient descent may be used.

[0159] Incorporation of states from other assets In addition to the grid frequency, the states of one or more other assets may be used as additional input(s) to the asset control. The asset states may communicate directly between the assets via a central control system, or the model sharing approach may be used as described with respect to Figure 1C.

[0160] In a typical example of incorporating the state of other assets, a given non-battery asset can use the charge state of another battery asset as an input to follow and can vary its own demand response based on the battery charge state. For example, when the charge level of a pair of battery assets is low, the asset can increase its own energy output.

[0161] Additional response curve(s) can be defined as part of the asset configuration that defines the asset response in dependence on the state of other assets. Next, the final control decision (i.e., the final setpoint) is made by the asset controller by combining (e.g., summing) the power regulation values defined by all applicable response curves described above.

[0162] Control device FIG. 7 shows, in an exemplary embodiment, the hardware and software architecture of the components of the system being described. The central server 200 is provided to act as a central control system for performing the optimization process being described and calculating the asset configuration. The server includes one or more processors 702, along with volatile / random access memory 704 for storing temporary data and software code being executed.

[0163] The network interface 706 (e.g., a wired or wireless interface) provides communication with other system components including energy assets through one or more communication networks 712 (e.g., a wide area network including a local network or the Internet).

[0164] Persistent storage 708 (e.g., in the form of hard disk storage, optical storage, etc.) permanently stores software for performing the described functions, including an optimization process 710, to calculate the local response function of the asset based on the learned neural network model of the asset. The simulation and training process 711 performs the techniques described to simulate the asset to generate training samples and use the training samples to train the neural network of the asset. Also, the persistent storage includes other server software and data (not shown) such as a server operating system.

[0165] The server includes other conventional hardware and software components known to those skilled in the art, and the components are interconnected by a data bus (which may actually consist of several individual buses such as a memory bus and an I / O bus).

[0166] The asset controller 410 is provided in the form of a computing device for controlling an energy asset 414 (e.g., a battery or other energy supply asset, or an industrial load or other energy consumption asset). The asset controller 410 is connected to a communication network 712 using a network interface 720 (e.g., a wired or wireless interface) to enable communication with a central server. The asset controller is also connected to one or more grid sensors 416 for locally detecting the grid frequency (optionally, other grid operating characteristics and / or asset operating characteristics (e.g., battery charge level)) for the asset.

[0167] The server includes one or more processors 716 (e.g., a built-in microprocessor such as an ARM Cortex CPU with associated RAM may be used) along with volatile / random access memory 718 for storing temporary data and software code being executed.

[0168] The persistent storage 722 (e.g., in the form of hard disk storage, optical storage, solid state storage, or flash memory, etc.) permanently stores software and data to perform the described functions of the asset controller, and includes an asset configuration 412 (including one or more response curves), and a device agent 724 that implements demand response / frequency control responses based on the stored asset configuration. The asset controller may include other conventional hardware / software elements known to those skilled in the art.

[0169] Typically, the system includes a plurality of such asset controllers 410 associated with various assets of the grid that are interconnected and / or communicable with the central server 200 via a communication network 712. Each asset managed in the asset pool may be associated with each asset controller. In some cases, a single asset controller may operate multiple assets. In a system without a central controller, the central server 200 may be omitted, and processes 710 and 711 may be performed in a distributed manner by, for example, one or more asset controllers.

[0170] Although a particular architecture is shown by way of example, any suitable hardware / software architecture may be used.

[0171] Furthermore, the functional components shown as being separate may be combined, and vice versa. For example, the functions of the server 200 may actually be performed by a plurality of separate server devices (e.g., the optimization process 710 and the simulation / training process 711 may be executed on different servers, or different servers may manage different asset pools or grid regions). In another example, the functions of the central server may be integrated into a selected asset controller.

[0172] It is understood that the present invention has been described above purely by way of example and that details may be modified within the scope of the present invention.

Claims

1. A computer-implemented method for controlling a plurality of assets connected to a grid to provide a demand response service, wherein the plurality of assets are configured to supply electrical energy to the grid and / or consume electrical energy from the grid, and are configurable to adjust the energy flow into or out of the grid in response to changes in the operating state of the grid, the computer-implemented method comprising: accessing, for each of the plurality of assets, a trained neural network model, the neural network model comprising: receiving, as an input, an asset demand response configuration that defines the demand response of the asset to one or more detected fluctuations in the operating state of the grid detected at the asset; using the asset demand response configuration to output, during operation, one or more performance metrics related to the operation of the asset with respect to the required demand response service; determining, by an optimization function, an optimization metric that includes a measure of how effective the plurality of assets are in implementing a desired demand response, the optimization function receiving, as an input, based on the performance metrics, output by the neural network model; performing a search process for determining a modified optimization metric by varying the asset demand response configuration of one or more of the plurality of assets, the search process continuing until an end criterion for the modified optimization metric is met; determining, based on the modified optimization metric, an updated asset demand response configuration for the plurality of assets; transmitting the updated asset demand response configuration to one or more asset control devices associated with the plurality of assets; and controlling, by the control device, the energy flow between the plurality of assets and the grid in accordance with the asset demand response configuration and in response to changes in the operating state of the grid.

2. The search process includes varying one or more asset demand response configurations to change the performance metric output by the neural network, thereby changing the value of the optimization metric. Preferably, the variation step iterates until the end criterion is met. The method according to claim 1.

3. The search process includes optimizing the optimization function with respect to the optimization metric. Preferably, the asset demand response configurations of the plurality of assets define a set of dimensions of the search space, and the search process performs a gradient descent search of the search space to optimize the output value of the optimization function. The method according to claim 1 or 2.

4. The optimization function is a cost function, and the search process includes minimizing the cost function by changing the asset demand response configuration. The method according to any one of claims 1 to 3.

5. The end criterion includes one or more of a predetermined threshold or an optimal value (e.g., a minimum or maximum value) of the optimization metric, a maximum number of iterations, and a maximum computation time. The method according to any one of claims 1 to 4.

6. The search process starts from an initial set of asset demand response configurations, and the initial configuration includes one or more of the current asset demand response configuration of the asset, the default asset demand response configuration of the asset, and a randomly generated asset demand response configuration. The method according to any one of claims 1 to 5.

7. For one or more of the plurality of assets, it includes training the neural network model by a process, and the process generates a plurality of asset demand response configurations; for each asset demand response configuration, simulates the operation of the asset according to the asset demand response configuration, and determines one or more performance metrics of the asset demand response configuration based on the simulation; uses a plurality of training samples to train the neural network model, where each training sample is based on an asset demand response configuration, and the corresponding performance metric is determined for the asset demand response configuration by the simulation; The method according to any one of claims 1 to 6.

8. The method according to claim 7, wherein generating the asset demand response configuration includes randomly selecting the asset demand response configuration. **Claim 9** The method includes training a neural network model and repeating the training of one or both of the neural network model and the search process periodically or in response to changes in the plurality of assets, the changes optionally including addition, removal, or change of operating characteristics of one or more assets. The method according to any one of claims 1 to 8. **Claim 10** The method according to any one of claims 1 to 9, wherein the asset demand response configuration includes configuration data defining how the energy flow between the asset and the grid should change in response to a change in the operating state measured at the asset. **Claim 11** The method according to claim 10, wherein the configuration data defines one or more response curves, and each response curve defines a required power flow level or power flow change according to a given operating state parameter. **Claim 12** The method according to claim 11, wherein the configuration data defines a plurality of response curves respectively defined for different operating state parameters and / or a plurality of response curves respectively defined for different value ranges of the same operating state parameter. **Claim 13** The method according to any one of claims 1 to 12, wherein the operating state includes one or more parameters regarding the local grid frequency measured at the asset. **Claim 14** The method according to claim 12, wherein the different value ranges of the same operating state parameter include different frequency bands of the grid frequency parameter. **Claim 15** The frequency parameter is the local grid frequency measured at the asset, and data derived from the local grid frequency, and includes at least one of them. The method according to claim 14. **Claim 16** The method according to any one of claims 10 to 15, wherein the configuration data defines a required power input value, power output value, or adjustment value of the asset for each of a plurality of individual grid frequency values. **Claim 17** The method according to any one of claims 1 to 16, wherein the one or more operating states include the operating state of the asset connected to the grid or another asset. **Claim 18** The search process according to any one of claims 1 to 17, including varying one or more power flow values or power flow regulation values, and / or one or more frequency thresholds of the power flow regulation values.

19. In a given control device, receiving one or more signals indicating an operating state related to the operation of the grid or an asset of the grid; determining a power flow level of an asset controlled by the control device based on the one or more signals and the asset demand response configuration of the asset; controlling the asset according to the determined power flow level; The method according to any one of claims 1 to 18, including.

20. Determining the power flow level optionally includes calculating the power flow level based on a response curve defined by the asset demand response configuration by interpolating values of the response curve of the operating state parameter from a set of data points of the curve defined by the asset demand response configuration. The method according to claim 19.

21. The method according to claim 19 or 20, wherein the one or more signals include local grid frequency measurement values and / or signals derived from local grid frequency measurement values.

22. In the control device, receiving a plurality of signals indicating respective operating state parameters; determining a plurality of power flow regulation values based on the signals, each power flow regulation value being derived using each response curve defined by the asset demand response configuration, and the response curve mapping each operating state parameter to a power flow regulation value; determining a total power flow regulation value based on the plurality of power flow regulation values; controlling the asset according to the determined total power flow regulation value; The method according to any one of claims 19 to 21, including.

23. The performance metric output by the neural network includes one or more measurements of the performance of the asset related to the required demand response service defined by the asset demand response configuration. The method according to any one of claims 1 to 22.

24. The performance metric is the availability of the asset providing the demand response; the amount of energy indicating the total amount of energy supplied or consumed over a certain period when providing the demand response; The response time of the asset to achieve the desired energy flow adjustment value, and The number of operating cycles or charge / discharge cycles of the asset over a certain period, and A measure of success for delivering the configured demand response, and A measure of the cost of providing the demand response service, and The method according to any one of claims 1 to 23, which defines one or more of the above. **Claim 25** A computer-implemented method for controlling a plurality of assets connected to a grid, wherein the plurality of assets are configured to supply electrical energy to the grid and / or consume electrical energy from the grid, and are configurable to adjust the energy flow into or out of the grid in response to changes in the operating state of the grid, the method comprising: For each of the plurality of assets to be configured, accessing a mapping, the mapping being: Receiving, as an input, an asset demand response configuration that defines the demand response of the asset to one or more detected fluctuations in the operating state of the grid detected by the asset; Using the asset demand response configuration to output, during operation, one or more performance indicators related to the operation of the asset for the required demand response service; and accessing configured to perform; Determining an optimization metric that includes a measurement of how effective the plurality of assets are when implementing a desired demand response by an optimization function, the optimization function receiving an input based on the performance indicator output by the mapping; and determining; Performing a search process for determining a modified optimization metric by varying the asset demand response configuration of one or more of the plurality of assets, the search process continuing until an end criterion for the modified optimization metric is met; and performing; Determining an updated asset demand response configuration for the plurality of assets based on the modified optimization metric. Transmitting the updated asset demand response configuration to one or more asset control devices associated with the plurality of assets; and controlling, by the control device, an energy flow between the plurality of assets and the grid in accordance with the asset demand response configuration and in response to a change in an operating state of the grid. A method comprising the steps of: **Claim 26** The method according to claim 25, further comprising additional steps or features according to any one of claims 2 to 24. **Claim 27** A computer system or apparatus optionally having means including one or more processors having associated memory for performing the method according to any one of claims 1 to 26. **Claim 28** A computer-readable medium including software code adapted to perform the method according to any one of claims 1 to 26 when executed on one or more data processing devices.

Images