System for demand response verification

Abstract

Methods and apparatus for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid. The methods comprising: obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; providing the data features as inputs to a model; and determining, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features.

Description

[0001]SYSTEM FOR DEMAND RESPONSE VERIFICATION FIELD OF THE INVENTION The present application relates to methods and systems for determining whether devices of an energy system have operated to provide a demand response service. BACKGROUND OF THE INVENTION Fluctuations in the AC (alternating current) transmission frequency on an electricity distribution / transmission grid can be countered by adjusting electrical power consumption from the grid or electrical power supply into the grid. Specifically, an increase in grid frequency above the standard expected value (e.g. 50 Hz in Europe), also referred to herein as the reference or nominal frequency value, can be countered by increasing consumption or reducing supply, whilst a decrease in grid frequency can be countered by decreasing consumption or increasing supply. In some cases, providers that have suitable energy consuming or energy producing assets (for example, batteries) connected to the grid can contract with the grid operator (e.g. a Transmission System Operator or TSO) to provide a demand response (DR) service (also referred to as a flexibility service) by adjusting supply or consumption of one or more energy devices on demand to counter a frequency fluctuation. For many reasons, including proper management of grid performance, it is desirable to determine whether the energy devices have properly fulfilled the demand response service. This typically requires collection of large volumes of energy data from participant systems and transmission of that data to the service operator’s computer systems. This can present a significant burden in terms of data collection / transmission equipment needed at each participant system and the network bandwidth for the transmission of the energy data. Additionally, privacy regulations may prohibit the sharing of detailed energy data with the service operator. Data protection laws may restrict this kind of data collection. SUMMARY OF THE INVENTION Embodiments of the invention aim to provide alternative approaches that allow determination of whether electronic devices of a participant system’s have operated in compliance with expected operation to fulfill the demand response actions required for a demand response service. Aspects of the invention are set out in the independent claims. Certain preferred features are set out in the dependent claims. According to the invention in an aspect, there is provided a method of verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the method comprising: obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; providing the data features as inputs to a model; and determining, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features. Verification of provision of a demand response service encompasses the determination of whether electrical devices of an energy system have operated in a way that would implement the demand response as required. That is, operated in a way that stabilises grid frequency. The term ‘energy asset’ encompasses an electrical device that is configured to consume energy from the grid and / or supply energy to the grid. In exemplary arrangements, determining whether the activation data is present in the energy data may involve determining whether one or more of time domain activation data, frequency domain activation data and magnitude of the activation data correctly corresponds to a respective one or more of time domain energy data, frequency domain energy data and magnitude of the energy data. Such correspondence may be within predetermined threshold limits. According to the invention in an aspect, there is provided a method for determining correct operation of one or more electrical devices of an electrical energy system in the provision of a demand response, wherein the electrical energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more electrical devices for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the electrical devices and the grid, the method comprising: obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; obtaining activation data indicative of one or more requests for changes in energy flow between at least one electrical device and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; providing the data features as inputs to a trained machine learning model; and determining, using the trained machine learning model, an output indicating whether the electrical devices of the energy system operated to provide the demand response service, based on the data features. In exemplary arrangements determining whether the electrical devices of the energy system operated to provide the demand response service comprises determining a correlation between at least a part of the activation data and the energy data in one or more of the time domain, the frequency domain and magnitude. Optionally, the energy data indicates a time-varying net quantity of energy consumed by the energy system from the grid or provided by the energy system to the grid; and / or the activation data indicates a time-varying quantity of energy to be supplied or consumed by the at least one energy asset for the demand response service. The energy data and / or the activation data are readily accessible on the supply side. Therefore, no transmission of data is necessary from the energy system. Optionally, the energy data and activation data comprise respective time series of electrical power values, optionally wherein the energy data comprises net electrical power flow between the energy system and the grid, and optionally wherein the activation data comprises power values requested to be consumed from and / or supplied to the grid by the energy system. Optionally, the energy data and / or the activation data is obtained from the energy system and / or a dynamic response system. Optionally, the activation data is generated in response to changes in an operating characteristic of the grid measured at the energy system and / or a dynamic response system, the operating characteristic optionally comprising a grid frequency. A measured change in grid frequency may be used to determine the activation data. Optionally, the verification result comprises: one of two binary classifications indicating whether the demand response has been supplied by the energy system; or one of more than two classifications, each classification indicative of a respective measure, degree and / or likelihood of whether the demand response has been supplied by the energy system. Optionally, the verification result includes one or more of: correct provision of the demand response service; provision of the demand response service at a power level different from the required level specified by the activation data; provision of the demand response service with a delay compared to the activation data; non-provision of the demand response service. Optionally, the data features derived from the energy data and the activation data include one or more correlation features indicative of correlation between a time window of the energy data and a time window of the activation data. Optionally, the one or more correlation features comprise: one or more features indicative of a time-domain correlation between the energy data and the activation data over the time window; and / or one or more features indicative of a frequency domain correlation between the energy data and the activation data over the time window. Optionally, the time-domain correlation features comprise at least one of: a correlation between the activation data and the energy data over the time window; a correlation between the activation data and base energy data over the time window, wherein the base energy data is based on a difference between the energy data and the activation data. Optionally, the frequency-domain correlation features comprise at least one of: a correlation between a frequency domain representation of the activation data and a frequency domain representation of the energy data, over the time window; a correlation between a frequency domain representation of the activation data and a frequency domain representation of base energy data over the time window, the base energy data based on a difference between the energy data and the activation data. Optionally, the method further comprises performing a frequency transform on the energy data or base energy data and the activation data over the time window, and evaluating a correlation between the resulting transforms. Optionally, the frequency domain correlation features comprise: one or more correlation features determined across a frequency range of the frequency transform; and / or respective separate correlations of frequency domain input vectors computed for at least two sub-ranges of the frequency range. Optionally, the method further comprises determining separate frequency domain correlations between the activation data, and the energy data and / or base energy data, for two respective frequency ranges divided at a predetermined threshold frequency. Optionally, the data features further comprise features determined based on a total magnitude of consumption or provision and / or a total magnitude of activation data over the time window, optionally comprising: a feature determined based on a ratio of the total magnitude of the activation data and the total magnitude of the energy data over the time window; and / or a feature determined based on a ratio of the total magnitude of the activation data and a magnitude of a maximum possible activation for the one or more energy assets over the time window. Optionally, the model comprises one of: a logistic regression model, a support vector machine, a random forest classifier and a neural network. Optionally, the energy system is provided at a building environment connected to the electrical grid. Optionally, at least the steps of obtaining the energy data and activation data and deriving the set of data features are performed at a local data processing system provided at the building environment, the local data processing system transmitting to a remote management system at least one of: the set of data features, and the verification result. Optionally, the steps of obtaining the energy data, obtaining the activation data, deriving the set of data features, and providing the data features are performed at the local data processing system, and further comprising transmitting, by the local data processing system, the verification result to the remote management system over a data communications network. Optionally, the energy data and / or activation data are processed by the local data processing system without being transmitted to the remote management system. Optionally, the method comprises one of repeating the steps of deriving the data features and applying a model for each of a plurality of time windows of the inputs, and outputting verification results for each time window. Optionally, the method comprises analysing the verification results, optionally at a remote management system, to evaluate a participation of the energy system in the demand response service over time, the analysing optionally comprising generating, based on the verification results, one of: a fault indication indicating a possible fault associated with the energy system or the one or more energy assets; financial compensation information relating to compensation for provision of the demand response service. According to the invention in an aspect, there is provided an apparatus for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving energy from and / or providing energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the apparatus comprising: an energy data unit for obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; an activation data unit for obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; and a computer processor configured to: derive a set of data features based on the energy data and the activation data; provide the data features as inputs to a model; and determine, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features. According to the invention in an aspect, there is provided an apparatus for determining correct operation of one or more electrical devices of an electrical energy system in the provision of a demand response service, wherein the electrical energy system is connected to an electrical energy grid for providing the demand response service by receiving energy from and / or providing energy to the grid, and wherein the energy system comprises one or more electrical devices for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the electrical devices and the grid, the apparatus comprising: an energy data unit for obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; an activation data unit for obtaining activation data indicative of one or more requests for changes in energy flow between at least one electrical device and the grid for provision of the demand response service; and a computer processor configured to: derive a set of data features based on the energy data and the activation data; provide the data features as inputs to a trained machine learning model; and determine, using the trained machine learning model, an output indicating whether the electrical devices of the energy system operated to provide the demand response service, based on the data features. According to the invention in an aspect, there is provided a method of training a machine learning model for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the method comprising: receiving an input training set comprising: grid frequency data covering a time period; and energy data covering the time period, the energy data indicating, at least partially, net energy flow between the grid and the energy system; determining activation data based on the grid frequency data, the activation data being indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; and providing the data features as inputs to the model for training thereof. According to the invention in an aspect, there is provided a method of training a machine learning model for determining correct operation of one or more electrical devices of an electrical energy system in the provision of a demand response service, wherein the electrical energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more electrical devices for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the electrical devices and the grid, the method comprising: receiving an input training set comprising: grid frequency data covering a time period; and energy data covering the time period, the energy data indicating, at least partially, net energy flow between the grid and the electrical energy system; determining activation data based on the grid frequency data, the activation data being indicative of one or more requests for changes in energy flow between at least one electrical device and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; and providing the data features as inputs to the model for training thereof. Optionally, the method comprises: determining correct outputs from the model based on the grid frequency and energy data; and providing the correct outputs to the model for training thereof. Optionally, the energy data and activation data represent compliant data in which the requested demand response was provided by the energy system, the method further comprising: creating non-compliant energy data and non-compliant activation data by introducing one or more of time, frequency and magnitude errors into the energy data and / or the activation data; deriving a set of non-compliant data features based on the non-compliant energy data and the non- compliant activation data; and providing the non-compliant data features as inputs to the model for training thereof. Optionally, substantially half the data used to train the model comprises compliant data, and substantially half the data used to train the model comprises non-compliant data. According to the invention in an aspect, there is provided a computer program, computer program product or non-transitory computer-readable medium comprising software code adapted, when executed by a data processing system, to perform a method as set out herein. The disclosure also encompasses a computer program, computer program product or tangible / non-transitory computer-readable medium comprising software code adapted, when executed by a data processing system, to perform any method as set out above or as described in more detail below. Features of one aspect or example may be applied to other aspects or examples, in any combination. For example, method features may be applied to system or computer program aspects or examples (and vice versa). Features implemented in software may be implemented in hardware and vice versa. BRIEF DESCRIPTION OF THE FIGURES Certain embodiments of the invention will now be described by way of example only, in relation to the Figures, wherein: Figure 1 illustrates a system for verifying demand response participation; Figure 2 shows an example of a control curve for controlling an energy asset as part of a demand response service; Figure 3A shows processing performed by a verification system; Figure 3B illustrates a method for verifying demand response participation; Figures 4A and 4B illustrate software and hardware architectures for a control system; and Figures 5A-5C, 6A-6D and 7A-7D illustrate evaluation results for an implementation of the demand response verification system. DETAILED DESCRIPTION Embodiments of the invention provide a system for determining whether electrical devices of an energy system have operated in accordance with the manner required to deliver a demand response. This involves determining if a user’s electrical energy system, which includes energy consuming and / or energy supplying devices (i.e. electrical devices) installed at a user’s premises, has participated correctly to provide a demand response in order to stabilise grid frequency. This may require that a certain amount of power was switched on / off at a specific time. Methods and apparatus may determine this by relying on local measurements and data- driven algorithms which are simple to obtain at a low computational burden and avoid privacy- sensitive information flowing outside the user premises. In the described embodiments, the techniques are applied in the context of home batteries participating in a frequency containment response (FCR) service, which is an exemplary demand response service. FCR (formerly known as R1), is a transmission system operator (TSO) service used to stabilize the frequency of electrical power grids after events that lead to a frequency deviation. Its goal is to limit this frequency deviation within the operational values until the problem is solved, or a slower reserve is activated. Thus, it typically requires activation of one or more electrical devices to consume and / or supply electrical power within a few seconds that is proportional to frequency deviations. However, the described principles may be applied to assets other than batteries and / or to demand response services other than FCR. A system for determining whether electrical devices of an energy system have operated as part of a demand response is illustrated in Figure 1. The system includes a home energy system 100 installed in a building environment such as a house, apartment or other dwelling. It is noted that the energy system 100 may be any other type of energy system and need not be associated with a domestic dwelling. The home energy system 100 is connected to an electricity grid, which may be a distribution grid or a transmission grid. In the example of Figure 1, the energy system 100 receives energy in the form of electricity from a public electricity distribution grid 120. Electricity is supplied to various energy consuming electrical devices 102 in the home, such as electrical appliances, lights etc. The home energy system 100 may also include a local electricity generator 104, such as a solar panel array, which may supply at least some of the energy needs of the local consuming devices 102. Figure 1 is purely illustrative and in practice any numbers of consumers and / or generators may be provided. The system may further include one or more electrical power storage apparatus, such as battery 106, as shown in the example of Figure 1. The battery 106 may be used to store electrical energy, for example excess generation capacity from local generator 104 and / or energy supplied from grid 120 (e.g. during times when energy cost is low). At other times, the battery 106 may also supply previously stored energy back to consumers 102 and / or the grid 120. Collectively, energy consuming and supplying devices such as consumers 102, generator 104 and battery 106 are referred to as electrical devices or energy assets. Charging and discharging of battery 106 is under control of a local control system 108, typically in the form of a computer running control software. The control system 108 receives energy data including information on energy that is consumed by the home energy system 100 from the grid 120 (or, in the case of provision of excess local supply to the grid 120, on energy supplied back to the grid 120). This may be obtained from an energy meter 110. It is generally assumed herein that the local consumption typically exceeds the local supply from generator 104 and / or battery 106 and thus the meter 110 typically measures net positive consumption from the grid 120. However, it is understood that local consumption may be less than the local supply, in which case the energy data may show that energy has been provided to the grid 120. The energy data indicates net energy flow between the grid 120 and the energy system 100. Net energy flow may be positive, negative (i.e. consumption of energy from a grid or supply of energy to a grid) or zero. Meter readings may be obtained at intervals, e.g. every 30 minutes for a low-resolution meter or every 30 seconds for a high-resolution meter, each meter reading giving a total energy consumption in the last meter interval, e.g. as a power consumption value. Control system 108 further receives a grid frequency measurement from a frequency sensor 112, measuring the frequency of the AC supply on the grid 120. In the illustrated example, the frequency is measured at the point where the electricity supply enters the home energy system 100. However, alternatively, the sensor 112 could be located outside the home energy system 100, e.g. at a suitable connection point to the grid 120. Frequency measurements may indicate an absolute measured frequency or a frequency deviation from a standard expected frequency (e.g.50Hz for Europe). Where the received measurement is an absolute value the local control system 108 may compute the frequency deviation, resulting in frequency deviation measurement Δ^^ (e.g. for a nominal frequency of 50Hz, an absolute measurement of 49.95 Hz would yield a frequency deviation of -0.05 Hz). During normal operation, the local control system 108 may control charging or discharging of the battery 106 based on local energy needs within the home, for example, to charge the battery 106 during excess generation by generator 104 and supply energy to consumers 102 at times of insufficient local generation. Additional required capacity is then drawn from grid 120. The local control system 108 is additionally configured to make the battery 106 available for participation in an FCR service. The FCR service is managed by an external demand response system 142. When configured to participate in the FCR service, the battery 106 may be controlled to charge or discharge, that is to consume additional electrical energy from the grid 120 or supply electrical energy to the grid 120 (or equivalently reduce overall consumption from the grid by the home energy system 100 by supplying at least some of the required energy directly to local consumers). While the focus here is on using the battery 106 to support FCR service provision, other electrical devices such as consumers 102 and generators 104 could also provide this function. Electrical devices that are configurable to participate in a demand response or flexibility service are also referred to as flexibility assets. For example, heat pumps, electric heaters or electric vehicle batteries (while connected to a home energy charger) may be used as flexibility assets. In general, any device capable of altering consumption or provision (flow from or to the grid) with a sufficiently rapid response to respond to frequency fluctuations on the grid may be utilised. The local control system 108 obtains the local frequency deviation measured using sensor 112 and controls energy flow to or from the battery 106 in order to counter any fluctuation in the measured frequency. Typically, the battery 106 is activated to charge from the grid 120 or discharge to the grid 120 or local energy system 100 whenever the local frequency deviation exceeds an activation threshold. The control response is configured by way of a droop curve, as depicted in Figure 2. Figure 2 shows the required power flow change as a function of measured frequency deviation from the nominal frequency. In this case the droop curve shows no response within a smaller range of frequency deviations between െ^^^and ^^^, with linear responses between െ^^^^௫and െ^^^and between ^^^and ^^^^௫respectively. The region 202 between െ^^^^௫and െ^^^corresponds to discharging of the battery 106 (indicated as a negative power consumption, i.e. power supply, value), while the region 206 between ^^^and ^^^^௫corresponds to charging of the battery 106 (indicated as a positive power consumption value). Below െ^^^^௫and above ^^^^௫the power flow adjustment reaches minimum / maximum values (െ^^^^௫and ^^^^௫) so that there is no further decrease / increase in the power flow adjustment (referred to as saturation of the response). These values are typically dependent on the battery type (i.e. the maximum charge or discharge power supported by the battery 106). Note that the depicted droop curve is symmetric and specified by the values ^^^, ^^^^௫, and ^^^^௫. These parameters may be specific to a particular battery 106 / home energy system 100, and different participating home energy systems may be configured with different droop curves. However, other more complex forms of droop curve could be defined (e.g. by defining additional points on the curve) and the curve could be asymmetrical (with different charging and discharging behaviour). Furthermore, the droop curve may take different forms for flexibility assets other than batteries. Returning to Figure 1, the flexibility service is managed by the remote demand response system 142 connected to the local control system 108 via a data network 140. The data network 140 may, for example, include one or more private and / or public networks, such as the Internet. The demand response system 142 may remotely configure the local control system 108 with the droop curve, e.g. by specifying the values ^^^, ^^^^௫and ^^^^௫. Alternatively, the droop curve may be fixed / configured at the local control system 108, for example based on the type of battery or other flexibility asset. When the flexibility service is active, the controller 108 monitors the grid frequency using sensor 112 and determines the deviation from nominal frequency (e.g.50 Hz in Europe). The controller 108 then determines the required power flow adjustment for the flexibility asset (e.g. the battery 106 and / or the relevant electrical device 102, 104) based on the power flow adjustment specified for the measured frequency deviation by the droop curve. As shown in the curve, the adjustment may be negative (e.g. for discharging the battery 106), positive (e.g. for charging the battery 106) or zero for measured frequency deviations close to zero (the “deadband” region 204 of the droop curve). The battery 106 is then controlled to charge or discharge in accordance with the calculated power flow adjustment value, resulting in increased or decreased net energy data, in this case for consumption from the grid 120. The control loop operates continuously and thus activation of the flexibility asset responds to changes in the grid frequency essentially in real-time. Activation of the flexibility asset ceases as soon as the frequency returns sufficiently closely to the nominal frequency (with a frequency deviation in the deadband 204). The demand response system 142 may use any number of home energy systems 100 such as the system of Figure 1 to provide the FCR service for the grid operator. The operator of the demand response system 142 may also be referred to an as “aggregator” since they provide an aggregate demand response service (in this case FCR) using electrical devices of a plurality of energy systems. In some exemplary systems, all or part of the functionality of the local control system 108 may be undertaken by the DR system 142. For example, the DR system 142 may receive frequency data to determine whether there has been a deviation in frequency of grid supply. The frequency data may be received by the DR system 142 from the sensor 112 and / or from a remote sensor located outside the energy system 100. In some arrangements, the DR system 142 may determine a required demand response service from the (and optionally further) energy system 100. This may be done using the droop curve described above. In some arrangements, the DR system 142 may instruct the local control system 108 to effect the demand response service in accordance with the determined demand response service. Determination of FCR participation Participation in a demand response service such as FCR may be required for proper management of the electrical grid, as well as being incentivised financially, e.g. by paying a homeowner (or providing an energy discount) for participation. It is therefore important to accurately measure whether a particular home energy system 100 actually did participate as expected / requested. This may allow the determination of whether the electrical grid has operated within acceptable limits. In addition, issues or faults with the energy system, the electrical devices and / or the grid may be determined. Normally, determination of participation would require provision of local energy metering data from each home energy system 100 to the DR system provider (e.g. DR system 142) to allow the DR system 142 to evaluate each energy system’s level of participation – i.e. how well each system 100 has met the required response. Transmission of such data may be undesirable for privacy and data protection reasons, as well as due to the consumed transmission bandwidth and requirements for more advanced data transmission and handling functionality in the local control system 108. To address this, a verification system may operate at the local control system 108, another local computer system within the home or at a DR system 142. The verification system may determine participation in the demand response service based on the locally measured energy data, which provides a net flow (consumption or provision) of energy between the grid 120 and the energy system 100. This may avoid the need for transmitting large volumes of data over the data network 140. Furthermore, to reduce the local processing requirements, the determination is based on a machine learning (ML) model which can be applied to local energy data efficiently. Figure 3A illustrates the verification algorithm in overview. The verification algorithm aims to detect whether each energy system ^^ operated correctly to provide the requested demand response service with the requested energy adjustment amount, activation time, and duration. The demand response service may be requested locally, e.g. by the control system 108, or remotely, e.g. by the DR system 142. Taking the privacy concerns into account, the algorithm may also: • be based on local control signals and smart meter data; • be computationally simple as it is to be deployed locally (within the home); • not reveal privacy-sensitive information. The proposed solution may formulate the verification task as a classification problem. Given the local net power consumption / provision ^^୬^^^^^^^(energy data 302) and the requested control^^ୟ^ୡ^^^^^(activation data 304), a local classification model (316) determines whether the latter time-series is contained in the former. The following assumes that each energy system ^^ has a PV (photovoltaic or solar) power system (generator 104) so the measurements obtained from smart meters at each energy system containpower consumption ^^^^୫^^^^ and PV production ^^୮^^ ^^^^. Their difference represents the net power consumption / provision of the energy system 100: ^^^୬^^ ^^^^ ൌ ^^ ^^୫ ^^^^ െ ^^^୮^ ^^^^The net power consumption signal (e.g. energy data 302) ^^୬^^^^^^^ thus comprises a time series of power consumption values. Of course, while described in relation to energy systems with PV power sources, this principle may be applied in energy systems with different or additional energy sources. Thus, the ^^^௩term should be understood to represent the total power supplied by any available generator(s) or other power source(s) (e.g. batteries, petrol generators etc.) The FCR control signals for a particular battery ^^ୟ^ୡ^^^^^may be generated based on 1-secondresolution frequency data for the distribution grid ^^^^^^, using a droop control curve as shown in Figure 2. The droop curve determines the activation power, i.e. the amount of power to be consumed from the grid 120 for charging the battery 106, or the amount of power to be supplied by the battery 106 to the grid 120 or to the energy system 100 to reduce net consumption from the grid 120. The activation signal ^^ୟ^ୡ^^^^^thus comprises a time series of negative or positivepower values defining set points for the battery 106, indicating activation of the battery 106 for discharging or charging respectively. The droop curve depends on the power capacity of the battery 106 ^^^^୫^ୟ^and its frequency activation range ^^^^^^,^^^^୫^ୟ^, which determine the slope ^^^as: Using this slope and the frequency deviation from the reference ^^୰^^(nominal grid frequency,e.g. 50 Hz in Europe), defined as ^^^^^^^^ ൌ ^^^^^^ െ ^^୰^^, the activation power ^^ୟ^ୡ^can be specified as a function of ^^^^^^ as follows: ^^^ୟୡ^ ൫^^^^^^൯ ൌmax൫െ^^^^^୫ୟ^ ,^^^ ⋅ ^^^^^^^^ ^ ^^^^^^൯, ^^^^^^^^ ^ െ^^^^^^, ^min൫^^^^^୫ୟ^ ,^^^ ⋅ ^^^^^^^^ െ ^^^^^^൯, ^^^^^^^^ ^ ^^^^^^, 0, |^^^^| ^ ^^^^^^, The local controller 108 uses the activation signal ^^ୟ^ୡ^to control operation of the energy asset (e.g. battery 106). With reference to Figure 2, within the activation ranges ^^^^^^,^^^^୫^ୟ^, the electrical device (e.g. battery 106) is activated linearly either by charging ^^^^^^(region 206) or discharging െ^^^^^(region 202), until the activation power saturates േ^^^^୫^ୟ^. Otherwise, if the deviation amplitude is within the dead band ^^^^^^(region 204), no dispatch signal is sent to the asset. The verifier is based on a classification model 316 that processes as an input a set of data features 308 derived from the energy data ^^୬^^^302 and the activation data ^^ୟ^ୡ^304 by a feature extractor 306. The classifier 316 generates an output 318 that indicates whether the activation data was detected in the energy data, meaning that the electrical device provided the requested demand response service (compliance), or whether the activation data could not be detected, meaning the energy asset either did not respond at all, or did so insufficiently and / or late (non- compliance). The determination of FCR delivery may be performed at regular intervals ^^^using a fixed time window ^^^^. Therefore, the 1-second timeseries ^^୬^^^^^^^and ^^ୟ^ୡ^^^^^may be sliced into vectorsof length ^^ ൌ ^^^^ ⋅ 60 considering that ^^^^ is in this example expressed in minutes.^^^୬^^ ൌ ^^ ^୬^^ ^^^^ െ ^^^^, ^^^^^^^ୟୡ^ ൌ ^^ ^ୟୡ^ ^^^^ െ ^^^^, ^^^^The resulting vectors represent the energy data and activation data for a given time window of length ^^^^ ending at time ^^^. These vectors provide the input to the feature extractor (306) and classifier (316) of the verification algorithm. The verification algorithm as depicted in Figure 3A is repeated for successive time windows, which may be overlapping (e.g. using a sliding window approach), non-overlapping but contiguous (to process all input samples), or non- contiguous (skipping some input samples between the processed windows, resulting in a verifier that samples the input signals at wider time intervals). As noted above, to allow for use of simpler ML models which reduce local processing requirements, these vectors may not be used directly as inputs, but instead a set of data features (308) may be derived from the vectors comprising the raw data for the current time window in a feature determining step by feature extractor 306. To obtain the data features, time-domain and frequency-domain correlations of the energy data and activation data may be calculated to summarize the input information, and obtain insights related to the delays and variability between signals. These computations produce a set of time- domain correlation features 310 and frequency domain correlation features 312 pertaining to the current time window. Correlations may, for example, be determined using the Pearson correlation coefficient between two arbitrary signals ^^ and ^^. Furthermore, in some examples the Fast Fourier Transform (FFT) algorithm may be used to map any input time-domain vector to the frequency domain ^^^^^^. ^^^^^^ ൌ FFT^^^^^^^^where ^^ is the sample frequency up to the maximum ^^. An additional component that may be considered is ^^^ୠୟ^^, the base energy data, which is simply defined as the expected clean net grid exchange without the activation data. ^^^ ^ୠୟ^^ ൌ ^^୬^^ െ ^^^ୟୡ^The time correlation of the activation data ^^^^௧with the energy data ^^୬^^and the base net demand ^^ୠୟ^^is calculated as: ^^௧ୟୡ^,ୠୟ^^ ൌ corr^^^ୟୡ^,^^ୠୟ^^^This results in two distinct time-domain correlation features (310). Similar correlations are computed in the frequency domain to produce frequency-domain correlation features 312. In an embodiment, a further distinction is established between low and high frequencies. In aspecific example, the division is set to ^^^ ൌ 3.33 mHz (i.e., 5-minute period), which results insix frequency-domain correlation features including correlation of the frequency-transformed activation data and energy data and correlation of the frequency-transformed activation data and base energy data, for the whole frequency range and for each of the upper and lower frequency ranges defined by the division ^^^: ^^^ୟୡ^,୬^^ ൌ corr^^^ୟୡ^,^^୬^^^^^^ୟୡ^,ୠୟ^^ ൌ corr^^^ୟୡ^,^^ୠୟ^^^^^^ି୪୭^ୟୡ^,୬^^ ൌ corr^^^ୟୡ^^0,^^^^, ^^୬^^^0,^^^^^^^^ି୪୭^ୟୡ^,ୠୟ^^ ൌ corr^^^ୟୡ^^0, ^^^^,^^ୠୟ^^^0,^^^^^ With these components a positive verification should lead to activation signals ^^ୟୡ^that show high correlation (close to 1) with the net consumption ^^୬^^, but low correlation (close to 0) with the base consumption ^^ୠୟ^^. In an exemplary embodiment, two additional energy features 314 are used as input features to inform the classifier 316 about the ratio of activation signal to net consumption ^^ୟୡ^,୬^^and to maximum capacity ^^ୟୡ^,୫ୟ^: Here, |⋅|^denotes the 1-norm of the vector. ^^^^௧,^^௫can be considered to give a normalizedactivation (with the denominator ^^ ⋅ ^^^^୫ୟ^ indicating the maximum possible activation over aperiod of ^^ samples). The obtained data features 308 are provided as input to a trained classifier 316, which produces an output classification indicating whether, and optionally to what extent, the (expected) activation data 304 was detected in the energy data 302. In an embodiment, the classification model is a logistic regression classifier. However, other types of machine learning model such as support vector machines, random forest classifiers or neural networks could alternatively be used. The classifier or other machine learning model 316 may output a classification directly or may output a numerical value (e.g. indicating an extent, strength or probability of presence of the activation signal in the net / base energy data) which is converted to a classification by applying a predetermined threshold to the numerical output. In an embodiment, the classifier outputs a binary classification indicating the activation data was present (detected) or absent (not detected) in the energy data, i.e. the classifier outputs one of two possible labels (YES / NO or DETECTED / NOT DETECTED) for each analysed time window of the input signals 302, 304. In the case of the logistic regression model, this computes a probability of a positive classification (indicating detection of the signal) and the positive (“detected”) classification may then be assigned when the probability meets / exceeds some threshold (e.g.50% or 80%). A confidence may be assigned to the classification depending on how close the probability is to 100% (or conversely to 0% for a “not detected” classification). For classifications with low confidence (e.g. close to 50%), further analysis could optionally be performed, e.g. using a more complex algorithm. In some embodiments, the classifier 316 could be extended to provide a more detailed classification 318 with additional possible outputs. In such an implementation, the classification output labels could specify a type of non-compliance, for example using one of the following possible outputs: ^ Non-provision (no DR provision detected) ^ Reduced power provision (DR provision was detected, but at a power level lower than was required) ^ Delayed provision (DR provision at the expected power level was detected, but the provision occurred later than expected) ^ Correct provision (DR provision was detected at the correct power level and time) Alternatively or additionally, the classifier 316 could classify the degree of non-compliance, e.g. as one of a set of percentage bands indicating a detected provision as a percentage of expected provision (in terms of the change in power level required) – for example indicating compliance as one of 0% (non-compliance), 25%, 50%, 75% or 100% (full compliance). As a further alternative, a simply binary classifier could be used initially, and additional analysis performed where the binary classifier output indicates non-compliance, using a follow-up algorithm (either locally at the controller 108, or at the central system by retrieving detailed energy / activation data for the affected time period from the local control system). As noted above, the verification algorithm of Figure 3A may be repeated for successive time windows of the input signals, at regular intervals, resulting in a time series of outputs of the classifier 316. This time series output provides an indication of the moment-to-moment participation of the energy system 100 in the demand response service and allows the participation to be monitored, evaluated and quantified over time. This in turn allows proper analysis of grid performance and the correction of poor grid performance. The size of the time window Δ^^ used for extracting the correlation features may be fixed or may be a configurable parameter. For example, the central system could change this parameter remotely and / or manage it based on current grid conditions. The verification process may be performed by a verification module integrated into the local control system 108, or could be run on a separate computing device located at or remote from the home energy system 100. The verification process performed by the verification module is summarized in Figure 3B. In step 350, the verification module obtains the energy data 302. For example, the energy data 302 may have been stored previously by the local control system 108 whilst performing DR control. The energy data 302 may have been obtained directly from the smart meter 110 or may have been derived from other data obtained from the smart meter 110 (e.g. separate power consumption and power supply measurements ^^ௗ^^and ^^^^as discussed above). In step 352, the verification module obtains the activation data 304. This may again be obtained as a stored activation signal previously generated and stored by the local control system 108 or the DR system 142 when performing DR control based on the measured grid frequency. Alternatively, the expected activation data 304 could be regenerated during verification based on stored grid frequency data using the configured droop curve. In step 354, the verification module analyses a current time window of the energy data 302 and activation data 304 and derives a set of data features 308 characterizing the signals over the time window. It is noted that this step may be undertaken prior to the operation of the classifier and, in some arrangements, may be undertaken as part of the classifier 316. This step may also involve deriving the base energy data from the energy data 302 and activation data 304 and using the base energy data to derive additional features. The features 308 include features indicative of correlation between energy data 302 and / or base energy data on the one hand, and the activation data 304 on the other hand. In step 356, the derived classification features 308 for the current time window are input to the (previously trained) model (classifier 316), which in this case is an ML model. The ML model outputs a verification result indicating whether the activation data 304 is present in the energy data 302, within the currently processed time window, based on the data features 308. It is noted that in other arrangements, the classifier 316 may itself analyse a current time window of the energy data 302 and activation data 304 and derive a set of data features 308 characterizing the signals over the time window as part of the classification. In step 357, the verification module determines whether there is further data (further time windows) to be processed. If yes, then derivation of the classification features 308 and classification by the classifier 316 (steps 354 / 356) are repeated for the next time window of the data. If not, e.g. if the current batch of input data has been fully processed, then the verification analysis ends. In step 358, the verification results 318 for the processed data are output, for example being transmitted to the remote DR system 142 over the network. In other arrangements, the verification may be undertaken by the DR system 142. At step 360, the DR system 142 can further process the verification results output by the classifier as needed to determine whether the electrical devices operated correctly for participation of the energy system 100 in the DR service. In particular, the DR system 142 can determine whether and / or to what extent the energy system 100 provided the expected response at each point in time, and can then further aggregate this information to measure DR participation over extended periods (e.g. as a percentage compliance over a given period such as a day, week or month). The DR system 142 may output compliance data indicating the results of the analysis, for example to summarise compliance over a time period being evaluated. The compliance data derived at the DR system 142 from the verification results may be used for various purposes. As an example, the DR system 142 can use the compliance data to detect technical problems with participants’ energy systems 100 or specifically with the flexibility assets (electrical devices) implementing the DR service. For example, persistent non-compliance, or a sudden change in compliance as indicated by the compliance data may be indicative of a faulty or degraded battery or other technical fault. The system 142 can notify the user of any compliance issues and possible technical faults based on the compliance data. As a further example, the DR system 142 can use compliance data from the energy systems 100 of a cohort of participants to assess portfolio response as a whole, for example to assess whether more and / or different flexibility assets are needed to meet the overall needs of the DR service for the grid 120. The remote DR system 142 can perform the necessary computations for any of these purposes purely based on the verification result generated by the classifier 316, without requiring access to the original energy consumption data and activation signal data, reducing bandwidth requirements and resource requirements for data handling / transmission at the local control system 108, whilst also improving data privacy and enabling participation by users who may not be willing or able to share the source data with the service operator. In one described approach, the processing of the energy data 302 and activation data 304, classification feature extraction 306, 308, and classification 316 based on the classification features 308, are all performed at the local system (energy system 100 / local control system 108). In an alternative embodiment, the energy system 100 could perform only the signal processing and feature extraction steps, transmitting the feature data 308 for each time window to the DR system 142, with the classification and / or other processing performed at the DR system 142. This may enable more advanced analysis methods or classification models to be applied at the DR system 142, whilst still avoiding the need for transmission of the source data (energy consumption signal and activation signal) to the DR system 142, maintaining the bandwidth and privacy advantages at least to some extent. The Figure 3B process may be repeated periodically, e.g. daily, with the local system reporting the processing results for that day’s data to the DR system 142 for further processing and analysis. Model training The machine learning model underlying classifier 316 can be trained on historical grid frequency data and historical energy data (net consumption data) from a plurality of houses or other buildings. Examples of datasets used in a proof of concept (PoC) evaluation, discussed in more detail below, included: ^ Grid Frequency measurements provided by Fingrid, the TSO of Finland: https: / / data.fingrid.fi / en / dataset / frequency-historical-data ^ Household power measurements data from ECO dataset, https: / / rossa-prod- ap21.ethz.ch:8443 / delivery / DeliveryManagerServlet?dps_pid=IE594964 The household power measurements data comprises the energy data and base net demand data. Using a droop curve, such as those discussed earlier and shown in Figure 2, activation data suitable for training the model can be generated from the grid frequency data. The energy data and the activation data are supplied to the feature extractor 306 for determination of relevant data features. The data features may include one or more of: a correlation between the energy data and the activation data in the time domain; a correlation between the energy data and the activation data in the frequency domain; a correlation between the base net demand data (present in the household power measurements data) and the activation data in the time domain; a correlation between the base net demand data and the activation data in the frequency domain; a ratio between the magnitude of the response requested in the activation data and the magnitude of the energy data; and a ratio between the magnitude of the response requested in the activation data and the magnitude of the energy storage capacity of the energy assets in the household. Frequency domain correlations may be performed for different frequency bands, e.g. as discussed above. The determined data features may then be supplied to a machine learning model 316, such as a linear regression, random forest and / or support vector machine model. This may be done, for example, using the Scikit-learn software package. The training may be supervised in that the correct classifications based on the data are known and are used to train the model. In exemplary arrangements, the raw data is fully compliant data, meaning that it is classified as providing the correct demand response that is requested in the activation data. To train the classification model, not only compliant data (positive activations) but also non- compliant data (negative activation examples) may be used. In examples. the set of possible distorted FCR activations is generally much richer than the set of correctly delivered FCR activations. In exemplary embodiments, non-compliant data may be created from the raw data sets of historical energy data and historical activation data. Non-compliant data may comprise data wherein the energy data is not correlated with the activation data, e.g. in the time domain, the frequency domain and / or based on power magnitude. To create non-compliant data, time, frequency and / or magnitude errors may be introduced. For example, the activation data and / or the energy data may be altered such that a time that a demand response is requested is out of synchronization with a corresponding feature of the energy data. The degree of the time error may be determined based on the desired error to be introduced. The time error may be positive or negative. Alternatively or in addition, the activation data and / or the energy data may be altered such that the magnitude of the requested demand response does not align with the magnitude of response requested in the activation data. The degree of the magnitude error may be determined based on the desired error to be introduced. The time error may be positive or negative. For training purposes three basic distortions may be considered: ^ FCR activation is not executed ^ FCR activation is executed with a 50% magnitude reduction of the requested power; ^ FCR activation is executed with a 30-second delay in the delivery. It will be appreciated that other distortions may be introduced into the energy data and / or the activation data. For model training purposes, it may further be assumed that the batteries’ FCR activations are either correct all the time or have the same distortion all the time. The performance of the classifier 316 may be evaluated with a set of unseen cases, i.e., a data set that differs from the data set used to train the classifier. In an embodiment, around a third of the available training samples are held back for use as a test set. Preferably, the training and test sets are from two non-overlapping time periods to avoid any data leakage. In the proof-of- concept implementation (discussed below), timeseries of energy data and activation data from 75 households were divided into a training set of 50 households and a test set of 25 households, using data from different time periods for each set. Implementation example Figure 4A illustrates a hardware and services architecture for an implementation of the described demand response verification system in accordance with an embodiment. The depicted architecture was used to demonstrate the applicability of the verification algorithms in a proof-of-concept (PoC) implementation. This PoC was carried out at the HomeLab in EnergyVille (Belgium) and consists of a single-phase installation that emulates a residential consumer. The system comprises an embedded controller 400 (e.g. implementing the control system 108 of Figure 1) implementing a private flexibility control architecture. The embedded controller 400 is connected to the electricity grid via a smart meter 422, which meters energy supply to various domestic appliances via the household distribution panel. The installation is monitored and controlled by a home energy management system (HEMS) 420, which interfaces with the embedded controller 400. The verification takes places locally using energy data 302 recorded by the smart meter 422. In the example implementation, a standard Belgian smart meter was used, the P1 port of which provides basic measurements with a 1-second granularity. This port takes the Dutch Smart Meter Requirements (DSMR) standard as the base implementation, providing a serial communication through a RJ12 connector located in the chassis and with unrestricted access for the user. An RJ12 to USB converter cable is used to connect the smart meter to a USB port of the embedded controller to read the data stream. Note the smart meter may measure the energy data (^^^^௧^ directly, which can then be used directly for the verification algorithm. Alternatively, the smart meter 422 may measure the consumption (demand) from the grid (^^ௗ^^) and the production, e.g. by the PV system or other generator (^^^^), separately, in which case the controller computes the net consumption as described previously. The grid frequency is obtained using the HEMS 420. The HEMS 420 is used to monitor the local grid based on a series of current and voltage measurements in the distribution panel. The real-time measurements can be accessed by means of an MQTT broker installed on the HEMS 420 under the topic servicelocation / uuid / realtime, which provides 1-second data on the total grid exchanges and the submetered circuits. Therefore, the grid frequency can be read by subscribing to this topic. The controller 400 controls an electrical device or flexibility asset, here implemented as programmable load 440, based on a droop curve as previously described. Droop curves may be configured depending on the flexibility asset. For evaluation purposes for the PoC implementation, each droop curve was created as a random combination of capacities ^^^^୫^ୟ^ൌ ^1,3,5^ ^^^^, and frequency ranges Theimplementation assumed that neither capacity nor the activation range of the flexibility asset changes during its participation. In the PoC, the FCR activation was emulated using a programmable AC load 440 functioning as the flexibility asset. The device has a TCP / IP server which supports the Standard Commands for Programmable Instruments (SCPI) protocol. This allows the controller to select the current consumption level using simple ASCII string commands. Other appliances in the setup can then be operated to generate background consumption. In the PoC, the programmable AC load did not support bidirectional power flow and thus, the frequency regulation was only performed on the right-hand plane of the droop curve. However, for other asset types supporting bidirectional power flow (such as the battery 106 of Figure 1), frequency regulation can be performed across the full droop curve. The embedded controller 400 runs software for implementing the local determination of correct participation in the demand response service. The software architecture consists of a number of services deployed on the embedded controller, each tackling a different part of the verification flow. A reader service 410 collects the data from the P1 port of the smart meter 422 and stores it to a database 412. The service itself was implemented as a thin wrapper around the dsmr_parser library, while the library psycopg2 was used to store the data in the database 412. A controller service 402 comprises an MQTT client 404 that fetches the grid frequency measurements from HEMS 420 using MQTT and supplies them to the database 412 for storage and to a droop curve controller 406 that controls the programmable load 440 by following the configured droop curve, via a load interface 408. The control service relies on the pyvisa package to interface with the load using SCPI via TCP / IP. The verifier service 414 implements the verification algorithm including the feature extractor 306 and classifier 316 described above and illustrated in Figure 3A. At periodic intervals ^^௩(e.g. at intervals of 1 minute), it retrieves the net consumption 302 and activation signals 304 for the time window being evaluated, computes the base consumption signal, computes the classification features 308 and applies the classifier 316 to the features to determine whether the user’s system complied with the FCR activations over the evaluated time window. The verifier service outputs a time series of classification outputs (318) ^^^^^^ indicating compliance or non-compliance at each time instant ^^ (as noted previously, in some embodiments the classification output may indicate more detailed classifications). A database service 412 stores all the measurements and estimations in a local database. In the PoC embodiment this was implemented using TimescaleDB (a database management system optimized for storing time series data) but other types of database may be used. In an embodiment, the data is stored in a set of database tables including tables for smart meter measurements, grid frequency samples, power set points calculated by the droop controller (forming the activation signal ^^^^௧^^^^) and the results of the verification process in the form of the classifier outputs ^^^^^^at each evaluated time instant / window. An external interface 416 provides for communication with the remote demand response management system 142 via appropriate data networks 140 (e.g. wired / wireless) to allow remote configuration of the demand response service (e.g. to configure the droop curve and enable / disable operation of the demand response service) and to retrieve data from the database 412, especially the outputs ^^^^^^ of the verifier service. A monitoring service may additionally be provided locally or in the remote system to allow for the visualization of the different measurements, e.g. using Grafana. For evaluation purposes in the PoC, to emulate not only positive scenarios but also cases in which the FCR service is not properly followed, a distortion module 407 was introduced between the droop controller 406 and the control signal to the programmable load 440. The distortion module 407 introduces artificial distortions into the activation signal, to emulate a sub-optimally performing flexibility asset 440. As noted above, two typical modes for non- compliance with the setpoints specified by the activation signal include 1) a lower power provision than expected, and 2) delayed activation. The distortion module 407 can be configured to apply both distortions to the activation signal independently or simultaneously and / or with different severity. The type of distortion may be applied randomly. Similarly, distortion can be applied all the time or at random times. In the PoC implementation, all services were written in Python and deployed using docker containers on a Raspberry Pi 4 with 4 GB of RAM. The external management system 142 receives the outputs ^^^^^^of the verifier service 414 from the external interface 416 over the network and uses that information to assess compliance with the demand response service as previously described. Figure 4B illustrates a processing device for implementing the local control system 108 of Figure 1 or embedded controller 400 of Figure 4A. The processing device 400 includes one or more processors 450 together with volatile / random access memory 452 for storing temporary data and software code being executed. A network interface 454 is provided for communication with other system components. For example, the processing device may communicate via the network interface with the external DR system 142 via data network 140 (Figures 1 / 4A). Communication may occur over one or more networks (e.g. Local and / or Wide Area Networks, including private networks and / or public networks such as the Internet. I / O interfaces 456 are provided for local input / output to / from locally connected devices. For example, this may include a USB interface and / or other local peripheral interfaces for communicating with the HEMS 420, smart meter 422 and programmable load 440 (Figure 4A). Persistent storage 460 (e.g. in the form of hard disk storage, FLASH memory, optical storage and the like) persistently stores software and data for performing various described functions. In an example, this includes various service modules 462 for implementing the services described in relation to Figure 4A and the database 412 for storing data used by the system, such as the frequency data, consumption data, activation signal data and verification output of the verifier. The persistent storage further includes a computer operating system 464 and any other software and data needed for operating the processing device. The device will include other conventional hardware components as known to those skilled in the art, and the components are interconnected by one or more data buses (e.g. a memory bus and I / O bus). While a specific architecture is shown and described by way of example, any appropriate hardware / software architecture may be employed to implement the system of Figures 4A / 4B. Furthermore, functional components indicated as separate may be combined and vice versa. For example, the various functions may be performed by a single device 400 or may be distributed across multiple devices. Evaluation The following section discusses experimental evaluation of the proof-of-concept implementation shown in Figure 4A. Various performance metrics may be used for evaluation of the classification model. The classifier can incur two types of errors, which impact the practical implementation differently. • Type-I error: misclassifying non-participating customer as participating (false positive); • Type-II error: misclassifying participating customer as non-participating (false negative). Minimising Type-I errors can be important for validation and settlement, as this error can result in penalty fees not being issued to non-participating customers. On the other hand, low Type- II errors are a concern for the privacy considerations since end-users will have to reveal more (possible private) data to prove their participation. A wide range of metrics exists to assess the performance of a classification model. They generally make use of the ratio between correct classification (original label equals predicted class) and incorrect classifications (original label differs from the predicted class). A common approach to define this relationship is with the confusion matrix as shown below. True class positive True class negative Pred. class positive True positive – ^^^^ False positive – ^^^^ Pred. class negative False negative – ^^^^ True negative – ^^^^ In the proof-of-concept case study performed by the inventors, both classes (true activation and distorted activations) were proportionally represented, so accuracy can be a good metric for the overall performance: However, the “precision” and “recall” metrics can be helpful for understanding the types of errors that the model incurs. High precision relates to low Type-I error while high recall indicates low Type-II error. precision ൌ^^^^ ^^^^ ^ ^^^^ Another useful metric is the Area Under the Curve (AUC) in the Receiver Operating Characteristic (ROC) curve. This curve is obtained by plotting the recall or true positive rate against false positive rate (FPR) at various threshold values of the classifier. The closer the AUC to one the higher the rate of correct classifications made by the classifier. ^^^^ ^^^^ FPR ൌ^^^^ ^ ^^^^ ൌ 1 െ specificity ൌ 1 െ^^^^ ^ ^^^^Various experimental evaluations were performed using the above metrics. First, the sensitivity of the classifiers for different ML models (random forest, support vector machine / SVM and logistic regression) was assessed with respect to the duration of the data aggregation windows ΔT={15,30,60} min. Fig.5A shows the performance of the classifier in terms of the performance metrics discussed above. It can be seen that shortening the duration of the aggregation window improves the performance and precision, but recall is negatively affected. On this dataset, the random forest classifier achieves highest scores followed by logistic regression and SVM. Secondly, each trained ML model was evaluated for each distortion with ΔT=60 min. That is, in the simulated dataset half of the households deliver FCR activations correctly and half of them have a given distortion. Fig.5B shows the results in terms of the different performance metrics for magnitude distortions (measured as percentage of requested power change actually delivered) and Fig.5C shows the results for different delay distortions. In both cases, the larger the distortion the better the model performance. The best overall model is again the random forest classifier, closely followed by logistic regression and SVM. The SVM has particularly good precision, but proportionally bad recall. The good relative performance of the logistic regression in all cases is worth mentioning, especially considering its simplicity. Thus, this model was selected for the PoC evaluation. For the PoC evaluation in the laboratory settings a droop curve was selected with ^^^^୫ୟ^ൌ1 kW, and activation in the range ^^^^^, ^^^^୫ୟ^ ൌ 50,100 mHz. We also considered verificationblocks ^^^^ ൌ ^5,15,30,60^ min. With these configurations, four scenarios were generated:without background loads, with magnitude and delay distortions, and with background loads. FCR compliance without background loads This first scenario aimed to provide a sanity check as the only activation is the FCR service during a 24-hour period. Fig. 6(a) illustrates the results for slice of 3 hours. The correlation between frequency and consumption can be appreciated on the top graph. Here, the “frequency” line (grey) shows the frequency deviation ^^^^^^^^from 50 Hz, the “consumption” line (orange) is the real grid consumption, and “set point” line (blue) line represents the theoretical set points for the flexibility asset (corresponding to the activation signal). Fig.7(a) shows the FCR compliance for the different aggregation periods. As can be observed, longer verification periods result in better performance. For this and the following scenarios, Table 1 lists a series of statistics calculated over the test window. These are the mean value (^^), the standard deviation (^^), the 10% and 90% deciles (^^^^,^^ଽ^) and the percentage of output above the 0.5 threshold. Scenario ΔT μ σ Q^^ Qଽ^ ^ 50%Baseline 5 0.94 0.18 0.80 1.00 97.4 15 0.80 0.10 0.99 1.00 99.2 30 1.00 0.00 0.99 1.00 100.0 60 1.00 0.00 1.00 1.00 100.0 50% magnitude 5 0.10 0.19 0.00 0.50 19.3 15 0.04 0.12 0.00 0.02 6.7 30 0.02 0.06 0.00 0.02 1.5 60 0.01 0.01 0.00 0.02 0.0 30s delay 5 0.16 0.26 0.00 0.50 18.7 15 0.33 0.35 0.00 0.91 30.6 30 0.48 0.35 0.03 0.96 48.2 60 0.62 0.30 0.13 0.96 66.0 Background 5 0.86 0.29 0.45 1.00 88.3 15 0.68 0.44 0.00 1.00 68.4 30 0.59 0.36 0.03 1.00 62.3 60 0.42 0.24 0.10 0.79 45.2 Table I: Verification performance for PoC scenarios. Numerical values confirm that the performance is even better than the one predicted by the theorical evaluation of the model (above 99% for the 15-min aggregation). Nevertheless, it should of course be noted that this scenario did not consider any type of distortion or background load. FCR compliance with activation distortions Once the baseline performance of the verifier was established, different distorted activations were tested. The first scenario with distortion considered a partial activation (50%) of the FCR mechanism as could be the case when a battery has conflicting objectives between local self- consumption maximisation and grid services participation. Fig.6(b) and Fig.7(b) show again 3 hours of a 1-day experiment when only half of the capacity was delivered. The verifier tagged most FCR actions as non-compliance. However, we can observe some higher detection probability but barely above the threshold of 0.5 and mostly during periods when no FCR was needed as the frequency deviation was below the threshold. Table 1 confirms the result numerically. For the entire analysed period the verifier outputs a very low mean value. Similarly, the percentage of values above the threshold is low with a 6.7% of values detected as activations for the 15-min period. The second distortion scenario investigated the delayed activation. For this case, the load carried out the activation as calculated by the droop controller but with a delay of 30 seconds. Fig. 6(c) clearly shows this delay between the set point (blue) and the net consumption (orange). As shown by the compliance plot in Fig.7(c) the decisions of the classifier are less clear and are highly affected by the aggregation window size. The numerical results in Table 1 confirm that for short verification windows the classifier tagged the events as non-compliant with relatively high confidence. However, for 30 and 60-min intervals the percentage of false positives increases. FCR compliance with background noise The previous scenarios considered the end-to-end functionality when the load works in isolation. Now we will analyse the performance of the system when other appliances are operating simultaneously. Note that the verifier uses the total consumption measurements from the smart meter. Thus, it has to discern FCR activation from the rest of background consumption. To provide a more challenging test, a load with a varying power was used, in particular a dishwasher that is active for an entire cycle. Furthermore, the FCR service uses activationparameters that are close to the nominal power of the dishwasher. Therefore, we selected ^^^^ ൌ2.3 kW, and range ^^^^^,^^^^୫ୟ^ ൌ 10,200 mHz. The load followed the droop curve between11:30-13:00, while the dishwasher was turned on between 12:05-12:55. Fig.6(d) shows the power consumption and set points, where the overlapping operation of the FCR service and the dishwasher can be seen. Similarly, Fig. 7(d) depicts the verifier estimations where several false negatives were reported for the overlapping periods. In this final scenario, the best results in Table 1 were obtained for the smallest aggregation window^^^^ ൌ 5-min as it was able to reject the raising and falling edges from the background loadactivation more robustly. For the other time windows the accuracy was worse than theoretically calculated. The results demonstrate that simple ML models such as logistic regression suffice to acknowledge the compliance with FCR control commands with a relatively high accuracy. Likewise, when distortions in the activation exist, the model was also able to tag these events as non-compliant although a higher number of false positives were found in this case. The concurrence of background loads with the FCR service activation has proved to be the most challenging scenario, as the verifier is not always able to clearly discern the power generated by the FCR controller, and the power consumed by other loads, especially for long aggregation windows. While a relatively simple verification algorithm based on a logistic regression classifier has been proposed to reduce processing demands at the embedded controller, more complex models could be adopted, using different classification features and / or more advanced ML classifiers. In one approach, features could still be computed locally at the embedded controller (e.g. by feature extractor 306 as shown in Figure 3) to obfuscate the users’ source data, but the calculated features are then sent to the central system which can apply more sophisticated models to verify the participation of the end-user. While described mainly in relation to home energy systems (e.g. operating in houses / apartments / residential premises) the described techniques can be applied in other contexts, e.g. commercial or industrial premises (offices, factories etc.) Generally speaking, the techniques can be applied in any context where energy systems connected to an energy distribution grid participate in a demand response service by varying supply / consumption of one or more flexibility assets. It will be understood that the present invention has been described above purely by way of example, and modification of detail can be made within the scope of the invention.

Claims

CLAIMS 1. A method of determining correct operation of one or more electrical devices of an electrical energy system in the provision of a demand response service, wherein the electrical energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more electrical devices for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the electrical devices and the grid, the method comprising: obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; obtaining activation data indicative of one or more requests for changes in energy flow between at least one electrical device and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; providing the data features as inputs to a trained machine learning model; and determining, using the trained machine learning model, an output indicating whether the electrical devices of the energy system operated to provide the demand response service, based on the data features.

2. A method according to claim 1, wherein: the energy data indicates a time-varying net quantity of energy consumed by the energy system from the grid or provided by the energy system to the grid; and / or the activation data indicates a time-varying quantity of energy to be supplied or consumed by the at least one electrical device for the demand response service.

3. A method according to claim 1 or 2, wherein the energy data and activation data comprise respective time series of electrical power values, optionally wherein the energy data comprises net electrical power flow between the energy system and the grid, and optionally wherein the activation data comprises power values requested to be consumed from and / or supplied to the grid by the energy system.

4. A method according to any of the preceding claims, wherein the energy data and / or the activation data is obtained from the energy system and / or a dynamic response system.

5. A method according to any of the preceding claims, wherein the activation data is generated in response to changes in an operating characteristic of the grid measured at the energy system and / or a dynamic response system, the operating characteristic optionally comprising a grid frequency.

6. A method according to any of the preceding claims, wherein the output comprises: one of two binary classifications indicating whether the demand response has been supplied by the energy system; or one of more than two classifications, each classification indicative of a respective measure, degree or likelihood of whether the demand response has been supplied by the energy system.

7. A method according to claim 6, wherein the output includes one or more of: correct provision of the demand response service; provision of the demand response service at a power level different from the required level specified by the activation data; provision of the demand response service with a delay compared to the activation data; non-provision of the demand response service.

8. A method according to any of the preceding claims, wherein the data features derived from the energy data and the activation data include one or more correlation features indicative of correlation between a time window of the energy data and a time window of the activation data.

9. A method according to claim 8, wherein the one or more correlation features comprise: one or more features indicative of a time-domain correlation between the energy data and the activation data over the time window; and / or one or more features indicative of a frequency domain correlation between the energy data and the activation data over the time window.

10. A method according to claim 9, wherein the time-domain correlation features comprise at least one of: a correlation between the activation data and the energy data over the time window; a correlation between the activation data and base energy data over the time window, wherein the base energy data is based on a difference between the energy data and the activation data.

11. A method according to claim 9 or 10, wherein the frequency-domain correlation features comprise at least one of: a correlation between a frequency domain representation of the activation data and a frequency domain representation of the energy data, over the time window; a correlation between a frequency domain representation of the activation data and a frequency domain representation of base energy data over the time window, the base energy data based on a difference between the energy data and the activation data.

12. A method according to any of claims 9 to 11 comprising performing a frequency transform on the energy data or base energy data and the activation data over the time window, and evaluating a correlation between the resulting transforms.

13. A method according to claim 12, wherein the frequency domain correlation features comprise: one or more correlation features determined across a frequency range of the frequency transform; and / or respective separate correlations of frequency domain input vectors computed for at least two sub-ranges of the frequency range.

14. A method according to any of claims 9 to 13, comprising determining separate frequency domain correlations between the activation data, and the energy data and / or base energy data, for two respective frequency ranges divided at a predetermined threshold frequency.

15. A method according to any of the preceding claims, wherein the data features further comprise features determined based on a total magnitude of consumption or provision and / or a total magnitude of activation data over the time window, optionally comprising:a feature determined based on a ratio of the total magnitude of the activation data and the total magnitude of the energy data over the time window; and / or a feature determined based on a ratio of the total magnitude of the activation data and a magnitude of a maximum possible activation for the one or more electrical devices over the time window.

16. A method according to any of the preceding claims, wherein the trained machine learning model comprises one of: a logistic regression model, a support vector machine, a random forest classifier and a neural network.

17. A method according to any of the preceding claims, wherein the energy system is provided at a building environment connected to the electrical grid.

18. A method according to claim 17, wherein at least the steps of obtaining the energy data and activation data and deriving the set of data features are performed at a local data processing system provided at the building environment, the local data processing system transmitting to a remote management system at least one of: the set of data features, and the verification result.

19. A method according to claim 18, wherein the steps of obtaining the energy data, obtaining the activation data, deriving the set of data features, and providing the data features are performed at the local data processing system, and further comprising transmitting, by the local data processing system, the verification result to the remote management system over a data communications network.

20. A method according to claim 18 or 19, wherein the energy data and / or activation data are processed by the local data processing system without being transmitted to the remote management system.

21. A method according to any of the preceding claims, further comprising repeating the steps of deriving the data features and applying a model for each of a plurality of time windows of the inputs, and determining an output for each time window.

22. A method according to claim 21, comprising analysing the outputs, optionally at a remote management system, to determine a participation of the energy system in the demandresponse service over time, the analysing optionally comprising generating, based on the outputs, one of: a fault indication indicating a possible fault associated with the energy system or the one or more electrical devices; financial compensation information relating to compensation for provision of the demand response service.

23. An apparatus for determining correct operation of one or more electrical devices of an electrical energy system in the provision of a demand response service, wherein the electrical energy system is connected to an electrical energy grid for providing the demand response service by receiving energy from and / or providing energy to the grid, and wherein the energy system comprises one or more electrical devices for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the electrical devices and the grid, the apparatus comprising: an energy data unit for obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; an activation data unit for obtaining activation data indicative of one or more requests for changes in energy flow between at least one electrical device and the grid for provision of the demand response service; and a computer processor configured to: derive a set of data features based on the energy data and the activation data; provide the data features as inputs to a trained machine learning model; and determine, using the trained machine learning model, an output indicating whether the electrical devices of the energy system operated to provide the demand response service, based on the data features.

24. A method of training a machine learning model for determining correct operation of one or more electrical devices of an electrical energy system in the provision of a demand response service, wherein the electrical energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more electrical devices for providing energy to and / or consuming energy from the grid, the energy systembeing configured to provide the demand response service by altering energy flow between one or more of the electrical devices and the grid, the method comprising: receiving an input training set comprising: grid frequency data covering a time period; and energy data covering the time period, the energy data indicating, at least partially, net energy flow between the grid and the electrical energy system; determining activation data based on the grid frequency data, the activation data being indicative of one or more requests for changes in energy flow between at least one electrical device and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; and providing the data features as inputs to the model for training thereof.

25. The method according to claim 24, further comprising: determining correct outputs from the model based on the grid frequency and energy data; and providing the correct outputs to the model for training thereof.

26. The method according to claim 24 or 25, wherein the energy data and activation data represent compliant data in which the requested demand response was provided by the energy system, the method further comprising: creating non-compliant energy data and non-compliant activation data by introducing one or more of time, frequency and magnitude errors into the energy data and / or the activation data; deriving a set of non-compliant data features based on the non-compliant energy data and the non-compliant activation data; and providing the non-compliant data features as inputs to the model for training thereof.

27. The method according to claim 26, wherein substantially half the data used to train the model comprises compliant data, and substantially half the data used to train the model comprises non-compliant data.

28. A computer program, computer program product or non-transitory computer-readable medium comprising software code adapted, when executed by a data processing system, to perform a method as set out in any of claims 1 to 22 and 24.

Images