System for demand response verification

Abstract

An apparatus for verifying provision of a demand response service (142, figure 1) by an electrical energy system (100, figure 1) connected to an electrical energy grid (120, figure 1). The system comprises energy assets (102, 104, figure 1) for providing and / or consuming energy to / from the grid, the system providing the service by altering energy flow between the assets and the grid. The apparatus having an energy data unit (140, figure 1) for obtaining energy data 302 indicating net energy flow between the grid and the system, and an activation data unit for obtaining activation data 304 indicative of one or more requests for changes in energy flow between energy assets and the grid for provision of the service. A computer processor (400, figure 4) is configured to derive a set of data features 308 based on the energy data and the activation data, providing the data features as inputs to a model 316. The model determines a verification result 318 indicative of whether the activation data is present in the energy data based on the data features. A method of use and a method of training a machine learning model are also disclosed.

Description

FIELD OF THE INVENTION The present application relates to methods and systems for verifying provision of a demand response service by an energy system. BACKGROUND OF THE INVENTION Fluctuations in the AC (alternating current) transmission frequency on an electricity distribution / transmission grid can be countered by adjusting power consumption from the grid or power supply into the grid. Specifically, an increase in grid frequency above the standard expected value (e.g. 50 Hz in Europe), also referred to herein as the reference or nominal frequency value, can be countered by increasing consumption or reducing supply, whilst a decrease in grid frequency can be countered by decreasing consumption or increasing supply. In some cases, providers that have suitable energy consuming or energy producing assets (for example, batteries) connected to the grid can contract with the grid operator (e.g. a Transmission System Operator or TSO) to provide a demand response (DR) service (also referred to as a flexibility service) by adjusting supply or consumption of one or more energy assets on demand to counter a frequency fluctuation. Typically, participation in such a demand response service is incentivised financially. For example, where the flexibility assets are energy assets such as batteries installed in individual energy consumer’s homes, this can be in the form of payments (or credits against energy bills). However, this requires the operator of the demand response service to be able to monitor each participant’s energy systems to confirm the level of participation. This typically requires collection of large volumes of energy data from participant systems and transmission of that data to the service operator’s computer systems. This can present a significant burden in terms of data collection / transmission equipment needed at each participant system and the network bandwidth for the transmission of the energy data. Additionally, participants may not be comfortable sharing detailed energy data with the operator. Data protection laws may also restrict this kind of data collection. SUMMARY OF THE INVENTION Embodiments of the invention aim to provide alternative approaches that allow verification of a participant system's compliance with expected demand response actions required for a demand response service. Aspects of the invention are set out in the independent claims. Certain preferred features are set out in the dependent claims. According to the invention in an aspect, there is provided a method of verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the method comprising: obtaining energy data indicating, at least partially, net energy flow between die grid and the energy system; obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; providing the data features as inputs to a model; and determining, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features. Optionally, the energy data indicates a time-varying net quantity of energy consumed by the energy system from the grid or provided by the energy system to the grid; and / or the activation data indicates a time-varying quantity of energy to be supplied or consumed by the at least one energy asset for the demand response service. Optionally, the energy data and activation data comprise respective time series of electrical power values, optionally wherein the energy data comprises net electrical power flow between the energy system and the grid, and optionally wherein the activation data comprises power values requested to be consumed from and / or supplied to the grid by the energy system. Optionally, the energy data and / or the activation data is obtained from the energy system and / or a dynamic response system. Optionally, the activation data is generated in response to changes in an operating characteristic of the grid measured at the energy system and / or a dynamic response system, the operating characteristic optionally comprising a grid frequency. Optionally, the verification result comprises: one of two binary classifications indicating whether the demand response has been supplied by the energy system; or one of more than two classifications, each classification indicative of a respective measure, degree or likelihood of whether the demand response has been supplied by the energy system. Optionally, the verification result includes one or more of: correct provision of the demand response service; provision of the demand response service at a power level different from the required level specified by the activation data; provision of the demand response service with a delay compared to the activation data; non-provision of the demand response service. Optionally, the data features derived from the energy data and the activation data include one or more correlation features indicative of correlation between a time window of the energy data and a time window of the activation data. Optionally, the one or more correlation features comprise: one or more features indicative of a time-domain correlation between the energy data and the activation data over the time window; and / or one or more features indicative of a frequency domain correlation between the energy-data and the activation data over the time window. Optionally, the time-domain correlation features comprise at 1 cast one of: a correlation between the activation data and the energy data over the time window; a correlation between the activation data and base energy data over the time window, wherein the base energy data is based on a difference between the energy data and the activation data. Optionally, die frequency-domain correlation features comprise at least one of: a correlation between a frequency domain representation of the activation data and a frequency domain representation of the energy data, over the time window; a correlation between a frequency domain representation of the activation data and a frequency domain representation of base energy data over the time window, the base energy data based on a difference between the energy data and the activation data. Optionally, the method further comprises performing a frequency transform on the energy data or base energy data and the activation data over the time window, and evaluating a correlation between the resulting transforms. Optionally, the frequency domain correlation features comprise: one or more correlation features determined across a frequency range of the frequency transform; and / or respective separate correlations of frequency domain input vectors computed for at least two sub-ranges of the frequency range. Optionally, the method further comprises determining separate frequency domain correlations between the activation data, and the energy data and / or base energy data, for two respective frequency ranges divided at a predetermined threshold frequency. Optionally, the data features further comprise features determined based on a total magnitude of consumption or provision and / or a total magnitude of activation data over the time window, optionally comprising: a feature determined based on a ratio of the total magnitude of the activation data and the total magnitude of the energy data over the time window; and / or a feature determined based on a ratio of the total magnitude of the activation data and a magnitude of a maximum possible activation for the one or more energy assets over the time window. Optionally, the model comprises one of: a logistic regression model, a support vector machine, a random forest classifier and a neural network. Optionally, the energy system is provided at a building environment connected to the electrical grid. Optionally, at least the steps of obtaining the energy data and activation data and deriving the set of data features are performed at a local data processing system provided at the building environment, the local data processing system transmitting to a remote management system at least one of: the set of data features, and the verification result. Optionally, the steps of obtaining the energy data, obtaining the activation data, deriving the set of data features, and providing the data features are performed at the local data processing system, and further comprising transmitting, by the local data processing system, the verification result to the remote management system over a data communications network. Optionally, the energy data and / or activation data are processed by the local data processing system without being transmitted to the remote management system. Optionally, the method comprises one of repeating the steps of deriving the data features and applying a model for each of a plurality of time windows of the inputs, and outputting verification results for each time window. Optionally, the method comprises analysing the verification results, optionally at a remote management system, to evaluate a participation of the energy system in the demand response service over time, the analysing optionally comprising generating, based on the verification results, one of: a fault indication indicating a possible fault associated with the energy system or the one or more energy assets; financial compensation information relating to compensation for provision of the demand response service. According to the invention in an aspect, there is provided an apparatus for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving energy from and / or providing energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the apparatus comprising: an energy data unit for obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system; an activation data unit for obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; and a computer processor configured to: derive a set of data features based on the energy data and the activation data; provide the data features as inputs to a model; and determine, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features. According to the invention in an aspect, there is provided a method of training a machine learning model for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the method comprising: receiving an input training set comprising: grid frequency data covering a time period; and energy data covering the time period, the energy data indicating, at least partially, net energy flow between the grid and the energy system; determining activation data based on the grid frequency data, the activation data being indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; deriving a set of data features based on the energy data and the activation data; and providing the data features as inputs to the model for training thereof. Optionally, the method comprises: determining correct outputs from the model based on the grid frequency and energy data; and providing the correct outputs to the model for training thereof. Optionally, the energy data and activation data represent compliant data in which the requested demand response was provided by the energy system, the method further comprising: creating non-compliant energy data and non-compliant activation data by introducing one or more of time, frequency and magnitude errors into the energy data and / or the activation data; deriving a set of non-compliant data features based on the non-compliant energy data and the non-compliant activation data; and providing the non-compliant data features as inputs to the model for training thereof. Optionally, substantially half the data used to train the model comprises compliant data, and substantially half the data used to train the model comprises non-compliant data. According to the invention in an aspect, there is provided a computer program, computer program product or non-transiton computer-readable medium comprising software code adapted, when executed by a data processing system, to perform a method as set out herein. The disclosure also encompasses a computer program, computer program product or tangible / non-transitory computer-readable medium comprising software code adapted, when executed by a data processing system, to perform any method as set out above or as described in more detail below. Features of one aspect or example may be applied to other aspects or examples, in any combination. For example, method features may be applied to system or computer program aspects or examples (and vice versa). Features implemented in software may be implemented in hardware and vice versa. BRIEF DESCRIPTION OF THE FIGURES Certain embodiments of the invention will now be described by way of example only, in relation to the Figures, wherein: Figure 1 illustrates a system for verifying demand response participation; Figure 2 shows an example of a control curve for controlling an energy asset as part of a demand response service; Figure 3A shows processing performed by a verification system; Figure 3B illustrates a method for verifying demand response participation; Figures 4A and 4B illustrate software and hardware architectures for a control system; and Figures 5A-5C, 6A-6D and 7A-7D illustrate evaluation results for an implementation of the demand response verification system. DETAILED DESCRIPTION Embodiments of the invention provide a system for verifying demand response participation. This involves verifying if auser’s energy system (e.g. a system of energy, e.g. electrical energy, consuming and / or energy supplying devices installed at a user’s premises) has participated in flexibility provision according to an aggregator’s request, i.e., that a certain amount of power was switched on / off at a specific time, by relying on local measurements and data-driven algorithms which avoid privacy-sensitive information flowing outside the user premises. In the described embodiment, the techniques are applied in the context of home batteries participating in a frequency containment response (FCR) service, which is an exemplary demand response service. FCR (formerly known as RI), is a transmission system operator (TSO) service used to stabilize the frequency of power systems after events that lead to a frequency deviation. Its goal is to limit this frequency deviation within the operational values until the problem is solved, or a slower reserve is activated. Thus, it typically requires activation within a few seconds that is proportional to frequency deviations. However, the described principles may be applied to assets other than batteries and / or to demand response services other than FCR. A system for verifying demand response participation is illustrated in Figure 1. The system includes a home energy system 100 installed in a building environment such as a house, apartment or other dwelling. The home energy system 100 is connected to an electricity grid, which may be a distribution grid or a transmission grid. In the example of Figure 1, the energy system 100 receives energy in the form of electricity from a public electricity distribution grid 120. Electricity is supplied to various energy consuming devices 102 in the home, such as electrical appliances, lights etc. The home energy system 100 may also include a local electricity generator 104, such as a solar panel array, which may supply at least some of the energy needs of the local consumers 102. Figure 1 is purely illustrative and in practice any numbers of consumers and / or generators may be provided. The system may further include one or more batteries 106, as shown in the example of Figure 1. The batten- 106 may be used to store electrical energy, for example excess generation capacity from local generator 104 and / or energy supplied from grid 120 (e.g. during times when energy cost is low). At other times, the battery 106 may also supply previously stored energy back to consumers 102 and / or the grid. Collectively, energy consuming and supplying devices such as consumers 102, generator 104 and battery? 106 are referred to as energy assets. Charging and discharging of battery 106 is under control of a local control system 108, typically in the form of a computer running control software. The control system 108 receives energy data including information on energy that is consumed by the home energy system 100 from the grid 120 (or, in the case of provision of excess local supply to the grid 120, on energy-supplied back to the grid 120), from an energy meter 110. It is generally assumed herein that the local consumption typically exceeds the local supply from generator 104 and / or battery 106 and thus the meter 110 typically measures net positive consumption from the grid 120. However, it is understood that local consumption may be less than the local supply, in which case the energy data may show that energy has been provided to the grid 120. The energy data indicates net energy flow between the grid 120 and the energy system 100. Net energy flow may be positive, negative (i ,e. consumption of energy from a grid or supply of energy to a grid) or zero. Meter readings may be obtained at intervals, e.g. every 30 minutes for a low-resolution meter or every 30 seconds for a high-resolution meter, each meter reading giving a total energy consumption in the last meter interval, e.g. as a power consumption value. Control system 108 further receives a grid frequency measurement from a frequency sensor 112, measuring the frequency of the AC supply on the grid 120. hi the illustrated example, the frequency is measured at the point where the electricity supply enters the home energy system 100. However, alternatively, the sensor 112 could be located outside the home energy system 100, e.g. at a suitable connection point to the grid 120. Frequency measurements may indicate an absolute measured frequency or a frequency deviation from a standard expected frequency (e.g. 50Hz for Europe). Where the received measurement is an absolute value the local control system 108 may compute the frequency deviation, resulting in frequency deviation measurement A / (e.g. for a nominal frequency of 50Hz, an absolute measurement of 49.95 Hz would yield a frequency deviation of -0.05 Hz). During normal operation, the local control system 108 may control charging or discharging of the battery 106 based on local energy needs within the home, for example, to charge the battery 106 during excess generation by generator 104 and supply energy to consumers 102 at times of insufficient local generation. Additional required capacity is then drawn from grid 120. The local control system 108 is additionally configured to make the battery 106 available for participation in an FCR service. The FCR service is managed by an external demand response system 142. When configured to participate in the FCR service, the battery 106 may be controlled to charge or discharge, that is to consume additional electrical energy from the grid 120 or supply electrical energy to the grid 120 (or equivalently reduce overall consumption from the grid by the home energy system 100 by supplying at least some of the required energy directly to local consumers). While the focus here is on using the battery 106 to support FCR service provision, other energy assets such as consumers and generators could also provide this function. Energy assets that are configurable to participate in a demand response or flexibility service are also referred to as flexibility assets. For example, heat pumps, electric heaters or electric vehicle batteries (while connected to a home energy charger) may be used as flexibility assets. In general, any device capable of altering consumption or provision (flow from or to the grid) with a sufficiently rapid response to respond to frequency fluctuations on the grid may be utilised. The local control system 108 obtains the local frequency deviation measured using sensor 112 and controls energy flow to or from the batten’ 106 in order to counter any fluctuation in the measured frequency. Typically, the batten 106 is activated to charge from the grid 120 or discharge to the grid 120 or local energy system 100 whenever the local frequency deviation exceeds an activation threshold. The control response is configured by way of a droop curve, as depicted in Figure 2. Figure 2 shows the required power flow change as a function of measured frequency deviation from the nominal frequency. In this case the droop curve shows no response within a smaller range of frequency deviations between — f0 and f0, with linear responses between —fmax and — f0 and between f0 and fmax respectively. The region 202 between —fmax and — f0 corresponds to discharging of the battery 106 (indicated as a negative power consumption, i.e. power supply, value), while the region 206 between f0 and fmax corresponds to charging of the battery 106 (indicated as a positive power consumption value). Below —fmax and above fmax the power flow adjustment reaches minimum / maximum values (—Pmax and Pmax) so that there is no further decrease / increase in the power flow adjustment (referred to as saturation of the response). These values are typically dependent on the battery type (i.e. the maximum charge or discharge power supported by the battery 106). Note that the depicted droop curve is symmetric and specified by the values f0, fmax, and Pmax. These parameters may be specific to a particular battery 106 / home energy system 100, and different participating home energy systems may be configured with different droop curves. However, other more complex forms of droop curve could be defined (e.g. by defining additional points on the curve) and the curve could be asymmetrical (with different charging and discharging behaviour). Furthermore, the droop curve may take different forms for flexibility assets other than batteries. Returning to Figure 1, the flexibility service is managed by the remote demand response system 142 connected to the local control system via a data network 140. The data network 140 may, for example, include one or more private and / or public networks, such as the Internet. The demand response system 142 may remotely configure the local control system 108 with the droop curve, e.g. by specifying the values f0, fmax and Pmax- Alternatively, the droop curve may be fixed / configured at the local control system 108, for example based on the type of battery or other flexibility asset. When the flexibility service is active, the controller 108 monitors the grid frequency using sensor 112 and determines the deviation from nominal frequency (e.g. 50 Hz in Europe). The controller 108 then determines the required power flow adjustment for the flexibility asset based on the power flow adjustment specified for the measured frequency deviation by the droop curve. As shown in the curve, the adjustment may be negative (for discharging the battery 106), positive (for charging the battery 106) or zero for measured frequency deviations close to zero (the "deadband" region 204 of the droop curve). The battery 106 is then controlled to charge or discharge in accordance with the calculated power flow adjustment value, resulting in increased or decreased net energy data, in this case for consumption from the grid 120. The control loop operates continuously and thus activation of the flexibility asset responds to changes in the grid frequency essentially in real-time. Activation of the flexibil ity asset ceases as soon as the frequency returns sufficiently closely to the nominal frequency (with a frequency deviation in the deadband 204). The demand response system 142 may use any number of home energy systems 100 such as the system of Figure 1 to provide the FCR service for the grid operator. The operator of the demand response system 142 may also be referred to an as “aggregator’ since they provide an aggregate flexibility service (in this case FCR) using energy assets of many home energy systems. In some exemplary7 systems, all or part of the functionality of the local control system 108 maybe undertaken by the DR system 142. For example, the DFR system 142 may receive frequency data to determine whether there has been a deviation in frequency of grid supply. The frequency data may be received by the DR system 142 from the sensor 112 and / or from a remote sensor located outside the energy system 100. In some arrangements, the DR system 142 may determine a required demand response service from the (and optionally further) energy system 100. This may be done using the droop curve described above. In some arrangements, the DR system 142 may instruct the local control system 108 to effect the demand response service in accordance with the determined demand response service. Verification of FCR participation Participation in a flexibility service such as FCR may be incentivised financially, e.g. by-paying a homeowner (or providing an energy discount) for participation. This necessitates accurately measuring whether a particular home energy system 100 actually did participate as expected / requested. Normally, this would require provision of local energy metering data from each home energy system 100 to the DR system provider (e.g. DR system 142) to allow the DR system 142 to evaluate each system’s level of participation - i.e. how well each system 100 has met the required response. Transmission of such data may be undesirable for privacy and data protection reasons, as well as due to the consumed transmission bandwidth and requirements for more advanced data transmission and handling functionality in the local control system 108. To address this, a verification system that may operate at the local control system 108, another local computer system within the home or at a DR system 142, may perform verification of participation based on the locally measured energy data, which provides a net flow (consumption or provision) of energy between the grid 120 and the system 100. This may avoid the need for transmitting large volumes of data over the data network 140. Furthermore, to reduce the local processing requirements, the verification is based on a machine learning (ML) model which can be applied to local energy data efficiently. Figure 3A illustrates the verification algorithm in overview. The verification algorithm aims to detect whether each end-user i provided the requested demand response service with the requested energy adjustment amount, activation time, and duration. The demand response service may be requested locally, e.g. by the control system 108, or remotely, e.g. by the DR system 142. Taking the privacy concerns into account, the algorithm may also: • be based on local control signals and smart meter data; • be computationally simple as it is to be deployed locally (within the home); • not reveal too much privacy-sensitive information. The proposed solution may formulate the verification task as a classification problem. Given the local net power consumption / provision P„et(t) (energy data 302) and the requested control Pact (0 (activation data 304), a local classification model (316) determines whether the latter time-series is contained in the former. The following assumes that each household i has a PV (photovoltaic or solar) power system (generator 104) so the measurements obtained from smart meters at each household contain power consumption PdemCO and PV production Pplv(t)- Their difference represents the net power consumption / provision of the home energy system 100: PnetCO = PdemCO — Ppv(0 The net power consumption signal (e.g. energy data 302) PAet(O thus comprises a time series of power consumption values. Of course, while described in relation to systems with PV power sources, this principle may be applied in systems with different or additional energy sources. Thus, the Ppv term should be understood to represent the total power supplied by any available generator(s) or other power source(s) (e.g. batteries, petrol generators etc.) Hie FCR control signals for a particular batten' are generated based on 1-second resolution frequency data for the distribution grid f(t), using a droop control curve as shown in Figure 2. The droop curve determines the activation power, i.e. the amount of power to be consumed from the grid 120 for charging the battery 106 or the amount of power to be supplied by the battery 106 to the grid 120 or to the local energy system 100 to reduce net consumption from the grid 120. The activation signal Pact(O thus comprises a time series of negative or positive power values defining set points for the battery 106, indicating activation of the battery 106 for discharging or charging respectively. The droop curve depends on the power capacity of the battery 106 dP^ax and its frequency activation range 4 / 0',d / n'ux. which determine the slope ml as: II t, — d / max - <' Using this slope and tire frequency deviation from the reference fref (nominal grid frequency, e.g. 50 Hz in Europe), defined as Af (t) = / (t) — fref, the activation power Pact can be specified as a function of / (t) as follows: ^ct(r(o) = fmax(—dP^ax, ml • Af^t} + Af^, Af(t) <-Aft, <min(dP^ax, m1 • Af(t) - Aft}, Af(t) >Aft, (o, \Af\ <Aft, The local controller 108 uses the activation signal Pact to control operation of the energy asset (e.g. battery’ 106). With reference to Figure 2, within the activation ranges Aft, Aftax, the asset is activated linearly either by charging +dPl (region 206) or discharging —APl (region 202), until the activation power saturates ±dPmax. Otherwise, if the deviation amplitude is within the dead band Aft (region 204), no dispatch signal is sent to the asset. Ure verifier is based on a classification model (316) that processes as an input a set of data features 308 derived from the energy data P„et (302) and the activation data Pact (304) by a feature extractor 306, and generates an output (318) that indicates whether the activation data was detected in the energy data, meaning that the energy asset provided the requested demand response service (compliance), or whether the activation data could not be detected, meaning the energy asset either did respond at all, or did so insufficiently and / or late (non-compliance). The verification of FCR delivery may be performed at regular intervals tv using a fixed time window AT. Therefore, the 1-second timeseries P„et(t) and ^act(0 niay be sliced into vectors of length N = AT • 60 considering that AT is in this example expressed in minutes. Pnet = ^nett^v ~ AT, tv] Pact = ^act[^v — ^T, The resulting vectors represent the energy data and activation data for a given time window of length AT ending at time tv. These vectors provide the input to the feature extractor (306) and classifier (316) of the verification algorithm. The verification algorithm as depicted in Figure 3A is repeated for successive time windows, which may be overlapping (e.g. using a sliding window approach), non-overlapping but contiguous (to process all input samples), or noncontiguous (skipping some input samples between the processed windows, resulting in a verifier that samples the input signals at wider time intervals). As noted above, to allow for use of simpler ML models which reduce local processing requirements, these vectors may not be used directly as inputs, but instead a set of data features (308) may be derived from the vectors comprising the raw data for the current time window in a feature determining step by feature extractor 306. To obtain the data features, time-domain and frequency-domain correlations of the energy data and activation data may be calculated to summarize the input information, and obtain insights related to the delays and variability between signals. These computations produce a set of timedomain correlation features 310 and frequency domain correlation features 312 pertaining to the current time window. Correlations may be determined using the Pearson correlation coefficient between two arbitrary signals x and y. Furthermore, the Fast Fourier Transform (FFT) algorithm may be used to map any input timedomain vector to the frequency domain P[fc], P[ / c] = FFT(p[n]) where k is the sample frequency up to the maximum K. An additional component that may be considered is Pbasc- the base energy data, which is simply defined as the expected clean net grid exchange without the activation data. Phase = Pnet “ Pact The time correlation of the activation data pact with the energy data pnet and the base net demand Phase is calculated as: Pact.net = corr(pact, Pnet) Pact,base COrr(pact, Pbase) This results in two distinct time-domain correlation features (310). Similar correlations are computed in the frequency domain to produce frequency-domain correlation features 312. In an embodiment, a further distinction is established between low and high frequencies. In a specific example, the division is set to kc = 3.33 mHz (i.e., 5-minute period), which results in six frequency-domain correlation features including correlation of the frequency-transformed activation data and energy data and correlation of the frequency-transformed activation data and base energy data, for the whole frequency range and for each of the upper and lower frequency ranges defined by the division kc: Pact.net = corr(Pact< Pnet) Pact,base COrr(Pact, Pbase) Pact,net = corr(Pact[0, / cc],Pnet[0,fcc]) Pact,'base = COrr(Pact[0japbase[0A]) PactS = corr(Pact[fcc + 1,K], Pnet[kc + 1,^]) PfctE = COrr(Pactfc + l.H Phased + 1,^]) With these components a positive verification should lead to activation signals pact that show high correlation (close to 1) with the net consumption pnet, but low correlation (close to 0) with the base consumption pbase. In an exemplary embodiment, two additional energy features 314 are used as input features to inform the classifier about the ratio of activation signal to net consumption ract,net and to maximum capacity ract,max: __ I Pact 11 Pict.net T“ i- I Pnet 11 Here, Mt denotes the 1-norm of the vector. ractimax can be considered to give a normalized activation (with the denominator N ■ dPmax indicating the maximum possible activation over a period of N samples). The obtained data features 308 are provided as input to atrained classifier 316, which produces an output classification indicating whether, and optionally to what extent, the (expected) activation data 304 was detected in the energy data 302. In an embodiment, the classification model is a logistic regression classifier. However, other types of machine learning model such as support vector machines, random forest classifiers or neural networks could alternatively be used. The classifier or other machine learning model 316 may output a classification directly or may output a numerical value (e.g. indicating an extent, strength or probability of presence of the activation signal in the net / base energy data) which is converted to a classification by applying a predetermined threshold to the numerical output. In an embodiment, the classifier outputs a binary classification indicating the activation data was present (detected) or absent (not detected) in the energy data, i.e. tire classifier outputs one of two possible labels (YES / NO or DETECTED / NOT DETECTED) for each analysed time window of the input signals 302,304. In the case of the logistic regression model, this computes a probability of a positive classification (indicating detection of the signal) and the positive (“detected”) classification may then be assigned when the probability meets / exceeds some threshold (e.g. 50% or 80%). A confidence may be assigned to the classification depending on how close the probability is to 100% (or conversely to 0% for a “not detected” classification). For classifications with low confidence (e.g. close to 50%), further analysis could optionally be performed, e.g. using a more complex algorithm. In some embodiments, the classifier could be extended to provide a more detailed classification with additional possible outputs. In such an implementation, the classification output labels could specify a type of non-compliance, for example using one of the following possible outputs: • Non-provision (no DR provision detected) • Reduced power provision (DR provision was detected, but at a power level lower than was required) • Delayed provision (DR provision at the expected power level was detected, but the provision occurred later than expected) • Correct provision (DR provision was detected at the correct power level and time) Alternatively or additionally, the classifier could classify the degree of non-compliance, e.g. as one of a set of percentage bands indicating a detected provision as a percentage of expected provision (in terms of the change in power level required) - for example indicating compliance as one of 0% (non-compliance), 25%, 50%, 75% or 100% (full compliance). As a further alternative, a simply binary classifier could be used initially, and additional analysis performed where the binary classifier output indicates non-compliance, using a follow-up algorithm (either locally at the controller, or at the central system by retrieving detailed energy / activation data for the affected time period from the local control system). As noted above, the verification algorithm of Figure 3A may be repeated for successive time windows of the input signals, at regular intervals, resulting in a time series of outputs of the classifier 316. This time series output provides an indication of the moment-to-moment participation of the energy system in the flexibility service and allows the participation to be monitored, evaluated and quantified over time. The size of the time window AT used for extracting the correlation features may be fixed or may be a configurable parameter. For example, the central system could change this parameter remotely and / or manage it based on current grid conditions. The verification process may be perfonned by a verification module integrated into the local control system 108, or could be run on a separate computing device located at or remote from the home energy system 100. The verification process performed by the verification module is summarized in Figure 3B. In step 350, the verification module obtains the energy data. For example, the energy data may have been stored previously by the local control system 108 whilst performing DR control. The energy data may have been obtained directly from the smart meter or may have been derived from other data obtained from the smart meter (e.g. separate power consumption and power supply measurements Pdem and Ppv as discussed above). In step 352, the verification module obtains the activation data. Tins may again be obtained as a stored activation signal previously generated and stored by the local control system 108 or the DR system 142 when performing DR control based on the measured grid frequency. Alternatively, the expected activation data could be regenerated during verification based on stored grid frequency data using the configured droop curve. In step 354, the verification module analyses a current time window of the energy data and activation data and derives a set of data features characterizing the signals over the time window. It is noted that this step may be undertaken prior to the operation of the classifier and, in some arrangements, may be undertaken as part of the classifier. This step may also involve deriving the base energy data from the energy data and activation data and using the base energy data to derive additional features. Ure features include features indicative of correlation between energy data and / or base energy data on the one hand, and the activation data on the other hand. In step 356, the derived classification features for the current time window are input to the (previously trained) model (classifier 316), which in this case is an ML model. The ML model outputs a verification result indicating whether the activation data is present in the energy data, within the currently processed time window, based on the input features. It is noted that in other arrangements, the classifier 316 may itself analyse a current time window of the energy data and activation data and derive a set of data features characterizing the signals over the time window as part of the classification. In step 357, the verification module determines whether there is further data (further time windows) to be processed. If yes, then derivation of the classification features and classification by the classifier (steps 354 / 356) are repeated for the next time window of the data. If not, e.g. if the current batch of input data has been fully processed, then the verification analysis ends. In step 358, tire verification results for the processed data are output, for example being transmitted to the remote DR system 142 over the network. In other arrangements, the verification may be undertaken by the DR system 142. At step 360, the DR system 142 can further process the verification results output by the classifier as needed to evaluate participation of the energy system 100 in the DR service. In particular, the system 142 can evaluate whether and / or to what extent the energy system 100 provided the expected response at each point in time, and can then further aggregate this information to measure DR participation over extended periods (e.g. as a percentage compliance over a given period such as a day, week or month). The system may output compliance data indicating the results of the analysis, for example to summarise compliance over a time period being evaluated. The compliance data derived at the DR management system 142 from the verification results may be used for various purposes. One usage is for financial settlement, where some sort of compensation is offered to users for participation in the demand response service. This may, for example, involve computing payments / credits (for compliance) and / or penalty fees (for non-compliance) in accordance with any agreed service contract and the obtained compliance data. As another example, the DR management system 142 can use the compliance data to detect technical problems with participants’ energy systems 100 or specifically with the flexibility assets implementing the DR service. For example, persistent non-compliance, or a sudden change in compliance as indicated by the compliance data may be indicative of a faulty or degraded battery or other technical fault. The system 142 can notify the user of any compliance issues and possible technical faults based on the compliance data. As a further example, the DR management system 142 can use compliance data from the energy systems 100 of a cohort of participants to assess portfolio response as a whole, for example to assess whether more and / or different flexibility assets are needed to meet the overall needs of the DR service. The remote DR management system 142 can perform the necessary computations for any of these purposes purely based on the verification result generated by the classifier 316, without requiring access to the original energy consumption data and activation signal data, reducing bandwidth requirements and resource requirements for data handling / transmission at the local control system 108, whilst also improving data privacy and enabling participation by users who may not be willing or able to share the source data with the service operator. In one described approach, the processing of the energy and activation data, classification feature extraction, and classification based on the classification features, are all performed at the local system (home energy system 100 / local control system 108). In an alternative embodiment, the local system could perform only the signal processing and feature extraction steps, transmitting the feature data for each time window to the remote system, with the classification and / or other processing performed at the remote system. This may enable more advanced analysis methods or classification models to be applied at the remote system, whilst still avoiding the need for transmission of the source data (energy consumption signal and activation signal) to the remote system, maintaining the bandwidth and privacy advantages at least to some extent. The Figure 3B process may be repeated periodically, e.g. daily, with the local system reporting the processing results forthat day’s data to the DR system for further processing and analysis. Model training The machine learning model underlying classifier 316 can be trained on historical grid frequency data and historical energy data (net consumption data) from a plurality of houses or other buildings. Examples of datasets used in a proof of concept (PoC) evaluation, discussed in more detail below, included: • Grid Frequency measurements provided by Fingrid, the TSO of Finland: https: / / data.fingrid.fi / en / dataset / frequency -historical-data • Household power measurements data from ECO dataset, https : / / rossa-prod-ap21 .ethz.ch:8443 / delivery / DeliveryManagerServlet?dps_pid=IE594964 The household power measurements data comprises the energy data and base net demand data. Using a droop curve, such as those discussed earlier and shown in Figure 2, activation data suitable for training the model can be generated from the grid frequency data. The energy data and the activation data are supplied to the feature extractor 306 for determination of relevant data features. The data features may include one or more of: a correlation between the energy data and the activation data in the time domain; a correlation between the energy data and the activation data in the frequency domain; a correlation between the base net demand data (present in the household power measurements data) and the activation data in the time domain; a correlation between the base net demand data and the activation data in the frequency domain; a ratio between the magnitude of the response requested in the activation data and the magnitude of the energy data; and a ratio between the magnitude of the response requested in the activation data and the magnitude of the energy storage capacity of the energy assets in the household. Frequency domain correlations may be performed for different frequency bands, e.g. as discussed above. The determined data features may then be supplied to a machine learning model, such as a linear regression, random forest and / or support vector machine model. This may be done, for example, using the Scikit-leam software package. The training may be supervised in that the correct classifications based on the data are known and are used to train the model. In exemplary' arrangements, the raw data is fully compliant data, meaning that it is classified as provided the correct demand response that is requested in the activation data. To train the classification model, not only compliant data (positive activations) but also non-compliant data (negative activation examples) may be used. In examples, the set of possible distorted FCR activations is generally much richer than the set of correctly delivered FCR activations. In exemplary embodiments, non-compliant data may be created from the raw data sets of historical energy data and historical activation data. Non-compliant data may comprise data wherein the energy data is not correlated with the activation data, e.g. in the time domain, the frequency domain and / or based on power magnitude. To create non-compliant data, time, frequency and / or magnitude errors may be introduced. For example, the activation data and / or the energy data may be altered such that a time that a demand response is requested is out of synchronization with a corresponding feature of the energy data. The degree of the time error may be determined based on the desired error to be introduced. Hie time error may be positive or negative. Alternatively or in addition, the activation data and / or the energy data may be altered such that the magnitude of the requested demand response does not align with the magnitude of response requested in the activation data. The degree of the magnitude error may be determined based on the desired error to be introduced. Hie time error may be positive or negative. For training purposes three basic distortions may be considered: • FCR activation is not executed • FCR activation is executed with a 50% magnitude reduction of the requested power; • FCR activation is executed with a 30-second delay in the delivery. It will be appreciated that other distortions may be introduced into the energy data and / or the activation data. For model training purposes, it may further be assumed that the batteries’ FCR activations are either correct all the time or have the same distortion all the time. The performance of the classifier may be evaluated with a set of unseen cases, i.e., a data set that differs from the data set used to train the classifier. In an embodiment, around a third of the available training samples are held back for use as a test set. Preferably, the training and test sets are from two non-overlapping time periods to avoid any data leakage. In the proof-of-concept implementation (discussed below), timeseries of energy data and activation data from 75 households were divided into a training set of 50 households and a test set of 25 households, using data from different time periods for each set. Implementation example Figure 4A illustrates a hardware and services architecture for an implementation of the described demand response verification system in accordance with an embodiment. The depicted architecture was used to demonstrate the applicability of the verification algorithms in a proof-of-concept (PoC) implementation. This PoC was carried out at the HomeLab in EnergyVille (Belgium) and consists of a single-phase installation that emulates a residential consumer. Hie system comprises an embedded controller 400 (e.g. implementing the control system 108 of Figure 1) implementing a private flexibility control architecture. The embedded controller 400 is connected to the electricity grid via a smart meter 422, which meters energy supply to various domestic appliances via the household distribution panel. The installation is monitored and controlled by a home energy management system (HEMS) 420, which interfaces with the embedded controller. The verification takes places locally using the energy data recorded by the smart meter 422. In the example implementation, a standard Belgian smart meter was used, the Pl port of which provides basic measurements with a 1-second granularity. This port takes the Dutch Smart Meter Requirements (DSMR) standard as the base implementation, providing a serial communication through a RJ12 connector located in the chassis and with unrestricted access for the user. An RJ12 to USB converter cable is used to connect the smart meter to a USB port of the embedded controller to read the data stream. Note the smart meter may measure the energy data (Pnet) directly, which can then be used directly for the verification algorithm. Alternatively, the smart meter may measure the consumption (demand) from the grid (Pdem) and the production, e.g. by the PV system or other generator (Ppv), separately, in which case the controller computes the net consumption «0 = ^(0-^ as described previously. The grid frequency is obtained using the HEMS 420. The HEMS is used to monitor the local grid based on a series of current and voltage measurements in the distribution panel. The realtime measurements can be accessed by means of an MQTT broker installed on the HEMS under the topic servicelocation / uuid / realtime, which provides 1-second data on the total grid exchanges and the submetered circuits. Therefore, the grid frequency can be read by subscribing to this topic. The controller controls the flexibility asset, here implemented as programmable load 440, based on a droop curve as previously described. Droop curves may be configured depending on the flexibility asset. For evaluation purposes for the PoC implementation, each droop curve was created as a random combination of capacities dPmax = {1,3,5} kW, and frequency ranges = {[10,100], [20,200]} mHz. Ure implementation assumed that neither capacity nor the activation range of the flexibility asset changes during its participation. In the PoC, the FCR activation was emulated using a programmable AC load 440 functioning as the flexibility asset. The device has a TCP / IP server which supports the Standard Commands for Programmable Instruments (SCPI) protocol. This allows the controller to select the current consumption level using simple ASCII string commands. Other appliances in the setup can then be operated to generate background consumption. In the PoC, the programmable AC load did not support bidirectional power flow and thus, the frequency regulation was only performed on the right-hand plane of the droop curve. However, for other asset types supporting bidirectional power flow (such as the battery 106 of Figure 1), frequency regulation can be performed across the full droop curve. The embedded controller 400 runs software for implementing the local verification of participation in the demand response service. The software architecture consists of a number of services deployed on the embedded controller, each tackling a different part of the verification flow. A reader sendee 410 collects the data from the Pl port of the smart meter 422 and stores it to the database 412. The service itself was implemented as athin wrapper around the dsmr_parser library, while the library psycopg2 w as used to store the data in the database. A controller service 402 comprises an MQTT client 404 that fetches the grid frequency measurements from HEMS 420 using MQTT and supplies them to the database 412 for storage and to a droop curve controller 406 that controls the programmable load 440 by following the configured droop curve, via a load interface 408. The control service relies on the pyvisa package to interface with the load using SCPI via TCP / IP. The verifier service 414 implements the verification algorithm including the feature extractor 306 and classifier 316 described above and illustrated in Figure 3A. At periodic intervals tv (e.g. at intervals of 1 minute), it retrieves the net consumption and activation signals for the time window being evaluated, computes the base consumption signal, computes the classification features and applies the classifier to the features to estimate whether the user’s system complied with the FCR activations over the evaluated time window. The verifier service outputs a time series of classification outputs v(t) indicating compliance or non-compliance at each time instant t (as noted previously, m some embodiments the classification output may indicate more detailed classifications). A database service 412 stores all the measurements and estimations in a local database. In the PoC embodiment this was implemented using TimescaleDB (a database management system optimized for storing time series data) but other types of database may be used. In an embodiment, the data is stored in a set of database tables including tables for smart meter measurements, grid frequency samples, power set points calculated by the droop controller (forming the activation signal Pact(O) and the results of the verification process in the form of the classifier outputs v(t) at each evaluated time instant / window. An external interface 416 provides for communication with the remote demand response management system 142 via appropriate data networks 140 (e.g. wired / wireless) to allow remote configuration of the demand response service (e.g. to configure the droop curve and enable / disable operation of the demand response service) and to retrieve data from the database 412, especially the outputs v(t) of the verifier service. A monitoring service may additionally be provided locally or in the remote system to allow for the visualization of the different measurements, e.g. using Grafana. For evaluation purposes in the PoC, to emulate not only positive scenarios but also cases in which the FCR service is not properly followed, a distortion module 407 was introduced between the droop controller 406 and the control signal to the programmable load 440. The distortion module 407 introduces artificial distortions into the activation signal, to emulate a sub-optimally performing flexibility asset 440. As noted above, two typical modes for non-compliance with the setpoints specified by the activation signal include 1) a lower power provision than expected, and 2) delayed activation. The distortion module 407 can be configured to apply both distortions to the activation signal independently or simultaneously and / or with different severity. The type of distortion may be applied randomly. Similarly, distortion can be applied all the time or at random times. In the PoC implementation, all services were written in Python and deployed using docker containers on a Raspberry- Pi 4 with 4 GB of RAM. The external management system 142 receives the outputs v(t) of the verifier sendee 414 from the external interface 416 over the network and uses that information to assess compliance with the demand response service as previously described. Figure 4B illustrates a processing device for implementing the local control system 108 of Figure 1 or embedded controller 400 of Figure 4A. The processing device 400 includes one or more processors 450 together with volatile / random access memory 452 for storing temporary data and software code being executed. A network interface 454 is provided for communication with other system components. For example, the processing device may communicate via the network interface with the external DR system 142 via data network 140 (Figures 1 / 4A). Communication may occur over one or more networks (e.g. Local and / or Wide Area Networks, including private networks and / or public networks such as the Internet. I / O interfaces 456 are provided for local input / output to / from locally connected devices. For example, this may include a USB interface and / or other local peripheral interfaces for communicating with the HEMS 420, smart meter 422 and programmable load 440 (Figure 4A). Persistent storage 460 (e.g. in the form of hard disk storage, FLASH memory, optical storage and the like) persistently stores software and data for performing various described functions. In an example, this includes various service modules 462 for implementing the services described in relation to Figure 4A and the database 412 for storing data used by the system, such as the frequency data, consumption data, activation signal data and verification output of the verifier. The persistent storage further includes a computer operating system 464 and any other software and data needed for operating the processing device. The device will include other conventional hardware components as known to those skilled in the art, and the components are interconnected by one or more data buses (e.g. a memory bus and I / O bus). While a specific architecture is shown and described by way of example, any appropriate hardware / software architecture may be employed to implement the system of Figures 4A / 4B. Furthermore, functional components indicated as separate may be combined and vice versa. For example, the various functions may be performed by a single device 400 or may be distributed across multiple devices. Evaluation The following section discusses experimental evaluation of the proof-of-concept implementation shown in Figure 4A. Various performance metrics may be used for evaluation of the classification model. Tire classifier can incur two types of errors, which impact the practical implementation differently. • Type-I error: misclassifying non-participating customer as participating (false positive); • Type-II error: misclassifying participating customer as non-participating (false negative). Minimising Type-1 errors can be important for validation and settlement, as this error can result in penalty fees not being issued to non-participating customers. On the other hand, low Type-II errors are a concern for the privacy considerations since end-users will have to reveal more (possible private) data to prove their participation. A wide range of metrics exists to assess the performance of a classification model. They generally make use of the ratio between correct classification (original label equals predicted class) and incorrect classifications (original label differs from the predicted class). A common approach to define this relationship is with the confusion matrix as shown below. True class positive True class negative Pred. class positive True positive - TP False positive - FP Pred. class negative False negative - FN True negative TN In the proof-of-concept case study performed by the inventors, both classes (true activation and distorted activations) were proportionally represented, so accuracy can be a good metric for the overall performance: TP + TN accuracy = rp + rjV + Fp + FJV However, the “precision” and “recall” metrics can be helpful for understanding the types of errors that the model incurs. High precision relates to low Type-I error while high recall indicates low Type-II error. TP precision =-------- 1 TP + FP TP recall =--------- pp _|_ p pi Another useful metric is the Area Under the Curve (AUC) in the Receiver Operating Characteristic (ROC) curve. This curve is obtained by plotting the recall or true positive rate against false positive rate (FPR) at various threshold values of the classifier. The closer the AUC to one the higher tine rate of correct classifications made by the classifier. FPR = TP TN + FP TN = 1 — specificity = 1-- 1 7 TN + FP Various experimental evaluations were performed using the above metrics. First, the sensitivity of the classifiers for different ML models (random forest, support vector machine / SVM and logistic regression) was assessed with respect to the duration of the data aggregation windows AT={ 15,30,60} min. Fig. 5A shows the performance of the classifier in terms of the performance metrics discussed above. It can be seen that shortening the duration of the aggregation window improves the performance and precision, but recall is negatively affected. On this dataset, the random forest classifier achieves highest scores followed by logistic regression and SVM. Secondly, each trained ML model was evaluated for each distortion with AT=60 min. That is, in the simulated dataset half of the households deliver FCR activations correctly and half of them have a given distortion. Fig. 5B shows the results in terms of the different performance metrics for magnitude distortions (measured as percentage of requested power change actually delivered) and Fig. 5C shows the results for different delay distortions. In both cases, the larger the distortion the better the model performance. The best overall model is again the random forest classifier, closely followed by logistic regression and SVM. The SVM has particularly good precision, but proportionally bad recall. The good relative performance of the logistic regression in all cases is worth mentioning, especially considering its simplicity. Thus, this model was selected for the PoC evaluation. For the PoC evaluation in the laboratory settings a droop curve was selected with 4Pmax = 1 kW, and activation in the range Af0, Afmax — 50,100 mHz. We also considered verification blocks AT = {5,15,30,60} min. With these configurations, four scenarios were generated: without background loads, with magnitude and delay distortions, and with background loads. FCR compliance without background loads This first scenario aimed to provide a sanity check as the only activation is the FCR service during a 24-hour period. Fig. 6(a) illustrates the results for slice of 3 hours. The correlation between frequency and consumption can be appreciated on the top graph. Here, the “frequency” line (grey) shows the frequency deviation Af(t) from 50 Hz, the “consumption” line (orange) is the real grid consumption, and “set point” line (blue) line represents the theoretical set points for the flexibility asset (corresponding to the activation signal). Fig. 7(a) shows tire FCR compliance for the different aggregation periods. As can be observed, longer verification periods result in better performance. For this and the following scenarios, Table 1 lists a series of statistics calculated over the test window. These are the mean value (jt), the standard deviation (a), the 10% and 90% deciles (Q10, Qyo) and the percentage of output above the 0.5 threshold. Scenario AT p o Qio Q90 > 50% Baseline 5 0.94 0.18 0.80 1.00 97.4 15 0.80 0.10 0.99 1.00 99.2 30 1.00 0.00 0.99 1.00 100.0 60 1.00 0.00 1.00 1.00 100.0 50% magnitude 5 0.10 0.19 0.00 0.50 19.3 15 0.04 0.12 0.00 0.02 6.7 30 0.02 0.06 0.00 0.02 1.5 60 0.01 0.01 0.00 0.02 0.0 30s delay 5 0.16 0.26 0.00 0.50 18.7 Scenario AT R o Qio Q90 > 50% 15 0.33 0.35 0.00 0.91 30.6 30 0.48 0.35 0.03 0.96 48.2 60 0.62 0.30 0.13 0.96 66.0 Background 5 0.86 0.29 0.45 1.00 88.3 15 0.68 0.44 0.00 1.00 68.4 30 0.59 0.36 0.03 1.00 62.3 60 0.42 0.24 0.10 0.79 45.2 Table I: Verification performance for PoC scenarios. Numerical values confirm that the performance is even better than the one predicted by the theorical evaluation of the model (above 99% for the 15-min aggregation). Nevertheless, it should of course be noted that this scenario did not consider any type of distortion or background load. FCR compliance with activation distortions Once the baseline performance of the verifier was established, different distorted activations were tested. The first scenario with distortion considered a partial activation (50%) of the FCR mechanism as could be the case when a battery has conflicting objectives between local selfconsumption maximisation and grid services participation. Fig. 6(b) and Fig. 7(b) show again 3 hours of a 1-day experiment when only half of the capacity was delivered. Hie verifier tagged most FCR actions as non-compliance. However, we can observe some higher detection probability but barely above the threshold of 0.5 and mostly during periods when no FCR was needed as the frequency deviation was below the threshold. Table 1 confirms the result numerically. For the entire analysed period the verifier outputs a very low mean value. Similarly, the percentage of values above the threshold is low with a 6.7% of values detected as activations for the 15-min period. The second distortion scenario investigated the delayed activation. For this case, the load carried out the activation as calculated by the droop controller but with a delay of 30 seconds. Fig. 6(c) clearly shows this delay between the set point (blue) and the net consumption (orange). As shown by the compliance plot in Fig. 7(c) the decisions of the classifier are less clear and are highly affected by the aggregation window size. The numerical results in Table 1 confirm that for short verification windows the classifier tagged the events as non-compliant with relatively high confidence. However, for 30 and 60-min intervals the percentage of false positives increases. FCR compliance with background noise The previous scenarios considered the end-to-end functionality when the load works in isolation. Now we will analyse the performance of the system when other appliances are operating simultaneously. Note that the verifier uses the total consumption measurements from the smart meter. Thus, it has to discern FCR activation from the rest of background consumption. To provide a more challenging test, a load with a varying power was used, in particular a dishwasher that is active for an entire cycle. Furthermore, the FCR service uses activation parameters that are close to the nominal power of the dishwasher. Therefore, we selected AP = 2.3 kW, and range Af0,Afmax = 10,200 mHz. The load followed the droop curve between 11:30-13:00, while the dishwasher was turned on between 12:05-12:55. Fig. 6(d) shows the power consumption and set points, where the overlapping operation of the FCR service and the dishwasher can be seen. Similarly, Fig. 7(d) depicts the verifier estimations where several false negatives were reported for the overlapping periods. In this final scenario, the best results in Table 1 were obtained for the smallest aggregation window AT = 5-min as it was able to reject the raising and falling edges from the background load activation more robustly. For the other time windows the accuracy was worse than theoretically calculated. The results demonstrate that simple ML models such as logistic regression suffice to acknowledge the compliance with FCR control commands with a relatively high accuracy. Likewise, when distortions in the activation exist, the model was also able to tag these events as non-compliant although a higher number of false positives were found in this case. Hie concurrence of background loads with the FCR service activation has proved to be the most challenging scenario, as the verifier is not always able to clearly discern the power generated by the FCR controller, and the power consumed by other loads, especially for long aggregation windows. While a relatively simple verification algorithm based on a logistic regression classifier has been proposed to reduce processing demands at the embedded controller, more complex models could be adopted, using different classification features and / or more advanced ML classifiers. In one approach, features could still be computed locally at the embedded controller 5 (e.g. by feature extractor 306 as shown in Figure 3) to obfuscate the users’ source data, but the calculated features are then sent to the central system which can apply more sophisticated models to verify the participation of the end-user. While described mainly in relation to home energy systems (e.g. operating in 10 houses / apartments / residential premises) the described techniques can be applied in other contexts, e.g. commercial or industrial premises (offices, factories etc.) Generally speaking, the techniques can be applied in any context where energy systems connected to an energy distribution grid participate in a demand response service by varying supply / consumption of one or more flexibility assets. 15 It will be understood that the present invention has been described above purely by way of example, and modification of detail can be made within the scope of the invention.

Claims

1. A method of verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid,and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid,the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid,the method comprising:obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system;obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service;deriving a set of data features based on the energy data and the activation data; providing the data features as inputs to a model; anddetermining, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features.

2. A method according to claim 1, wherein:the energy data indicates a time-varying net quantity of energy consumed by the energy system from the grid or provided by the energy system to the grid; and / orthe activation data indicates a time-varying quantity of energy to be supplied or consumed by the at least one energy asset for the demand response service.

3. A method according to claim 1 or 2, wherein the energy data and activation data comprise respective time series of electrical power values, optionally wherein the energy data comprises net electrical power flow between the energy system and the grid, and optionally wherein the activation data comprises power values requested to be consumed from and / or supplied to the grid by the energy system.

4. A method according to any of the preceding claims, wherein tire energy data and / or the activation data is obtained from the energy system and / or a dynamic response system.

5. A method according to any of the preceding claims, wherein the activation data is generated in response to changes in an operating characteristic of the grid measured at the energy system and / or a dynamic response system, the operating characteristic optionally comprising a grid frequency.

6. A method according to any of the preceding claims, wherein the verification result comprises:one of two binary classifications indicating whether the demand response has been supplied by the energy system; orone of more than two classifications, each classification indicative of a respective measure, degree or likelihood of whether the demand response has been supplied by the energy system.

7. A method according to claim 6, wherein the verification result includes one or more of: correct provision of the demand response service;provision of the demand response service at a power level different from the required level specified by the activation data;provision of the demand response service with a delay compared to the activation data; non-provision of the demand response service.

8. A method according to any of the preceding claims, wherein tire data features derived from the energy data and the activation data include one or more correlation features indicative of correlation between a time window of the energy data and a time window of the activation data.

9. A method according to claim 8, wherein the one or more correlation features comprise: one or more features indicative of a time-domain correlation between the energy data and the activation data over the time window; and / orone or more features indicative of a frequency domain correlation between the energy data and the activation data over the time window.

10. A method according to claim 9, wherein the time-domain correlation features comprise at least one of:a correlation between the activation data and the energy data over the time window;a correlation between the activation data and base energy data over the time window, wherein the base energy data is based on a difference between the energy data and the activation data.

11. A method according to claim 9 or 10, wherein the frequency-domain correlation features comprise at least one of:a correlation between a frequency domain representation of the activation data and a frequency domain representation of the energy data, over the time window;a correlation between a frequency domain representation of the activation data and a frequency domain representation of base energy data over the time window, the base energy data based on a difference between the energy data and the activation data.

12. A method according to any of claims 9 to 11 comprising performing a frequency transform on the energy data or base energy data and the activation data over the time window, and evaluating a correlation between the resulting transforms.

13. A method according to claim 12, wherein the frequency domain correlation features comprise:one or more correlation features determined across a frequency range of the frequency transform; and / orrespective separate correlations of frequency domain input vectors computed for at least two sub-ranges of the frequency range.

14. A method according to any of claims 9 to 13, comprising determining separate frequency domain correlations between the activation data, and the energy data and / or base energy data, for two respective frequency ranges divided at a predetermined threshold frequency.

15. A method according to any of the preceding claims, wherein the data features further comprise features determined based on a total magnitude of consumption or provision and / or a total magnitude of activation data over the time window, optionally comprising:a feature determined based on a ratio of the total magnitude of the activation data and the total magnitude of the energy data over the time window; and / ora feature determined based on a ratio of the total magnitude of the activation data and a magnitude of a maximum possible activation for the one or more energy assets over the time window.

16. A method according to any of the preceding claims, wherein the model comprises one of: a logistic regression model, a support vector machine, a random forest classifier and a neural network.

17. A method according to any of the preceding claims, wherein the energy system is provided at a building environment connected to the electrical grid.

18. A method according to claim 17, wherein at least the steps of obtaining the energy data and activation data and deriving the set of data features are performed at a local data processing system provided at the building environment, the local data processing system transmitting to a remote management system at least one of: the set of data features, and the verification result.

19. A method according to claim 18, wherein the steps of obtaining the energy data, obtaining the activation data, deriving the set of data features, and providing the data features are performed at the local data processing system,and further comprising transmitting, by the local data processing system, the verification result to the remote management system over a data communications network.

20. A method according to claim 18 or 19, wherein the energy data and / or activation data are processed by the local data processing system without being transmitted to the remote management system.

21. A method according to any of the preceding claims, further comprising repeating the steps of deriving the data features and applying a model for each of a plurality of time windows of the inputs, and outputting verification results for each time window.

22. A method according to claim 21, comprising analysing the verification results, optionally at a remote management system, to evaluate a participation of the energy system in the demand response service over time, the analysing optionally comprising generating, based on the verification results, one of:a fault indication indicating a possible fault associated with the energy system or the one or more energy assets;financial compensation information relating to compensation for provision of the demand response service.

23. An apparatus for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving energy from and / or providing energy to the grid,and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid,the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid,the apparatus comprising:an energy data unit for obtaining energy data indicating, at least partially, net energy flow between the grid and the energy system;an activation data unit for obtaining activation data indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service; anda computer processor configured to:derive a set of data features based on the energy data and the activation data; provide the data features as inputs to a model; anddetermine, using the model, a verification result indicative of whether the activation data is present in the energy data based on the data features.

24. A method of training a machine learning model for verifying provision of a demand response service by an electrical energy system, wherein the energy system is connected to an electrical energy grid for providing the demand response service by receiving electrical energy from and / or providing electrical energy to the grid, and wherein the energy system comprises one or more energy assets for providing energy to and / or consuming energy from the grid, the energy system being configured to provide the demand response service by altering energy flow between one or more of the energy assets and the grid, the method comprising:receiving an input training set comprising:grid frequency data covering a time period; andenergy data covering the time period, the energy data indicating, at least partially, net energy flow between the grid and the energy system;determining activation data based on the grid frequency data, the activation data being indicative of one or more requests for changes in energy flow between at least one energy asset and the grid for provision of the demand response service;deriving a set of data features based on the energy data and the activation data; and providing the data features as inputs to the model for training thereof.

25. The method according to claim 24, further comprising:determining correct outputs from the model based on the grid frequency and energy-data; andproviding the correct outputs to the model for training thereof.

26. The method according to claim 24 or 25, wherein the energy data and activation data represent compliant data in which the requested demand response was provided by the energy-system, the method further comprising:creating non-compliant energy data and non-compliant activation data by introducing one or more of time, frequency and magnitude errors into the energy data and / or the activation data;deriving a set of non-compliant data features based on the non-compliant energy data and the non-compliant activation data; andproviding the non-compliant data features as inputs to the model fortraining thereof.

27. The method according to claim 26, wherein substantially half the data used to train the model comprises compliant data, and substantially half the data used to train the model comprises non-compliant data.

28. A computer program, computer program product or non-transitory- computer-readable medium comprising software code adapted, when executed by a data processing system, to perform a method as set out in any of claims 1 to 22 and 24.41