A method of extracting appliance fingerprints from aggregated load data

Abstract

The present application belongs to the technical field of non-intrusive load decomposition, and particularly relates to a method for extracting an electrical appliance fingerprint from aggregated load data. The present application improves the existing neural network used for the non-intrusive load monitoring (NILM) task, proposes a concurrent load decoupler network model to extract the electrical appliance fingerprint. The model contains a residual position sub-network sequence, and adds a multi-head self-attention layer at the end of the sequence. A global pooling layer is applied to map the input matrix to an electrical appliance fingerprint vector. Finally, the corresponding electrical appliance label is output through a fully connected feedforward network layer, so as to extract the electrical appliance fingerprint from the aggregated load data and determine the running condition of the electrical appliance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of non-intrusive load decomposition, and particularly relates to a method for extracting an electrical appliance fingerprint from aggregated load data. BACKGROUND

[0002] The electric meter can only report the total power load data of the family, so inferring the power load of a specific device and performing power load decomposition are the main research goals at the present stage, and how to use deep learning related methods to improve the accuracy of load decomposition is a popular research direction at present. The network model at the present stage is limited by the number of concurrent electrical appliance loads in the public data set, and the aggregated information has strong distribution, which leads to low robustness of the existing network model, and the load decomposition accuracy is also low, so there is currently a lack of a network model capable of extracting features from aggregated data of more concurrent electrical appliances. SUMMARY

[0003] The application aims to provide a method for extracting an electrical appliance fingerprint from aggregated load data, to solve the problems of difficulty in disaggregating the existing aggregated load data in the NILM field and low accuracy of electrical appliance fingerprint extraction,

[0004] The method for extracting an electrical appliance fingerprint from aggregated load data provided by the application improves the existing neural network for the task of non-intrusive load monitoring (Non-Intrusive Load Monitoring, NILM), and proposes a concurrent load decoupler (Concurrent Loads Disaggregator, CoLD) network model for electrical appliance fingerprint extraction. The model includes a sequence of residual position-wise sub-networks (Residual Position-Wise Sub-Networks, RPSN), and adds a multi-head self-attention layer at the end of the sequence, applies a global pooling layer to map the input matrix to an electrical appliance fingerprint vector, and finally outputs the corresponding electrical appliance label through a fully connected feedforward network layer, realizing the extraction of electrical appliance fingerprints from aggregated load data to determine the running condition of electrical appliances, and improving the accuracy of the network model.

[0005] The method for extracting an electrical appliance fingerprint from aggregated load data provided by the application, the specific steps are:

[0006] Step 1: Synthesize an aggregated load data set, and divide it into training, validation and test sets;

[0007] Step 2: Establish the main component of the model, the residual position-wise sub-network;

[0008] Step 3: Build a concurrent load decoupler network model and determine the input and output formats;

[0009] Step 4: Hyperparameter search is performed to find the most suitable parameter space;

[0010] Step 5: Aggregate data decomposition is performed to extract the appliance fingerprint and determine the running state of the appliance.

[0011] The following further describes each step.

[0012] Step 1: Synthesize the aggregate load data set, and divide the training, validation, and test sets

[0013] The existing public data sets in the NILM field have problems such as lack of unified standardization and small number of maximum concurrent loads. In order to ensure that the method of the present application can achieve better decomposition effect, it is necessary to construct an appliance fingerprint aggregate data set with rich types.

[0014] First, select a public data set as the initial data source, normalize the public data set to ensure that the appliance load signals in different measurement environments can be combined, then determine the types of appliances in the aggregate data set, combine the normalized public data set to obtain a basic data set, and finally determine the maximum number of concurrent loads, synthesize the aggregate signal data set, and divide the training, validation, and test sets.

[0015] Step 2: Establish the main component residual position subnetwork of the model;

[0016] ① Position-wise feedforward layers

[0017] The position-wise feedforward layers used in the present application are established based on the idea proposed in "Attention is all you need", mainly consisting of two fully connected layers and a normalization layer, and using a ReLU activation function.

[0018] The input is a spectrogram matrix X0(t*v), which is obtained by Fourier transform of the aggregate load data, each row is a time slice, i.e. the first dimension t is related to the sequential nature of the data (signal duration); each column is a frequency window, i.e. the second dimension v is related to the feature (Fourier transform result), and the matrix X0 represents a period of aggregate load data.

[0019] The calculation of the specific j position layer is represented as:

[0020] h j (X j-1 )=X j-1 ·W j T -Θ j

[0021] Where X j-1The activation output of the j-1 layer; W j The input matrix X of the j layer is projected onto a q-dimensional space, q being the number of input neuron nodes; Θ j-1 The input matrix X of the j layer is projected onto a q-dimensional space, q being the number of input neuron nodes; Θ j The bias matrix; the output of this part in the network The j layer output is represented as an unactivated j layer output, and the final j layer activation output is obtained through a ReLU activation function That is:

[0022]

[0023] ② Network structure of the residual position subnetwork

[0024] The first layer of the residual position subnetwork is the position feedforward layer mentioned above, followed by a ReLU activation function, and then a position feedforward layer, followed by a normalization layer, which is a fully connected layer, and finally a ReLU activation function; The residual network connection contained in it connects the input r(X j-1 of the residual position subnetwork to the normalization layer, where the data is added and normalized to obtain the non-activated input of the next residual position subnetwork module is represented as:

[0025]

[0026] Step 3, build a concurrent load decoupler network model and determine the input and output formats; The specific process is as follows:

[0027] ① Build a residual position subnetwork sequence

[0028] The k-layer residual position subnetwork module is connected in series in the concurrent load decoupler network model to form a residual position subnetwork sequence, each layer is recorded as RPSN 2i+1 , i∈(1, k]. There is a separate position feedforward layer at the head of the concurrent load decoupler network, which maps the input spectral graph matrix X0 to a q-dimensional space to form the input h1(X0) of the first layer RPSN3 and the residual module input r(X0).

[0029] ② Set up a multi-head self-attention layer

[0030] The multi-head self-attention layer is connected after the RPSN 2k+1 module, and the purpose is to summarize the information in the matrix X j+1 time dimension obtained through the residual position subnetwork sequence, and jointly process the information from different representation subspaces in different positions. The multi-head self-attention layer performs a scaled dot-product attention function in parallel time, and then connects and projects the results to another learnable space, represented as:

[0031]

[0032]

[0033] where Q represents the queried information, i.e., the fingerprint of the relevant appliance, K represents the attribute related to the fingerprint of the appliance, such as the voltage and current waveform, and V represents the content vector obtained according to the current query and the key attribute, and in the self-attention of the application, Q=K=V=X j+1 ; is the projection matrix of each head, respectively, is the output projection matrix; the calculation of the scaled dot-product attention function in the multi-head self-attention mechanism is realized by the following function:

[0034]

[0035] where, is the soft-max output value to avoid the saturation (minimum gradient occurs) region thereof.

[0036] ③ Global pooling and classification

[0037] The role of the global pooling and classification layer is to convert the feature matrix X into an output vector x, and the vector x contains the activated label (device) l in the given time window. The expectation E is introduced to process the input matrix to realize dimension reduction, so the global pooling is represented as:

[0038] g: R t×q → R q

[0039] g(X j-1 )=E m [(X j-1 ) m,n ]=x j-1

[0040] At the same time, another nonlinear transformation is introduced to reduce the features represented by the vector x j-1 , and the classification output is:

[0041]

[0042] where W Pred ∈ R q×|L| , |L| represents the number of all appliance categories in the current data set, and a learnable vector a is used to adjust the Sigmoid function to realize the output of a specific label class:

[0043]

[0044] ④ Form representation of the concurrent load decoupler network structure

[0045] The dropout function is also involved in the overall structure of the network:

[0046] h d,j = Dropout j (h j (X j-1 ))

[0047] The residual position subnetwork can be represented by the following function form:

[0048]

[0049] Therefore, the concurrent load decoupler overall network can be represented as:

[0050]

[0051] Step 4, perform hyperparameter search to find the most suitable parameter space;

[0052] According to the concurrent load decoupler network model structure described above, the parameter space is <q, k, n head , p d >, wherein p d is the screening probability of the dropout function. In the present application, the ASHA algorithm is used to iterate the parameter space and different learning rate parameters {η}, weight decay {λ} and batch size {B} in the hyperparameter search process, which adopts an aggressive strategy to stop training weaker models than other models. Therefore, compared with the standard grid search or random search strategy, hundreds of training processes for different hyperparameters can be completed very quickly.

[0053] In the hyperparameter search process, the network model under various different hyperparameters is evaluated to find the model with the best load decomposition effect. In the present application, the F1-score method is used to evaluate the decomposition effect of different models, and the judgment formula of the method is The final result is the harmonic mean of the precision PR w and recall RE w of different parameter subsets S w :

[0054]

[0055] Wherein, #· represents the cardinality |·|; the probability value in the prediction vector is whether it is a specific label, so a threshold should be set, when the prediction probability is greater than the threshold, it is represented that the label is activated in the current signal, recorded as 1, otherwise 0. In order to find an optimal parameter set, the tuning process is performed to calculate the weighted average of F1 score in the interval (0, 1) threshold, and then the parameters under the threshold setting with the highest weighted average are selected as the final parameter space set.

[0056] Step 5 carries out the polymeric data decomposition, extracts the electric appliance fingerprint, and judges the electric appliance running state

[0057] According to the network model building and the optimal parameter space searching according to the above steps, the optimized concurrent load decoupler network model can be obtained, the data decomposition of the multi-concurrent electric appliance load signal can be carried out, the electric appliance fingerprint can be extracted, the current running state of the electric appliance can be judged, the running state of the known household electric appliance can be judged according to the electric meter data of the household, and the electric appliance running under the current load can be identified.

[0058] Compared with the prior art, the beneficial effects of the present application are as follows:

[0059] The present application improves the existing neural network used for the non-intrusive load monitoring task, proposes a concurrent load decoupler network model for extracting the electric appliance fingerprint. The model includes a residual position sub-network sequence, and a multi-head self-attention layer is added at the end of the sequence. A global pooling layer is applied to map the input matrix to the electric appliance fingerprint vector. Finally, the corresponding electric appliance label is output through the fully connected feedforward network layer, so as to extract the electric appliance fingerprint from the aggregated load data and judge the running state of the electric appliance. The accuracy of identifying the electric appliance running feature by the neural network in the current NILM task is improved, and the number of concurrent loads of different electric appliances processed by the network is improved. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 The flowchart of the present application for extracting the electric appliance fingerprint from the aggregated load data.

[0061] Figure 2 The network structure diagram of the concurrent load decoupler of the present application. DETAILED DESCRIPTION

[0062] The technical solutions of the present application are described in detail below in combination with the drawings and examples.

[0063] Example:

[0064] The present application proposes a method for extracting the electric appliance fingerprint from the aggregated load data, and the flowchart is as shown in Figure 1 The method can be divided into the following steps:

[0065] Step 1: Synthesize the aggregated load data set, and divide the training, verification and test sets;

[0066] Step 2: Establish the main component residual position sub-network of the model;

[0067] Step 3: Construct the concurrent load decoupler network model, and determine the input and output formats;

[0068] Step 4: Perform hyperparameter search to find the most suitable parameter space;

[0069] Step 5: Perform the disaggregated data, extract the appliance fingerprint, and determine the running state of the appliance.

[0070] The following further describes each step.

[0071] 1. Synthesize the aggregated load data set, divide the training, validation, and test sets

[0072] There are problems such as lack of unified standardization and small number of maximum concurrent loads in existing public data sets in the NILM field. In order to ensure that the method of the present application can achieve better disaggregation effect, it is necessary to construct an aggregated data set of rich types of appliance fingerprints.

[0073] First, select a public data set as the initial data source, normalize the public data set to ensure that the appliance load signals in different measurement environments can be combined, then determine the types of appliances in the aggregated data set, combine the normalized public data set to obtain a basic data set, and finally determine the maximum number of concurrent loads, synthesize the aggregated signal data set, and divide the training, validation, and test sets.

[0074] The aggregated signal data set mainly includes three types of data of current, voltage and power when the appliance is running, and records the information of the running appliance contained in the current aggregated signal data, such as running state, running time and running operation (such as switching from on to off, from one to two gears, etc.). The following 2-3 introduces the concurrent load decoupler network model used, and the modeling of deep neural network is more from the understanding of data and target, so when implementing, only the network needs to be constructed, and the appropriate method is used to find the best parameter setting, and finally a network model is obtained to realize the task of load disaggregation.

[0075] 2. Main components of the established model: residual position subnetwork

[0076] ① Position-wise feedforward layers

[0077] The position-wise feedforward layers used in the present application are established based on the idea proposed in "Attention is all you need", mainly composed of two fully connected layers and a normalization layer, and using ReLU activation function.

[0078] The input is a spectrogram matrix X0(t*v), which is obtained by Fourier transform of the aggregated load data, each row is a time slice, i.e. the first dimension t is related to the sequential nature of the data (signal duration); each column is a frequency window, i.e. the second dimension v is related to the feature (Fourier transform result), and the matrix X0 represents a period of aggregated load data.

[0079] The calculation of the layer at a specific position j is expressed as:

[0080] h j (X j-1 )=X j-1 W j T -Θ j

[0081] where X j-1 is the activation output of layer j-1; W j is a learnable weight matrix, and the input matrix X of layer j is j-1 Projected onto the q-dimensional space, q is the number of input neuron nodes; Θ j is the bias matrix; the output of this part in the network Represented as the unactivated j-layer output, the final j-layer activation output is obtained through the ReLU activation function Right now:

[0082]

[0083] ②Network structure of residual position sub-network

[0084] The first layer of RPSN is a position feedforward layer, followed by a ReLu activation function, and then a position feedforward layer, followed by a normalization layer, and finally a ReLU activation function. The residual network included in it is connected to the input r(X j-1 ) to the normalization layer, where the data is added and normalized to obtain the inactive input of the next RPSN module Expressed as:

[0085]

[0086] 3. Build a concurrent load decoupler network model and determine the input and output formats

[0087] ①Construct residual position sub-network sequence

[0088] In the concurrent load decoupler network model, k layers of residual position sub-network modules are connected in series to form a residual position sub-network sequence, and each layer is recorded as RPSN 2i+1 , i∈(1,k]. There is a separate positional feedforward layer at the head of the concurrent load decoupler network, which maps the input spectrogram matrix X0 to the q-dimensional space to form the input h1(X0) of the first layer RPSN3 and the residual module input r(X0).

[0089] ②Set up a multi-head self-attention layer

[0090] Multi-head self-attention layer connected to RPSN 2k+1The purpose of the module is to summarize the matrix X obtained by the residual position subnetwork sequence j+1 information in the time dimension, and jointly process information from different representation subspaces of different positions, the multi-head self-attention layer performs a scaled dot-product attention function in parallel time, then connects and projects the results onto another learnable space, represented as:

[0091]

[0092]

[0093] where Q represents the information queried, i.e. the relevant appliance fingerprint, K represents the attributes related to the appliance fingerprint, such as voltage and current waveforms, and V represents the content vector obtained according to the current query and key attributes, in the self-attention of the present application Q = K = V = X j+1 ; are the projection matrices of each head respectively, is the projection matrix of the output; the calculation of the scaled dot-product attention function in the multi-head self-attention mechanism is realized by the following function:

[0094]

[0095] where, is the soft-max output value in order to avoid the saturation (minimum gradient occurs) region thereof.

[0096] ③ Global pooling and classification

[0097] The role of the global pooling and classification layer is to convert the feature matrix X into an output vector x, which contains the activated label (device) l in the given time window. The expectation E is introduced to process the input matrix to achieve dimension reduction, so the global pooling is represented as:

[0098] g: R t×q → R q

[0099] g(X j-1 ) = E m [(X j-1 ) m,n ] = x j-1

[0100] At the same time, another nonlinear transformation is introduced to reduce the features represented by the vector x j-1 , and the classification output is:

[0101]

[0102] where W Pred ∈ R q×|L||L| represents the number of all kinds of appliances in the current data set, and a learnable vector a is used to adjust the Sigmoid function to realize the output of a specific label class:

[0103]

[0104] 4. Concurrent load decoupler network structure

[0105] The dropout function is also involved in the overall structure of the network:

[0106] h d,j = Dropout j (h j (X j-1 ))

[0107] The residual position subnetwork can be represented by the following function form:

[0108]

[0109] Therefore, the overall network of the concurrent load decoupler can be represented as:

[0110]

[0111] 4. Perform hyperparameter search to find the most suitable parameter space

[0112] According to the concurrent load decoupler network model structure described above, the parameter space is <q, k, n head , p d >, where p d is the screening probability of the dropout function. In the present application, hyperparameter search is performed, and the ASHA algorithm is used to iterate the parameter space and different learning rate parameters {η}, weight decay {λ} and batch size {B}. The ASHA algorithm uses an aggressive strategy to stop training weaker models than other models. Therefore, compared with the standard grid search or random search strategy, hundreds of training processes for different hyperparameters can be completed very quickly.

[0113] In the hyperparameter search process, the network model under various different hyperparameters is evaluated to find the model with the best load decomposition effect. In the present application, the decomposition effect of different models is evaluated by the F1-score method, and the evaluation formula of the method is The final result is the harmonic mean of the precision PR w and recall RE w of different parameter subsets S w :

[0114]

[0115] where #· represents the cardinality |·|; for the probability value in the prediction vector is not a specific label, so a threshold should be set, when the prediction probability is greater than the threshold, it means that the label is activated in the current signal, recorded as 1, otherwise 0, in order to find an optimal parameter set, the tuning process is performed to calculate the weighted average of F1 score in the interval (0, 1), and then the parameters under the threshold with the highest weighted average are selected as the final parameter space set.

[0116] In this embodiment, according to the description of the concurrent load decoupler network, the network is built, more than 100 parameter combinations are executed by ASHA algorithm, and the following optimal parameter space solution is obtained: q = 256, k = 14, n head = 8, p d = 0.2, η = 3·10 -4 , λ = 2.8·10 -2 and B = 64, and the best threshold δ = 0.74 is obtained.

[0117] 5. Perform aggregated data decomposition, extract appliance fingerprints, and determine the running state of the appliance

[0118] According to the above steps, the optimized concurrent load decoupler network model is obtained by building the network model and searching for the optimal parameter space, which can perform data decomposition on the multi-concurrent appliance load signal and extract the appliance fingerprint to determine the current running state of the appliance. The running state of the known household appliance can be determined according to the meter data of the household, and the currently running appliance under the load can be identified. The network model can achieve an identification accuracy of more than ninety percent (the number of aggregated devices is less than 10) on the validation set of the aggregated load data set obtained in 1. When the number of aggregated devices increases, the accuracy decreases significantly. In the case of a large number of aggregated devices, the hyperparameter search can be performed according to experience to compare more parameter combinations, and in the actual application scenario, the parameters of the network can be adjusted as the number of decomposed aggregated data increases, so that the accuracy of the model can be further improved.

Claims

1. A method of extracting an appliance fingerprint from aggregated load data, the method comprising: The specific steps are as follows: Step 1, synthesize the aggregated load data set, and divide the training, validation and test sets; specifically including: First, select a public data set as the initial data source, normalize the public data set to ensure that the appliance load signals in different measurement environments can be combined; then, determine the types of appliances in the aggregated data set, combine the normalized public data set to obtain the basic data set; finally, determine the maximum number of concurrent loads, synthesize the aggregated signal data set, and divide the training, validation and test sets; Step 2, establish the residual position sub-network, denoted as RPSN; The first layer of the RPSN is a position feed-forward layer followed by a ReLu activation function, then a position feed-forward layer, followed by a normalization layer, and finally followed by a ReLu activation function; which contains a residual network connection RPSN input r(X j-1 ) to the normalization layer, where the data is added and normalized to obtain the non-activated input of the next RPSN module is represented as: The position feedforward layer mainly consists of two fully connected layers and a normalization layer, and uses the ReLU activation function; the input is the spectral graph matrix X0(t*v), which is obtained by Fourier transform of the aggregated load data, each row is a time slice, i.e. the first dimension t is related to the sequence property of the data; each column is a frequency window, i.e. the second dimension v is related to the feature, and the matrix X0 represents the aggregated load data in a period of time; The calculation of the specific j position layer is represented as: h j (X j-1 )=X j-1 ·W j T -Θ j where X j-1 is the j-1 layer activation output; W j is a learnable weight matrix, projecting the input matrix X j-1 of the j layer to a q-dimensional space, q being the number of input neuron nodes; Θ j is a bias matrix; the output of this part in the network is denoted as the j layer output without activation, and the final j layer activation output is obtained through the ReLU activation function i.e.: Step 3, build the concurrent load decoupler network model and determine the input and output formats; the specific process is as follows: ① Build the RPSN network sequence The RPSN network sequence is formed by connecting the k RPSN modules in series, and each layer is recorded as RPSN 2i+1 , i ∈ (1, k]; there is a separate position feedforward layer at the head of the COLD network, which maps the input spectral image matrix X0 onto a q-dimensional space to form the input h1(X0) of the first layer RPSN3 and the residual module input r(X0); ② Set the multi-head self-attention layer The multi-head self-attention layer is arranged in the RPSN 2k+1 After the module, the purpose is to obtain the matrix X through the RSPN sequence j+1 Information in the time dimension, and jointly processing information from different representation subspaces of different positions, the multi-head self-attention layer performs a scaled dot-product attention function in parallel time, and then connects and projects the result onto another learnable space, which is represented as: wherein Q represents the queried information, i.e., the fingerprint of the relevant appliance, K represents the attribute related to the fingerprint of the appliance, and V represents the content vector obtained according to the current query and the key attribute, wherein is the projection matrix of each head, respectively, is the output projection matrix; wherein the calculation of the scaled dot-product attention function in the multi-head self-attention mechanism is realized by the following function: wherein is the soft-max output value in order to avoid saturation of the minimum gradient occurrence region thereof; ③ Global pooling and classification The role of the global pooling and classification layer is to convert the feature matrix X into an output vector x, which contains the activated label l in the given time window; the expected value E is introduced to process the input matrix to realize dimension reduction, so the global pooling is represented as: g: R t×q → R q g(X j-1 ) = E m [(X j-1 ) m,n ] = x j-1 At the same time, another non-linear transformation is introduced to reduce the vector x j-1 The features are represented and a classification output is made: where W Pred ∈ R q×|L| |L| denotes the number of all appliance categories in the current dataset, and a learnable vector a is used to adjust the Sigmoid function to realize the output of a specific label class: ④ Formally represent the structure of the concurrent load decoupler network The overall network structure also involves the dropout function: h d,j = Dropout j (h j (X j-1 )) The RPSN network is represented by the following function form: Therefore, the overall network of the concurrent load decoupler is represented as: Step 4, search for the most suitable parameter space by searching for the hyperparameters; According to the concurrent load decoupler network model structure described above, the parameter space is <q, k, n head , p d >, wherein p d is the screening probability of the dropout function; hyperparameter search is performed, and the ASHA algorithm is used to iterate the parameter space and different learning rate parameters {η}, weight decay {λ}, and batch size {B}, the algorithm uses an aggressive strategy to stop training models that are weaker than other models; In the hyperparameter search process, the network model under various different hyperparameters is evaluated to find the model with the best load decomposition effect. The F1-score method is used to evaluate the decomposition effect of different models, and the evaluation formula of the method is The final result is the harmonic mean of the precision PR w and recall RE w under different parameter subsets S w : Where #· represents the cardinality |·|; for the probability value in the prediction vector, a threshold value is set, when the prediction probability is greater than the threshold value, it means that the label is activated in the current signal, denoted as 1, otherwise 0; in order to find an optimal parameter set, the tuning process is performed to calculate the weighted average of the F1 score in the interval (0, 1), and then the parameters under the threshold value with the highest weighted average are selected as the final parameter space set; Step 5, aggregate data decomposition, extract appliance fingerprints, and determine the running state of the appliance The optimized COLD network model obtained in the above steps is used to perform data decomposition on the multi-concurrent appliance load signal, extract the appliance fingerprint, and determine the current running state of the appliance; for the running state judgment of known household appliances based on the household meter data, the currently running appliance under the load is identified.

Images