Non-intrusive load disaggregation method based on mathematical morphology and improved transformer

Abstract

The application discloses a non-intrusive load decomposition method based on mathematical morphology and improved Transform, mainly uses a non-intrusive load identification device to acquire power consumption information of a family, carries out data preprocessing and constructs a data set, filters the data by constructing a multi-scale mathematical morphology filter, and then inputs the data into an improved Transform neural network for training and learning; the improvement points mainly include introducing a LocalBiLSTM layer to replace a position coding layer, using a probability sparse attention mechanism to replace a traditional attention mechanism, and using a 1D convolution network as a forward feedback layer; during training, a large amount of data is combined with a Bayesian optimizer to continuously adjust network parameters; finally, the optimal model parameters with the minimum loss are selected, the optimal model is used to complete a load decomposition task, and a non-intrusive load decomposition result is analyzed; and the load information with strong noise resistance and strong stability can be obtained, and the load decomposition result has the advantages of high precision and short decomposition time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power load decomposition, and particularly to a non-intrusive load decomposition method based on mathematical morphology and improved Transformer. BACKGROUND

[0002] The intelligent energy management system optimizes the use of electricity by using sensor data to achieve efficient residential energy consumption. In order to better understand the household electricity consumption under limited monitoring data, non-intrusive load monitoring (NILM) or energy decomposition is proposed, which aims to decompose the aggregated electricity into individual devices. Using smart metering technology, the NILM algorithm can predict the electricity consumption of individual appliances in real time by monitoring the total consumption of the household. This sensor-based power analysis provides feedback on electricity consumption, thus supporting sustainable energy consumption.

[0003] It is estimated that up to 12% of residential energy savings can be achieved by providing household-level electricity consumption feedback, so NILM can benefit the general public in community research and public utilities, including residential and commercial energy use, device innovation, energy-saving marketing, and project evaluation.

[0004] In summary, it is of high practical value to invent a non-intrusive load decomposition method that meets high accuracy and real-time requirements. SUMMARY

[0005] The present application aims to overcome the shortcomings and deficiencies of the prior art and proposes a non-intrusive load decomposition method based on mathematical morphology and improved Transformer, which can be applied to online monitoring of the specific power consumption of household appliances. It has the characteristics of low monitoring cost, high decomposition accuracy, etc. Through the present application, load data with strong noise resistance and stable waveform can be obtained, which can capture the close relationship between the front and rear load data, reduce the training time when performing load decomposition tasks, and improve the accuracy of the results.

[0006] To achieve the above-mentioned purpose, the technical scheme provided by the present application is as follows: a non-intrusive load decomposition method based on mathematical morphology and improved Transformer, comprising the following steps:

[0007] 1) Use a non-intrusive load identification device to obtain the power sequence of the target appliance to be tested and the total household power sequence, and perform data preprocessing and construct a data set, and divide the data set into a training set and a test set;

[0008] 2) Construct a filter based on multi-scale mathematical morphology processing data, called multi-scale mathematical morphology filter, and filter the data in the data set;

[0009] 3) input the filtered data into the constructed non-intrusive load model for decomposition training, use the data in the training set to continuously adjust and update the model parameters combined with the Bayesian optimizer, and finally input the data in the test set into the trained model to obtain the power curve of the target electrical appliance to be tested, and use the corresponding index for evaluation; wherein the constructed non-intrusive load model is an improved Transformer neural network, and the improvement points of the network include: first, a LocalBiLSTM layer is introduced instead of a position encoding layer, which not only does not need to design an effective position embedding layer according to different tasks, but also the LocalBiLSTM layer can fully capture the sequential information within each window, directly perform calculation parallelization, thereby greatly reducing the complexity of calculation; second, a kind of probabilistic sparse attention mechanism is introduced to sparsify the calculation process of attention to improve the inference speed and reduce the space consumption; third, unlike the traditional Transformer neural network which uses two layers of dense layer network as the forward feedback layer, it uses two layers of 1D convolution network and ReLU activation, the motivation is that for the non-intrusive load decomposition task, the correlation information between adjacent hidden elements can be extracted, thereby further improving the accuracy.

[0010] Further, in step 1), the power sequence of the target electrical appliance to be tested and the total power sequence of the household are obtained by using the non-intrusive load identification device, only using the low-frequency power of 1Hz at the power inlet, the original data obtained has large noise and many abnormal data, and data preprocessing operation is needed, specifically using the improved box plot for data outlier processing; the steps of the improved box plot for outlier processing include: step 1.1, solving the upper quantile Q U , lower quantile Q L and median of the data; step 1.2, setting the maximum value Max = Q U +1.5(Q U -Q L ) and the minimum value Min = Q L -1.5(Q U -Q L ); step 1.3, if the data date satisfies: date>Max or date<Min, it is determined as an outlier; step 1.4, deleting the outlier and using Lagrange interpolation method to complete the vacancy value thus generated; step 1.5, repeating steps 1.1-1.4 to check the outlier processing condition;

[0011] After cleaning the abnormal data, the data is divided into training set and test set according to the proportion, and the construction of the data set is completed;

[0012] Wherein, the Lagrange interpolation method is to use polynomial function to approximate the relationship between input and output, for n+1 sample points (x0, y0), (x1, y1), ···, (x i ,y i ), ···, (x n ,y n ), wherein, i = 0, 1,..., n, x i is the i-th sample point representing the total power of the load, y i is the i-th sample point representing the device power, given any data x in the interval [a, b], the corresponding estimated value at x can be calculated, the formula is as follows:

[0013]

[0014]

[0015] In the formula, L n (x) is the Lagrange interpolation polynomial, i is n+1 number, n represents the n-th sampling point, j is an integer value between 0 and n, not equal to i, y i is the function value at x i , x i , x j are different nodes, and l i (x) is the base function.

[0016] Further, in step 2), the input signal is z(n), n∈{0,1,…,N-1}, there are N data in total, the structure element SE is γ(m), m∈{0,1,…,M-1}, the length of the structure element is M, and the dilation and erosion operation of the input signal is defined as:

[0017]

[0018]

[0019] In the formula, is the dilation operator; is the erosion operator;

[0020] Based on the cascade of dilation and erosion, the two operators of open and close are represented as:

[0021]

[0022]

[0023] In the formula, and · represent the open and close operators respectively;

[0024] The MMM (multi-scale mathematical morphology) realizes the open and close operations by selecting SEs of different time scales, and the open and close operations of the MMM are given as follows:

[0025]

[0026]

[0027] where s t is a time scale;

[0028] The open and close operations of the morphology have low-pass characteristics, and the cascading thereof can filter out positive and negative pulses simultaneously, and according to the order of the cascading, an OC (open-close) filter or a CO (close-open) filter can be formed, and due to the expansibility of the open operation and the anti-expansibility of the close operation, both filters are statistically biased, so an average combination of the two filters is adopted, and the output h(x) of the filter is represented as:

[0029] h(x) = [OC(f(x)) + CO(f(x))] / 2

[0030] where f(x) is the input data;

[0031] Therefore, a MMMF (multi-scale mathematical morphology filter) is proposed, which realizes waveform decomposition in a flexible fluctuation period by weighting the filtering results of a target signal and multi-scale SEs, and the MMMF is represented as:

[0032]

[0033]

[0034] where k is the number of time scales; is a weight value of the SE at each scale, s i represents the i-th time scale, and in order to reduce the noise influence in the filtering results of the SE of a small scale, the weight value is determined by the variance value of the filtering noise at each scale;

[0035]

[0036]

[0037] where is the filtering mean square error of the SE at the scale s i ;

[0038] Finally, the constructed MMMF is used to filter the data in the data set.

[0039] Further, in step 3), first, in order to fully capture the sequential information of the data and consider the front and back relationship of the data, a LocalBiLSTM layer is used instead of the position encoding layer in the traditional Transformer neural network;

[0040] The LocalBiLSTM layer is different from the previous BiLSTM layer, the BiLSTM layer is applied to the entire sequence, while the original load power sequence is divided into many short fragments, which only contain short-time load information; the local power sequence of the window size G is extracted from the filtered data set through the sliding window, the load power sequence extracted by the local sliding window forms a local power short sequence, a shared BiLSTM layer is trained from the front and back directions to learn the potential information, through the above-mentioned manner, the local power information of each local area of the entire load data sequence is explicitly included in the learned latent representation, and the shared BiLSTM layer is called LocalBiLSTM layer, the LocalBiLSTM layer only focuses on the local short-term dependency relationship, and does not consider any long-term dependency relationship;

[0041] Specifically, a series of power sequences x t-(M-1) ,x t-(M-2) ,…,x t are selected in turn by a sliding window with a length of G, the LocalBiLSTM layer processes the short-term power sequence and generates G hidden states, wherein the last power point x t is used as a representation of the local short sequence h t ;

[0042] h t =LocalBiLSTM(x t-(G-1) ,x t-(G-2) ,…,x t )

[0043] In order to enable the model to process the sequence in an autoregressive manner, and ensure that the sequence length before and after processing is consistent, the input sequence is increased by (G-1) in length before being processed by the sliding window, from the perspective of the sequence, the LocalBiLSTM layer accepts an input sequence x1,x2,…,x N and needs to output a hidden representation sequence h1,h2,…,h N containing local area information:

[0044] h1,h2,…,h N =LocalBiLSTM(x1,x2,…,x N )

[0045] The LocalBiLSTM layer is similar to a one-dimensional convolutional neural network, however, the convolution operation completely ignores the sequential information of the position within the local window, although the traditional Transformer neural network proposes a position embedding layer to alleviate this problem, the effectiveness of the position embedding is limited, and it also needs a lot of additional efforts to design an effective position embedding according to different tasks; on the other hand, the LocalBiLSTM layer can completely capture the sequential information within each window, and the sliding operation naturally contains the global sequential information, so the influence of the front and rear power information is considered during training, thereby improving the training accuracy; at the same time, the calculation of processing short sequences is independent of each other, therefore, the calculation parallelization is also relatively simple, which greatly reduces the computational complexity;

[0046] Secondly, in order to improve the inference speed and reduce the space consumption, a kind of probability sparse attention mechanism is introduced to replace the traditional self-attention mechanism, which sparsifies the calculation process of self-attention, specifically, the sparsity measure based on Kullback-Leibler divergence is used to determine whether to calculate the attention function of each query Q, the traditional self-attention mechanism Attention is represented as:

[0047]

[0048] In the formula, Q is the target vector, K is the source vector, V is the value corresponding to the source vector, d k is the dimension of K, the similarity calculation adopts the dot product of vectors, and d k is scaled to the result, and then the weight is obtained by using the softmax function, in the self-attention mechanism, Q and K are the same, so Q=K=V, the size of the dimension is d k .

[0049] And the probability sparse attention mechanism only replaces K with is a sparse vector, the size is the same as K, but only contains the first u elements, the others are filled with zero, the size of u is controlled by the sampling factor c:

[0050] u=clnL

[0051] In the formula, L represents the length of K, the first u elements are taken as the standard, that is, the method of distinguishing important contacts and unimportant contacts is a variant of KL divergence, which is called maximum-mean measure, the greater the sparse value M' of the query indicates that it plays a more important role in the self-attention mechanism:

[0052]

[0053] In the formula, q i represents the i-th element in Q, k jdenotes the jth element in K, so as long as the elements in Q are randomly selected and K is calculated M', and then u elements are selected from M' to form K, the calculation resources and memory resources can be greatly simplified, and the performance loss of the self-attention mechanism is small;

[0054] Finally, the two layers of dense layer in the traditional Transformer forward feedback layer are improved into two layers of 1D convolution network, the motivation is to extract the correlation information between adjacent hidden elements, make the connection between the front and rear power sequences more close, in order to improve the precision of non-intrusive load decomposition task.

[0055] Further, the improved Transformer neural network comprises the following modules:

[0056] Module 1, feature extraction layer, composed of separable 1D convolution layer and activation function ReLU;

[0057] Module 2, 2x2 average pooling layer;

[0058] Module 3, LocalBiLSTM layer;

[0059] Module 4, multi-head probability sparse attention mechanism module, composed of h probability sparse attention mechanism layers;

[0060] Module 5, forward feedback layer, composed of two 1D convolution layers;

[0061] Module 6, residual layer;

[0062] Module 7, layer normalization layer;

[0063] Module 8, linear output layer, composed of dense layer;

[0064] The architecture of the whole improved Transformer neural network is expressed as:

[0065] h1,h2,…,h T =LocalBiLSTM(x1,x2,…,x T )

[0066]

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] where x1, x2, …, x T is the input load sequence, h1, h2, …, h T is the output sequence after the LocalBiLSTM layer, is the sequence after the LayerNorm1 layer normalization layer, u1, u2, …, u T is the sequence after the MultiHeadAttention multi-head sparse probability attention mechanism layer, is the sequence after the LayerNorm2 layer normalization layer, m1, m2, …, m T is the sequence after the 1D convolution, T is the sequence after the LayerNorm2 layer normalization layer, y1, y2, …, y T is the final output sequence after the dense layer;

[0073] Therefore, the total loss function is represented as:

[0074] loss total = loss(Dense(Conv1D(Add&Norm(MultiheadAttention(LocalBiLSTM(feature(X))))))

[0075] where loss total represents the total loss function, Dense represents the dense layer, Add&Norm represents the residual layer and layer normalization layer, feature represents the feature extraction layer, and X represents the input load power sequence.

[0076] Further, in step 3), the filtered data is input into the constructed non-intrusive load model for decomposition training. First, the initial neural network parameters for training are set, including the optimizer, Batchsize, initial learning rate, maximum number of training times, and sliding window size. Then, the data in the training set is used to continuously adjust and update the model hyperparameters with the Bayesian optimizer. Finally, the optimal Batchsize is 256, the maximum number of training times is 200, the optimal sliding window size is 5, Adam is selected as the optimizer, the initial learning rate is 0.0001, the loss function uses the least mean square error MSE, and in order to further prevent overfitting, the early stopping earlystopping mechanism is added, that is, when the mean square error stops decreasing for 10 iteration times, the model will automatically stop training and a learning rate decay mechanism is introduced, with a decay rate of 0.1 and a decay step of 1000. Through the continuous optimization and adjustment of the Bayesian optimizer on the model hyperparameters, the model accuracy can be improved. Similarly, the parameters in the convolutional layer, dense layer, LocalBiLSTM layer, and multi-head sparse probability attention mechanism layer can also be selected by the Bayesian optimizer to obtain the best values. After training is completed, the model parameters with the smallest loss are selected from the saved training parameters as the model parameters of the detection model. Then, the data in the test set is input into the trained model, and the user's decomposition result can be obtained.

[0077] Further, in step 3), in order to better evaluate the performance of the model in performing the non-intrusive load decomposition task, the mean absolute error MAE, the normalized signal total error SAE, and F1, accuracy Accuracy, precision Precision, and recall Recall are used for evaluation, respectively as follows:

[0078]

[0079] wherein the predicted power value and the actual power value of a certain device at time t are represented by and x t , respectively, where N' is the number of samples;

[0080]

[0081] wherein the true consumed power and the predicted total power of a certain device in a period of time t are represented by and , respectively, that is, and

[0082]

[0083]

[0084]

[0085] where precision(c j ) is the accuracy; recall(c j ) is the recall; F1-Score(c j ) is the F1 index; TP(c j ) represents the number of correct identification of class c j ; FN(c j ) represents the number of failure to correctly identify class c j ; FP(c j ) is the number of not belonging to class c j , but being identified as c j .

[0086] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0087] 1. The multi-scale mathematical morphological filtering method of the present application can remove measurement abnormal points caused by non-intrusive load identification devices or environmental factors, so that the data is more resistant to noise, more stable, and the load characteristics are more obvious.

[0088] 2. The present application provides a new perspective for solving the task of non-intrusive load decomposition. Unlike RNN and CNN models, the model proposed by the present application is an improved Transformer neural network with parallel computing capability, and the probability sparse attention mechanism is used to further reduce the amount of calculation and storage, providing an efficient load decomposition method based on deep learning.

[0089] 3. The present application proposes to use LocalBiLSTM layer instead of position embedding layer to fully capture the local information of the load and enhance the connection between the front and rear load data to improve the recognition of the local mode of the load by the model.

[0090] 4. The present application proposes to use a two-layer 1D convolution network instead of a traditional forward feedback network, which can extract the correlation information between adjacent hidden elements, further improving the accuracy of network decomposition. BRIEF DESCRIPTION OF DRAWINGS

[0091] Figure 1 is a flowchart of the method of the present application.

[0092] Figure 2An illustration of filtering operation on data using multiscale mathematical morphology, where raw signal represents the original signal, denoised signal represents the filtered signal, multiscale morphology analysis represents multiscale analysis on the signal, adaptive selection of SEs represents adaptive selection of structure elements, and weighted summation represents weighted summation.

[0093] Figure 3 A structural diagram of the LocalBiLSTM layer.

[0094] Figure 4 A structural diagram of the improved Transformer neural network, where Input layer represents the input load sequence, Feature extract layer represents the feature extraction layer, Multihead Attention represents the multi-head sparse probability attention mechanism layer, Add&Norm represents the residual layer and layer normalization layer, Conv1d represents the 1D convolution layer, Regressor represents linear regression using the dense layer, and Outputs represents the final input load decomposition result. DETAILED DESCRIPTION

[0095] The application will be further described in conjunction with the embodiments and the accompanying drawings, but the embodiments of the application are not limited thereto.

[0096] As shown in Figure 1 , the embodiment discloses a non-intrusive load decomposition method based on mathematical morphology and improved Transformer, and the specific conditions are as follows:

[0097] 1) The power sequence of the target electric appliance to be measured and the total power sequence of the household are obtained by using the non-intrusive load identification device, and data preprocessing and data set construction are performed, and the data set is divided into a training set and a test set, and the specific conditions are as follows:

[0098] The power sequence of the target electric appliance to be measured and the total power sequence of the household are obtained by using the non-intrusive load identification device, and only the low-frequency power of 1Hz at the power inlet is used, the acquisition cost is low, the original data obtained is usually noisy, and there are more abnormal data, so data preprocessing operation is required, and an improved box plot is used for data outlier processing;

[0099] The main steps of the improved box plot for outlier processing are as follows: step 1.1, solving the upper quantile Q U , the lower quantile Q L , and the median of the data; step 1.2, setting the maximum value Max=QU +1.5(Q U -Q L ), minimum value Min = Q L -1.5(Q U -Q L ); Step 1.3, if the data date satisfies date > Max or date < Min, it is determined as an outlier; Step 1.4, delete the outlier and use the Lagrange interpolation method to fill in the missing values thus generated; Step 1.5, repeat Step 1.1 to Step 1.4 to check the outlier handling situation;

[0100] Among them, the Lagrange interpolation method approximates the relationship between input and output by using a polynomial function. For n + 1 sample points (x0, y0), (x1, y1), ···, (x i , y i ), ···, (x n , y n ), where i = 0, 1,... n, x i is the i-th sample point representing the total load power, y i is the i-th sample point representing the equipment power. Given any data x within the interval [a, b], the corresponding estimated value at x can be calculated, and the formula is as follows:

[0101]

[0102]

[0103] In the formula, L n (x) is the Lagrange interpolation polynomial, i is n + 1 numbers, n represents the n-th sampling point, j is an integer value between 0 and n that is not equal to i, y i is the function value at x i , x i , x j are distinct nodes, and l i (x) is the basis function;

[0104] After cleaning the abnormal data, perform data standardization. The purpose is to make the data easier to be trained by the neural network and is conducive to obtaining better results. The standardization formula is expressed as follows:

[0105]

[0106] Among them, x t represents the power value at time t; and σ respectively represent the average power and standard deviation of the electrical equipment, and x is the power data after standardization. The average power and standard deviation of the kettle, refrigerator, washing machine, dishwasher, and microwave oven are shown in the following table;

[0107] Device Average power (w) Standard deviation Kettle 700 1000 Microwave 500 800 Refrigerator 200 400 Dishwasher 700 1000 Washing machine 400 700

[0108] Finally, the data is divided into training set and test set according to the ratio of 7:3, and the construction of the data set is completed. In order to make the result of load decomposition more obvious and show the advantage of the proposed method, we take several typical load devices in UK-DALE data set including: kettle, refrigerator, washing machine, dishwasher and microwave oven for decomposition, and complete the construction of database.

[0109] 2) Construct a filter based on multi-scale mathematical morphological processing data, called multi-scale mathematical morphological filter, and filter the data in the data set, as follows:

[0110] Let the input signal be z(n), n∈{0,1,…,N-1}, there are N data in total, and the structure element SE is γ(m), m∈{0,1,…,M-1}, the length of the structure element is M, and the dilation and erosion operation of the input signal is defined as:

[0111]

[0112]

[0113] In the formula, is the dilation operator; is the erosion operator;

[0114] Based on the cascade of dilation and erosion, the two operators of open and close can be represented as:

[0115]

[0116]

[0117] In the formula, and·represent the open and close operators respectively;

[0118] Multi-scale mathematical morphological MMM can realize the open and close operation by selecting SE of different time scales, and the open and close operation of multi-scale mathematical morphological can be given as follows:

[0119]

[0120]

[0121] In the formula, s t is the time scale;

[0122] The on and off operation of morphology has low-pass characteristic, and the cascade of which can filter out positive and negative pulses simultaneously. According to the order of cascade, on-off OC or off-on CO filter can be formed respectively. Due to the scalability of on operation and the anti-scalability of off operation, both filters are statistically biased. Therefore, the average combination of the two filters is adopted in this paper, and the output h(x) of the filter can be expressed as:

[0123] h(x) = [OC(f(x)) + CO(f(x))] / 2

[0124] In the formula, f(x) is the input data.

[0125] Therefore, we propose a multi-scale mathematical morphology filter MMMF, which realizes waveform decomposition in flexible fluctuation period by weighting and multi-scale SE of the filtering result of target signal. MMMF can be expressed as:

[0126]

[0127]

[0128] In the formula, k is the number of time scales; is the weight value of SE at each scale, S i represents the i-th time scale. In order to reduce the noise influence in the filtering result of small scale SE, the weight value is determined by the variance value of filtering noise at each scale;

[0129]

[0130]

[0131] In the formula, is the filtering mean square error of SE at scale s i ;

[0132] Finally, the constructed multi-scale mathematical morphology filter is used to filter the data in the data set. The results before and after filtering and the process are shown in Figure 2 . In the figure, raw signal represents the original signal, denoised signal represents the filtered signal, multiscale morphology analysis represents multi-scale analysis of the signal, adaptive selection of SEs represents adaptive selection of structure elements, and weighted summation represents weighted summation.

[0133] 3) the filtered data is input into the constructed non-intrusive load model for decomposition training, the data in the training set is combined with the Bayesian optimizer to continuously adjust and update the model parameters, and finally the data in the test set is input into the trained model to obtain the power curve of the target electric appliance to be tested, and the corresponding indicators are used for evaluation; wherein the constructed non-intrusive load model is an improved Transformer neural network, and the improvement points of the network include: first, a LocalBiLSTM layer is introduced instead of a position encoding layer, which not only does not need to design an effective position embedding layer according to different tasks, but also the LocalBiLSTM layer can fully capture the sequential information within each window, directly perform calculation parallelization, thereby greatly reducing the complexity of calculation; second, a kind of probabilistic sparse attention mechanism is introduced to sparsify the calculation process of attention to improve the inference speed and reduce the space consumption; third, unlike the traditional Transformer neural network which uses two layers of dense layer network as the forward feedback layer, we use two layers of 1D convolution network and ReLU activation, the motivation is that for the non-intrusive load decomposition task, the correlation information between adjacent hidden elements can be extracted, thereby further improving the accuracy.

[0134] First, in order to fully capture the sequential information of the data and consider the connection between the data before and after, a LocalBiLSTM layer is used instead of the position encoding layer in the traditional Transformer, and the structure is as shown in Figure 3

[0135] The LocalBiLSTM layer is different from the previous BiLSTM layer, the BiLSTM layer is usually applied to the entire sequence, and we divide the original load power sequence into many short fragments, which only contain short-time load information; further, we extract a local power sequence with a window size of K from the filtered data set through a sliding window, the load power sequence extracted through the local sliding window forms a local power short sequence, and a shared BiLSTM layer is trained from the front and back directions to learn the latent information, through the above method, the local power information of each local area of the entire load data sequence is explicitly included in the learned latent image representation, and we call the shared BiLSTM layer LocalBiLSTM layer. Compared with the classic BiLSTM layer, the LocalBiLSTM layer only focuses on local short-term dependencies and does not consider any long-term dependencies;

[0136] Specifically, a series of power sequences x t-(M-1) ,x t-(M-2) ,…,x t ​Selected sequentially by a sliding window of length G, the LocalBiLSTM layer processes short-term power sequences and generates G hidden states, where the last power point x t Used as a local short sequence h t The representation of;

[0137] h t =LocalBiLSTM(x t-(G-1) ,x t-(G-2) ,…,x t )

[0138] To enable the model to process sequences in an autoregressive manner and ensure that the sequence length remains consistent before and after processing, we increase the length of the input sequence by (G-1) units before using the sliding window. From the sequence's perspective, the LocalBiLSTM layer accepts an input sequence x1, x2, ..., x... N Finally, a hidden representation sequence h1, h2, ..., h containing local region information needs to be output. N :

[0139] h1,h2,…,h N =LocalBiLSTM(x1,x2,…,x) N )

[0140] LocalBiLSTM is similar to a one-dimensional convolutional neural network. However, convolutional operations completely ignore the sequential information of positions within a local window. Although traditional Transformer neural networks have proposed position embedding layers to alleviate this problem, the effectiveness of position embedding is limited, and designing effective position embeddings for different tasks requires considerable additional effort. On the other hand, LocalBiLSTM layers can completely capture the sequential information within each window, and the sliding operation naturally includes global sequential information. Therefore, the influence of power information before and after the operation is taken into account during training, thereby improving training accuracy. At the same time, the computations for processing short sequences are independent of each other, so computational parallelization is relatively simple, greatly reducing computational complexity.

[0141] Secondly, to improve inference speed and reduce space consumption, we introduce a probabilistic sparse attention mechanism to replace the traditional self-attention mechanism. This mechanism sparsifies the computation process of self-attention. Specifically, we use a sparsity metric based on Kullback-Leibler divergence for each query Q to determine whether to compute the attention function for that query Q. The traditional self-attention mechanism can be represented as:

[0142]

[0143] Where Q is the target vector, K is the source vector, V is the value corresponding to the source vector, and dk is the dimension of K, the similarity calculation uses the dot product of vectors, and d k The result is scaled and then weighted with the softmax function. In the self-attention mechanism, Q and K are the same, and in the general attention mechanism, K and V are the same unless otherwise specified, so Q = K = V, and the dimension size is d k ;

[0144] The probability sparse attention mechanism only replaces K with is a sparse vector, the size is the same as K, but only contains the first u elements, and the others are filled with zeros, and the size of u is controlled by the sampling factor c:

[0145] u = clnL

[0146] where L represents the length of K, the first u elements are taken, that is, the method of distinguishing important connections and unimportant connections is a variant of KL divergence, called maximum-mean metric, and the greater the sparse value M' represents the more important role in the self-attention mechanism:

[0147]

[0148] In the above formula, q i represents the i-th element in Q, k j represents the j-th element in K, so as long as u elements in Q are randomly selected and K is calculated M', and then u elements are selected from M' to form K, which can greatly simplify the calculation resources and memory resources, and the performance loss of the self-attention mechanism will not be too much;

[0149] Finally, we improve the two layers of dense layer in the traditional Transformer forward feedback layer to two layers of 1D convolution network, the motivation is to extract the correlation information between adjacent hidden elements, make the connection between the front and back power sequences more close, in order to improve the precision of non-intrusive load decomposition task.

[0150] The structure of the whole improved Transformer neural network is shown in Figure 4 , where Input layer represents the input load sequence, Feature extract layer represents the feature extraction layer, Multihead Attention represents the multi-head sparse probability attention mechanism layer, Add&Norm represents the residual layer and layer normalization layer, Conv1d represents the 1D convolution layer, Regressor represents the linear regression with dense layer, and Outputs represents the final input load decomposition result.

[0151] In summary, the improved Transformer neural network comprises the following modules:

[0152] Module 1, feature extraction layer, composed of separable 1D convolution layer, activation function ReLU;

[0153] Module 2, 2x2 average pooling layer;

[0154] Module 3, LocalBiLSTM layer;

[0155] Module 4, multi-head probability sparse attention mechanism module, composed of h probability sparse attention mechanism layers;

[0156] Module 5, forward feedback layer, composed of two 1D convolution layers;

[0157] Module 6, residual layer;

[0158] Module 7, layer normalization layer;

[0159] Module 8, linear output layer, composed of dense layer;

[0160] The architecture of the entire improved Transformer neural network can be expressed as:

[0161] h1,h2,...,h T =LocalBiLSTM(x1,x2,...,x T )

[0162]

[0163]

[0164]

[0165]

[0166]

[0167] y1,y2,...,y T =Dense(n1,n2,...,n T )

[0168] In the formula, x1,x2,...,x T is the input load sequence, h1,h2,...,h T is the sequence output after the LocalBiLSTM layer, is the sequence after the LayerNorm1 layer normalization layer, u1,u2,...,u Tfor a sequence of through a MultiHeadAttention multi-head sparse probability attention mechanism layer, for a sequence of through a LayerNorm2 layer normalization layer, m1, m2,..., m T for a sequence of through a 1D convolution, T for a sequence of through a LayerNorm2 layer normalization layer, y1, y2,..., y T for a sequence of final output through a dense layer;

[0169] Therefore, the total loss function is represented as:

[0170] loss total =loss(Dense(Conv1D(Add&Norm(MultiheadAttention(LocalBiLSTM(feature(X))))))

[0171] In the formula, loss total represents the total loss function, Dense represents the dense layer, Add&Norm represents the residual layer and the layer normalization layer, feature represents the feature extraction layer, and X represents the input load power sequence.

[0172] The filtered data is input into the constructed non-intrusive load model for decomposition training. First, the initial neural network parameters for training are set, including the optimizer, Batchsize, initial learning rate, maximum number of training times, sliding window size, etc. Then, the data in the training set is used to continuously adjust and update the model hyperparameters with the Bayesian optimizer. Finally, the optimal Batchsize is 256, the maximum number of training times is 200, the optimal sliding window size is 5, Adam is selected as the optimizer, the initial learning rate is 0.0001, and the loss function adopts the least mean square error MSE. In order to further prevent overfitting, an early stopping mechanism is added, that is, when the mean square error stops decreasing for 10 iteration times, the model will automatically stop training and a learning rate decay mechanism is introduced, with a decay rate of 0.1 and a decay step of 1000. Through the continuous optimization and adjustment of the Bayesian optimizer on the model hyperparameters, the model accuracy can be improved. Similarly, the parameters in the convolution layer, dense layer, LocalBiLSTM layer and multi-head sparse probability attention mechanism layer can also be selected by the Bayesian optimizer to obtain the best values. After training is completed, the model parameters with the smallest loss are selected from the saved training parameters as the model parameters of the detection model. Then, the data in the test set is input into the trained model, and the user's decomposition result can be obtained.

[0173] In order to better evaluate the performance of the model in performing the non-intrusive load decomposition task, the mean absolute error MAE, the normalized signal total error SAE and F1, accuracy, precision, recall are used for evaluation, respectively:

[0174]

[0175] In the formula, the predicted power value and the actual power value of the device at time t are respectively represented by and x t , wherein N' is the number of samples;

[0176]

[0177] In the formula, the real consumed power and the predicted total power of the device in a period of time t are respectively represented by and , that is, and

[0178]

[0179]

[0180]

[0181] In the formula, precision(c j ) is the accuracy; recall(c j ) is the recall; F1-Score(c j ) is the F1 index; TP(c j ) represents the number of correctly identified class c j ; FN(c j ) represents the number of class c j that cannot be correctly identified; FP(c j ) is the number of class c j that is not identified as c j ; it can be verified by the above various regression and decomposition index formulas that the model of the application is feasible, and the results of the model on the UK-DALE data set are as shown in the following table, wherein room 2 is the verification set, and rooms 1, 3, 4 and 5 are the training set.

[0182] Error Microwave Refrigerator Dishwasher Washing machine Kettle Average MAE 7.31 11.60 15.26 15.83 6.71 11.34±4.28 SAE 0.305 0.051 0.114 0.154 0.057 0.136±0.103 Recall 0.628 0.914 0.960 0.575 0.933 0.802 Accuracy 0.902 0.973 0.992 0.989 0.996 0.970 Precision 0.424 0.931 0.777 0.586 0.974 0.738 F1 0.506 0.922 0.859 0.581 0.953 0.764

[0183] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications, etc. made without departing from the spirit and principles of the present application should be equivalent replacement manners and should be included in the protection scope of the present application.

Claims

1. A non-intrusive load decomposition method based on mathematical morphology and an improved Transformer, characterized in that, Includes the following steps: 1) Use a non-invasive load identification device to obtain the power sequence of the target appliance and the total household power consumption sequence, and perform data preprocessing and build a dataset, dividing the dataset into training set and test set; 2) Construct a filter based on multi-scale mathematical morphology to process data, called a multi-scale mathematical morphology filter, to filter the data in the dataset; 3) The filtered data is input into the constructed non-intrusive load model for decomposition training. The model parameters are continuously adjusted and updated using data from the training set combined with a Bayesian optimizer. Finally, the data from the test set is input into the trained model to obtain the power curve of the target appliance, which is then evaluated using relevant indicators. The constructed non-intrusive load model is an improved Transformer neural network, and the improvements include: First, a LocalBiLSTM layer is used to replace the position encoding layer in the traditional Transformer neural network; The LocalBiLSTM layer divides the original load power sequence into many short segments, which contain only short-term load information. A continuous local power sequence with a window size of K' is extracted from the filtered dataset through a sliding window. The load power sequence extracted by the local sliding window forms a short local power sequence. A shared BiLSTM layer is trained from both the front and back directions to learn the latent information. The local power information of each local region of the entire load data sequence is explicitly incorporated into the learned latent image representation. This shared BiLSTM layer is called the LocalBiLSTM layer. The LocalBiLSTM layer only focuses on local short-term dependencies and does not consider any long-term dependencies. Specifically, a series of power sequences x t-(M-1) ,x t-(M-2) ,…,x t The sequence is selected sequentially by a sliding window of length G, where M is the length of the structuring element. The LocalBiLSTM layer processes the short-term power sequence and generates G hidden states, where the last power point x... t Used as a local short sequence h t The representation of; h t =LocalBiLSTM(x t-(G-1) ,x t-(G-2) ,…,x t ) Before using sliding window processing, the input sequence is lengthened by (G-1) units. From the sequence perspective, the LocalBiLSTM layer accepts an input sequence x1, x2, ..., x... N Finally, a hidden representation sequence h1, h2, ..., h containing local region information needs to be output. N : h1,h2,…,h N =LocalBiLSTM(x1,x2,…,x N ) In the formula, N represents the index of the sequence; Secondly, a probabilistic sparse attention mechanism is introduced to replace the traditional self-attention mechanism. This mechanism sparsifies the computation process of self-attention. Specifically, for each query Q, a sparsity metric based on Kullback-Leibler divergence is used to determine whether to compute the attention function for that query Q. The traditional self-attention mechanism is represented as: In the formula, Q is the target vector, K is the source vector, V is the value corresponding to the source vector, and d is the dimension of K. The similarity calculation uses the dot product of the vectors, and the result is scaled by d. Then the softmax function is used to calculate the weight. In the self-attention mechanism, Q and K are the same, so Q = K = V, and the size of the dimension is d. The probabilistic sparse attention mechanism simply replaces K with... It is a sparse vector with the same size as K, but it only contains the first u elements, and the rest are filled with zeros. The size of u is controlled by the sampling factor c. u=clnL In the formula, L represents the length of K, and the standard for the first u elements, i.e., the method for distinguishing important and unimportant relationships, is a variant of the KL divergence, known as the maximum-mean measure. The sparsity value M′ is expressed as follows: In the formula, q i Let k represent the i-th element in Q. j Let represent the j-th element in K. In this way, we only need to randomly select u elements from Q and calculate M′ with K, and then select u elements from M′ to form K. Finally, the two dense layers in the traditional Transformer feedforward layer are improved into two 1D convolutional networks to extract the correlation information between adjacent hidden elements.

2. The non-intrusive load decomposition method based on mathematical morphology and improved Transformer according to claim 1, characterized in that, In step 1), the non-intrusive load identification device is used to obtain the power sequence of the target electrical appliance to be measured and the total household power consumption sequence. Only the low-frequency power of 1 Hz at the power inlet is used, and the original data obtained has large noise and many abnormal data, so data preprocessing operations need to be carried out. Specifically, an improved box plot is used to process data outliers. The steps for processing outliers using the improved box plot include: Step 1.1, calculate the upper quantile Q U , lower quantile Q L and the median; Step 1.2, set the maximum value Max = Q U + 1.5(Q U - Q L ), minimum value Min = Q L - 1.5(Q U - Q L ); Step 1.3, if the data date satisfies: date > Max or date < Min, then it is determined as an outlier; Step 1.4, delete the outlier and use the Lagrange interpolation method to fill the resulting missing value; Step 1.5, repeat steps 1.1 to 1.4 to check the outlier processing situation; After cleaning up the abnormal data, the data is divided into training and test sets according to the proportions to complete the construction of the dataset; Among them, the Lagrange interpolation method approximates the relationship between input and output using a polynomial function. For n+1 sample points (x0, y0), (x1, y1), ..., (x... i ,y i ), ···, (x n ,y n ), where i = 0, 1, ..., n, x i Let y represent the i-th sample point of the total load power. i For the i-th sample point representing the device power, given any data x in the interval [a, b], the estimated value at x can be calculated using the following formula: In the formula, L n (x) represents the Lagrange interpolation polynomial, where i is an (n+1)th number, n represents the nth sampling point, j is an integer value between 0 and n that is not equal to i, and y i It is x i The function value at x i x j For mutually distinct nodes, l i (x) is a basis function.

3. The non-intrusive load decomposition method based on mathematical morphology and improved Transformer according to claim 2, characterized in that, In step 2), let the input signal be z(n), n∈{0,1,…,N-1}, with a total of N data points, and the structuring element SE be γ(m), m∈{0,1,…,M-1}, with a length of M. The dilation and erosion operations on this input signal are defined as follows: In the formula, It is an expansion operator; It is an erosion operator; Based on the cascading of expansion and erosion, the two operators for opening and closing are represented as follows: In the formula The '+' and '·' operators represent the opening and closing operators, respectively. Multiscale mathematical morphology (MMM) implements opening and closing operations by selecting SEs at different time scales. The opening and closing operations of MMM are given as follows: In the formula, s t Time scale; Morphologically, the on and off operations have low-pass characteristics. Cascading them can simultaneously filter out both positive and negative pulses. Depending on the cascading order, on-off (OC) or off-on (CO) filters can be formed, respectively. Due to the scalability of the on operation and the inverse scalability of the off operation, both filters have statistical biases. Therefore, the average combination of the two filters is used, and the filter output h(x) is expressed as: h(x) = [OC(f(x)) + CO(f(x))] / 2 In the formula, f(x) represents the input data; Therefore, a multi-scale mathematical morphology filter (MMMF) is proposed. It achieves waveform decomposition under flexible fluctuation periods by weighting and multi-scale SE of the filtered target signal. MMMF is expressed as: In the formula, k is the number of time scales; These are the weights of SE at each scale, s i This represents the i-th time scale. To reduce the noise impact in the filtering results of small-scale SE, the weight value is determined by the variance of the filtering noise at each scale. In the formula, It is s on scale i The mean square error of the filter; Finally, the constructed multi-scale mathematical morphology filter is used to filter the data in the dataset.

4. The non-intrusive load decomposition method based on mathematical morphology and improved Transformer according to claim 3, characterized in that, The improved Transformer neural network includes the following modules: Module 1, the feature extraction layer, consists of separable 1D convolutional layers and the ReLU activation function; Module 2 is a 2×2 average pooling layer; Module 3, LocalBiLSTM layer; Module 4, Multi-head Probabilistic Sparse Attention Mechanism Module, consists of h probabilistic sparse attention mechanism layers; Module 5, the feedforward layer, consists of two 1D convolutional layers; Module 6, Residual Layer; Module 7, Layer Normalization Layer; Module 8, linear output layer, consists of dense layers; The architecture of the entire improved Transformer neural network is described as follows: h1,h2,…,h T =LocalBiLSTM(x1,x2,…,x T ) y1,y2,...,y T =Dense(n1,n2,...,n T ) In the formula, x1, x2, ..., x T The input load sequence is h1, h2, ..., h T The sequence output after passing through the LocalBiLSTM layer. The sequence is normalized by LayerNorm1, u1, u2, ..., u T The sequence is processed through the MultiHeadAttention multi-head sparse probabilistic attention mechanism layer. The sequence is a normalized sequence after passing through LayerNorm2 layers, m1, m2, ..., m T Given a sequence n1, n2, ..., n after 1D convolution. T The sequence y1, y2, ..., y is normalized by LayerNorm2. T This is the final output sequence after passing through the dense layer; Therefore, the total loss function is expressed as: loss total =loss(Dense(Conv1D(Add&Norm(MultiheadAttention(LocalBiLSTM(feature(X))))))) In the formula, loss total represents the total loss function, Dense represents the dense layer, Add&Norm represents the residual layer and the layer normalization layer, feature represents the feature extraction layer, and X represents the input load power sequence.

5. The non-intrusive load decomposition method based on mathematical morphology and improved Transformer according to claim 4, characterized in that, In step 3), the filtered data is input into the constructed non-intrusive load model for decomposition training. First, the initial neural network parameters are set, including the optimizer, batch size, initial learning rate, maximum number of training iterations, and sliding window size. Then, the model hyperparameters are continuously adjusted and updated using data from the training set combined with a Bayesian optimizer. The optimal batch size is 256, the maximum number of training iterations is 200, and the optimal sliding window size is 5. Adam is selected as the optimizer, with an initial learning rate of 0.0001. The loss function is the minimum mean squared error (MSE). To further prevent overfitting, an early stopping mechanism is added, i.e., when… After the mean squared error stops decreasing for 10 iterations, the model will automatically stop training and introduce a learning rate decay mechanism with a decay rate of 0.1 and a decay step size of 1000. By continuously optimizing and adjusting the model's hyperparameters through the Bayesian optimizer, the model's accuracy can be improved. Similarly, the parameters in the convolutional layer, dense layer, LocalBiLSTM layer, and multi-head sparse probabilistic attention mechanism layer can also be optimized using the Bayesian optimizer. After training, the model parameters with the minimum loss are selected from the saved training parameters as the model parameters for the detection model. Then, the data in the test set is input into the trained model to obtain the user's decomposition results.

6. The non-intrusive load decomposition method based on mathematical morphology and improved Transformer according to claim 5, characterized in that, In step 3), to better evaluate the model's performance in performing the non-intrusive load decomposition task, the mean absolute error (MAE), normalized total signal error (SAE), F1 score, accuracy, precision, and recall are used for evaluation, respectively: In the formula, the predicted power value and the actual power value of a certain device at time t are respectively represented by... and x t ' represents N', where N' is the number of samples; In the formula, the actual power consumption and the predicted total power of a device within a time period t are respectively represented by... and Indicates, that is and In the formula, precision(c j ) represents accuracy; recall(c j F1-Score(c) represents recall rate; j ) is the F1 index; TP(c) j This indicates that category c was correctly identified. j The number of FN(c) j This indicates that category c could not be correctly identified. j The number of FP(c) j ) does not belong to c j Category, but identified as c j The number of.

Images