Non-intrusive load identification method based on linear complexity self-attention mechanism

Abstract

The application belongs to the field of non-intrusive load identification, and discloses a non-intrusive load identification method based on a linear complexity self-attention mechanism, which comprises the following steps: step 1: using a current sensor to collect the running data of a target electrical appliance in a laboratory, and using the data set for establishing a model; step 2: based on deep learning, constructing a non-intrusive load identification PTPNet model architecture of the linear complexity self-attention mechanism; step 3: using the collected data set and a public data set UK-DALE to perform training, verification and testing; the method disclosed by the application significantly improves the model performance in the field of non-intrusive load monitoring (NILM) by virtue of the innovative UFO module, and has excellent performance in processing high-power load events.

Description

Technical Field

[0001] This invention relates to the field of non-invasive load recognition, specifically a non-invasive load recognition method based on a linear complexity self-attention mechanism. Background Technology

[0002] With the development of smart grids, the monitoring and management of energy consumption is becoming increasingly important. To achieve this goal, Non-Intrusive Load Monitoring (NILM) technology has emerged, which can identify the energy consumption patterns of individual electrical appliances from a single total power consumption data set, thereby providing users with power consumption feedback and energy-saving suggestions. However, NILM technology faces many challenges, such as the diversity of electrical load characteristics, data noise and sparsity, algorithm complexity, and scalability.

[0003] To address these issues, NILM researchers employed various approaches, most commonly model-based and data-based methods. Model-based methods require a deep understanding of the characteristics of electrical loads, such as on / off states, steady-state and non-steady-state power consumption, and then use probabilistic or optimization models to decompose the total load. A representative example of this type of method is the Hidden Markov Model (HMM), which can model and infer the hidden states of electrical appliances. However, model-based methods also have limitations, such as overly simplistic assumptions about appliance characteristics, inaccurate parameter estimations, and difficulty in expanding the applicability to different appliance types.

[0004] Data-driven methods do not require much prior knowledge of appliance load characteristics. Instead, they learn and extract features directly from raw energy consumption data, and then use machine learning or deep learning models to identify appliances. Representative methods of this type include Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), which can utilize spatial and temporal information, respectively, to capture the characteristics of appliance loads. In recent years, data-driven methods have received increasing attention because they can overcome some of the shortcomings of model-based methods, such as dependence on appliance features, sensitivity to parameters, and limitations on appliance types.

[0005] However, data-based methods also have some shortcomings, such as high requirements for data volume and quality, high consumption of computing resources, and weak ability to process long-term series.

[0006] Based on the above problems, the applicant proposes a non-intrusive load identification method based on a linear complexity self-attention mechanism. Summary of the Invention

[0007] To reduce the impact of time scale and low sampling rate on event detection accuracy during event recognition, this invention proposes a non-intrusive load recognition method based on a linear complexity self-attention mechanism.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0009] A non-intrusive load identification method based on a linear complexity self-attention mechanism includes the following steps:

[0010] Step 1: Preprocess the public dataset UK-DALE by dividing it into training, validation, and test sets according to its length.

[0011] Step 2: Construct a non-intrusive load recognition PTPNet model based on a linear complexity self-attention mechanism. The non-intrusive load recognition PTPNet model includes an input layer, a UFO intermediate layer, a temporary pooling layer, and an output layer. The input layer includes convolution processing, normalization processing, deactivation processing, and pooling processing. The UFO layer includes normalization processing, UFO model processing, and convolution processing. The temporary pooling layer includes pooling processing, convolution processing, batch normalization processing, and deactivation processing. The output layer includes one-dimensional deconvolution, batch normalization processing, and one-dimensional convolution processing.

[0012] Step 3: Put the training set, validation set and test set into the non-invasive load recognition PTPNet model for iterative model training to obtain the non-invasive load recognition PTPNet model with the best recognition results.

[0013] Step 4: Use the model with the best identification results for non-intrusive load event identification to identify the type of electrical appliance.

[0014] Furthermore, in step 1, the training set, validation set, and test set consist of the first 80% of the data for each house, the validation set consists of 80% to 90% of the data, and the test set consists of the last 10% of the data.

[0015] Furthermore, in step 2, the workflow of the non-invasive load identification PTPNet model includes the following steps:

[0016] S1 involves refining and encoding the total power time series features of the input dataset, specifically including:

[0017] S1.1, Perform convolution on the total power of the input dataset to obtain the convolution output;

[0018] S1.2, The output of the convolution process is processed by an activation function to obtain the output of the activation process;

[0019] S1.3, normalize the output of the activation process to obtain the normalized output;

[0020] S1.4, the output of the normalization process is randomly deactivated to obtain the output of the first coding layer, which consists of convolution processing, activation processing, normalization processing and random deactivation processing;

[0021] S1.5, the output of the first coding layer is processed by max pooling and average pooling and then added together to obtain the first pooled output;

[0022] S1.6, the pooling output result is processed again from S1.1 to S1.5 to obtain the next layer pooling output result, until the fourth layer does not perform pooling processing, that is, the output result of the fourth layer only goes up to S1.4 and does not go through S1.5;

[0023] S1.7, Continue the pooling output result through steps S1.1 to S1.6 until the fourth coding layer that has not undergone pooling is output;

[0024] S2, the output of the fourth coding layer is processed through five steps: normalization, UFO model processing, normalization, convolution processing, and normalization, to obtain the UFO intermediate layer output;

[0025] S3, perform 7 time pooling operations at different scales on the UFO intermediate layer output to obtain 7 time pool outputs; concatenate the UFO intermediate layer output and the 7 time pool outputs to obtain a large tensor;

[0026] S4 performs one-dimensional deconvolution and batch normalization on the tensor to obtain the transposed convolution normalized output; then performs one-dimensional convolution on the transposed convolution normalized output to obtain the final output data.

[0027] Furthermore, the encoding layer module consists of an ordered sequence of one-dimensional convolution, batch normalization and regularization, max pooling and average pooling.

[0028] Furthermore, the UFO intermediate layer is an ordered process of layer normalization, UFO modeling, and cross-normalization.

[0029] Furthermore, the UFO model introduces a cross-normalization (XNorm) strategy to optimize the self-attention mechanism. In cross-normalization, the direct multiplication between keys and values ​​maps the input data to h clusters using a linear kernel method. Then, the dot product terms are scaled and weighted by projection weights. The mathematical expression for cross-normalization is:

[0030]

[0031]

[0032]

[0033]

[0034] Where i, j, n, N, h represent the sequence of elements in the matrix, XNorm represents the cross-normalization operation, and K T V represents the transpose of the matrix of keys in the self-attention process, V represents the matrix of values ​​in the attention process, and A is the attention operator. Represents the query vector. Q represents the key vector, and Q represents the matrix composed of query vectors.

[0035] The formula for cross-normalization can be expressed as:

[0036]

[0037] Based on this, the attention operation of the UFO module is defined by the formula:

[0038]

[0039] It becomes:

[0040] A(x)=XNormdim=filter(Q)·

[0041] (XNormdim=space(KT)·V)

[0042] Where Q, K, and V represent the query vector matrix, key matrix, and value matrix, respectively, and d k It is the dimension of the key vector, A(x) is the output attention weight matrix, XNorm dim＝filter (Q) represents the normalization of the query matrix Q after dimensional filtering, XNorm dim＝space (K T ·V) represents the normalization of the product of the transpose of the key matrix and the value matrix in the attention matrix in a specific dimensional space.

[0043] Furthermore, in S3, the temporary pooling layer is a quantitative splicing of the time-series pool output and the UFO intermediate layer output.

[0044] Furthermore, in S4, the output layer is an ordered process of one-dimensional transposed convolution, batch normalization, and one-dimensional convolution.

[0045] Furthermore, in step 3, during the training process, the non-intrusive load recognition PTPNet model will iterate multiple times. Each iteration will perform forward and backward propagation on the training set, validation set, and test set, calculate the loss value, and save the average training loss, average validation loss, and average test loss of each iteration. Based on this, the model with the best performance will be saved.

[0046] Furthermore, the network parameters for training, validation, and testing are optimized using the Adam optimization algorithm with appropriate learning rate, batch size, and period. During training, the network parameters are saved whenever the objective function calculated in the data validation part reaches a new minimum value. Finally, a certain number of different instances are trained to derive the overall average performance, verifying the superiority of the non-intrusive load identification PTPNet model.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] 1) It can avoid multiple recognition events caused by data fluctuations;

[0049] 2) It can reduce the impact of low sampling rate on the accuracy of event recognition. Attached Figure Description

[0050] Figure 1 This is a flowchart of the present invention.

[0051] Figure 2 This is a schematic diagram of the PTPNet model architecture for non-intrusive load identification in this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Reference Figure 1 and Figure 2 A non-intrusive load identification method based on a linear complexity self-attention mechanism includes the following steps:

[0054] Step 1: Perform data preprocessing on the public dataset UK-DALE. Based on the length of the public dataset UK-DALE, divide the dataset into training set, validation set, and test set. The training set is the collection of the first 80% of the data for each house, the validation set is the collection of the 80% to 90% of the data, and the test set is the collection of the last 10% of the data.

[0055] Step 2: Construct a non-intrusive load recognition PTPNet model based on a linear complexity self-attention mechanism. The non-intrusive load recognition PTPNet model includes an input layer, a UFO intermediate layer, a temporary pooling layer, and an output layer. The input layer includes convolution processing, normalization processing, deactivation processing, and pooling processing. The UFO layer includes normalization processing, UFO model processing, and convolution processing. The temporary pooling layer includes pooling processing, convolution processing, batch normalization processing, and deactivation processing. The output layer includes one-dimensional deconvolution, batch normalization processing, and one-dimensional convolution processing.

[0056] The workflow of the non-invasive load identification PTPNet model includes the following steps:

[0057] S1 involves refining and encoding the total power time series features of the input dataset, specifically including:

[0058] S1.1, Perform convolution on the total power of the input dataset to obtain the convolution output;

[0059] S1.2, The output of the convolution process is processed by an activation function to obtain the output of the activation process;

[0060] S1.3, normalize the output of the activation process to obtain the normalized output;

[0061] S1.4, the output of the normalization process is randomly deactivated to obtain the output of the first coding layer, which consists of convolution processing, activation processing, normalization processing and random deactivation processing;

[0062] S1.5, the output of the first coding layer is processed by max pooling and average pooling and then added together to obtain the first pooled output;

[0063] S1.6, the pooling output result is processed again from S1.1 to S1.5 to obtain the next layer pooling output result, until the fourth layer does not perform pooling processing, that is, the output result of the fourth layer only goes up to S1.4 and does not go through S1.5;

[0064] S1.7, Continue the pooling output result through steps S1.1 to S1.6 until the fourth coding layer that has not undergone pooling is output;

[0065] S2, the output of the fourth coding layer is processed through five steps: normalization, UFO model processing, normalization, convolution processing, and normalization, to obtain the UFO intermediate layer output;

[0066] S3 performs 7 time pooling operations at different scales on the UFO intermediate layer output to obtain 7 time pool outputs; concatenate the UFO intermediate layer output and the 7 time pool outputs to obtain a large tensor.

[0067] The temporary pooling layer is a quantitative splicing of the output of the time-series pool and the output of the UFO intermediate layer.

[0068] S4 performs one-dimensional deconvolution and batch normalization on the tensor to obtain the transposed convolution normalized output; then performs one-dimensional convolution on the transposed convolution normalized output to obtain the final output data.

[0069] The output layer consists of an ordered sequence of one-dimensional transposed convolution, batch normalization, and one-dimensional convolution.

[0070] The encoding layer module consists of ordered steps of one-dimensional convolution, batch normalization and regularization, max pooling and average pooling.

[0071] The intermediate layer of the UFO consists of an ordered process of layer normalization, UFO model, and cross-normalization.

[0072] The UFO model introduces a cross-normalization (XNorm) strategy to optimize the self-attention mechanism. In cross-normalization, the direct multiplication between keys and values ​​maps the input data to h clusters using a linear kernel method. Then, the dot product terms are scaled and weighted by projection weights. The mathematical expression for cross-normalization is:

[0073]

[0074]

[0075]

[0076]

[0077] Where i, j, n, N, h represent the sequence of elements in the matrix, XNorm represents the cross-normalization operation, and K T V represents the transpose of the matrix of keys in the self-attention process, V represents the matrix of values ​​in the attention process, and A is the attention operator. Represents the query vector. Q represents the key vector, and Q represents the matrix composed of query vectors.

[0078] The formula for cross-normalization can be expressed as:

[0079]

[0080] Based on this, the attention operation of the UFO module is defined by the formula:

[0081]

[0082] It becomes:

[0083] A(x) = XNormdim＝filter (Q)·(XNorm dim＝space (K T )·V)

[0084] Where Q, K, and V represent the query vector matrix, key matrix, and value matrix, respectively, and d k It is the dimension of the key vector, A(x) is the output attention weight matrix, XNorm dim＝filter (Q) represents the normalization of the query matrix Q after dimensional filtering, XNorm dim＝space (K T ·V) represents the normalization of the product of the transpose of the key matrix and the value matrix in the attention matrix in a specific dimensional space.

[0085] Step 3: Put the training set, validation set and test set into the non-invasive load recognition PTPNet model for iterative model training to obtain the non-invasive load recognition PTPNet model with the best recognition results.

[0086] During training, the non-intrusive load recognition PTPNet model iterates multiple times. Each iteration performs forward and backward propagation on the training, validation, and test sets, calculates the loss value, and saves the average training loss, average validation loss, and average test loss of each iteration. Based on this, the model with the best performance is saved.

[0087] The network parameters for training, validation, and testing are optimized using the Adam optimization algorithm with appropriate learning rate, batch size, and period. During training, the network parameters are saved whenever the objective function calculated in the data validation part reaches a new minimum value. Finally, a certain number of different instances are trained to derive the overall average performance, verifying the superiority of the non-intrusive load identification PTPNet model.

[0088] Step 4: Use the model with the best identification results for non-intrusive load event identification to identify the type of electrical appliance.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A non-intrusive load identification method based on a linear complexity self-attention mechanism, characterized in that, Includes the following steps: Step 1: Preprocess the public dataset UK-DALE by dividing it into training, validation, and test sets according to its length. Step 2: Construct a non-intrusive load recognition PTPNet model based on a linear complexity self-attention mechanism. The non-intrusive load recognition PTPNet model includes an input layer, a UFO intermediate layer, a temporary pooling layer, and an output layer. The input layer includes convolution processing, normalization processing, deactivation processing, and pooling processing. The UFO intermediate layer includes normalization processing, UFO model processing, and convolution processing. The temporary pooling layer includes pooling processing, convolution processing, batch normalization processing, and deactivation processing. The output layer includes one-dimensional deconvolution, batch normalization processing, and one-dimensional convolution processing. The workflow of the non-invasive load identification PTPNet model includes the following steps: S1 involves refining and encoding the total power time series features of the input dataset, specifically including: S1.1, Perform convolution on the total power of the input dataset to obtain the convolution output; S1.2, The output of the convolution process is processed by an activation function to obtain the output of the activation process; S1.3, normalize the output of the activation process to obtain the normalized output; S1.4, the output of the normalization process is randomly deactivated to obtain the output of the first coding layer, which consists of convolution processing, activation processing, normalization processing and random deactivation processing; S1.5, the output of the first coding layer is processed by max pooling and average pooling and then added together to obtain the first pooled output; S1.6, the pooling output result is processed again from S1.1 to S1.5 to obtain the next layer pooling output result, until the fourth layer does not perform pooling processing, that is, the output result of the fourth layer only goes up to S1.4 and does not go through S1.5; S1.7, Continue the pooling output result through steps S1.1 to S1.6 until the fourth coding layer that has not undergone pooling is output; S2, the output of the fourth coding layer is processed through five steps: normalization, UFO model processing, normalization, convolution processing, and normalization, to obtain the UFO intermediate layer output; S3, perform 7 time pooling operations at different scales on the UFO intermediate layer output to obtain 7 time pool outputs; concatenate the UFO intermediate layer output and the 7 time pool outputs to obtain a large tensor; S4. Perform one-dimensional deconvolution and batch normalization on the tensor to obtain the transposed convolution normalized output; perform one-dimensional convolution on the transposed convolution normalized output to obtain the final output data. Step 3: Put the training set, validation set and test set into the non-invasive load recognition PTPNet model for iterative model training to obtain the non-invasive load recognition PTPNet model with the best recognition results. Step 4: Use the model with the best identification results for non-intrusive load event identification to identify the type of electrical appliance.

2. The non-intrusive load identification method based on a linear complexity self-attention mechanism as described in claim 1, characterized in that, In step 1, the training set, validation set, and test set are as follows: the training set is the collection of the first 80% of the data for each house; the validation set is the collection of the 80% to 90% of the data; and the test set is the collection of the last 10% of the data.

3. The non-intrusive load identification method based on a linear complexity self-attention mechanism as described in claim 1, characterized in that, The encoding layer module consists of an ordered sequence of one-dimensional convolution, batch normalization and regularization, max pooling and average pooling.

4. The non-intrusive load identification method based on a linear complexity self-attention mechanism as described in claim 1, characterized in that, The UFO model introduces a cross-normalization strategy to optimize the self-attention mechanism. In cross-normalization, the direct multiplication between keys and values ​​maps the input data to h clusters using a linear kernel method. Then, the dot product terms are scaled and weighted by projection weights. The mathematical expression for cross-normalization is: Where i, j, n, N, h represent the sequence of elements in the matrix, XNorm represents the cross-normalization operation, and K T V represents the transpose of the matrix of keys in the self-attention process, V represents the matrix of values ​​in the attention process, and A is the attention operator. Represents the query vector. Q represents the key vector, and Q represents the matrix composed of the query vectors. The formula for cross-normalization can be expressed as: Based on this, the attention operation of the UFO module is defined by the formula: It becomes: Where Q, K, and V represent the query vector matrix, key matrix, and value matrix, respectively, and d k It is the dimension of the key vector, A(x) is the output attention weight matrix, XNorm dim=filter (Q) represents the normalization of the query matrix Q after dimensional filtering, XNorm dim=space (K T ·V) represents the normalization of the product of the transpose of the key matrix and the value matrix in the attention in a specific dimensional space.

5. The non-intrusive load identification method based on a linear complexity self-attention mechanism as described in claim 1, characterized in that, In S3, the temporary pooling layer is a quantitative splicing of the time-series pool output and the UFO intermediate layer output.

6. The non-intrusive load identification method based on a linear complexity self-attention mechanism as described in claim 1, characterized in that, In step 3, during the training process, the non-intrusive load recognition PTPNet model will iterate multiple times. Each iteration will perform forward and backward propagation on the training set, validation set, and test set, calculate the loss value, and save the average training loss, average validation loss, and average test loss of each iteration. Based on this, the model with the best performance will be saved.

7. The non-intrusive load identification method based on a linear complexity self-attention mechanism as described in claim 6, characterized in that, The network parameters for training, validation, and testing are optimized using the Adam optimization algorithm with appropriate learning rate, batch size, and period. During training, the network parameters are saved whenever the objective function calculated in the data validation part reaches a new minimum value. Finally, a certain number of different instances are trained to derive the overall average performance, verifying the superiority of the non-intrusive load identification PTPNet model.

Images