A non-intrusive load disaggregation method based on global-local attention and pooling scaling

Abstract

The application provides a non-intrusive load decomposition method based on global-local attention and pooling scaling. The method proposes a non-intrusive load decomposition network, and the load decomposition process comprises: inputting load data into a feature extraction module for feature extraction to generate a feature sequence, inputting the feature sequence into a first feature enhancement module to perform rotation position coding and learn global features in the feature sequence, performing a pooling operation on the global features and sequentially inputting the global features into the first two second feature enhancement modules to learn local features in the feature sequence layer by layer, performing an upsampling operation on the local features and fusing the local features with the global features, inputting the fused features into the last second feature enhancement module to obtain final fused features; and a output module performs load decomposition according to the final fused features. The application can effectively extract global and local features in the load data sequence, thereby improving the accuracy of load decomposition.

Description

Technical Field

[0001] This invention relates to the field of smart grid technology, and in particular to a non-intrusive load decomposition method based on global local attention and pooling scaling. Background Technology

[0002] Smart grids have become an inevitable trend in energy development. A key technology is understanding detailed electricity consumption, followed by optimizing power usage and distribution to achieve rational energy utilization. This process is known as load monitoring. Currently, non-intrusive load monitoring has become the mainstream monitoring method due to its low economic cost and ease of installation.

[0003] Non-intrusive load monitoring collects information such as voltage, current, and active / reactive power within a residence by installing monitoring instruments at the main electricity meter. After feature extraction, this data is compared with a pre-established database of electrical load characteristics to identify the power consumption of each appliance in the house, thus achieving load decomposition. This method helps reduce redundant data acquisition and improves monitoring efficiency.

[0004] Currently, non-intrusive load decomposition methods are mainly based on deep learning, utilizing various network layers such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) for feature extraction. However, although these network models have shown good decomposition accuracy on specific datasets, CNNs cannot effectively extract time-series information, while RNNs have weak channel information extraction capabilities. Furthermore, these models have poor generalization ability, requiring retraining for new datasets. Summary of the Invention

[0005] In order to at least partially address the problems of limited feature extraction capability and poor model generalization ability of existing deep learning-based load decomposition methods, this invention proposes a non-intrusive load decomposition method based on global local attention and pooling scaling.

[0006] This invention provides a non-intrusive load decomposition method based on global local attention and pooling scaling, comprising:

[0007] Step 1: Construct a non-intrusive load decomposition network, including a feature extraction module, a first feature enhancement module, three second feature enhancement modules, and an output module connected in sequence; wherein, pooling operations are performed after the first feature enhancement module and the first second feature enhancement module; and upsampling operations are performed between the last two second feature enhancement modules; correspondingly, the load decomposition process of the non-intrusive load decomposition network includes:

[0008] The load data is input to the feature extraction module for feature extraction to generate a feature sequence. The feature sequence is then input to the first feature enhancement module for rotational position encoding and learning of global features in the feature sequence. After pooling, the global features are sequentially input to the first two second feature enhancement modules to learn local features in the feature sequence layer by layer. The local features are then upsampled and fused with the global features. The fused features are then input to the last second feature enhancement module to obtain the final fused features. The output module performs load decomposition based on the final fused features.

[0009] Step 2: Train the non-intrusive load decomposition network using load data of the target type of electrical appliances to obtain a non-intrusive load decomposition model;

[0010] Step 3: Input the load data in the target room into the non-intrusive load decomposition model to obtain the load decomposition results.

[0011] Furthermore, the feature extraction module includes a convolutional layer and a pooling layer connected in sequence; the convolutional layer has a kernel size of 5 and a stride of 1; the pooling layer has a kernel size of 2 and a stride of 2.

[0012] Furthermore, the second enhancement module employs a GL-Transformer network, which is a network formed by adding a local window layer to a Transformer network;

[0013] Correspondingly, the GL-Transformer network's processing of the input feature sequence includes: inputting the original feature sequence into a multi-head self-attention layer in the Transformer network to calculate global attention features; inputting the original feature sequence into a local window layer to obtain a new feature sequence; and inputting the new feature sequence into a multi-head self-attention layer in the Transformer network to obtain local attention features. Specifically, inputting the original feature sequence into the local window layer to obtain a new feature sequence includes: defining a local window size l. s The current position in the feature sequence and its leftmost position are located at l. s The elements within the range are set to 1, and the elements to the right of the current position are set to 0 to generate a local window mask. The local window mask is multiplied with the original feature sequence to obtain a new feature sequence. The global attention features and the local attention features are weighted and fused using the given weights to obtain a fused representation.

[0014] Furthermore, the first feature enhancement module employs a Ro-Transformer network; the Ro-Transformer network is a network formed by adding a RoPE layer before the multi-head self-attention layer in the GL-Transformer network; wherein, the RoPE layer is used to perform rotation position encoding on the input feature sequence.

[0015] Furthermore, the output module includes a transposed convolutional layer and a fully connected layer connected in sequence.

[0016] Furthermore, during the training process in step 2, the following formula is used as the loss function:

[0017]

[0018] Where L is the total loss, JSD represents the JS distance function, and p and These represent the actual load data and the predicted load data, respectively. τ is the absolute error adjustment value between the actual load data and the predicted load data, and T represents the time step.

[0019] Furthermore, before feature extraction from the raw load data, the method further includes: data preprocessing of the raw load data; the data preprocessing includes:

[0020] The data is resampled to form a one-dimensional sequence;

[0021] The one-dimensional sequence is subjected to data cleaning operations, including deleting abnormal data and filling in missing data;

[0022] Standardize all data in the one-dimensional sequence after data cleaning.

[0023] Furthermore, the filling of missing data includes: for missing data with a duration less than a first duration, a positive filling method is used; for missing data with a duration greater than a second duration, a zero filling method is used.

[0024] The beneficial effects of this invention are:

[0025] (1) By constructing a first feature enhancement module containing rotation position encoding, a unique encoding is provided for each position in the load data sequence so that the model can distinguish the labels of different positions and prepare for further learning of shallow and deep features in the sequence.

[0026] (2) By constructing a second feature enhancement module to learn global and local attention features of the input load data sequence, the model can better learn the global trends and local details in the load data sequence, thereby improving the accuracy of load identification;

[0027] (3) By constructing the first and second feature enhancement modules on a traditional Transformer network, the multi-head self-attention mechanism can be fully utilized to extract features of load data sequences with temporal characteristics. Furthermore, it fully considers both global and local features of the load data, enabling the model to capture the overall energy consumption patterns and trends, while also identifying and distinguishing the energy consumption patterns of different devices at a more detailed level. In addition, the local attention mechanism can more effectively process real signals, reduce the sensitivity of local areas to noise, help counteract noise interference in the data, and improve the robustness of the model. Simultaneously, this invention can reduce computational resource consumption, significantly shorten load identification time, and improve the accuracy of load identification. Attached Figure Description

[0028] Figure 1 This is one of the flowcharts of a non-intrusive load decomposition method based on global local attention and pooling scaling provided in an embodiment of the present invention;

[0029] Figure 2 This is a second flowchart illustrating a non-intrusive load decomposition method based on global local attention and pooling scaling, provided as an embodiment of the present invention.

[0030] Figure 3 This is a diagram of the Ro-Transformer network structure provided in an embodiment of the present invention;

[0031] Figure 4 This is a diagram of the GL-Transformer network structure provided in an embodiment of the present invention;

[0032] Figure 5 This is a structural diagram of the output module provided in an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0034] Combination Figure 1 and Figure 2 As shown, this embodiment of the invention provides a non-intrusive load decomposition method based on global local attention and pooling scaling, including the following steps:

[0035] S101: Construct a non-intrusive load decomposition network, including a feature extraction module, a first feature enhancement module, three second feature enhancement modules, and an output module connected in sequence; wherein, pooling operations are performed after the first feature enhancement module and the first second feature enhancement module; in this embodiment of the invention, the window size and stride of the pooling operation are 2. An upsampling operation is performed between the last two second feature enhancement modules to ensure that the local features output by the second feature enhancement module have the same shape as the global features, facilitating their fusion.

[0036] Correspondingly, the load decomposition process of the non-intrusive load decomposition network includes: inputting load data into a feature extraction module for feature extraction to generate a feature sequence; inputting the feature sequence into a first feature enhancement module for rotational position encoding and learning global features in the feature sequence; performing pooling operations on the global features and then sequentially inputting them into the first two second feature enhancement modules to learn local features in the feature sequence layer by layer; performing upsampling operations on the local features and then fusing them with the global features; inputting the fused features into the last second feature enhancement module to obtain the final fused features; and the output module performing load decomposition based on the final fused features.

[0037] In this embodiment of the invention, the feature extraction module includes a convolutional layer and a pooling layer connected in sequence; the convolutional layer has a kernel size of 5 and a stride of 1; the pooling layer has a kernel size of 2 and a stride of 2.

[0038] As one possible implementation, the structure of the first feature enhancement module is as follows: Figure 3 As shown, the structure of the second feature enhancement module is as follows: Figure 4 As shown. Since the first feature enhancement module is a further improvement on the second feature enhancement module, the structure and feature processing of the second feature enhancement module will be described in detail here first.

[0039] In this embodiment of the invention, the second enhancement module adopts a GL-Transformer network, which is a network formed by adding a local window layer to a Transformer network; where G and L represent the abbreviations of Global and Local, respectively.

[0040] Specifically, traditional Transformer networks can only perform global attention calculations on the input data, ignoring local details. Therefore, this invention proposes the GL-Transformer network to enable the model to learn local details within the load data. The main difference between the GL-Transformer network and traditional Transformer networks lies in the introduction of a local window layer. This local window layer masks the input data to obtain multiple local sub-input data points, and then performs attention calculations based on these temporally related sub-input data points to obtain local attention.

[0041] Furthermore, the GL-Transformer network processes the input feature sequence as follows:

[0042] Step A1: Input the original feature sequence into the multi-head self-attention layer in the Transformer network to calculate the global attention feature G. m ;

[0043] Step A2: Input the original feature sequence into the local window layer to obtain a new feature sequence, and input the new feature sequence into the multi-head self-attention layer in the Transformer network to obtain the local attention feature L. m The process involves inputting the original feature sequence into a local window layer to obtain a new feature sequence, specifically including: defining the local window size l. s The current position in the feature sequence and its leftmost position are located at l. s The elements within the range are set to 1, and the elements to the right of the current position are set to 0 to generate a local window mask. The local window mask is then multiplied by the original feature sequence to obtain a new feature sequence.

[0044] Specifically, the process by which the multi-head self-attention layer calculates attention features based on the input data is as follows:

[0045]

[0046] head i =Attention(QW i Q ,KW i K VW i V )

[0047] MultiHead(Q,K,V)=Concat(head1,…,head h W O

[0048] Where Q, K, and V represent the Query, Key, and Value matrices, respectively, and d k Let x be the dimension of the matrix, and W be the input signal data. q W k W v The input signal data is converted into a coefficient matrix of Q, K, and V. i W represents the i-th head in the multi-head attention mechanism. i Q W i K W i V W represents the coefficient matrix of Q, K, and V corresponding to each head. O This indicates that the join result is mapped.

[0049] Step A3: Using the given weights G w and L w For the global attention feature G m and the local attention feature L m We perform weighted fusion to obtain the fused representation F. a =G w ×G m +L w ×L m .

[0050] The entire process described above achieves a fusion of global attention, local attention, and linearity. By setting weights, the contributions of both can be balanced, enabling the model to better handle global trends and local details, and helping to model energy consumption patterns more accurately.

[0051] Furthermore, the first feature enhancement module employs a Ro-Transformer network; the Ro-Transformer network is a network formed by adding a RoPE layer before the multi-head self-attention layer in the GL-Transformer network; wherein, the RoPE layer is used to perform rotation position encoding on the input feature sequence.

[0052] Specifically, the core idea of ​​rotational position encoding is to transform the Query and Key formed in the Transformer model to carry positional information, allowing the model to automatically perceive the relative positional relationships of different elements in the sequence without requiring additional modifications in subsequent dot product operations. The specific calculation method of rotational position encoding is as follows:

[0053] First, add the absolute position information m,n to q,k:

[0054]

[0055] in, and Carry absolute position information m, n;

[0056] Furthermore, in order to and The result of the inner product contains relative positional information. We need to find an equivalent method to encode these positions so that the above relationship holds, assuming a constant relationship exists:

[0057] f(q,m),f(k,n)=g(q,k,mn)

[0058] After formula transformation, the relative position code represented by complex numbers in the two-dimensional case is finally obtained:

[0059] f(q,m)=qe imθ

[0060] According to the geometric meaning of complex multiplication, this transformation actually corresponds to a vector rotation operation, hence it is called rotational position embedding, which means that the vector q is multiplied on the left by a rotation matrix, written in matrix form as follows:

[0061]

[0062] Left-multiply the vector q at position m by matrix R. m Multiply the vector k at position n by matrix R on the left. n We obtain the transformed q and k, and use them to calculate the attention mechanism, enabling attention to automatically suggest relative positional information. This yields the following constant:

[0063]

[0064] Where nm represents relative position information.

[0065] Because rotational position encoding represents relative position information in the manner of absolute position encoding and integrates relative position information into the self-attention mechanism, the Transformer can more effectively focus on the relationship between different times. Therefore, it improves the model's sensitivity to the relationship between power load changes at different time points, helps the model to more accurately capture the energy consumption patterns of electrical equipment, and improves model performance.

[0066] Furthermore, such as Figure 5As shown, the output module consists of a transposed convolutional layer and a fully connected layer connected sequentially. The transposed convolutional layer and the fully connected layer restore the feature dimension, ultimately predicting a result consistent with the output power dimension. The transposed convolution, also known as deconvolution, restores the dimension because the input power sequence typically shrinks in terms of output channels and dimensions after passing through a series of neural network layers; therefore, it needs to be restored to its original size for further computation. The fully connected layer primarily performs an identity mapping, with the following formula:

[0067] Y(x) = Wx + b

[0068] Where W is the weight matrix, whose parameters can be learned through backpropagation of the neural network, and b is the bias term.

[0069] S102: The non-intrusive load decomposition network is trained using load data of the target type of electrical appliances to obtain a non-intrusive load decomposition model;

[0070] Specifically, considering that using a single mean squared error or mean absolute error is insufficient to assess the gap between the true and predicted values, this embodiment of the invention employs a combined loss function for comprehensive evaluation during the training process. This combined loss function consists of three parts: the standard mean squared error loss function, the Jensen-Shannon divergence, and the probability distribution between the measured and predicted values.

[0071]

[0072] Where L is the total loss, p and These represent the actual load data (e.g., actual power value) and the predicted load data (e.g., predicted power value), respectively. τ is the absolute error adjustment value between the actual and predicted load data, and T represents the time step. In this loss function, the first term is the mean squared error loss function, and the second term is the JS distance function, used to evaluate the probability distribution of the dispersion between the predicted and actual values.

[0073] S103: Input the load data in the target room into the non-intrusive load decomposition model to obtain the load decomposition results.

[0074] The non-intrusive load decomposition method provided in this invention first performs convolution and pooling processing on the input load data sequence, and then embeds it into a Ro-Transformer. Next, pooling is performed on the output of the Ro-Transformer to extract global features from the sequence by downsampling and preserving key feature information; this process passes positional information, extracts and learns shallow features, laying the foundation for subsequent deep feature learning. These global features are then passed to a GL-Transformer, which performs pooling again to emphasize local feature information in the sequence. This is then passed to another GL-Transformer to further extract more important feature information, resulting in a more abstract and refined representation. This process progressively extracts and emphasizes key features at different levels in the input sequence, enhancing the model's understanding and representation of key features during the operation of electrical appliances.

[0075] Based on the above embodiments, in order to improve the accuracy of load decomposition, both the training and testing processes also include the following steps:

[0076] Before feature extraction from the raw load data, the method further includes: data preprocessing of the raw load data; the data preprocessing includes:

[0077] The data is resampled to form a one-dimensional sequence;

[0078] The one-dimensional sequence is subjected to data cleaning operations, including deleting abnormal data and filling in missing data;

[0079] For example, if the power value at a certain moment in a one-dimensional sequence is negative or exceeds the maximum power that the appliance can withstand, then the power value is abnormal data and should be deleted. In this embodiment of the invention, when filling missing data, the following two cases are used: First, for missing data with a duration less than a first duration (e.g., 4 minutes), a positive filling method is used, that is, the power data of the previous moment is used to fill the missing data to maintain consistency; for missing data with a duration greater than a second duration (e.g., 2 minutes), a zero-filling method is used.

[0080] The standardization operation is performed on all data in the one-dimensional sequence after data cleaning, using the following formula:

[0081]

[0082] Where x is the input sequence, μ is the mean, and σ is the standard deviation.

[0083] The non-intrusive load decomposition method based on global-local attention and pooling scaling provided in this invention mainly includes three stages: data preprocessing, constructing and training a load decomposition model (including Ro-Transformer and GL-Transformer networks), and using the trained model for load decomposition. Specifically, by adding rotational position encoding to the constructed Ro-Transformer network, a unique code is provided for each position in the sequence, enabling the model to distinguish the labels of different positions. The constructed GL-Transformer network is used to learn features globally and locally, allowing the model to better learn the global trends and local details in the sequence. This load decomposition method of the present invention can reduce computational resource costs, shorten load identification time, and improve the accuracy of load identification.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A non-intrusive load decomposition method based on global local attention and pooling scaling, characterized in that, include: Step 1: Construct a non-intrusive load decomposition network, comprising a feature extraction module, a first feature enhancement module, three second feature enhancement modules, and an output module connected in sequence. Pooling operations are performed after the first feature enhancement module and the first second feature enhancement module. Upsampling operations are performed between the last two second feature enhancement modules. The feature extraction module comprises convolutional layers and pooling layers connected in sequence. The convolutional layer has a kernel size of 5 and a stride of 1. The pooling layer has a kernel size of 2 and a stride of 2. The first feature enhancement module uses a Ro-Transformer network. The Ro-Transformer network is formed by adding a RoPE layer before the multi-head self-attention layer in a GL-Transformer network. The RoPE layer is used to encode the rotation position of the input feature sequence. The second enhancement modules use a GL-Transformer network, which is formed by adding a local window layer to a Transformer network. The output module comprises a transposed convolutional layer and a fully connected layer connected in sequence. The load decomposition process of the non-intrusive load decomposition network includes: The load data is input to the feature extraction module for feature extraction to generate a feature sequence. The feature sequence is then input to the first feature enhancement module for rotational position encoding and learning of global features in the feature sequence. After pooling, the global features are sequentially input to the first two second feature enhancement modules to learn local features in the feature sequence layer by layer. The local features are then upsampled and fused with the global features. The fused features are then input to the last second feature enhancement module to obtain the final fused features. The output module performs load decomposition based on the final fused features. Step 2: Train the non-intrusive load decomposition network using load data of the target type of electrical appliances to obtain a non-intrusive load decomposition model; Step 3: Input the load data in the target room into the non-intrusive load decomposition model to obtain the load decomposition results.

2. The non-intrusive load decomposition method based on global local attention and pooling scaling according to claim 1, characterized in that, The GL-Transformer network's processing of the input feature sequence includes: inputting the original feature sequence into a multi-head self-attention layer in the Transformer network to calculate global attention features; inputting the original feature sequence into a local window layer to obtain a new feature sequence; and inputting the new feature sequence into a multi-head self-attention layer in the Transformer network to obtain local attention features. Specifically, inputting the original feature sequence into the local window layer to obtain a new feature sequence includes: defining a local window size l. s The current position in the feature sequence and its leftmost position are located at l. s The elements within the range are set to 1, and the elements to the right of the current position are set to 0 to generate a local window mask. The local window mask is multiplied with the original feature sequence to obtain a new feature sequence. The global attention features and the local attention features are weighted and fused using the given weights to obtain a fused representation.

3. The non-intrusive load decomposition method based on global local attention and pooling scaling according to claim 1, characterized in that, During the training process in step 2, the following formula is used as the loss function: in, For the total loss, This represents the JS distance function. and These are the actual load data and the predicted load data, respectively. It is the absolute error adjustment value between the actual load data and the predicted load data, where T represents the time step.

4. The non-intrusive load decomposition method based on global local attention and pooling scaling according to claim 1, characterized in that, Before feature extraction from the raw load data, the method further includes: data preprocessing of the raw load data; the data preprocessing includes: The data is resampled to form a one-dimensional sequence; The one-dimensional sequence is subjected to data cleaning operations, including deleting abnormal data and filling in missing data; Standardize all data in the one-dimensional sequence after data cleaning.

5. A non-intrusive load decomposition method based on global local attention and pooling scaling according to claim 4, characterized in that, The filling of missing data includes: for missing data with a duration less than a first duration, a positive filling method is used; for missing data with a duration greater than a second duration, a zero filling method is used.

Images