Load decomposition method and system based on deep learning attention mechanism

Abstract

The present invention provides a load decomposition method and system based on a deep learning attention mechanism, relating to the technical field of load monitoring. The present invention processes the total power signal through a pre-built TransUNet-NILM model to obtain high-resolution data. The model introduces the self-attention mechanism of the Transformer into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. During the construction of the model, the input sequence is decoded through the upsampling block to obtain multi-scale features, and the multi-scale features are cropped to obtain subsequences. A sequence-to-subsequence method proposed in the present invention strikes a balance between seq2seq and seq2point to balance the convergence difficulties in deep neural networks, making training easier and reducing the amount of computation during inference.

Description

Technical Field

[0001] The present invention relates to the field of load monitoring technology, and in particular to a load decomposition method and system based on a deep learning attention mechanism. Background Art

[0002] Load disaggregation (also known as non-intrusive load monitoring or NILM) is a computational technique used to estimate the electricity demand of individual appliances from a single meter that measures the combined demand of multiple appliances. The ultimate goal of load disaggregation is to help reduce residential energy consumption, thus minimizing the impact on user privacy, as the term "non-intrusive" suggests; to help operators manage the grid; to identify faulty appliances; or to investigate appliance usage behavior. Load disaggregation methods derive information about appliance energy usage from total electricity consumption without requiring additional sensors. Therefore, this method reduces the cost of sensing infrastructure and relies on load disaggregation methods to monitor the load of appliances. Load disaggregation algorithms can inform end consumers of potential energy savings and can also be used for demand response management. They also allow for the development of new fair pricing policies, particularly for the electricity market as a whole, which favors green credits.

[0003] With the recent success of deep learning methods in many different fields, load decomposition research has also migrated from traditional signal processing methods to deep learning architectures. In the deep learning framework, NILM is considered as a sequence-to-sequence (seq2seq), sequence-to-point (seq2point), or sequence-to-subsequence (seq2subseq) problem with single-label or multi-label state classification and energy usage prediction tasks.

[0004] However, existing methods still have certain problems. For example, for the seq2seq method, when the length (time window) of the input (power supply) and output (device) sequences becomes longer, the learning will make it difficult for the training process to converge. That is, in the existing deep learning framework, it is difficult to converge during the training process. Summary of the Invention

[0005] (1) Technical problems solved

[0006] In response to the shortcomings of the existing technology, the present invention provides a load decomposition method and system based on a deep learning attention mechanism, which solves the technical problem of difficulty in convergence during training in the existing methods.

[0007] (2) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0009] In a first aspect, the present invention provides a load decomposition method based on a deep learning attention mechanism. The load decomposition method processes the total power signal through a pre-built TransUNet-NILM model to obtain high-resolution data. The TransUNet-NILM model introduces the Transformer self-attention mechanism into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The construction process of the TransUNet-NILM model includes:

[0010] S1. Obtain a total power signal with a true label, and process the total power signal to obtain a training set;

[0011] S2, extract features from the training set through the downsampling block to obtain the embedding matrix;

[0012] S3. Encode the embedding matrix through the Transformer block to obtain the input sequence;

[0013] S4. Decode the input sequence through the upsampling block to obtain multi-scale features, and crop the multi-scale features to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data;

[0014] S5. Use the mean absolute error between the true label and high-resolution data as the optimization target to optimize the TransUNet-NILM model.

[0015] Preferably, the downsampling block includes multiple convolutional layers and pooling layers in sequence from input to output;

[0016] The feature extraction of the training set is performed through the downsampling block to obtain an embedding matrix, including:

[0017] Multiple convolutional layers are used to increase the hidden size of the data in the training set, position embedding is introduced, and the learning L in the pooling layer is used to 2 The norm pooling operation pools the convolution outputs with increasing hidden sizes to obtain an embedding matrix.

[0018] Preferably, the Transformer block includes a multi-head attention mechanism layer, two LayerNorm layers, and a feedforward neural network;

[0019] The embedding matrix is ​​encoded through the Transformer block to obtain the input sequence, including:

[0020] S301, performing a linear transformation on the embedded matrix to obtain Q, K, and V matrices of the embedded matrix;

[0021] S302. Obtain multiple subspaces of multiple Q, K, and V matrices through a multi-head attention layer, and concatenate the multiple subspaces to obtain an output sequence.

[0022] S303, normalizing the output sequence through the first layer LayerNorm;

[0023] S304. Perform residual connection on multiple normalized output sequences through a multi-head attention layer and a position feedforward neural network to obtain a spliced ​​sequence, and perform normalization operation on the spliced ​​sequence through the second layer LayerNorm to obtain an input sequence.

[0024] Preferably, the upsampling block includes, from input to output, a deconvolution layer, a convolution layer, a window cropping layer, and an output layer;

[0025] The upsampling block is used to decode the input sequence to obtain multi-scale features, and the multi-scale features are cropped to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data, including:

[0026] S401, by using multiple upsampling blocks, where each upsampling block includes a one-dimensional deconvolution layer and a convolution layer, the encoding input to the upsampling block first passes through the deconvolution layer, and the high-level features obtained by the downsampling block are concatenated with the high-resolution features obtained by convolution of multiple convolution layers in the upsampling block to obtain a multi-scale feature;

[0027] S402, cropping the multi-scale features through a sub-window cropping layer to obtain a sub-sequence;

[0028] S403. Repeat steps S401 to S402. The subsequence is finally input to the output layer. The output layer includes a convolutional layer and an MLP.

[0029] Preferably, the step of clipping the multi-scale features through a sub-window clipping layer to obtain a sub-sequence includes:

[0030] The center of the clipping window in the sub-window clipping layer is aligned with the center of the main window, and W'≤W / 2, where W' represents the clipping window and W represents the main window.

[0031] Preferably, the method further comprises:

[0032] S6. Test the TransUNet-NILM model using the test set to further optimize the TransUNet-NILM model.

[0033] In a second aspect, the present invention provides a load decomposition system based on a deep learning attention mechanism. The load decomposition system includes a training subsystem of a TransUNet-NILM model for training and a signal processing subsystem for calling the TransUNet-NILM model. The signal processing subsystem calls the TransUNet-NILM model to process the total power signal to obtain high-resolution data. The TransUNet-NILM model introduces the Transformer self-attention mechanism into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The training subsystem includes:

[0034] A training set acquisition module is used to obtain a total power signal with a real label, and process the total power signal to obtain a training set;

[0035] The downsampling module is used to extract features from the training set through the downsampling block to obtain the embedding matrix;

[0036] Transformer module, which encodes the embedding matrix through the Transformer block to obtain the input sequence;

[0037] The upsampling module is used to decode the input sequence through the upsampling block to obtain multi-scale features, and then crop the multi-scale features to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data;

[0038] The optimization module is used to optimize the TransUNet-NILM model using the mean absolute error of the true label and high-resolution data as the optimization target.

[0039] Preferably, the system further comprises:

[0040] The testing module is used to test the TransUNet-NILM model through a test set to further optimize the TransUNet-NILM model.

[0041] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program for load decomposition based on a deep learning attention mechanism, wherein the computer program enables a computer to execute the load decomposition method based on a deep learning attention mechanism as described above.

[0042] In a fourth aspect, the present invention provides an electronic device, comprising:

[0043] one or more processors;

[0044] Memory; and

[0045] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the load decomposition method based on the deep learning attention mechanism as described above.

[0046] (3) Beneficial effects

[0047] This invention provides a load decomposition method and system based on a deep learning attention mechanism. Compared with the existing technology, it has the following advantages:

[0048] The present invention processes the total power signal using a pre-built TransUNet-NILM model to obtain high-resolution data. This model incorporates the Transformer self-attention mechanism into the U-Net architecture as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The model construction process includes: obtaining a total power signal with a true label, processing the total power signal to obtain a training set; extracting features from the training set using the downsampling block to obtain an embedding matrix; encoding the embedding matrix using the Transformer block to obtain an input sequence; decoding the input sequence using the upsampling block to obtain multi-scale features, and cropping the multi-scale features to obtain subsequences. The subsequences pass through the output layer to obtain high-resolution data; and optimizing the TransUNet-NILM model using the mean absolute error between the true label and the high-resolution data as the optimization target. The present invention proposes a sequence-to-subsequence method that balances the seq2seq and seq2point approaches to address convergence difficulties in deep neural networks, making training easier and reducing the amount of computation during inference. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 is a block diagram of a load decomposition method based on a deep learning attention mechanism in an embodiment of the present invention;

[0051] Figure 2 This is a block diagram of a load decomposition system based on a deep learning attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0053] The embodiments of the present application solve the technical problem of difficulty in convergence during training in existing methods by providing a load decomposition method and system based on a deep learning attention mechanism, and achieve a trade-off between seq2seq and seq2point to balance the problem of convergence difficulties in deep neural networks and accelerate the convergence speed.

[0054] The technical solution in the embodiments of the present application is to solve the above technical problems, and the overall idea is as follows:

[0055] Currently, many different methods have been applied to load decomposition. Many methods based on signal processing and machine learning techniques have been proposed, such as the hidden Markov model (HMM) and its variations, graph signal processing, and combinatorial optimization methods. However, when the number of decomposed devices increases, HMM-based techniques are usually inefficient and have high computational complexity.

[0056] However, as deep learning methods have achieved good results in many different fields in recent years, load decomposition research has also migrated from traditional signal processing methods to deep learning architectures. Among the deep learning methods applied to load decomposition, recurrent neural networks (RNNs), denoising autoencoders (dAEs), long short-term memory (LSTMs), convolutional neural networks (CNNs), and generative adversarial networks (GANs) have shown success in power prediction. Generally, researchers view NILM as a sequence-to-sequence (seq2seq), sequence-to-point (seq2point), or sequence-to-subsequence (seq2subseq) problem, with single-label or multi-label state classification and energy usage prediction tasks.

[0057] The sequence-to-sequence model learns nonlinear regression between power reading sequences and appliance readings with the same time window. The sequence-to-point model does not train the network to predict device readings from a window, but instead uses a sliding window method to only predict the output signal at the midpoint of the window. This method utilizes all neighboring areas of the input sequence, namely the past and future, which allows the neural network representation capability to be concentrated on the midpoint of the window rather than on the more difficult edge, thereby producing more accurate output.

[0058] While recent preliminary research has demonstrated the significant potential of NILM, many challenges remain. The first is the trade-off between model computational complexity and the long-term dependencies of tracking energy consumption data, which often contain rich daily, seasonal, and even annual patterns. Most deep learning models require large amounts of high-quality labeled data for training, particularly per-device power consumption data. Data collection costs, such as through sub-metering, can be high. Furthermore, many users are reluctant to share their device information due to privacy concerns.

[0059] For load decomposition tasks, the power consumption patterns of different devices typically have different scales, and the aggregate consumption of multiple devices often has a more complex shape, thus requiring the ability to handle scale variations. In addition to data information within a timeframe, it is important to consider the contextual dependencies of consumption patterns, as energy consumption behaviors contain higher-level semantics, such as the dryer running after the washing machine, or people turning on the microwave multiple times until cooking is complete.

[0060] From the above description, it can be seen that the existing method has the following defects:

[0061] 1. For the seq2seq method, when the length (time window) of the input (power) and output (device) sequences becomes longer, the learning will make it difficult for the training process to converge. For the seq2point method, each forward process of the model only produces one output signal, thus increasing the amount of computation during inference.

[0062] 2. The RNN-based sequence-to-sequence model is not very capable of capturing the long-term dependencies of power signals within devices during load decomposition, resulting in long backpropagation paths and a high risk of vanishing or exploding gradients. The LSTM model approach is generally more akin to a Markov decision process, making it difficult to extract global information. The CNN-based model fails to leverage the relationship between appliance usage, leading to a high false positive / negative error rate in the decomposition results. Deep learning is used in decomposition by applying a neural network to each device in the target environment. However, achieving more accurate decomposition results for each device requires increasing the number of convolutional layers and filter resolution. This results in excessive computational cost and complexity as the number of decomposed devices increases, making it unsuitable for practical application.

[0063] 3. The Transformer model lacks the ability to generalize bias (the ability to capture local features) when extracting power consumption signal features. It does not have the translation invariance and locality of CNN. Therefore, it cannot generalize well to load decomposition tasks when data is insufficient, that is, it is difficult to achieve accurate decomposition.

[0064] In response to the shortcomings of the existing technology, the embodiments of the present invention propose a load decomposition method and system based on a deep learning attention mechanism.

[0065] In order to better understand the above technical solution, the above technical solution will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0066] The embodiment of the present invention provides a load decomposition method based on a deep learning attention mechanism. The method processes the total power signal through a pre-built TransUNet-NILM model to obtain high-resolution data. The model introduces the Transformer self-attention mechanism into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. Figure 1 As shown in Figure 2, the model construction process includes:

[0067] S1. Obtain a total power signal with a true label, and process the total power signal to obtain a training set;

[0068] S2, extract features from the training set through the downsampling block to obtain the embedding matrix;

[0069] S3. Encode the embedding matrix through the Transformer block to obtain the input sequence;

[0070] S4. Decode the input sequence through the upsampling block to obtain multi-scale features, and crop the multi-scale features to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data;

[0071] S5. Use the mean absolute error between the true label and high-resolution data as the optimization target to optimize the TransUNet-NILM model.

[0072] A sequence-to-subsequence method proposed in an embodiment of the present invention strikes a balance between seq2seq and seq2point to balance the convergence difficulties in deep neural networks, make training easier, and reduce the amount of computation during inference.

[0073] The following is a detailed description of each step:

[0074] The TransUNet-NILM model pre-built in this embodiment of the present invention introduces the Transformer self-attention mechanism into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The downsampling block includes multiple convolutional layers and pooling layers from input to output; the Transformer block includes a multi-head attention mechanism layer, two LayerNorm layers, and a feedforward neural network; and the upsampling block includes a deconvolution layer, a convolution layer, a window cropping layer, and an output layer from input to output. The model construction process is as follows:

[0075] In step S1, a total power signal with a true label is obtained and processed to obtain a training set. The specific implementation process is as follows:

[0076] In the embodiment of the present invention, let y(t)∈R W×d represents a set of input features derived from the total power consumption of N electrical devices, W represents the window size, X∈R W×N Represents the power signal of the relevant electrical equipment, and at time t each electrical equipment has k states, represented by s i (t) = {s i (t) 1 ,s i (t) 2 ,…,s i (t) k}, where s i (t) k ∈{0,1}.

[0077] S∈R W×N Represents the associated multi-tag state X∈R of N devices W×N is the corresponding power consumption, then the training set D can be expressed as follows:

[0078] D={y(t),s(t)|t=1,2,…,W}.

[0079] In step S2, the training set is feature extracted through the downsampling block to obtain the embedding matrix. The specific implementation process is as follows:

[0080] The training set is processed through multiple convolutional layers and pooling layers in the downsampling block to obtain the embedding matrix. Specifically, it includes:

[0081] Before the training set is fed into the Transformer block, feature extraction is performed in a downsampling section of the U-Net model. Multiple convolutional layers are used to increase the hidden size of the input data, and then the learning L in the pooling layer is used to extract the feature. 2The norm pooling operation pools the convolution output with increasing hidden size, which applies squared average pooling on the input data to preserve features and obtain patchy i (t). To encode the patch spatial information, position embeddings are introduced in embodiments of the present invention. These embeddings are added to the patch embeddings to capture the position sequence, as shown below:

[0082] z0=LPPooling(E)+E pos

[0083]

[0084] Where E∈R W×d is the patch embedding, E pos ∈R N×d It is position embedding, LPPooling represents LP pooling calculation, and Conv represents convolution operation.

[0085] The role of patch embedding is to convert the original 2D image into a 1D patch embedding, which is to do a convolution. Unlike RNN, the Transformer-based model allows for parallel computing, and all power consumption data for a period of time can be input into the model at the same time, introducing E pos Its function is to sort the time sequence of input power data. The specific formula is as follows:

[0086]

[0087]

[0088] Among them, pos represents the position, d model Indicates the length of a vector.

[0089] In step S3, the embedding matrix is ​​encoded through the Transformer block to obtain the input sequence. The specific implementation process is as follows:

[0090] The convolution and pooling embedding matrix is ​​fed into the Transformer block of TransUNet-NILM for encoding. The encoding part of the Transformer block includes two LayerNorm layers, a multi-head attention layer, and a feedforward neural network. The specific process is as follows:

[0091] S301: Perform a linear transformation on the embedded matrix to obtain the Q, K, and V matrices of the embedded matrix. Specifically, the following steps are involved:

[0092] For single-head attention (scaled dot product attention), it can be represented by Q (Query), K (Key), and V (Value) matrices, which are obtained by linear transformation of the embedding matrix. Q and K are first multiplied and divided by the dimension d of the K vector k The square root of , and then construct SoftAttention through softmax operation, and then multiply by V and return the weighted value matrix, the formula is as follows:

[0093]

[0094] S302: Obtain multiple subspaces of multiple Q, K, and V matrices through a multi-head attention layer, and concatenate the multiple subspaces to obtain an output sequence. This specifically includes:

[0095] Similarly, for multi-head attention, the hidden space is divided into multiple subspaces with parameter matrices and the same calculation is performed, thereby generating multiple Q, K, and V matrices. Therefore, through multi-head attention, information can be obtained in multiple subspaces, and the output result z is obtained in each subspace. i Connect them to get the final output sequence, which can be expressed as follows:

[0096] MultiHead(Q,K,V)=Concat(head1,head2,…,head h )W O

[0097] head i =Attention(QW i Q ,KW i K ,VW i V )

[0098] in: are all trained weight matrices. is the model weight matrix for joint training.

[0099] S303: Normalize the output sequence through the first layer LayerNorm. Specifically, it includes:

[0100] The matrix MultiHead calculated by the multi-head attention layer is denoted as z MH Normalization is first performed. LayerNorm is used here to speed up the training and improve the stability of the training. The normalized output is fed into the position feedforward neural network. This layer is relatively simple and consists of two fully connected layers. The first layer performs a linear transformation first, and the second layer uses the activation function ReLU to perform a nonlinear transformation before performing a linear transformation. The specific role of these two layers is to transform the input zMH Perform high-dimensional mapping, then use the nonlinear function ReLU for filtering, and then restore it to the original dimension. The specific formula is as follows:

[0101]

[0102]

[0103] in: represents the normalized output from the multi-head attention layer in the j-th encoder; z j-1 represents the input sequence obtained in the j-1th encoder; PFFN represents the feedforward neural network operation; W1 and W2 represent weight parameters, and b1 and b2 represent offsets.

[0104] Considering that the encoding block of the Transformer block is composed of 6 encoders stacked together, It represents the normalized output from the multi-head attention layer in the j-th encoder.

[0105] S304, perform residual connection on multiple normalized output sequences through the multi-head attention layer and the position feedforward neural network to obtain a spliced ​​sequence, and perform normalization operation on the spliced ​​sequence through the second layer LayerNorm to obtain the input sequence z j , the specific formula is as follows:

[0106]

[0107] It should be noted that, in the embodiment of the present invention, since the coding block uses 6 identical encoders, the method further includes: S305, repeating steps S303 to S304.

[0108] In step S4, the input sequence is decoded by the upsampling block to obtain multi-scale features, and the multi-scale features are cropped to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data. The specific implementation process is as follows:

[0109] S401, by using multiple upsampling blocks, where each upsampling block includes a one-dimensional deconvolution and a convolution layer, the encoding input to the upsampling block first passes through the deconvolution layer, and then the obtained high-level features are convolved with the multiple convolution layers in the upsampling block to obtain the high-resolution feature patchy i (t) Splicing to obtain a multi-scale feature.

[0110] S402: Crop the multi-scale features through a sub-window cropping layer to obtain a sub-sequence.

[0111] Specifically:

[0112] Considering the window size W, and the portion of interest during load decomposition near the window midpoint, the power consumption signal at t = 0 and t = W has little impact on the decomposition results and increases training time. Therefore, taking into account both the data dimension and the window size, this embodiment of the present invention introduces a smaller window cropping layer W', whose center is aligned with the center of the main window, W' = W / 2. Finally, the middle portion of the last layer of the decoder generates a "subsequence" as the final output.

[0113] S403. Repeat steps S401 to S402. The subsequence is finally input to the output layer. This layer includes a convolutional layer and an MLP. The MLP includes a deconvolutional layer and two linear layers. The formula is as follows:

[0114] s t =softmax(MLP(z))

[0115] MLP(z)=Tanh(Deconv(z)W1+b1)W2+b2

[0116] Among them, Tanh is the activation function, Deconv represents deconvolution calculation, W1 and W2 represent weight parameters, and b1 and b2 represent offsets.

[0117] In step S5, the TransUNet-NILM model is optimized using the mean absolute error between the true label and the high-resolution data as the optimization target. The specific implementation process is as follows:

[0118] During decomposition, the minimum loss function is selected. In this embodiment of the present invention, the mean absolute error is used as the minimum loss function to ensure the accuracy of decomposition. The cross entropy between the decomposed device distribution and the state of each device is used to minimize the loss function of the device decomposition state by using softmax prediction. The specific formula is as follows:

[0119]

[0120]

[0121] Where x = f(y), represents the true value, s and They represent the true state label and the predicted state respectively, and N represents the number of electrical devices.

[0122] Note: What is considered here is the length of the output subsequence, that is, the above steps finally determine the small window to be W'=W / 2, then the model represents

[0123] It should be noted that, in an embodiment of the present invention, the method further includes:

[0124] S6. Test the TransUNet-NILM model using the test set to further optimize the TransUNet-NILM model.

[0125] The embodiment of the present invention also provides a load decomposition system based on deep learning attention mechanism, such as Figure 2 As shown in the figure, the load decomposition system includes a training subsystem for the TransUNet-NILM model and a signal processing subsystem for calling the TransUNet-NILM model. The signal processing subsystem calls the TransUNet-NILM model to process the total power signal to obtain high-resolution data. The model introduces the Transformer self-attention mechanism into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The training subsystem of the model includes:

[0126] A training set acquisition module is used to obtain a total power signal with a real label, and process the total power signal to obtain a training set;

[0127] The downsampling module is used to extract features from the training set through the downsampling block to obtain the embedding matrix;

[0128] Transformer module, which encodes the embedding matrix through the Transformer block to obtain the input sequence;

[0129] The upsampling module is used to decode the input sequence through the upsampling block to obtain multi-scale features, and then crop the multi-scale features to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data;

[0130] The optimization module is used to optimize the TransUNet-NILM model using the mean absolute error of the true label and high-resolution data as the optimization target.

[0131] It can be understood that the load decomposition system based on the deep learning attention mechanism provided in the embodiment of the present invention corresponds to the above-mentioned load decomposition method based on the deep learning attention mechanism. The explanation, examples, beneficial effects, etc. of the relevant contents can refer to the corresponding contents in the load decomposition method based on the deep learning attention mechanism, and will not be repeated here.

[0132] An embodiment of the present invention also provides a computer-readable storage medium that stores a computer program for load decomposition of a deep learning attention mechanism, wherein the computer program enables a computer to execute the load decomposition method based on the deep learning attention mechanism as described above.

[0133] An embodiment of the present invention further provides an electronic device, including:

[0134] one or more processors;

[0135] Memory; and

[0136] One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the load decomposition method based on the deep learning attention mechanism as described above.

[0137] In summary, compared with the existing technology, the present invention has the following beneficial effects:

[0138] 1. A sequence-to-subsequence method proposed in an embodiment of the present invention strikes a balance between seq2seq and seq2point to balance the convergence difficulties in deep neural networks, making training easier and reducing the amount of computation during inference.

[0139] 2. This embodiment of the present invention combines Transformer and CNN, adopting the TransUNet model to compensate for the shortcomings of Transformer and CNN. The Transformer block encodes the tokenized image blocks in the convolutional neural network feature map into an input sequence for extracting the global context; the decoder upsamples the encoded features and then combines them with the high-resolution CNN feature map to achieve accurate decomposition.

[0140] 3. The embodiment of the present invention proposes a method based on a deep learning attention mechanism, which can learn the relationship between features in the input long text data through the attention mechanism. It does not rely entirely on the data itself, effectively solves the problem of excessive dependence of existing technologies on raw data, and further improves the accuracy of decomposition.

[0141] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0142] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A load decomposition method based on deep learning attention mechanism, characterized in that: The load decomposition method processes the total power signal through a pre-built TransUNet-NILM model to obtain high-resolution data. The TransUNet-NILM model introduces the Transformer self-attention mechanism into the U-Net architecture and acts as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The construction process of the TransUNet-NILM model includes: S1. Obtain a total power signal with a true label, and process the total power signal to obtain a training set; S2, extract features from the training set through the downsampling block to obtain the embedding matrix; S3. Encode the embedding matrix through the Transformer block to obtain the input sequence; S4. Decode the input sequence through the upsampling block to obtain multi-scale features, and crop the multi-scale features to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data; S5. Optimize the TransUNet-NILM model using the mean absolute error between the true label and the high-resolution data as the optimization target. The upsampling block includes, from input to output, a deconvolution layer, a convolution layer, a window cropping layer, and an output layer. The upsampling block is used to decode the input sequence to obtain multi-scale features, and the multi-scale features are cropped to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data, including: S401, by using multiple upsampling blocks, where each upsampling block includes a one-dimensional deconvolution layer and a convolution layer, the encoding input to the upsampling block first passes through the deconvolution layer, and the high-level features obtained by the downsampling block are concatenated with the high-resolution features obtained by convolution of multiple convolution layers in the upsampling block to obtain a multi-scale feature; S402, cropping the multi-scale features through a sub-window cropping layer to obtain a sub-sequence; S403. Repeat steps S401 to S402. The subsequence is finally input to the output layer. The output layer includes a convolutional layer and an MLP.

2. The load decomposition method based on deep learning attention mechanism according to claim 1, characterized in that: The downsampling block includes multiple convolutional layers and pooling layers from input to output; The feature extraction of the training set is performed through the downsampling block to obtain an embedding matrix, including: Multiple convolutional layers are used to increase the hidden size of the data in the training set, position embedding is introduced, and the learning L in the pooling layer is used to 2 The norm pooling operation pools the convolution outputs with increasing hidden sizes to obtain an embedding matrix.

3. The load decomposition method based on deep learning attention mechanism according to claim 1, characterized in that: The Transformer block includes a multi-head attention mechanism layer, two LayerNorm layers, and a feedforward neural network; The embedding matrix is ​​encoded through the Transformer block to obtain the input sequence, including: S301, performing a linear transformation on the embedded matrix to obtain Q, K, and V matrices of the embedded matrix; S302. Obtain multiple subspaces of multiple Q, K, and V matrices through a multi-head attention layer, and concatenate the multiple subspaces to obtain an output sequence. S303, normalizing the output sequence through the first layer LayerNorm; S304. Perform residual connection on multiple normalized output sequences through a multi-head attention layer and a position feedforward neural network to obtain a spliced ​​sequence, and perform normalization operation on the spliced ​​sequence through the second layer LayerNorm to obtain an input sequence.

4. The load decomposition method based on deep learning attention mechanism according to claim 1, characterized in that: The multi-scale features are cropped by the sub-window cropping layer to obtain a subsequence, including: The center of the clipping window in the sub-window clipping layer is aligned with the center of the main window, and W'≤W / 2, where W' represents the clipping window and W represents the main window.

5. The load decomposition method based on deep learning attention mechanism according to any one of claims 1 to 4, characterized in that: The method further comprises: S6. Test the TransUNet-NILM model using the test set to further optimize the TransUNet-NILM model.

6. A load decomposition system based on deep learning attention mechanism, characterized in that: The load decomposition system includes a training subsystem for the TransUNet-NILM model and a signal processing subsystem for calling the TransUNet-NILM model. The signal processing subsystem calls the TransUNet-NILM model to process the total power signal to obtain high-resolution data. The TransUNet-NILM model introduces the Transformer self-attention mechanism into the U-Net architecture and serves as an encoder. The model includes a downsampling block, a Transformer block, and an upsampling block. The training subsystem includes: A training set acquisition module is used to obtain a total power signal with a real label, and process the total power signal to obtain a training set; The downsampling module is used to extract features from the training set through the downsampling block to obtain the embedding matrix; Transformer module, which encodes the embedding matrix through the Transformer block to obtain the input sequence; The upsampling module is used to decode the input sequence through the upsampling block to obtain multi-scale features, and then crop the multi-scale features to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data; The optimization module is used to optimize the TransUNet-NILM model using the mean absolute error of the true label and high-resolution data as the optimization target; The upsampling block includes, from input to output, a deconvolution layer, a convolution layer, a window cropping layer, and an output layer. The upsampling block is used to decode the input sequence to obtain multi-scale features, and the multi-scale features are cropped to obtain sub-sequences. The sub-sequences pass through the output layer to obtain high-resolution data, including: S401, by using multiple upsampling blocks, where each upsampling block includes a one-dimensional deconvolution layer and a convolution layer, the encoding input to the upsampling block first passes through the deconvolution layer, and the high-level features obtained by the downsampling block are concatenated with the high-resolution features obtained by convolution of multiple convolution layers in the upsampling block to obtain a multi-scale feature; S402, cropping the multi-scale features through a sub-window cropping layer to obtain a sub-sequence; S403. Repeat steps S401 to S402. The subsequence is finally input to the output layer. The output layer includes a convolutional layer and an MLP.

7. The load decomposition system based on deep learning attention mechanism according to claim 6, characterized in that: The system further comprises: The testing module is used to test the TransUNet-NILM model through a test set to further optimize the TransUNet-NILM model.

8. A computer-readable storage medium, characterized in that It stores a computer program for load decomposition based on a deep learning attention mechanism, wherein the computer program enables a computer to execute the load decomposition method based on a deep learning attention mechanism as described in any one of claims 1 to 5.

9. An electronic device, characterized in that: include: one or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the programs include a method for executing the load decomposition method based on the deep learning attention mechanism as described in any one of claims 1 to 5.

Images