TransUNet-based non-intrusive load monitoring method and system

Abstract

The invention discloses a non-intrusive load monitoring method and system based on TransUNet. The non-intrusive load monitoring method comprises the following steps: acquiring a historical total power load and power consumption data of single equipment; using a non-intrusive load monitoring model to learn from historical total power load and power consumption data of a single device, and identifying features of different devices in use; and according to the trained non-intrusive load monitoring model, analyzing the collected real-time total power load data, and outputting an identification result of the non-intrusive load. And the feature extraction capability is obviously improved, so that the power change and time feature discovery capability is improved. Meanwhile, a sequence-to-subsequence method is provided, so that the calculation complexity is reduced, and the calculation efficiency and the reasoning speed of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of non-intrusive load monitoring and deep learning, and specifically to a non-intrusive load monitoring method and system based on TransUNet. Background Art

[0002] Non-intrusive load monitoring (NILM) involves identifying the power consumption of individual appliances in a system with a master meter. In residential applications, NILM enables intelligent appliance identification by monitoring residential electricity usage. By leveraging smart meter data, NILM enables customers to understand the actual power consumption of each appliance and its contribution to total power consumption. Consequently, NILM can better help customers reduce residential electricity consumption, assist power system operators in managing the power system, and detect appliance failures.

[0003] NILM technology captures the operating status and power consumption of each appliance from data collected from a home's main meter. The key to this process is extracting effective features of household appliance power usage from operating power data. Power data can be divided into two categories: high-sampling rate data and low-sampling rate data. High-sampling rate data contains transient events in the power signal. However, as data volume increases, collecting this data becomes prohibitively expensive. Low-sampling rate data loses detailed information about the device load, which significantly impacts the performance of the NILM model. Specifically, since low-sampling data can extract fewer load features, the accuracy and stability of the NILM model are compromised. Furthermore, noise in the power signal can affect the accuracy of NILM feature extraction. Noise can cause an appliance's power signal to be undetectable within the context of the home's total power consumption or introduce erroneous spikes in the power signal. These erroneous spikes can be mistaken for device activation, leading to inaccurate identification. Noise can also be identified as features, resulting in poor model performance. Therefore, NILM methods for low-sampling data require more effective feature extraction techniques to improve the accuracy and reliability of load identification.

[0004] Research on NILM methods falls into two main categories: event-based and non-event-based methods. Event-based NILM methods require three steps to complete load monitoring: event detection, feature extraction, and load classification. Non-event-based NILM methods do not rely on event detection and load classification. Common methods include factorial hidden Markov models (FHMMs), graph signal processing, and machine learning. However, FHMMs can be computationally complex when dealing with a large number of devices. Machine learning methods often require manual feature extraction. Existing non-event-based methods often face problems such as limited NILM accuracy and prolonged model training time. Therefore, it is necessary to find more effective methods to improve the efficiency of load feature extraction from aggregated signals. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are the most representative. CNN models perform well on local features, also known as spatial features, such as the power amplitude of an appliance. RNNs are capable of extracting temporal relationships between sequential data, such as changes in the operating status of household appliances. For NILM problems, the focus is on improving the performance of single feature extraction. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is that the feature extraction capability of the existing NILM-based solution is insufficient and cannot fully represent the relationship between electrical devices, and the false positive and false negative rates of the decomposition results are very high.

[0007] To solve the above technical problems, the present invention provides the following technical solution: a non-intrusive load monitoring method based on TransUNet, comprising:

[0008] Obtain historical total power load and power consumption data for individual devices;

[0009] Using a non-intrusive load monitoring model to learn from historical total power load and individual device power consumption data, it can identify the characteristics of different devices when in use.

[0010] According to the trained non-intrusive load monitoring model, the collected real-time total power load data is analyzed and the identification result of the non-intrusive load is output.

[0011] As a preferred embodiment of the non-intrusive load monitoring method based on TransUNet of the present invention, after the non-intrusive load monitoring model is trained, the model predicts the power consumption of a specific appliance based on the observed total load pattern, and the relationship equation is:

[0012]

[0013] Among them, si (t0 represents the switch state of the i-th electrical appliance, x i (t0 represents the power consumption of a single device, ε(t) represents the measurement noise, t represents the current moment, and y(t) represents the total loss;

[0014] Non-intrusive load monitoring can be expressed as:

[0015] f(y(t))=[x1(t),x2(t),…,x N (t)]

[0016] Among them, the function f represents the mapping of the NILM process;

[0017] The dataset D is represented as:

[0018] D={y(t),s(t)|t=1,2,…W}

[0019] Where W represents the window size and s(t) represents the switch status of all electrical appliances.

[0020] As a preferred solution of the non-intrusive load monitoring method based on TransUNet described in the present invention, the non-intrusive load monitoring model includes preprocessing the selected data parameters and introducing the residual network theory to solve the gradient vanishing problem;

[0021] Combining the Transformer model and the U-Net architecture, a non-intrusive load monitoring model based on the TransUNet neural network is formed.

[0022] As a preferred solution of the non-intrusive load monitoring method based on TransUNet described in the present invention, the preprocessing includes data cleaning, time synchronization, data normalization, data input and output as a sequence to a point, and data storage and management.

[0023] As a preferred solution of the non-intrusive load monitoring method based on TransUNet described in the present invention, wherein: the residual network includes, x represents the output of the previous layer, the original output of the next layer is F(x), relu(F(x)+x) is the result of the final residual connection, and x is input to the next layer and directly connected to the output of the next layer;

[0024] When the model depth has reached the optimal value, the residual connection can adaptively learn F(x) to be all 0;

[0025] relu(F(x)+x) is converted to relu(x);

[0026] If the distance between x and the optimal effect does not meet expectations, the residual connection adaptively allows the next layer F(x) to fully learn;

[0027] Let the output of the residual block be H(x), then it is expressed as H(x)=F(x)+x;

[0028] Where F(x) represents the fitting function stacked with the relu activation function;

[0029] The specific steps to build a Resnet-based encoder for local feature extraction are:

[0030] Step 1: Construct two 3x3 convolutional layers; the first 3x3 convolutional layer is responsible for extracting local features in the input feature map; the second 3x3 convolutional layer is responsible for refining and processing features and enhancing the expressiveness of features while maintaining spatial locality;

[0031] Step 2: Construct a 1x1 convolutional layer directly connected to the output to ensure that the input is unchanged after convolution;

[0032] Step 3: Use the relu activation function between the input and the convolutional network;

[0033] Step 4: Obtain local features based on the output of the residual network.

[0034] As a preferred solution of the non-intrusive load monitoring method based on TransUNet of the present invention, the Transformer model includes: Step 1: configuring a LayerNorm layer;

[0035] Step 2: Configure an Add&Norm layer;

[0036] Step 3: Configure a multi-head self-attention layer;

[0037] Step 4: Configure a feedforward neural network layer.

[0038] A single-head self-attention unit is represented by a query matrix Q, a key matrix K, and a value matrix V, all of which are obtained by linearly transforming the input matrix;

[0039] Steps to construct the self-attention layer:

[0040] Step 1: Multiply matrices Q and K;

[0041] Step 2: Divide the result of the previous step by the square root of the dimension of the key matrix Zoom in or out;

[0042] Step 3: Apply the Softmax operation to the result of the previous step to convert it into a probability distribution;

[0043] Step 4: By calculating the similarity between the query Q and the key Ke, the Softmax weight is used to perform weighted summation on the values, and finally the weighted feature vector is output. The specific expression is as follows:

[0044]

[0045] Among them, K T is the transposed matrix of the key matrix, is the square root of the dimension of the key matrix.

[0046] As a preferred solution of the non-intrusive load monitoring method based on TransUNet described in the present invention, the deep learning model of TransUNet includes:

[0047] Step 1: Use l2 norm to pool the sequence to retain features;

[0048] Step 2: Encode the patch spatial information and introduce the position embedding E pos , and added to the patch embedding;

[0049] Step 3: Perform normalization calculation on the obtained feature data;

[0050] Step 4: The normalized output is fed into a feedforward network consisting of a LayerNorm layer and an Add&Norm layer; the LayerNorm layer performs linear transformation; the Add&Norm layer uses the activation function relu to perform nonlinear transformation and linear transformation; the two layers map the feature vector to a higher dimension and then restore it to its original dimension size;

[0051] Step 5: During the decoding process, multiple upsampling blocks are used; each upsampling block consists of a one-dimensional deconvolution layer and a convolution layer; the input of the upsampling block passes through the deconvolution layer to generate high-level features;

[0052] Step 6: Combine the generated high-level features with the previously calculated local high-resolution features to obtain multi-scale features;

[0053] Step 7: Feed the multi-scale feature vector to the output layer, which consists of a convolutional layer and a multi-layer perceptron; the MLP consists of a deconvolution layer and two linear layers;

[0054] Step 8: Use a small time window to reduce training time;

[0055] Step 9: The middle part of the last layer of the decoder generates a subsequence as the final output.

[0056] A non-intrusive load monitoring system based on TransUNet using any of the methods described in the present invention, characterized in that:

[0057] The acquisition unit obtains historical total power load and power consumption data of individual devices;

[0058] The recognition unit uses a non-intrusive load monitoring model to learn from historical total power load and power consumption data of individual devices to identify the characteristics of different devices when they are in use;

[0059] The analysis unit analyzes the collected real-time total power load data according to the trained non-intrusive load monitoring model and outputs the identification result of the non-intrusive load.

[0060] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein: when the processor executes the computer program, the steps of any one of the methods of the present invention are implemented.

[0061] A computer-readable storage medium stores a computer program, wherein: when the computer program is executed by a processor, the steps of any one of the methods of the present invention are implemented.

[0062] Beneficial effects of the present invention: The non-intrusive load monitoring method based on TransUNet provided by the present invention combines the advantages of the Transformer model and the U-Net model to propose an improved NILM model based on TransUNet. This model significantly improves the feature extraction capability by using the residual network and the attention mechanism, thereby improving the ability to discover power changes and time characteristics. In addition, the U-Net structure enhances the ability of the proposed model to accurately identify various electrical appliances. The use of the Transformer module enables the model to capture long-term dependencies and extract relevant features from large-scale data sets. In addition, the present invention proposes a sequence to subsequence method to reduce computational complexity and improve the computational efficiency and inference speed of the model. The test results on the REDD and UK-DALE datasets show its superiority over traditional CNN models and RNN models. Finally, through simulation and comparative analysis with a variety of other types of models, the model can more efficiently extract local features of electrical appliances. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] Figure 1 An overall flow chart of a non-intrusive load monitoring method based on TransUNet provided in the first embodiment of the present invention;

[0065] Figure 2 A non-intrusive load monitoring method based on TransUNet according to a first embodiment of the present invention includes two input and output modes;

[0066] Figure 3 A residual network flow chart of a non-intrusive load monitoring method based on TransUNet provided in the first embodiment of the present invention;

[0067] Figure 4 A residual module diagram of a non-intrusive load monitoring method based on TransUNet provided in the first embodiment of the present invention;

[0068] Figure 5 A flow chart of a Transformer module of a non-intrusive load monitoring method based on TransUNet provided in the first embodiment of the present invention;

[0069] Figure 6 A TransUNet model diagram of a non-intrusive load monitoring method based on TransUNet provided in the first embodiment of the present invention. DETAILED DESCRIPTION

[0070] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0071] Example 1, with reference to Figures 1-6 , as one embodiment of the present invention, provides a non-intrusive load monitoring method based on TransUNet, comprising:

[0072] Existing deep learning-based feature extraction methods still have some shortcomings. Power consumption behavior involves higher-level semantics, and CNN-based models do not effectively consider the relationships between power-consuming devices, resulting in high false positive and false negative rates in the decomposition results. Increasing the number of convolutions and filter resolution can improve the accuracy of the decomposition results, but this also incurs higher computational cost and complexity. Furthermore, many deep learning-based NILM methods still have limitations in extracting high-level semantic features and need improvement in model training efficiency.

[0073] This paper presents a non-intrusive load monitoring method based on the highly efficient TransUNet deep learning model, combining the advantages of the Transformer and U-Net models. By utilizing a residual network and self-attention mechanism, this model significantly improves feature extraction capabilities, thereby enhancing the ability to detect power variations and temporal features. Furthermore, a sequence-to-subsequence approach is proposed to reduce computational complexity, improving the model's computational efficiency and inference speed.

[0074] The following is an introduction to the non-intrusive load monitoring method based on TransUNet proposed in the present invention, which is mainly divided into five steps as follows:

[0075] Step 1: Preprocess the initial data.

[0076] Step 2: Introduce the residual network (ResNet) and Transformer modules to alleviate the gradient vanishing problem and more effectively capture long-range dependencies in the sequence.

[0077] Step 3: Propose the TransUNet architecture and apply it to NILM.

[0078] Step 4: Experimental simulation and model comparison.

[0079] In step 1, the specific description is as follows:

[0080] The NILM task observes a given total power load to determine the state of appliances and their corresponding power consumption. It decomposes the total load power y(t) obtained from the smart meter into the power consumption of individual appliances. The NILM model learns from historical total power load and individual appliance power consumption data, identifying unique features and patterns associated with the usage of different appliances. After training, the model predicts the power consumption of specific appliances based on the observed total load patterns. The specific relationship equation is:

[0081]

[0082] Among them, s i (t) represents the switch state of the i-th appliance, x i (t) represents the power consumption of a single device, and ε(t) represents the measurement noise. Thus, NILM can be expressed as:

[0083] f(y(t))=[x1(t),x2(t),…,x N (t)] (2)

[0084] Here, the function f represents the mapping of the NILM process.

[0085] Therefore, the dataset D is represented as:

[0086] D={y(t),s(t)|t=1,2,…W} (3)

[0087] Where W represents the window size.

[0088] Further, the data is preprocessed as follows:

[0089] Step 1: Perform data cleaning to deal with noise, missing values and outliers in the original data.

[0090] 1) To reduce noise, the simple moving average (SMA) method is used to smooth the power data. The specific method is as follows:

[0091] i) Select a window size n, which contains the most recent n data points;

[0092] ii) For each time point, calculate the average value of the data at that time point and the n-1 time points before it using the formula:

[0093]

[0094] Among them, X i Represents the data value at time point i.

[0095] 2) For missing power data, linear interpolation is used to fill the missing values. The specific steps are as follows:

[0096] i) Determine the location of missing values in the dataset by observing the data records;

[0097] ii) For each missing value, find the two known data points before and after it (i.e. (t0, X0) and (t1, X1));

[0098] iii) Substitute the coordinates of the known data points and the missing value t and use the linear interpolation formula to calculate the missing value X. The formula for linear interpolation is as follows:

[0099]

[0100] iiii) Place the calculated results into the corresponding positions in the original dataset and update the dataset.

[0101] 3) For outliers, use the mean and standard deviation to detect outliers. Any data point that deviates from the mean by more than 3 times the standard deviation can be considered an outlier. At the same time, analyze the changes in the data within the specified time window. Excessive fluctuations may indicate an abnormality. If the difference between the previous and next data is more than 3 times, it can be considered an outlier and the data can be cleared;

[0102] Step 2: In this invention, interpolation synchronization technology is used to align all timestamps to achieve time synchronization. For example, if the reading interval of one device is 1 second and that of another is 0.5 seconds, the data with the lower sampling rate is interpolated. That is, the average value of the previous and next data is inserted into the data with the reading interval of 1 second so that the data have the same time step.

[0103] Step 3: Normalize the data. Use the Min-Max method to normalize the data. Scale the data to a fixed range, usually [0,1]. The specific conversion formula is as follows:

[0104]

[0105] Among them, X represents the original data, min(X) represents the minimum value in the data set, max(X) represents the maximum value in the data set, and X norm Indicates the value of the normalized data.

[0106] Step 4: Data input and output. In the present invention, the input and output method from sequence to subsequence is adopted, such as Figure 2 As shown. The input of this method is a sequence, and the output is a subsequence that is smaller than the input sequence. Assuming that the length of the input sequence is window W, and the length of the output sequence is window W', when traversing the output, for the two input and output methods of sequence to point and sequence to subsequence, the time required is L, It can be seen that the sequence-to-subsequence method is faster in performing output traversal and is more suitable for neural networks.

[0107] Step 5: Data storage and management. NILM's raw data is usually very large, so an effective data storage and management strategy is required for subsequent processing.

[0108] Step 2: Introduce the residual network and Transformer self-attention mechanism to extract local features, alleviate the gradient vanishing problem, and capture long-term dependencies in time series data.

[0109] First, as the depth of the neural network increases, the effectiveness of feature extraction does not improve significantly, and it also causes gradient vanishing. This paper introduces the residual network (ResNet) to alleviate the gradient vanishing problem.

[0110] Residual Network ResNet (such as Figure 3 ) contain residual blocks that use residual connections, also known as “shortcut connections”, to mitigate gradient vanishing by skipping input data of certain layers.

[0111] F(x)=relu(x) (7)

[0112] H(x)=relu(F(x)+x)=relu(x) (8)

[0113] Among them, x represents the output of the previous layer, F(x) represents the original output of the next layer, H(x) represents the result of the final residual connection, and x is directly connected to the output of the next layer while being input into the next layer.

[0114] Secondly, consider x as an upper-layer output that gradually approaches the optimal result, and F(x) represents the residual, which indicates the network structure that approaches the optimal effect.

[0115] When the model depth has reached the optimal value, due to the existence of ReLU, the residual connection adaptively learns F(x) to be all 0.

[0116] Then, relu(F(x)+x) is converted to relu(x). Due to the properties of relu:

[0117] relu(x)=x (9)

[0118] At this point, x has already been transformed by the rule of the previous layer, and after another relu, it is still x, thus achieving the identity transformation.

[0119] Furthermore, if x is still far from the optimal effect, the residual connection also adaptively allows the next layer F(x) to fully learn.

[0120] Let the output of the residual block be H(x), then it can be expressed as

[0121] H(x)=F(x)+x (10)

[0122] Among them, F(x) is the residual, which represents the fitting function formed by stacking with the relu activation function.

[0123] In this paper, a Resnet-based encoder is constructed for local feature extraction, such as Figure 4 The specific steps are as follows:

[0124] Step 1: Construct two 3x3 convolutional layers. The first 3x3 convolutional layer is responsible for extracting local features from the input feature map; the second 3x3 convolutional layer is responsible for further refining and processing features while maintaining spatial locality and enhancing the expressiveness of features.

[0125] Step 2: Construct a 1x1 convolutional layer directly connected to the output to ensure that the input does not change after convolution.

[0126] Step 3: Use the relu activation function between the input and the convolutional network;

[0127] Step 4: Obtain local features based on the output of the residual network.

[0128] Furthermore, the Transformer module is introduced into NILM to capture long-term dependencies in time series data.

[0129] Although recurrent neural networks and long short-term memory networks can also process time series data, they are prone to the gradient vanishing problem when processing long sequences. In contrast, the Transformer can more effectively capture long-range dependencies in the sequence through the self-attention mechanism, and performs better when processing complex power consumption patterns. The Transformer module contains Encoder and Decoder modules, which are composed of multiple repeated Blocks. The Encoder Block contains a multi-head self-attention layer and a feedforward neural network layer, and the Decoder Block has an additional Encoder-Decoder attention layer. The multi-head self-attention allows the model to capture information from different angles, and the feedforward neural network layer is used to enhance feature extraction. The Add&Norm module is used for information transfer and normalization, and the position encoder adds position information to the sequence. The constructed Transformer module is as follows Figure 5 The specific steps are as follows:

[0130] Step 1: Configure a LayerNorm layer;

[0131] Step 2: Configure an Add&Norm layer;

[0132] Step 3: Configure a multi-head self-attention layer (MSA);

[0133] Step 4: Configure a feedforward neural network layer.

[0134] The multi-head self-attention layer is composed of multiple single-head self-attention units. A single-head self-attention unit is represented by a query matrix Q, a key matrix K, and a value matrix V, which are obtained by linearly transforming the input matrix.

[0135] The specific steps are as follows:

[0136] i) The input matrix X represents a sequence of data, and each element can be a vector. Assume that the input matrix is Where N is the length of the input sequence, d input is the feature dimension of the input;

[0137] ii) Construct three weight matrices,

[0138] iii) Generate the query matrix Q, key matrix K and value matrix V, as shown below.

[0139] Q=XW Q (11)

[0140] K=XW k (12)

[0141] V=XW v (13)

[0142] Among them, Q is the query matrix, which represents the "query" of the current input sequence at each time step for other time steps; K is the key matrix, which represents the "key" of each time step in the input sequence; V is the value matrix, which represents the "value" of each time step in the input sequence. It is weighted and aggregated according to the self-attention weight to finally generate the output.

[0143] The self-attention layer is the core component of the Transformer. It is used to calculate the similarity between each element in the input sequence and weight the input features based on these similarities to obtain a weighted output feature vector. The steps to construct the self-attention layer are as follows:

[0144] Step 1: Multiply matrices Q and K;

[0145] Step 2: Divide the result of the previous step by the square root of the dimension of the key matrix Scaling is performed to maintain numerical stability;

[0146] Step 3: Apply the Softmax operation to the results of the previous step to convert them into a probability distribution. Softmax converts the similarity score corresponding to each query into a weight, indicating the relevance of the current query to each key.

[0147] Step 4: Calculate the similarity between the query and the key, perform a weighted sum on the value using Softmax weights, and ultimately output a weighted feature vector. This mechanism enables the model to dynamically focus on the portion of the input sequence most relevant to the current task, effectively capturing long-term dependencies within the sequence. In the NILM task, this mechanism helps capture the temporal characteristics of the power signal and its changing patterns.

[0148] The specific expression is as follows:

[0149]

[0150] Among them, Q is the query matrix, K is the key matrix, V is the value matrix, Softmax represents the normalization function, K T is the transposed matrix of the key matrix, is the square root of the dimension of the key matrix.

[0151] Furthermore, when dealing with multi-head self-attention layers, the process involves dividing the latent space into several subspaces, that is, into multiple single-head self-attention units, each of which calculates its own parameter matrix, derives multiple sets of Q, K and V, constructs its own self-attention layer, and then derives the self-attention layer that merges each single-head self-attention unit.

[0152] The use of multi-head attention layers makes it possible to obtain information in multiple subspaces, and the final output sequence can be expressed as:

[0153] MultiHead(Q,K,V)=Concat(head1,head2,…,head n )W o (15)

[0154] in, Represents a single-head self-attention unit.

[0155] Both represent training weight matrices,

[0156] Represents the joint training model weight matrix.

[0157] Step 3: Build the TransUNet architecture, add the residual network and Transformer modules to it and apply them to NILM to build a non-intrusive detection deep learning model based on TransUNet.

[0158] The architecture of the non-intrusive detection deep learning model based on TransUNet is a one-dimensional CNN framework with a symmetrical encoding and decoding structure. The structure consists of a downsampling block, a Transformer as an encoder, and an upsampling block as a decoder, using a U-shaped design, such as Figure 6 shown.

[0159] Before the input sequence data is fed into the Transformer encoder, feature extraction is performed using the downsampling block of the U-Net model.

[0160] The specific steps are as follows:

[0161] Step 1: Use the l2 norm to pool the sequence to retain features. l2 norm pooling is a technique that reduces dimension and pools data by calculating the l2 norm of the data in a local area. The l2 norm, also known as the Euclidean norm, is expressed as the square root of the sum of the squares of each element of a vector. The specific steps of l2 norm pooling are as follows:

[0162] i) Divide the sequence into multiple fixed-size windows, each containing m consecutive time steps. For example, we can divide a sequence of length N into several small windows of length m;

[0163] ii) Calculate the l2 norm of each window, which represents the "total amount" or "energy" of all data points in the window;

[0164] iii) The l2 norm results of each window are merged into a new pooled sequence. The dimension of this pooled sequence is smaller than that of the original sequence and the comprehensive features of each window are retained.

[0165] Step 2: To encode the patch spatial information, we introduce the position embedding Epos and add it to the patch embedding.

[0166] Patch spatial information refers to the position and spatial structure of each input data block (usually a local portion of time series data) in the input sequence. Since the Transformer model itself does not have built-in position awareness, position embedding is required to tell the model the relative or absolute position of each input patch in the sequence.

[0167] Furthermore, patch embedding involves segmenting the input data into several blocks and using the learned vector representation to preserve the characteristics of each patch. Through patch embedding, the model learns a high-dimensional feature representation of each small block, which helps it capture local information in the data. Patch embedding also converts the input data into vectors of a certain dimension, enabling the model to process input data of any size, regardless of input size restrictions.

[0168] Then, by adding position embedding to each patch, TransUNet is able to capture the position information of each part in the sequence, and then use this spatial information in the Transformer encoder to better model the changes and patterns of temporal data. The formula for patch embedding is:

[0169] E patch =ResNet(y i ) (16)

[0170] Among them, y i represents the sequence after l2 norm pooling, W represents the window size, d input Indicates the dimension of the input data.

[0171] z0=LPPooling(E patch )+E pos (17)

[0172] in, z0 is input as a feature into the Transformer layer, and patch embedding encodes the power consumption characteristics of the appliance into a vector representation. At the same time, position embedding enables the model to consider temporal correlations and dependencies when learning data representations.

[0173] Step 3: Perform normalization calculation on z0;

[0174] Step 4: The normalized output is fed into a feedforward network consisting of a LayerNorm layer and an Add&Norm layer. The LayerNorm layer performs a linear transformation. The Add&Norm layer performs a nonlinear transformation using the ReLU activation function, followed by a linear transformation. These two layers map the feature vector to a higher dimension and then restore it to its original dimension size. The specific formula is:

[0175]

[0176] in, Represents the normalized feature data, W1, b1 are the parameters of the first linear transformation layer, W2, b2 are the parameters of the second linear transformation layer. FFN represents the function of the feedforward network.

[0177] Final output z output After normalization, it is input to the upsampling block.

[0178] Step 5: During the decoding process, multiple upsampling blocks are used. Each upsampling block consists of a one-dimensional deconvolution layer and a convolution layer. The input of the upsampling block first passes through the deconvolution layer to generate high-level features.

[0179] Step 6: Combine the generated high-level features with the previously calculated local high-resolution features to obtain multi-scale features. The specific formula is as follows:

[0180] z=Conv(Concat(Deconv(z),E patch )) (19)

[0181] Among them, Concat(Deconv(z)) means to generate high-level features through the deconvolution layer, E patch denotes patch embedding and Conv denotes convolution.

[0182] Step 7: Feed the multi-scale feature vector to the output layer, which consists of a convolutional layer and a multi-layer perceptron (MLP). The MLP consists of a deconvolution layer and two linear layers. The final output is given by the equation:

[0183] MLP(z)=Sigmoid(Deconv(z)W1+b1)W2+b2 (20)

[0184] W1, b1 are the parameters of the first linear transformation layer, and W2, b2 are the parameters of the second linear transformation layer.

[0185] Step 8: Use a smaller time window to reduce training time. Given the window size W, the NILM process primarily focuses on the portion near the midpoint of the window. Therefore, the power consumption at times t = 0 and t = W has little impact on the decomposition results, increasing training time. Therefore, we introduce a smaller window W', whose center is aligned with the center of the main window, to reduce training time.

[0186] Step 9: The middle part of the last layer of the decoder generates a "subsequence" as the final output.

[0187] In order to select the minimum loss function during disaggregation, we use the mean square error as the minimum loss function to ensure the accuracy of disaggregation. The specific formula is:

[0188]

[0189] Among them, x i is the power signal, The early stopping mechanism is used during model training to reduce training time and prevent model overfitting.

[0190] Step 4: Through simulation, the proposed method is analyzed and verified, which includes the following process:

[0191] First, two public datasets, REDD and UK-DALE, are used to evaluate the performance of TransUNet.

[0192] The REDD dataset contains load monitoring data from six buildings, including high-frequency and low-frequency data. In this example, only the low-frequency data is used.

[0193] In addition, since the power consumption of personal household appliances is relatively stable, we focus on four common household appliances: kettle, refrigerator, washing machine, and microwave oven. These devices were selected from the REDD dataset for the experiment.

[0194] The UK-DALE dataset contains load monitoring data for five buildings. This example uses low-frequency data, recording the total energy consumption of each building and the power consumption of each appliance every 6 seconds. Five common household appliances were selected: kettle, refrigerator, washing machine, microwave oven, and television.

[0195] Since the electricity consumption data in the REDD dataset is more comprehensive, it is selected as the test set to decompose the four electricity-consuming devices. For the UK-DALE dataset, one building is selected as the test set and the others are used as training sets.

[0196] Secondly, this embodiment selected three models for performance comparison with the model proposed in this invention. These three models are: Transformer model, U-NET model and CNN-BiGRU model.

[0197] Furthermore, the present invention uses a resnet-based encoder, such as Figure 3 All models were implemented using PyTorch 1.12.1 and trained on an NVIDIA RTX3090 GPU. When training the models on the Adam optimizer, a learning rate of 0.0001 was used to update the configuration. The relevant parameters are shown in Table 1.

[0198] Table 1 Encoder parameters

[0199]

[0200] Next, we need to understand the on-off threshold of each appliance. In this embodiment, the on-off threshold of each appliance from different data sets is used to predict the final on / off state of a single appliance, as shown in Tables 2 and 3.

[0201] Table 2 Evaluation results on the U.K.-DALE dataset

[0202]

[0203] Table 3 Appliance start-up thresholds in the U.K.-DALE dataset

[0204] electrical appliances kettle refrigerator washing machine Micro-wave oven TV set Opening threshold 100 60 25 250 40

[0205] Then, specific indicators should be set to evaluate the performance. In order to verify the feasibility of the model proposed in this invention, this embodiment uses three performance evaluation indicators: F1 score, mean absolute error (MAE) and normalized signal aggregation error (SAE).

[0206] in,

[0207] Here, P is the precision and R is the number of true values in the dataset that are correctly predicted, which is called recall.

[0208] Let TP, FP, and FN represent true positives, false positives, and false negatives, respectively. Thus, P and R can be expressed as:

[0209]

[0210]

[0211] Furthermore, for MAE and SAE, there are the following expressions:

[0212]

[0213] In the above formula, T is the time of predicted output, x i is the power consumption value of the i-th device predicted by the model, The actual power consumption of the device.

[0214] Furthermore, the larger the F1 score, the better the prediction effect. The smaller the MAE, the closer the predicted waveform is to the actual waveform. The smaller the SAE, the smaller the total energy error of the prediction.

[0215] In addition, the impact of noise needs to be considered. The signal-to-noise ratio (SNR) is an important indicator that defines the ratio of signal strength to noise strength. The signal-to-noise ratio is usually expressed as:

[0216]

[0217] Among them, P signal Indicates signal power, P noise Represents the noise power.

[0218] The final step is to control some parameters by the control variable method to compare the performance of several models. The specific steps are as follows:

[0219] We first considered the performance of two models, TransUNet and Transformer, on refrigerator power consumption data from the UK-DALE dataset under varying signal-to-noise ratios. We calculated recall and precision under varying noise levels to evaluate the performance of each model. We added different levels of Gaussian noise to the original data, setting the signal-to-noise ratio to 45dB, 35dB, and 25dB, respectively. The experimental results are shown in Table 4.

[0220] Table 4 Precision and recall of three models at different signal-to-noise ratios

[0221]

[0222] The results show that TransUNet outperforms the Transformer model at all signal-to-noise ratio levels. As the signal-to-noise ratio decreases, the precision and recall of all models decrease, but the performance drop of TransUNet is relatively small. This shows that TransUNet is more robust in dealing with noise.

[0223] Tables 5 and 6 present the comparative results of the TransUNet model with other models. Clearly, TransUNet performs better on all evaluation metrics. Although the evaluation results for individual appliances vary slightly, the proposed method generally outperforms the other models, demonstrating its robustness and adaptability. Compared to other models, it achieves a higher F1 score while maintaining lower MAE and SAE values.

[0224] Table 5 Evaluation results of REDD dataset

[0225]

[0226] Table 6 Evaluation results on the U.K.-DALE dataset

[0227]

[0228] The good performance of the TransUNet model is mainly attributed to several factors:

[0229] First, the model's encoder-decoder structure effectively extracts and reconstructs features from input power data. Meanwhile, its skip connections combine low-level and high-level features, improving the model's ability to recognize complex power load patterns.

[0230] Secondly, supervised learning with labeled data was used to train the model. In addition, the performance of the model was optimized by changing the parameters of the training process and adjusting the model architecture.

[0231] Finally, the model adopts a step-by-step learning approach during inference, enabling it to identify home appliances with higher accuracy and precision.

[0232] Performance evaluation results for residential electricity classification demonstrate that TransUNet effectively processes electricity load data. The data used in this example are all real-world and contain measurement uncertainty and noise. Experimental results demonstrate that the proposed method accurately infers appliance power consumption from this data, incorporating spatiotemporal features and demonstrating greater robustness compared to methods such as U-Net, CNN-BiGRU, and Transformer. Through its deep encoder and decoder and cross-layer connections, the TransUNet model better captures local features than other methods.

[0233] In contrast, although the Transformer model uses a self-attention mechanism to learn dependencies between input sequences, it has difficulty capturing spatial features in time series data.

[0234] Similarly, although the U-Net model performs well in image processing, it is not suitable for time series data. Therefore, the Transformer block is used in conjunction with the U-Net model as a time series encoder to enhance finer details by recovering local spatial information.

[0235] Example 2: This embodiment further provides a non-intrusive load monitoring system based on TransUNet, which includes:

[0236] The collection unit obtains historical total power load and power consumption data of individual devices.

[0237] The identification unit uses a non-intrusive load monitoring model to learn from the historical total power load and power consumption data of individual devices to identify the characteristics of different devices when they are in use.

[0238] The analysis unit analyzes the collected real-time total power load data according to the trained non-intrusive load monitoring model and outputs the identification result of the non-intrusive load.

[0239] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0240] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0241] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0242] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0243] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A non-intrusive load monitoring method based on TransUNet, characterized in that: include: Obtain historical total power load and power consumption data for individual devices; Using a non-intrusive load monitoring model to learn from historical total power load and individual device power consumption data, it can identify the characteristics of different devices when in use. According to the trained non-intrusive load monitoring model, the collected real-time total power load data is analyzed and the identification result of the non-intrusive load is output.

2. The non-intrusive load monitoring method based on TransUNet according to claim 1, characterized in that: After the non-intrusive load monitoring model is trained, the model predicts the power consumption of a specific appliance based on the observed total load pattern, and the relationship equation is: Among them, s i (t) represents the switch state of the i-th appliance, x i (t) represents the power consumption of a single device, ε(t) represents the measurement noise, t represents the current time, and y(t) represents the total loss; Non-intrusive load monitoring can be expressed as: f(y(t))=[x1(t),x2(t),…,x N (t)] Among them, the function f represents the mapping of the NILM process; The dataset D is represented as: D={y(t),s(t)t=1,2,…W} Where W represents the window size, and s(t0) represents the switch status of all electrical appliances.

3. The non-intrusive load monitoring method based on TransUNet according to claim 2, characterized in that: The non-intrusive load monitoring model includes preprocessing the selected data parameters and introducing residual network theory to solve the gradient vanishing problem; Combining the Transformer model and the U-Net architecture, a non-intrusive load monitoring model based on the TransUNet neural network is formed.

4. The non-intrusive load monitoring method based on TransUNet according to claim 3, wherein: The preprocessing includes data cleaning, time synchronization, data normalization, data input and output in sequence, and data storage and management.

5. The non-intrusive load monitoring method based on TransUNet according to claim 4, characterized in that: The residual network includes, x represents the output of the previous layer, the original output of the next layer is F(x), relu(F(x)+x) is the result of the final residual connection, and x is directly connected to the output of the next layer while being input to the next layer; When the model depth has reached the optimal value, the residual connection can adaptively learn F(x) to be all 0; relu(F(x)+x) is converted to relu(x); If the distance between x and the optimal effect does not meet expectations, the residual connection adaptively allows the next layer F(x) to fully learn; Let the output of the residual block be H(x), then it is expressed as H(x)=F(x)+x; Where F(x) represents the fitting function stacked with the relu activation function; The specific steps to build a Resnet-based encoder for local feature extraction are: Step 1: Construct two 3x3 convolutional layers; the first 3x3 convolutional layer is responsible for extracting local features in the input feature map; the second 3x3 convolutional layer is responsible for refining and processing features and enhancing the expressiveness of features while maintaining spatial locality; Step 2: Construct a 1x1 convolutional layer directly connected to the output to ensure that the input is unchanged after convolution; Step 3: Use the relu activation function between the input and the convolutional network; Step 4: Obtain local features based on the output of the residual network.

6. The non-intrusive load monitoring method based on TransUNet according to claim 5, characterized in that: The Transformer model includes,Step 1: configuring a LayerNorm layer; Step 2: Configure an Add&Norm layer; Step 3: Configure a multi-head self-attention layer; Step 4: Configure a feedforward neural network layer. A single-head self-attention unit is represented by a query matrix Q, a key matrix K, and a value matrix V, all of which are obtained by linearly transforming the input matrix; Steps to construct the self-attention layer: Step 1: Multiply matrices Q and K; Step 2: Divide the result of the previous step by the square root of the dimension of the key matrix Zoom in or out; Step 3: Apply the Softmax operation to the result of the previous step to convert it into a probability distribution; Step 4: By calculating the similarity between the query Q and the key Ke, the Softmax weight is used to perform weighted summation on the values, and finally the weighted feature vector is output. The specific expression is as follows: Among them, K T is the transposed matrix of the key matrix, is the square root of the dimension of the key matrix.

7. The non-intrusive load monitoring method based on TransUNet according to claim 6, characterized in that: The TransUNet deep learning model includes: Step 1: Use l2 norm to pool the sequence to retain features; Step 2: Encode the patch spatial information and introduce the position embedding E pos , and added to the patch embedding; Step 3: Perform normalization calculation on the obtained feature data; Step 4: The normalized output is fed into a feedforward network consisting of a LayerNorm layer and an Add&Norm layer; the LayerNorm layer performs linear transformation; the Add&Norm layer uses the activation function relu to perform nonlinear transformation and linear transformation; the two layers map the feature vector to a higher dimension and then restore it to its original dimension size; Step 5: During the decoding process, multiple upsampling blocks are used; each upsampling block consists of a one-dimensional deconvolution layer and a convolution layer; the input of the upsampling block passes through the deconvolution layer to generate high-level features; Step 6: Combine the generated high-level features with the previously calculated local high-resolution features to obtain multi-scale features; Step 7: Feed the multi-scale feature vector to the output layer, which consists of a convolutional layer and a multi-layer perceptron; the MLP consists of a deconvolution layer and two linear layers; Step 8: Use a small time window to reduce training time; Step 9: The middle part of the last layer of the decoder generates a subsequence as the final output.

8. A non-intrusive load monitoring system based on TransUNet using the method according to any one of claims 1 to 7, characterized in that: The acquisition unit obtains historical total power load and power consumption data of individual devices; The recognition unit uses a non-intrusive load monitoring model to learn from historical total power load and power consumption data of individual devices to identify the characteristics of different devices when they are in use; The analysis unit analyzes the collected real-time total power load data according to the trained non-intrusive load monitoring model and outputs the identification result of the non-intrusive load.

9. A computer device comprising: A memory and a processor; the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images