A load identification method combining multi-scale Mamba and attention mechanisms

Abstract

This invention discloses a load identification method combining multi-scale Mamba and attention mechanisms. By introducing virtual electrical appliances, it effectively addresses numerical fluctuations in low-power scenarios. A multi-scale composite Mamba module extracts the periodic power consumption characteristics of the total circuit at different time scales to capture long-term features. A multi-task model combining the Mamba module and Transformer technology accurately captures short-term power fluctuations. The multi-task model is optimized by designing a multi-task loss function and combining single-appliance loss, multi-appliance aggregation loss, and sample quantization loss. During operation, end-to-end regression synchronous fitting is used to obtain the ratio of the active power of each individual appliance to the total active power of the circuit, thereby deriving the true value of the active power of each individual appliance. This achieves non-intrusive and accurate identification of low-frequency power loads. The method enables non-intrusive and accurate identification of low-frequency power loads, improving the accuracy of low-frequency power load identification.

Description

Technical Field

[0001] This invention belongs to the field of non-intrusive power load monitoring, specifically involving a load identification method that combines multi-scale Mamba and attention (Transformer) mechanisms. Background Technology

[0002] In recent years, with the continuous growth of global electricity demand, the importance of power load monitoring technology has become increasingly prominent. Traditional power load monitoring methods mainly include Intrusive Load Monitoring (ILM) and Non-Intrusive Load Monitoring (NILM). Among them, NILM technology has become a research hotspot due to its advantages such as low cost, strong privacy protection, and ease of promotion. NILM analyzes electrical parameters such as current and power of the total circuit to decompose the power consumption information of each appliance, providing important basis for power companies to optimize load distribution and for users to adjust their electricity consumption behavior.

[0003] However, existing NILM technology still faces the following challenges:

[0004] 1) Difficulty in extracting features from low-frequency data: Smart meters generally use low-frequency sampling (e.g., once per minute), resulting in limited data information and difficulty in capturing the dynamic changes of electrical appliances. Although high-frequency NILM methods can extract richer features (such as frequency domain features, VI curves, etc.), their practical application is limited by hardware conditions.

[0005] 2) It is difficult to simultaneously capture both long-term and short-term fluctuations: Electrical appliance consumption behavior has both long-term regularity (such as daily cycle and weekly cycle) and short-term irregular fluctuations. Traditional methods are difficult to capture both characteristics at the same time.

[0006] 3) Low identification accuracy of low-power appliances: The signals of low-power appliances are easily masked by high-power appliances, and the regression analysis has a poor fitting effect on zero or near-zero values, resulting in a large identification error.

[0007] To address these issues, existing research has proposed several improvement methods, such as:

[0008] Traditional machine learning methods, such as Hidden Markov Models (HMM), Support Vector Machines (SVM), and Decision Trees, have some effectiveness, but their ability to model complex electricity consumption patterns is limited.

[0009] Deep learning methods, including convolutional neural networks (CNN), temporal convolutional networks (TCN), long short-term memory networks (LSTM), and Transformers, can capture temporal dependencies, but they have high computational complexity when processing long-period signals and are insufficient in feature extraction for low-frequency data.

[0010] Multi-task learning framework: Improve model performance by jointly optimizing multiple electrical parameters (such as current, power, etc.), but existing methods do not fully consider the aggregation constraints and zero-value fitting problem among electrical appliances in the design of loss functions.

[0011] The Mamba structure, proposed in 2024, has shown potential in time-series signal processing due to its selective input mechanism and linear computational complexity, but its application in the field of NILM has not been fully explored. In addition, existing methods mostly focus on single-scale feature extraction and lack effective fusion of multi-scale periodic electricity consumption information. Summary of the Invention

[0012] To address the shortcomings of existing technologies, a load identification method combining multi-scale Mamba and attention mechanisms is proposed. This load identification method can achieve non-invasive and accurate identification of low-frequency power loads, improve the accuracy of low-frequency power load identification, and effectively solve problems such as difficulty in extracting features from low-frequency data, difficulty in balancing long-period and short-term feature fluctuations, and low identification accuracy of low-power appliances.

[0013] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a load identification method combining multi-scale Mamba and attention mechanisms, comprising the following steps:

[0014] S1. Collect training data on the current, active power and reactive power of the total circuit, introduce virtual electrical appliances into the total circuit and perform normalized preprocessing on the training data to obtain the ratio of the active power of each single electrical appliance to the active power of the total circuit in the training data.

[0015] S2. Construct a multi-scale composite Mamba module, extract the periodic power consumption characteristics of the total circuit at different time scales through forward, reverse and in-order traversal, and combine it with preprocessed training data to construct and train a multi-task model combining the Mamba module and Transformer technology, which is used to calculate the predicted value of the ratio of the active power of each single appliance to the active power of the total circuit.

[0016] S3. Design a multi-task loss function, combine single-device loss, multi-device aggregation loss and sample quantization loss to optimize the multi-task model and obtain the optimized multi-task model.

[0017] S4. Collect actual data on the current, active power, and reactive power of the total circuit during operation. Introduce virtual electrical appliances into the total circuit using the method in S1 and perform normal standardization preprocessing on the actual data. Input the preprocessed actual data into the optimized multi-task model. The optimized multi-task model obtains the ratio of the active power of each individual electrical appliance to the active power of the total circuit through end-to-end regression synchronous fitting. From this, the active power of each individual electrical appliance is derived.

[0018] This invention's load identification method effectively addresses numerical fluctuation issues in low-power scenarios by introducing virtual electrical appliances. It utilizes a multi-scale composite Mamba module to extract periodic power consumption characteristics of the total circuit at different time scales to capture long-term features. A multi-task model combining the Mamba module and Transformer technology accurately captures short-term power fluctuation characteristics. By designing a multi-task loss function and combining single-appliance loss, multi-appliance aggregation loss, and sample quantization loss to optimize the multi-task model, the method simultaneously obtains the ratio of the active power of each individual appliance to the total active power of the circuit through end-to-end regression fitting. From this, the active power of each individual appliance is derived, achieving non-intrusive and accurate identification of low-frequency power loads.

[0019] As a preferred option, the specific process of step S1 is as follows:

[0020] S1.1 Collect training data on the total circuit's current, active power, and reactive power. Let the training data at any time t be set as The active power of each individual appliance in the total circuit can be obtained by decomposition, that is:

[0021] ,

[0022] Where N is the total number of electrical appliances in the circuit. This represents the active power of the k-th electrical appliance in the total circuit;

[0023] S1.2, Introduce an active power of [value] into the total circuit. The virtual appliance is assumed to be always in operation, and its active power is a constant value C, i.e. The introduction of virtual electrical appliances can effectively stabilize numerical fluctuations in low-power scenarios and improve the robustness of the model to fluctuations in low-power signals.

[0024] S1.3. Perform normalization preprocessing on the training data, that is, use the ratio standardization method to calculate the active power of the k-th appliance in the total circuit. Active power of the total circuit ratio The preprocessed training data is obtained, namely:

[0025] ,

[0026] Let constant value , here ,but .

[0027] Preferably, the multi-task model in step S2 comprises four parts: the first part consists of a linear layer (i.e., a Linear layer), an activated GELU layer, and a random blocking layer (i.e., a Dropout layer); the second part is a multi-scale Mamba transform accumulation part, which consists of five branches with the same structure but different scales. Each branch first performs average pooling downsampling, then performs Mamba transform through a multi-scale composite Mamba module, then aligns the sizes of all signals through upsampling, and finally accumulates all signals; the third part is a Transformer transformation part, which consists of two Transformer layers; and the fourth part consists of a linear layer, an activated GELU layer, a random blocking layer, a connection layer (i.e., a Concat layer), and two linear layers.

[0028] As a preferred option, the specific process of step S3 is as follows:

[0029] S3.1 Design a multi-task loss function:

[0030] ,

[0031] Loss1, Loss2, and Loss3 are the single-electrical appliance loss, the multi-electrical appliance aggregation loss, and the sample quantization loss, respectively.

[0032] S3.2 The calculation formula for the design loss of a single electrical appliance is as follows:

[0033] ,

[0034] Where M is a constant, and sign is the sign function. It is the ratio of the i-th appliance calculated by the multi-task model. The predicted value, It is the ratio of the i-th appliance. The true value, It is the ratio of the i-th appliance. The absolute value of the error between the predicted value and the actual value. It is the ratio of the i-th appliance. The average value;

[0035] S3.3 The calculation formula for the aggregation loss of multiple electrical appliances is as follows:

[0036] ,

[0037] S3.4 The formula for calculating the sample quantization loss is as follows:

[0038] ,

[0039] in It is the ratio of the i-th appliance. The maximum value of the true value, the symbol "[ ]" indicates rounding;

[0040] S3.5. The multi-task model is optimized by using the Adam optimization algorithm to jointly minimize the single-device loss, multi-device aggregation loss and sample quantization loss, resulting in the optimized multi-task model.

[0041] Compared with existing technologies, this invention has the following advantages: The load identification method of this invention constructs a multi-task model combining Mamba modules and Transformer technology, synergistically integrating the two technologies to accurately extract the long-term transformation signal and short-term fluctuation characteristics of the overall circuit, achieving non-intrusive and accurate identification of low-frequency power loads. This load identification method improves the accuracy of low-frequency power load identification and effectively solves problems such as the difficulty in extracting low-frequency data features, the difficulty in balancing long-period and short-term characteristic fluctuations, and the low accuracy of identifying low-power appliances. Compared with existing methods, this load identification method exhibits significant advantages in balancing long-period patterns and short-term fluctuations, effectively improving load identification performance. Attached Figure Description

[0042] Figure 1 This is a structural diagram of the multi-scale composite Mamba module constructed in the embodiment;

[0043] Figure 2 This is a structural diagram of the multi-task model combining the Mamba module and Transformer technology constructed in the embodiment.

[0044] Figure 3 This is a comparison chart of the actual and predicted values ​​of the active power of the rental room sub-meter in the embodiment;

[0045] Figure 4 This is a comparison chart of the actual and predicted active power values ​​of the heat pump sub-meter in the embodiment.

[0046] Figure 5 This is a comparison chart of the actual and predicted values ​​of the active power of the refrigerator sub-meter in the embodiment;

[0047] Figure 6 This is a comparison chart of the actual and predicted active power values ​​of the power amplifier sub-table in the embodiment.

[0048] Figure 7 This is a comparison chart of the actual and predicted values ​​of the active power of the four electrical appliances / sub-meters in the total circuit of the embodiment;

[0049] In the diagram, Sum represents the main circuit, App1 represents the rental room sub-meter, App2 represents the heat pump sub-meter, App3 represents the refrigerator sub-meter, and App4 represents the power amplifier sub-meter. Detailed Implementation

[0050] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0051] The load identification method combining multi-scale Mamba and attention mechanisms in this embodiment includes the following steps:

[0052] S1. Collect training data on the current, active power, and reactive power of the total circuit, introduce virtual electrical appliances into the total circuit, and perform normal standardization preprocessing on the training data to obtain the ratio of the active power of each single electrical appliance to the active power of the total circuit in the training data.

[0053] The specific process of step S1 is as follows:

[0054] S1.1 Assume there are N=8 appliances / sub-meters in the main circuit, namely the rental room sub-meter, heat pump sub-meter, refrigerator sub-meter, amplifier sub-meter, basement sub-meter, dishwasher sub-meter, washing machine sub-meter, and restaurant sub-meter. Collect training data on the current, active power, and reactive power of the main circuit. Let the training data at any time t be set as The active power of each individual appliance in the total circuit can be obtained by decomposition, that is:

[0055] ,

[0056] in, This represents the active power of the k-th electrical appliance in the total circuit;

[0057] Figure 7 The blue vertical line in (a) represents the total active power of the circuit during training;

[0058] S1.2, Introduce an active power of [value] into the total circuit. The virtual appliance is assumed to be always in operation, and its active power is a constant value C, i.e. ;

[0059] S1.3. Perform normalization preprocessing on the training data, that is, use the ratio standardization method to calculate the active power of the k-th appliance in the total circuit. Active power of the total circuit ratio The preprocessed training data is obtained, namely:

[0060] ,

[0061] Let constant value , here ,but .

[0062] S2, Construction Figure 1The multi-scale composite Mamba module shown (this multi-scale composite Mamba module in Figure 2 (referred to as CMB in Chinese) Figure 1 In this model, "Mamba" represents the Mamba module, "Inverse" represents reverse traversal, "MidTravel" represents inorder traversal, "Input x" represents input, and "+" represents addition. The model extracts the periodic power consumption characteristics of the total circuit at different time scales through forward, reverse, and inorder traversals. Combined with preprocessed training data, a multi-task model combining the Mamba module and Transformer technology is constructed and trained to calculate the predicted ratio of the active power of each individual appliance to the total active power of the circuit. The Transformer employs an 8-head self-attention mechanism with a hidden dimension of 512.

[0063] like Figure 2 As shown, the multi-task model obtained in step S2 consists of four parts: the first part consists of a linear layer, an activation GELU layer, and a random blocking layer; the second part is a multi-scale Mamba transform accumulation part, which consists of five branches with the same structure but different scales. The sliding window sizes Ksize in the pooling operation are 1, 5, 10, 30, and 60, respectively. Each branch first performs average pooling downsampling, then performs Mamba transform through a multi-scale composite Mamba module, then aligns the sizes of all signals through upsampling, and finally accumulates all signals; the third part is a Transformer transformation part, which consists of two Transformer layers (i.e., Transformer Layer × 2); the fourth part consists of a linear layer, an activation GELU layer, a random blocking layer, a connection layer, and two linear layers (i.e., Linear × 2).

[0064] S3. Design a multi-task loss function, combine single-device loss, multi-device aggregation loss and sample quantization loss to optimize the multi-task model and obtain the optimized multi-task model.

[0065] The specific process of step S3 is as follows:

[0066] S3.1 Design a multi-task loss function:

[0067] ,

[0068] Loss1, Loss2, and Loss3 are the single-electrical appliance loss, the multi-electrical appliance aggregation loss, and the sample quantization loss, respectively.

[0069] S3.2 The calculation formula for the design loss of a single electrical appliance is as follows:

[0070] ,

[0071] Where M is a constant, M=5, and sign is the sign function. It is the ratio of the i-th appliance calculated by the multi-task model. The predicted value, It is the ratio of the i-th appliance. The true value, It is the ratio of the i-th appliance. The absolute value of the error between the predicted value and the actual value. It is the ratio of the i-th appliance. The average value;

[0072] S3.3 The calculation formula for the aggregation loss of multiple electrical appliances is as follows:

[0073] ,

[0074] S3.4 The formula for calculating the sample quantization loss is as follows:

[0075] ,

[0076] in It is the ratio of the i-th appliance. The maximum value of the true value, the symbol "[ ]" indicates rounding;

[0077] S3.5. The multi-task model is optimized by using the Adam optimization algorithm to jointly minimize the single-device loss, multi-device aggregation loss and sample quantization loss, resulting in the optimized multi-task model.

[0078] S4. Collect actual data on the current, active power, and reactive power of the total circuit during operation. Introduce virtual electrical appliances into the total circuit using the method in S1 and perform normal standardization preprocessing on the actual data. Input the preprocessed actual data into the optimized multi-task model. The optimized multi-task model obtains the ratio of the active power of each individual electrical appliance to the active power of the total circuit through end-to-end regression synchronous fitting. From this, the active power of each individual electrical appliance is derived.

[0079] Figure 3 This is a comparison chart of the actual and predicted values ​​of the active power of the sub-meter for the rented room. Figure 3 In the table, (a) represents the actual value of the active power in the rented room sub-table. Figure 3 (b) in the table represents the predicted active power of the leased room sub-table.

[0080] Figure 4 This is a comparison chart of the actual and predicted active power values ​​of the heat pump sub-meter. Figure 4 In the diagram, (a) represents the actual value of the active power of the heat pump sub-meter. Figure 4(b) in the figure represents the predicted value of the active power of the heat pump submeter.

[0081] Figure 5 This is a comparison chart of the actual and predicted values ​​of the active power of the refrigerator's sub-meter. Figure 5 (a) in the figure represents the actual value of the active power of the refrigerator's sub-meter. Figure 5 (b) in the figure represents the predicted value of the active power of the refrigerator submeter.

[0082] Figure 6 This is a comparison chart of the actual and predicted active power values ​​of the power amplifier sub-meter. Figure 6 In the table, (a) represents the actual value of the active power in the power amplifier sub-table. Figure 6 (b) in the table represents the predicted value of the active power of the power amplifier sub-table.

[0083] Figure 7 This is a comparison chart of the actual and predicted active power values ​​of the four electrical appliances / sub-meters in the main circuit. Figure 7 In the diagram, (a) represents the actual active power of the total circuit and the four sub-meters. Figure 7 (b) represents the predicted active power of the total circuit and the four sub-meters. Figure 7 In (a) and (b), the vertical lines of the same color represent the actual and predicted values ​​of active power in the same sub-table.

[0084] In this invention, the data units for current, active power, and reactive power are as follows: current: ampere (A), active power: watt (W), reactive power: volt (V).

Claims

1. A load identification method combining multi-scale Mamba and attention mechanisms, characterized in that, Includes the following steps: S1. Collect training data on the current, active power and reactive power of the total circuit, introduce virtual electrical appliances into the total circuit and perform normalized preprocessing on the training data to obtain the ratio of the active power of each single electrical appliance to the active power of the total circuit in the training data. S2. Construct a multi-scale composite Mamba module, extract the periodic power consumption characteristics of the total circuit at different time scales through forward, reverse and in-order traversal, and combine it with preprocessed training data to construct and train a multi-task model combining the Mamba module and Transformer technology, which is used to calculate the predicted value of the ratio of the active power of each single appliance to the active power of the total circuit. S3. Design a multi-task loss function, combine single-device loss, multi-device aggregation loss and sample quantization loss to optimize the multi-task model and obtain the optimized multi-task model. S4. Collect actual data on the current, active power, and reactive power of the total circuit during operation. Introduce virtual electrical appliances into the total circuit using the method in S1 and perform normal standardization preprocessing on the actual data. Input the preprocessed actual data into the optimized multi-task model. The optimized multi-task model obtains the ratio of the active power of each individual electrical appliance to the active power of the total circuit through end-to-end regression synchronous fitting. From this, the active power of each individual electrical appliance is derived.

2. The load identification method combining multi-scale Mamba and attention mechanisms according to claim 1, characterized in that, The specific process of step S1 is as follows: S1.1 Collect training data on the total circuit's current, active power, and reactive power. Let the training data at any time t be set as The active power of each individual appliance in the total circuit can be obtained by decomposition, that is: ， Where N is the total number of electrical appliances in the circuit. This represents the active power of the k-th electrical appliance in the total circuit; S1.2, Introduce an active power of [value] into the total circuit. The virtual appliance is assumed to be always in operation, and its active power is a constant value C, i.e. ; S1.

3. Perform normalization preprocessing on the training data, that is, use the ratio standardization method to calculate the active power of the k-th appliance in the total circuit. Active power of the total circuit ratio The preprocessed training data is obtained, namely: ， Let constant value , here ,but .

3. The load identification method combining multi-scale Mamba and attention mechanisms according to claim 1, characterized in that, The multi-task model described in step S2 consists of four parts: the first part consists of a linear layer, an activated GELU layer, and a random blocking layer; the second part is a multi-scale Mamba transform accumulation part, which consists of five branches with the same structure but different scales. Each branch first performs average pooling downsampling, then performs Mamba transform through a multi-scale composite Mamba module, then aligns the sizes of all signals through upsampling, and finally accumulates all signals; the third part is a Transformer transformation part, which consists of two Transformer layers; and the fourth part consists of a linear layer, an activated GELU layer, a random blocking layer, a connection layer, and two linear layers.

4. The load identification method combining multi-scale Mamba and attention mechanisms according to claim 1, characterized in that, The specific process of step S3 is as follows: S3.1 Design a multi-task loss function: ， Loss1, Loss2, and Loss3 are the single-electrical appliance loss, the multi-electrical appliance aggregation loss, and the sample quantization loss, respectively. S3.2 The calculation formula for the design loss of a single electrical appliance is as follows: ， Where M is a constant, and sign is the sign function. It is the ratio of the i-th appliance calculated by the multi-task model. The predicted value, It is the ratio of the i-th appliance. The true value, It is the ratio of the i-th appliance. The absolute value of the error between the predicted value and the actual value. It is the ratio of the i-th appliance. The average value; S3.3 The calculation formula for the aggregation loss of multiple electrical appliances is as follows: ， S3.4 The formula for calculating the sample quantization loss is as follows: ， in It is the ratio of the i-th appliance. The maximum value of the true value, with the symbol "[ ]" indicating rounding; S3.

5. The multi-task model is optimized by using the Adam optimization algorithm to jointly minimize the single-device loss, multi-device aggregation loss and sample quantization loss, resulting in the optimized multi-task model.

Images