Power utilization abnormal behavior detection method and system based on deep learning

Abstract

The invention discloses a deep learning-based power utilization abnormal behavior detection method and system. The method comprises the following steps of: obtaining electrical quantity modal, environment modal and log modal data of a target system, and performing data cleaning, adaptive segmentation standardization, missing value filling and timestamp adding preprocessing; respectively inputting each piece of preprocessed modal data into a Transform model based on time decay attention and time perception position coding to carry out deep feature extraction, and obtaining features of corresponding modals; the electrical quantity modal features are enhanced and spliced with the environment modal features and the log modal features to obtain fusion features, the fusion features are utilized to train an anomaly detection model based on deep learning, and the anomaly detection model is updated in real time based on a dynamic threshold strategy to judge the anomaly standard; and using the trained anomaly detection model to carry out electricity consumption anomaly detection. According to the invention, the safety and stability of the power system can be significantly improved, and the intelligent development of the power industry is promoted.

Description

Technical Field

[0001] The present invention belongs to the field of electricity consumption behavior analysis and monitoring, and more specifically, relates to a method and system for detecting abnormal electricity consumption behavior based on deep learning. Background Art

[0002] With the acceleration of the modern industrialization and informatization processes, electricity, as an infrastructure, is widely used in all walks of life, especially playing an indispensable role in the manufacturing, service, and high-tech industries. With the continuous growth of electricity demand and the continuous expansion of the power system, how to accurately monitor, analyze, and prevent abnormal electricity consumption behavior has become an important issue for ensuring the safety of electricity supply and improving energy utilization efficiency.

[0003] Abnormal electricity consumption behaviors usually include illegal electricity use, equipment failures, electricity fraud, and human errors. These abnormal behaviors not only affect the normal operation of the power system, increase the operating costs of enterprises, but also may cause energy waste and even lead to safety accidents. Therefore, timely and accurately detecting abnormal electricity consumption behaviors is of great significance for ensuring the stability of the power system and improving energy management efficiency.

[0004] Currently, most traditional methods for detecting abnormal electricity consumption behaviors rely on rule-based detection or simple statistical analysis. Although these methods can detect abnormal behaviors to a certain extent, they have many limitations. For example, rule-based detection methods require manually setting a large number of thresholds and rules, and it is usually difficult for these rules to cover all types of abnormal situations. Moreover, with the dynamic changes in electricity consumption patterns, these rules need to be adjusted frequently. Statistical analysis methods can summarize and generalize data to a certain extent, but it is difficult to handle high-dimensional, non-linear, and complex patterns in the data, resulting in low detection accuracy and the inability to detect potential abnormal behaviors in a timely manner.

[0005] Driven by deep learning technologies, in recent years, data-driven methods for detecting abnormal electricity consumption behaviors have gradually become a research hotspot. Deep learning models, especially neural networks based on time-series data modeling, can automatically learn complex patterns in electricity consumption data and effectively handle high-dimensional data and time-series dependence problems. However, existing deep learning-based detection methods often face certain challenges in processing multi-dimensional and multi-source data, and model training requires a large amount of labeled data, which brings great cost pressure to practical applications.

[0006] Further research shows that the detection of abnormal electricity consumption behavior not only depends on historical electricity consumption data, but also needs to consider multi-dimensional spatio-temporal context information, such as seasonal changes, regional differences, user behavior patterns, etc. Traditional deep learning models have certain limitations in processing these multi-scale information. Especially when facing the differences in electricity consumption behavior at different time scales and different regions, the adaptability and generalization ability of the models are insufficient. Therefore, how to effectively combine spatio-temporal features and multi-scale data to improve the flexibility and accuracy of the model has become a key problem to be solved urgently in the field of abnormal electricity consumption behavior detection. Summary of the Invention

[0007] To solve the deficiencies in the prior art, the present invention provides a method and system for detecting abnormal electricity consumption behavior based on deep learning. Through an improved Transformer model and a multi-scale feature fusion mechanism, it can effectively capture the differences in electricity consumption behavior at different time periods and different regions, thereby improving the detection ability and generalization ability of the model. It not only improves the accuracy of abnormal behavior detection, but also has strong adaptability, and can cope with the dynamic changes of electricity consumption behavior and complex multi-modal data, thus providing strong support for the construction of smart grids and the electricity management of enterprises, and is applicable to the abnormal monitoring and prediction of electricity consumption of industrial and commercial users.

[0008] The present invention adopts the following technical solutions.

[0009] In the first aspect of the present invention, a method for detecting abnormal electricity consumption behavior based on deep learning is proposed, including:

[0010] Obtain the electrical quantity modality, environmental modality, and log modality data of the target system, and perform data cleaning, adaptive piecewise normalization, missing value filling, and timestamp addition preprocessing;

[0011] Input the preprocessed data of each modality into a Transformer model based on time decay attention and time-aware position encoding for deep feature extraction to obtain the features corresponding to each modality;

[0012] Enhance the electrical quantity modality features and splice them with the environmental modality and log modality features to obtain fusion features. Use the fusion features to train a deep learning-based anomaly detection model, and update the criteria for determining anomalies of the anomaly detection model in real time based on a dynamic threshold strategy;

[0013] Use the trained anomaly detection model to detect abnormal electricity consumption.

[0014] Preferably, the adaptive piecewise normalization is specifically: divide the data into high, medium, and low load condition segments according to the load rate, and calculate the normalization values independently for each condition segment:

[0015]

[0016] where esg ∈ {high, medium, low} is the working condition segmentation; x i，z is the z-th eigenvalue in the i-th data point; x' i，z is the standardized value; μ z，seg is the mean of the z-th feature in the seg-th working condition segment; σ z，seg is the standard deviation of the z-th feature in the seg-th working condition segment; β seg is the learnable scaling factor for the seg-th working condition segment; z ∈ {1, 2,..., m}, and m is the number of features in each data point.

[0017] Preferably, inputting the preprocessed multi-modal data into the Transformer model based on time decay attention and time-aware positional encoding for deep feature extraction to obtain the features corresponding to the modality, including:

[0018] Inputting the preprocessed multi-modal data into the Transformer model respectively, and performing embedding representation on the preprocessed data and injecting time-aware positional encoding to obtain the data after time-aware positional encoding where is the time-aware positional encoding matrix;

[0019] Based on perform time decay self-attention layer calculation, and stack the calculations of each time decay self-attention layer to obtain the final output, where the time decay self-attention layer calculation formula is as follows:

[0020]

[0021] where Q, K, and V respectively represent the query, key, and value matrices, which are generated by linear projection; d k is the dimension of the key; is the learnable decay coefficient; ⊙ represents the Hadamard product; Δt is the time difference matrix, and the elements of this matrix are Δt uv = |t u - t v | (u, v = 1, 2,..., N), t u , t v are the timestamps of the u-th and v-th data points, and N is the total number of data points; Softmax(·) is the Softmax function; Attention represents the time decay self-attention layer calculation.

[0022] Preferably, the calculation method of the time-aware positional encoding matrix PE(t) is:

[0023]

[0024] Among them, pos ∈ [1, n] represents the temporal position index of the data point, t is the absolute timestamp of the data point, γ is a learnable time decay coefficient, r is the dimension index of the position encoding; n is the number of time points, m is the feature dimension; PE (pos,2r) is the element value of the pos-th row and the 2r-th column in PE(t); PE (pos,2r+1) is the element value of the pos-th row and the (2r + 1)-th column in PE(t).

[0025] Preferably, enhancing the electrical quantity modal features and splicing them with the environmental modal and log modal features to obtain the fusion features includes:

[0026] Calculating the correlation coefficient matrices of the electrical quantity modal feature matrix and the environmental modal feature matrix and the log modal feature matrix respectively:

[0027]

[0028] Among them, s is the dimension of the output features of the Transformer model, n is the number of time points, T represents matrix transpose, and Softmax(·) is the Softmax function;

[0029] Enhancing the electrical quantity modal features according to the correlation coefficient matrix to obtain the enhanced electrical quantity modal feature matrix:

[0030]

[0031] Splicing the enhanced electrical quantity modal feature matrix with the environmental modal feature matrix and the log modal feature matrix to obtain the fusion feature matrix H fusion .

[0032] Preferably, training the deep learning-based anomaly detection model using the fusion features includes:

[0033] Based on the fusion feature matrix H fusion , constructing a time-modal weighted dynamic distance, and training the deep learning-based anomaly detection model by minimizing the loss function based on the dynamic distance, and its time-modal weighted dynamic distance is:

[0034]

[0035] Among them, x i represents the fusion feature vector of the i-th data point in H fusion ; x j represents H fusionThe j-th data point fusion feature vector in; H fusion，i，k is H fusion The k-th eigenvalue of the i-th data point in H fusion，j，k is H fusion The k-th eigenvalue of the j-th data point in; is the modal importance weight of the k-th feature dimension; is the time decay factor, λ is the learnable decay coefficient, t i , t j is the timestamp of the i-th and j-th data points, d is the total number of feature dimensions of H fusion

[0036] Preferably, the loss function based on dynamic distance is:

[0037]

[0038] where is the indicator function, y i is the label of the i-th data point, y j is the label of the j-th data point, N represents the total number of data points.

[0039] Preferably, the criterion for the real-time update of the anomaly detection model to determine anomalies based on the dynamic threshold strategy includes:

[0040] Iteratively calculate the dynamic threshold according to the device health decay factor and the uncertainty adaptation term. When the dynamic distance of the data point exceeds the dynamic threshold, it is determined as an anomaly. The iterative calculation formula for the dynamic threshold is:

[0041]

[0042] where T p , T p+1 are the dynamic thresholds of the current round and the next round, α is the uncertainty adjustment coefficient, η is the base learning rate, is the gradient of the loss function, H(p) is the device health index, λ1 is the health decay weight, and σ(p) represents the prediction uncertainty of the anomaly detection model.

[0043] The second aspect of the present invention proposes a power consumption anomaly behavior detection system based on deep learning, including:

[0044] A data processing module for acquiring electrical quantity modality, environmental modality, and log modality data of the target system and performing preprocessing such as data cleaning, adaptive piecewise normalization, missing value filling, and timestamp addition;

[0045] ​A feature extraction module, which is used to input the preprocessed multi-modal data into a Transformer model based on time decay attention and time-aware positional encoding respectively for deep feature extraction, so as to obtain the features corresponding to each modality;

[0046] A model training module, which is used to enhance the electrical quantity modality features and splice them with the environment modality and log modality features to obtain fused features, use the fused features to train an anomaly detection model based on deep learning, and update the criteria for the anomaly detection model to determine anomalies in real time based on a dynamic threshold strategy;

[0047] An anomaly detection module, which is used to perform power consumption anomaly detection by using the trained anomaly detection model.

[0048] The third aspect of the present invention proposes a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0049] The fourth aspect of the present invention proposes a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method are implemented.

[0050] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0051] 1. Multi-modal data fusion: By fusing multi-modal data from different data sources (such as electrical quantities, environmental data, historical power consumption data, etc.), the present invention can comprehensively explore the potential associations in the data, improve the accuracy and comprehensiveness of anomaly detection. This multi-modal data fusion method overcomes the limitations of traditional single data source methods and effectively improves the reliability of the detection results.

[0052] The multi-modal data fusion mechanism of the present invention can fully explore and utilize the information from different data sources. These data are interrelated and jointly reveal the complex patterns of enterprise power consumption behavior. Therefore, the model trained based on this can identify abnormal behaviors that cannot be captured by a single data source, thereby improving the comprehensiveness and accuracy of detection.

[0053] 2. Adaptive piecewise normalization: By adopting a learnable scaling factor, the traditional data preprocessing is upgraded to a differentiable calculation layer, making it a part of the model to participate in end-to-end optimization. On the one hand, this design can break through the separation between preprocessing and model training, enabling the normalization strategy to be dynamically adjusted according to the model objective; on the other hand, by normalizing high / medium / low load data respectively, it can eliminate the distribution shift caused by load fluctuations.

[0054] 3. Deep feature extraction: Use a Transformer model based on time-decaying attention and time-aware positional encoding for deep feature extraction. Among them, the time-decaying attention mechanism uses the time difference Δt matrix as the structured prior for attention calculation, realizing physical meaning-driven, conforming to the power equipment fault law that "recent events have a greater impact". The decay coefficient λ is directly related to the development speed of equipment faults (such as λ small → focus on long-term trends), realizing parameter interpretability. λ is automatically learned through gradient descent without manual parameter tuning, achieving end-to-end optimization; the time-aware positional encoding introduces the absolute timestamp t, encoding the equipment operation duration (such as cumulative hours) into the features, reflecting long-term trends such as aging and seasonal loads; the decay coefficient γ controls the decay rate of the positional encoding, matching the actual aging speed of the equipment, enhancing the model's sensitivity to abnormal electricity consumption.

[0055] The time-series data processing framework based on Transformer of the present invention can effectively capture the time-series features and dynamic changes in electricity consumption data, and then realize in-depth analysis of different electricity consumption patterns. It can flexibly focus on important time points and patterns in the data, automatically capture complex patterns in the data, and dig out subtle changes in the data, thus providing a more accurate feature representation for anomaly detection, avoiding the problem of insufficient local information processing in traditional methods, and improving the performance of the model in complex electricity consumption data.

[0056] 4. Compared with directly splicing features of different scales, the present invention superimposes interaction features on the original electrical quantity features. By enhancing the modal features of electrical quantities and splicing them with environmental modal and log modal features, fusion features are obtained, retaining the basic information while strengthening the key modal associations, capable of handling the differences in electricity consumption behaviors in different time periods and regions, and improving the adaptability and generalization ability of the model. When detecting abnormal electricity consumption, it not only has strong real-time performance but also can handle various complex scenarios, such as the processing of large-scale user data and the electricity consumption fluctuations during special time periods such as seasons and holidays.

[0057] 5. Dynamic threshold optimization mechanism: In order to better adapt to the electricity consumption characteristics of different regions, the present invention proposes a dynamic threshold adjustment mechanism based on the equipment health decay factor and the uncertainty adaptive term, which can dynamically adjust the threshold of anomaly detection according to the spatio-temporal distribution of electricity consumption behaviors, enabling the model to adapt to the distribution characteristics of different data, ensuring the accuracy and robustness of anomaly detection. By continuously optimizing the threshold, the model can adapt to the electricity consumption behaviors in different time periods, industries, and regions, flexibly adjust the anomaly detection strategy, and has stronger generalization ability, further improving the accuracy of the detection results.

[0058] In summary, by combining an improved Transformer model, multi-modal data fusion, and a dynamic threshold optimization mechanism, the present invention proposes an intelligent anomaly detection solution that is efficient, accurate, and highly robust. This method can not only enhance the ability to detect abnormal electricity consumption behaviors of enterprises but also be widely applied to the anomaly identification of various complex data, having important practical significance and application prospects. By combining the improved Transformer model and self-supervised learning technology, and utilizing the deep features of multi-modal data, the present invention can more accurately detect potential anomalies without relying on a large amount of labeled data. This method not only improves the accuracy of anomaly detection but also significantly enhances the robustness and adaptability of the model. Experimental results show that the detection method of the present invention performs excellently on multiple real electricity consumption datasets and can accurately identify various types of abnormal electricity consumption behaviors, such as illegal electricity use, equipment failures, electricity fraud, etc. Compared with traditional methods, the present invention has higher accuracy, recall rate, and stronger generalization ability. This method is widely applicable to fields such as smart grids, electricity consumption monitoring, and energy management, and can significantly improve the safety and stability of the power system, promoting the intelligent development of the power industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 is a schematic diagram of the overall framework process of the method of the present invention;

[0060] Figure 2 is a schematic diagram of normal and abnormal electrical quantity data curves;

[0061] Figure 3 is a schematic diagram of the anomaly detection process. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. The embodiments described in this application are only a part of the embodiments of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0063] As Figure 1 shown, Embodiment 1 of the present invention provides a method for detecting abnormal electricity consumption behaviors based on deep learning, including:

[0064] S1: Obtain multi-modal data of the target system and preprocess the obtained data;

[0065] Further preferably, the data collection and preparation process is as follows: collect various types of data related to the target system, including but not limited to equipment monitoring data, sensor data, system logs, etc. These data are sourced from different sensors, devices, and monitoring systems, including but not limited to electrical quantity data, environmental data, policy data, historical electricity consumption data, etc. These data may involve multiple feature dimensions, such as temperature, pressure, current, voltage, operating status, etc., aiming to comprehensively reflect the operating status of the target system.

[0066] In specific implementation, it is first necessary to collect data related to the target system. These data include but are not limited to equipment monitoring data, sensor data, and system logs, etc. The sources of the data can be multiple fields, such as energy management, industrial production, environmental monitoring, etc. The specific data items to be collected include:

[0067] (1) Equipment monitoring data: Through sensors deployed at distribution nodes of smart grids, industrial motor sets, or energy supply equipment in commercial buildings, real-time operating parameters of the equipment are collected, such as temperature, pressure, current, voltage, rotational speed, etc.;

[0068] (2) Sensor data: including the mechanical vibration quantity (vibration amplitude, frequency, phase) of the monitored equipment, environmental temperature and humidity, and spatial electromagnetic field intensity;

[0069] (3) System log data: covering operation logs, maintenance records, and fault waveform data generated by power monitoring systems (SCADA), equipment management systems (EMS), and fault recording devices.

[0070] Set the data set as D = {d1, d2,..., d n}, where each data point d i is a vector containing multiple feature dimensions. For example: d1 = (temperature = 30°C, pressure = 101 kPa, current = 10 A, voltage = 220 V) contains the numerical values of four features of the equipment, namely temperature, pressure, current, and voltage.

[0071] Among them, in order to ensure the consistency and quality of the data, the collected raw data is preprocessed, including data cleaning, adaptive piecewise normalization, missing value filling, and timestamp addition, etc., to provide standardized features and accurate timestamps for the subsequent time decay attention mechanism, as follows:

[0072] (1) Data cleaning: Detect and process noise and outliers, and adopt the box plot method combined with the real-time load rate of the equipment to dynamically adjust the outlier threshold. When the equipment is operating at high load, relax the outlier judgment thresholds for current and voltage parameters; tighten the thresholds when operating at low load to avoid misjudgment; and perform multiple imputation processing on the outliers.

[0073] (2) Adaptive Piecewise Standardization: Divide the data into high / medium / low load condition segments according to the load rate, and calculate the standardized values independently for each condition segment:

[0074]

[0075] where seg ∈ {high, medium, low} is the condition segment; x i，z is the z-th eigenvalue in the i-th data point; x′ i，z is the standardized value; x i，z is the original input, which does not depend on the condition segment and is only used as the starting point for calculation. x′ i，z as the final standardized value also does not depend on the condition segment, and the input and output data maintain global consistency, avoiding symbol confusion caused by condition segmentation; μ z，seg is the mean of the z-th feature in the seg-th condition segment; σ z，seg is the standard deviation of the z-th feature in the seg-th condition segment; β seg is the learnable scaling factor for the seg-th condition segment; z ∈ {1, 2,..., m}, where m is the number of features in each data point.

[0076] In specific implementation, the load rates corresponding to the high, medium, and low load condition segments are less than or equal to 40%, between 40% and 80%, and greater than or equal to 80% respectively. The learnable scaling factor β seg is randomly initialized and then determined through deep learning of the data in the corresponding condition segment.

[0077] The essence of the learnable scaling factor of adaptive piecewise standardization is to upgrade traditional data preprocessing to a differentiable calculation layer, making it part of the model to participate in end-to-end optimization. This design can, on the one hand, break through the separation between preprocessing and model training, enabling the standardization strategy to be dynamically adjusted according to the model objective; on the other hand, it can eliminate the distribution shift caused by load fluctuations by standardizing high / medium / low load data respectively.

[0078] (3) Missing Value Imputation: Missing values refer to the clustering, grouping, censoring, or truncation of data in rough data due to lack of information. It means that the values of one or some attributes in the existing dataset are incomplete. For power grid data, data missing problems may occur due to data collection or data storage failures at metering points. For missing data points, mean imputation, median imputation, or similarity-based imputation methods can be used.

[0079] (4) Finally, add timestamps to obtain the preprocessed dataset D′ = {d′1, d′2,..., d′ n}.

[0080] Each preprocessed data point d′ i ∈ D′ contains:

[0081] Feature vector: Data that has been cleaned, standardized, and filled with missing values;

[0082] Timestamp: The original acquisition time retained.

[0083] Example: For the monitoring data of a distribution transformer:

[0084] d′ i =(x′ i =[temperature, current, vibration], t i =1630453200)

[0085] where t i is the Unix timestamp.

[0086] The preprocessed dataset D′={d′1, d′2,..., d′ n} provides standardized features and accurate timestamps for the time decay attention mechanism.

[0087] S2: Input the preprocessed data of each modality into a Transformer model based on time decay attention and time-aware positional encoding for deep feature extraction to obtain the features corresponding to the modality;

[0088] Further preferably, the Transformer architecture is improved through time-aware positional encoding and stacked time decay self-attention layers to capture the temporal relationships and potential abnormal patterns in the data, and generate the feature vectors of each sample. This process can effectively extract the deep features of the data and provide support for subsequent anomaly detection. The specific process of the improved Transformer model for extracting deep features through time-aware positional encoding and stacked time decay self-attention layers is as follows:

[0089] (1) Input embedding and time-aware positional encoding: Embed the preprocessed data and inject time-aware positional encoding to obtain the data after time-aware positional encoding:

[0090]

[0091] where, is the time-aware positional encoding matrix, and its calculation method is:

[0092]

[0093] Among them, pos ∈ [1, n] represents the temporal position index of the data point, t is the absolute timestamp of the data point, γ is a learnable time decay coefficient, and r is the dimension index of the position encoding; n is the number of time points, and m is the feature dimension; it can be understood that PE (pos,2r) is a scalar, representing the element value at the pos-th row and the 2r-th column in the time-aware position encoding matrix PE(t), which is calculated from the parameter t. Similarly, PE (pos,2r+1) is a scalar, representing the element value at the pos-th row and the 2r + 1-th column in the time-aware position encoding matrix PE(t). PE(t) is the complete position encoding matrix, and each element in it is calculated from the parameter t.

[0094] (2) Perform the calculation of the l-th layer of time decay self-attention layer based on :

[0095]

[0096] Among them, Q, K, and V represent the query, key, and value matrices respectively, which are generated by linear projection from ; d k is the dimension of the key; is the learnable decay coefficient; ⊙ represents the Hadamard product (element-wise multiplication); Δt is the time difference matrix, and the elements of the matrix are Δt uv = |t u - t v | (u, v = 1, 2,..., N), where t u , t v are the timestamps of the u-th and v-th data points, and N is the total number of data points.

[0097] By taking the time difference Δt as the structured prior for attention calculation, the time decay attention mechanism realizes:

[0098] Physical meaning driven: It conforms to the power equipment failure law that "recent events have a greater impact".

[0099] Parameter interpretable: The decay coefficient λ is directly related to the development speed of equipment failures (e.g., small λ → focus on long-term trends).

[0100] End-to-end optimization: λ is automatically learned through gradient descent without manual parameter tuning.

[0101] (3) Stack the calculations of each time decay self-attention layer to obtain the final output.

[0102] The final output after stacking L layers is:

[0103]

[0104] Among them, is the input data, H∈R n×s is the output feature matrix, s is the feature dimension after the improved Transformer model, n is the number of time points, and f l (·) represents the time-decayed attention calculation of the lth layer based on the formula described in (2) (l=1,2,...,L), and Transformer(·) is the Transformer function.

[0105] S3: Enhance the electrical quantity modal features and combine them with the environmental modal and log modal features to obtain fused features. Use the fused features to train an anomaly detection model, and update the model's anomaly judgment criteria in real time based on a dynamic threshold strategy.

[0106] S3.1: Obtain the multimodal feature representation generated by inputting the preprocessed multimodal data into a unified time-decayed Transformer model:

[0107] H elec =Transformer time (X′ elec ) (Electrical characteristics: current, voltage, speed)

[0108] H env =Transformer time (X′ env ) (Environmental characteristics: temperature and humidity, vibration, electromagnetic field strength)

[0109] H log =Transformer time (X′ log ) (Log features: SCADA logs, EMS records, fault waveforms)

[0110] Then calculate the modal characteristics of electrical quantities and environmental modal characteristics Log mode features The correlation coefficient matrix of :

[0111]

[0112] The electrical quantity enhancement feature is then generated:

[0113]

[0114] Finally, the enhanced electrical modal features are spliced with the original environmental modal and log modal features to obtain the fusion feature matrix:

[0115]

[0116] Among them, Helec , H env , H log are features obtained at different scales. Compared with directly concatenating features of different scales, the present invention superimposes interaction features on the original electrical feature H elec , while retaining the basic information, strengthening the key modal associations.

[0117] S3.2: Use the fused features to train a deep learning-based anomaly detection model, specifically as follows:

[0118] (1) Based on the fused feature matrix construct a time-modal weighted dynamic distance metric:

[0119]

[0120] where x i represents the fused feature vector of the i-th sample, that is, the i-th row vector in H fusion , x j represents the fused feature vector of the j-th sample, that is, the j-th row vector in H fusion , H fusion，i，k is the k-th eigenvalue of the i-th sample in the fused feature matrix H fusion , H fusion，j，k is the k-th eigenvalue of the j-th sample in the fused feature matrix H fusion ; is the modal importance weight of the k-th feature dimension, and its initial value is determined by the variance ratio of H fusion :

[0121]

[0122] where d represents the total number of feature dimensions of the fused feature matrix H fusion .

[0123] is the time decay factor, and the learnable decay coefficient λ in the time decay self-attention formula is reused.

[0124] (2) During the training process, train the model by minimizing the weighted distance loss function:

[0125]

[0126] where, is the indicator function (its value is 1 for same-class samples and 0 for different-class samples), y i is the label of the i-th sample, indicating its class (normal or abnormal), y j is the label of the j-th sample, also indicating its class, and N represents the total number of samples in the training set, that is, the number of data points participating in the model training.

[0127] S3.3: Update the criteria for the model to determine anomalies in real time based on the dynamic threshold strategy, as follows:

[0128] (1) On the basis of the traditional dynamic threshold formula, introduce the device health attenuation factor and the uncertainty adaptation term. The improved iterative formula is:

[0129]

[0130] where T p , T p+1 are the dynamic thresholds for the current round and the next round, α is the uncertainty adjustment coefficient (a learnable parameter, with an initial value of 0.05), η is the base learning rate (with an initial value of 0.01 - 0.1), is the gradient of the loss function, H(p) is the device health index (which decays with the running time, H(p) = H0e -βp , β = 0.001 / day, and H0 = 1 indicates that the device is brand new), λ1 is the health attenuation weight (a learnable parameter, with an initial value of 0.1), and σ(p) represents the model prediction uncertainty (the standard deviation of the predicted values for the current batch).

[0131] (2) When the dynamic distance of the data point exceeds the dynamic threshold, it is determined as an anomaly:

[0132] D(x i , x j ) > T p

[0133] S4: Use the trained anomaly detection model to perform power consumption anomaly detection and feedback optimization of the model.

[0134] Further preferably, in practical applications, the results of the anomaly detection model will be fed back to the management system in real time so as to take corresponding measures. By dynamically adjusting the detection threshold and updating the model parameters, the system can continuously adapt to the changing power consumption behavior patterns, ensuring the efficiency and accuracy of anomaly detection. In addition, the present invention also designs a feedback mechanism, which can further optimize the model based on the detection results, continuously improving the detection accuracy and robustness.

[0135] Based on the above steps, perform power consumption anomaly detection, output the detection results and perform feedback optimization. The detection results can not only identify potential abnormal data, but also can be alarmed, classified or further processed according to needs so as to take corresponding measures in time. Finally, the model is optimized through the feedback mechanism to improve the accuracy and efficiency of subsequent detections.

[0136] In specific implementation, the present invention performs power consumption anomaly detection and determines which data points are abnormal based on the output of the above steps. When the Euclidean distance of a data point exceeds the dynamic threshold, it is determined as abnormal; once abnormal data is detected, further alarm processing, classification, or optimization feedback can be carried out. These anomaly detection results will be fed back to the model for subsequent optimization. By combining the anomaly detection results with the system feedback, the thresholds and training data of the model can be continuously optimized to improve the accuracy and robustness of anomaly detection.

[0137] Through the above specific implementation scheme, the present invention can ensure that the whole process from data collection to anomaly detection and feedback optimization can be accurately and efficiently executed, ensuring the ability to handle complex and variable industrial data, and the model has high adaptability and scalability, improving the accuracy, robustness, and real-time performance of anomaly detection.

[0138] Figure 2 The change curves of normal electrical quantity data and abnormal electrical quantity data are shown. In this figure, we simulate a typical electrical quantity data set, showing the fluctuation range of electrical quantities under normal conditions and the change trend of electrical quantities when anomalies occur. Normal electrical quantity data (solid line curve): represents the change of electrical quantities (such as current, voltage, etc.) over time under normal working conditions. Usually, the fluctuations of electrical quantities will fluctuate within a certain range, conforming to the expected working mode. In the figure, this curve shows smooth fluctuations, following a seasonal or periodic change pattern. Abnormal electrical quantity data (dashed line curve): represents the drastic fluctuations of electrical quantities in the case of equipment failures, load fluctuations, or other anomalies. Usually, abnormal data shows peaks or troughs that deviate significantly from the normal fluctuation mode, or continuous abnormal fluctuations. These abnormal data may be caused by equipment failures, illegal power consumption, or other sudden situations. By comparing the two curves, the differences between abnormal electrical quantity data and normal electrical quantity data can be intuitively seen, providing a clear basis for judgment for the subsequent anomaly detection model.

[0139] Figure 3 The process of anomaly detection in the present invention is shown. When the dynamic distance of a data point is greater than the dynamic threshold, it is determined as abnormal data. The anomaly detection results are used to optimize the model and adjust the threshold. The detected abnormal data will be marked and an alarm message will be generated. Further optimization processes will help improve the accuracy and robustness of the model, thereby enhancing the performance of future anomaly detection.

[0140] Embodiment 2 of the present invention provides a power consumption abnormal behavior detection system based on deep learning, including:

[0141] A data processing module, configured to obtain multi-modal data of the target system and preprocess the obtained data;

[0142] A feature extraction module, which is used to input the preprocessed data into an improved Transformer model for feature extraction;

[0143] A model training module, which is used to perform multi-scale feature fusion on the extracted features, train an anomaly detection model using the features after multi-scale fusion, and update the criterion for the model to determine anomalies in real time based on a dynamic threshold strategy;

[0144] An anomaly detection module, which is used to perform power consumption anomaly detection and feedback optimization of the model by using the trained anomaly detection model.

[0145] Embodiment 3 of the present invention provides a terminal, including a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0146] Embodiment 4 of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method are implemented.

[0147] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0148] 1. Multi-modal data fusion: By fusing multi-modal data from different data sources (such as electrical quantities, environmental data, historical power consumption data, etc.), the present invention can comprehensively explore the potential associations in the data, improve the accuracy and comprehensiveness of anomaly detection. This multi-modal data fusion method overcomes the limitations of traditional single-data-source methods and effectively improves the reliability of the detection results.

[0149] The multi-modal data fusion mechanism of the present invention can fully explore and utilize the information from different data sources. These data are interrelated and jointly reveal the complex patterns of the enterprise's power consumption behavior. Therefore, the model trained based on this can identify the abnormal behaviors that cannot be captured by a single data source, thereby improving the comprehensiveness and accuracy of the detection.

[0150] 2. Adaptive piecewise normalization: By using a learnable scaling factor, the traditional data preprocessing is upgraded to a differentiable calculation layer, making it a part of the model to participate in end-to-end optimization. On the one hand, this design can break through the separation between preprocessing and model training, enabling the normalization strategy to be dynamically adjusted according to the model objective; on the other hand, by normalizing high / medium / low load data respectively, the distribution shift caused by load fluctuations can be eliminated.

[0151] 3. Deep feature extraction: Use a Transformer model based on time-decaying attention and time-aware positional encoding for deep feature extraction. Among them, the time-decaying attention mechanism uses the time difference Δt matrix as the structured prior for attention calculation, achieving physical meaning-driven, conforming to the power equipment fault law that "recent events have a greater impact". The decay coefficient λ is directly related to the development speed of equipment faults (such as λ small → focus on long-term trends), realizing parameter interpretability. λ is automatically learned through gradient descent without manual parameter tuning, achieving end-to-end optimization; the time-aware positional encoding introduces the absolute timestamp t, encoding the equipment operation duration (such as cumulative hours) into the features, reflecting long-term trends such as aging and seasonal loads; the decay coefficient γ controls the decay rate of positional encoding, matching the actual aging speed of the equipment, enhancing the model's sensitivity to abnormal electricity consumption.

[0152] The time-series data processing framework based on Transformer of the present invention can effectively capture the time-series features and dynamic changes in electricity consumption data, and then realize in-depth analysis of different electricity consumption patterns. It can flexibly focus on important time points and patterns in the data, automatically capture complex patterns in the data, and dig out subtle changes in the data, thereby providing a more accurate feature representation for anomaly detection, avoiding the problem of insufficient local information processing in traditional methods, and improving the performance of the model in complex electricity consumption data.

[0153] 4. Compared with directly splicing features of different scales, the present invention superimposes interaction features on the original electrical quantity features. By enhancing the modal features of electrical quantities and splicing them with environmental modal and log modal features, fusion features are obtained, retaining the basic information while strengthening the key modal associations, capable of handling the differences in electricity consumption behaviors in different time periods and regions, and improving the adaptability and generalization ability of the model. When detecting abnormal electricity consumption, it not only has strong real-time performance but also can handle various complex scenarios, such as the processing of large-scale user data and the electricity consumption fluctuations in special time periods such as seasons and holidays.

[0154] 5. Dynamic threshold optimization mechanism: In order to better adapt to the electricity consumption characteristics of different regions, the present invention proposes a dynamic threshold adjustment mechanism based on the equipment health decay factor and the uncertainty adaptation term, which can dynamically adjust the threshold of anomaly detection according to the spatio-temporal distribution of electricity consumption behaviors, enabling the model to adapt to the distribution characteristics of different data, ensuring the accuracy and robustness of anomaly detection. By continuously optimizing the threshold, the model can adapt to the electricity consumption behaviors in different time periods, industries, and regions, flexibly adjust the anomaly detection strategy, and has stronger generalization ability, further improving the accuracy of the detection results.

[0155] In summary, by combining an improved Transformer model, multi-modal data fusion, and a dynamic threshold optimization mechanism, the present invention proposes an intelligent anomaly detection solution that is efficient, accurate, and highly robust. This method can not only enhance the ability to detect abnormal electricity consumption behaviors of enterprises but also be widely applied to the anomaly recognition of various complex data, having important practical significance and application prospects. By combining the improved Transformer model and self-supervised learning technology and leveraging the deep features of multi-modal data, the present invention can more accurately detect potential anomalies without relying on a large amount of labeled data. This method not only improves the accuracy of anomaly detection but also significantly enhances the robustness and adaptability of the model. Experimental results show that the detection method of the present invention performs excellently on multiple real electricity consumption datasets and can accurately identify various types of abnormal electricity consumption behaviors, such as illegal electricity use, equipment failures, and electricity fraud. Compared with traditional methods, the present invention has higher accuracy, recall rate, and stronger generalization ability. This method is widely applicable to fields such as smart grids, electricity consumption monitoring, and energy management, can significantly improve the security and stability of the power system, and promote the intelligent development of the power industry.

[0156] The present disclosure can be a system, method, and / or computer program product. The computer program product can include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0157] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or raised structures in a groove having instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not to be construed as an instantaneous signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0158] The computer-readable program instructions described herein can be downloaded to various computing / processing devices from a computer-readable storage medium or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0159] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: modifications or equivalent replacements can still be made to the specific embodiments of the present invention, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered by the protection scope of the claims of the present invention.

Claims

1. A method for detecting abnormal electricity consumption behavior based on deep learning, characterized in that, Including: Obtain the electrical quantity mode, environmental mode and log mode data of the target system, and perform data cleaning, adaptive segmented standardization, missing value filling and timestamp addition preprocessing; Input the preprocessed modal data into the Transformer model based on time decay attention and time-aware position encoding respectively for deep feature extraction to obtain the features of the corresponding mode; Enhance the electrical quantity mode features and splice them with the environmental mode and log mode features to obtain fused features, use the fused features to train a deep learning-based anomaly detection model, and update the anomaly determination criteria of the anomaly detection model in real time based on the dynamic threshold strategy; Use the trained anomaly detection model to detect electrical anomalies.

2. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 1, wherein: The adaptive segmented standardization is specifically: divide the data into high, medium and low load condition segments according to the load rate, and calculate the standardized values independently for each condition segment: where seg ∈ {high, medium, low} is the operating condition segment; x i，z is the z-th eigenvalue in the i-th data point; x' i，z is the standardized value; μ z，seg is the mean of the z-th feature in the seg-th operating condition segment; σ z，seg is the standard deviation of the z-th feature in the seg-th operating condition segment; β seg is the learnable scaling factor for the seg-th operating condition segment; z ∈ {1, 2,..., m}, where m is the number of features in each data point.

3. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 1, wherein: The input of the preprocessed modal data into the Transformer model based on time decay attention and time-aware position encoding respectively for deep feature extraction to obtain the features of the corresponding mode includes: Input the preprocessed multi-modal data into the Transformer model respectively, and for the preprocessed data perform embedding representation and inject time-aware positional encoding to obtain the data after time-aware positional encoding where is the time-aware positional encoding matrix; Based on perform time decay self-attention layer calculations, and stack the calculations of each time decay self-attention layer to obtain the final output. The calculation formula of the time decay self-attention layer is as follows: Among them, Q, K, and V represent the query, key, and value matrices respectively, which are generated by linear projection; d k is the dimension of the key; is the learnable attenuation coefficient; ⊙ represents the Hadamard product; Δt is the time difference matrix, and the elements of this matrix are Δt uv = |t u - t v | (u, v = 1, 2,..., N), where t u , t v are the timestamps of the u-th and v-th data points, and N is the total number of data points; Softmax(·) is the Softmax function; Attention represents the calculation of the time-decaying self-attention layer.

4. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 3, wherein: The calculation method of the time-aware position encoding matrix PE(t) is: Among them, pos ∈ [1, n] represents the temporal position index of the data point, t is the absolute timestamp of the data point, γ is a learnable time decay coefficient, and r is the dimension index of the positional encoding; n is the number of time points, m is the feature dimension; PE (pos,2r) is the element value of the pos-th row and the 2r-th column in PE(t); PE (pos,2r+1) is the element value of the pos-th row and the (2r + 1)-th column in PE(t).

5. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 1, wherein: The enhancement of the electrical quantity mode features and the splicing with the environmental mode and log mode features to obtain fused features includes: Calculate the modal feature matrices of electrical quantities separately and the modal feature matrix of the environment The modal feature matrix of the log The correlation coefficient matrix of: Among them, s is the dimension of the output features of the Transformer model, n is the number of time points, T represents matrix transpose, and Softmax(·) is the Softmax function; Enhance the electrical quantity mode features according to the correlation coefficient matrix to obtain the enhanced electrical quantity mode feature matrix: Concatenate the enhanced electrical quantity modal feature matrix with the environmental modal feature matrix and the log modal feature matrix to obtain the fusion feature matrix H fusion .

6. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 1, wherein: The training of a deep learning-based anomaly detection model using the fused features includes: Based on the fused feature matrix H fusion , construct a time-modal weighted dynamic distance, and train a deep learning-based anomaly detection model by minimizing the loss function based on the dynamic distance. The time-modal weighted dynamic distance is as follows: Among them, x i represents the fused feature vector of the i-th data point in H fusion ; x j represents the fused feature vector of the j-th data point in H fusion ; H fusion,i,k is the k-th eigenvalue of the i-th data point in H fusion ; H fusion，j，k is the k-th eigenvalue of the j-th data point in H fusion ; is the modal importance weight of the k-th feature dimension; is the time decay factor, λ is the learnable decay coefficient, t i , t j are the timestamps of the i-th and j-th data points, and d is the total number of feature dimensions of H fusion .

7. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 6, wherein: The loss function based on dynamic distance is: Among them, is an indicator function, y i is the label of the i-th data point, y j is the label of the j-th data point, and N represents the total number of data points.

8. A method for detecting abnormal electrical consumption behavior based on deep learning according to claim 1, wherein: The real-time update of the anomaly determination criteria of the anomaly detection model based on the dynamic threshold strategy includes: Iteratively calculate the dynamic threshold according to the device health decay factor and the uncertainty adaptation term. When the dynamic distance of the data point exceeds the dynamic threshold, it is determined as an anomaly, where the dynamic threshold iterative calculation formula is: where T p , T p+1 are the dynamic thresholds for the current and next rounds, α is the uncertainty adjustment coefficient, η is the base learning rate, is the gradient of the loss function, H(p) is the device health index, λ1 is the health decay weight, and σ(p) represents the prediction uncertainty of the anomaly detection model.

9. A power consumption abnormal behavior detection system based on deep learning, for running the method according to any one of claims 1-8, characterized in that, The system includes: A data processing module for obtaining the electrical quantity mode, environmental mode and log mode data of the target system, and performing data cleaning, adaptive segmented standardization, missing value filling and timestamp addition preprocessing; A feature extraction module, which is used to separately input the preprocessed multi-modal data into a Transformer model based on time decay attention and time-aware position encoding for deep feature extraction, so as to obtain features corresponding to each modality; A model training module, which is used to enhance the electrical quantity modality features and splice them with the environment modality and log modality features to obtain fused features, use the fused features to train a deep learning-based anomaly detection model, and update the criteria for the anomaly detection model to determine anomalies in real time based on a dynamic threshold strategy; An anomaly detection module, which is used to perform power consumption anomaly detection by using the trained anomaly detection model.

10. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to execute the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps of the method according to any one of claims 1-8 are implemented.

Images