Double-criterion abnormal electricity utilization detection method based on dimension reduction clustering

Abstract

The invention discloses a double-criterion abnormal electricity consumption detection method based on dimension reduction clustering, and relates to the technical field of electricity consumption detection, and the method comprises the following steps: building a daily and monthly electricity consumption data set of electricity consumption users based on original intelligent electric meter data, constructing feature indexes, and carrying out the normalization to obtain a high-dimensional daily and monthly electricity consumption feature set; performing dimensionality reduction processing on the high-dimensional daily and monthly electricity consumption feature set by using a uniform manifold approximation and projection algorithm to obtain a low-dimensional projection feature set, and setting a double-criterion detection rule based on an unsupervised clustering algorithm to construct an abnormal electricity consumption detection model; and based on the abnormal power consumption detection model, inputting the low-dimensional projection feature set, outputting an abnormal power consumption detection result, and evaluating output by using a model evaluation index. According to the method, data can be better clustered, the parameter selection difficulty is reduced, and accurate optimal clustering center selection is realized, so that the accuracy of an abnormal power utilization detection model is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electricity consumption detection, and in particular to a dual-criteria abnormal electricity consumption detection method based on dimensionality reduction clustering. Background Art

[0002] Non-technical loss (NTL) refers to the part of power transmission and distribution loss that cannot be explained by technology. Abnormal power consumption is the main cause of non-technical loss in power systems, which directly affects the safe and stable operation of power grids. Traditional abnormal power consumption detection adopts manual inspection, which consumes a lot of manpower and material resources and can no longer meet actual needs. In recent years, with the development of smart grids, power companies have deployed a large number of data acquisition sensors such as smart meters at the power generation, transmission, distribution and power consumption ends, which can collect and record relevant data from all aspects of the power system at a frequency of dozens of times a day. These data contain information on the power consumption behavior of different users, and also provide indirect evidence for abnormal power consumption detection. How to effectively mine and utilize this information to quickly and accurately locate abnormal power users is a difficult problem that power companies need to solve urgently and is also a hot topic of current research.

[0003] The power load detection algorithms used in existing abnormal power consumption detection technologies can be divided into supervised and unsupervised categories according to whether data labels are required. Among them, supervised algorithms learn the characteristics of abnormal power users through training sets, thereby obtaining the ability to identify abnormal power users, such as support vector machines and neural networks. These algorithms require a large amount of labeled sample data as a training basis, and power load data often faces the problem of category imbalance, that is, the proportion of abnormal power users is small, which may lead to underfitting during the training process, and ultimately affect the accuracy of the detection results.

[0004] Compared with supervised algorithms, the advantage of unsupervised algorithms is that they do not need to rely on data labels, and because the training process is omitted, the impact of data category imbalance on model performance is relatively small. At present, unsupervised algorithms based on deep learning are usually used as a supplement to supervised detection algorithms to reduce the negative impact of category imbalance data on detection results; however, unsupervised algorithms based on outlier factors are not suitable for detecting user loads with multiple power consumption patterns, while clustering-based unsupervised algorithms can achieve unsupervised detection of abnormal power consumption throughout the entire process, but they also face some challenges in anomaly detection, such as parameter selection usually relies on experience, and unsupervised algorithms are difficult to evaluate the pros and cons of selected parameters due to the lack of abnormal user labels, which makes it more difficult to obtain the optimal parameters.

[0005] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0006] In response to the problems in the related technology, the present invention proposes a dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering, which has an unsupervised clustering algorithm that is improved through optimization, and can automatically and accurately select cluster centers, avoiding reliance on manual experience and achieving the advantages of unsupervised clustering throughout the entire process. This solves the problem in the prior art that supervised algorithms may suffer from underfitting during the training of power load data, ultimately affecting the accuracy of the detection results, while unsupervised algorithms lack abnormal user labels, making it difficult to evaluate the pros and cons of the selected parameters and obtaining the optimal parameters.

[0007] To this end, the specific technical solution adopted by the present invention is as follows:

[0008] A dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering, the dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering comprises the following steps:

[0009] S1. Based on the original smart meter data, a daily and monthly electricity consumption dataset of electricity users is established, and characteristic indicators are constructed and normalized to obtain a high-dimensional daily and monthly electricity consumption feature set;

[0010] S2. Use uniform manifold approximation and projection algorithm to reduce the dimension of high-dimensional daily and monthly electricity consumption feature set to obtain a low-dimensional projection feature set, and set dual-criteria detection rules based on unsupervised clustering algorithm to build an abnormal electricity consumption detection model;

[0011] S3. Based on the abnormal power consumption detection model, input the low-dimensional projection feature set, output the abnormal power consumption detection result, and use the model evaluation index to evaluate the output.

[0012] Furthermore, based on the original smart meter data, a daily and monthly electricity consumption data set of electricity users is established, and characteristic indicators are constructed. Normalization to obtain a high-dimensional daily and monthly electricity consumption feature set includes the following steps:

[0013] S11, preprocessing the original smart meter data to generate daily and monthly electricity consumption data sets of electricity users;

[0014] S12. construct characteristic indicators based on the daily and monthly electricity consumption dataset of electricity users;

[0015] S13. Normalize the characteristic indicators to obtain a high-dimensional daily and monthly electricity consumption feature set.

[0016] Furthermore, preprocessing the original smart meter data to generate a daily and monthly electricity consumption data set of electricity users includes the following steps:

[0017] S111. According to the time period information, the power consumption of each user in the original smart meter data is accumulated on a daily basis to obtain daily power consumption data;

[0018] S112, identifying a data point in the daily power consumption data where the power consumption is 0, and replacing the data point with a preset value to generate a user daily power consumption data set;

[0019] S113, based on the user's daily electricity consumption data set, extract and accumulate the electricity consumption data within a preset time span to generate the user's monthly electricity consumption data set, and establish the user's daily and monthly electricity consumption data set in combination with the user's daily electricity consumption data set.

[0020] Furthermore, characteristic indicators include morphological indicators, volatility indicators, trend indicators and correlation indicators.

[0021] Furthermore, the uniform manifold approximation and projection algorithm are used to reduce the dimension of the high-dimensional daily and monthly electricity consumption feature set to obtain a low-dimensional projection feature set. Based on the unsupervised clustering algorithm, a dual-criteria detection rule is set to construct an abnormal electricity consumption detection model, which includes the following steps:

[0022] S21. Based on the high-dimensional daily and monthly electricity consumption feature set, construct a high-dimensional topological weight, and perform low-dimensional projection to generate a low-dimensional projection feature set;

[0023] S22. Construct an exponential density distance evaluation index, introduce a silhouette coefficient, and formulate a multi-stage clustering strategy, wherein the multi-stage clustering strategy includes a pre-clustering stage strategy and an optimization stage strategy;

[0024] S23. According to the δ-ρ decision diagram and the actual coordinates of the data points, a dual-criteria detection rule is established, and combined with a multi-stage clustering strategy, an abnormal power consumption detection model is constructed, wherein the dual-criteria detection rule includes a first abnormal point judgment rule and a second abnormal point judgment rule.

[0025] Furthermore, based on the high-dimensional daily and monthly electricity consumption feature set, a high-dimensional topological weight is constructed, and a low-dimensional projection is performed. The generation of a low-dimensional projection feature set includes the following steps:

[0026] S211. Construct high-dimensional topological weights based on data points and introduce a weighted adjacency matrix to ensure the consistency of weights between any two data points.

[0027] S212. Construct a cross entropy function and optimize the cross entropy loss;

[0028] S213. Use the stochastic gradient descent method to optimize the cross entropy function, obtain the weights of all low-dimensional data points, and establish a low-dimensional projection feature set.

[0029] Furthermore, an exponential density distance evaluation index is constructed, and the silhouette coefficient is introduced. The multi-stage clustering strategy is formulated, including the following steps:

[0030] S221, extracting preliminary clustering of the data set and setting a dynamic range for pre-clustering;

[0031] S222, for each pre-clustering result within the dynamic range, calculate the corresponding silhouette coefficient and identify the optimal number of clusters;

[0032] S223. Construct an exponential density distance evaluation index to identify potential cluster centers.

[0033] Furthermore, according to the δ-ρ decision diagram and the actual coordinates of the data points, a dual-criteria detection rule is established, and combined with a multi-stage clustering strategy, the abnormal power consumption detection model is constructed, which includes the following steps:

[0034] S231, calculating the reference density and reference distance of each cluster based on the δ-ρ decision diagram;

[0035] S232, if the data point is located at the upper left of the δ-ρ decision diagram, it is determined that the first abnormal point criterion rule is satisfied;

[0036] S233, defining the boundary points of each cluster, and calculating the boundary density of each cluster;

[0037] S234, if the density of the data point is less than the boundary density of the cluster to which it belongs, it is determined that the second outlier criterion rule is met;

[0038] S235. Combine the multi-stage clustering strategy to build an abnormal power consumption detection model, and mark the data points that meet both the first abnormal point judgment rule and the second type of abnormal point judgment rule as abnormal points.

[0039] Furthermore, defining the boundary points of each cluster and calculating the boundary density of each cluster includes the following steps:

[0040] S2331, define the boundary points of each cluster, and set the initial value of the boundary density of all clusters to 0;

[0041] S2332, taking the average density of the two points that meet the boundary point condition as the boundary density of the cluster to which the two points belong;

[0042] S2333, repeat step S2332 until the boundary densities of all clusters are obtained;

[0043] S2334. Select the maximum boundary density in each cluster as the boundary density of the cluster.

[0044] Furthermore, based on the abnormal power consumption detection model, inputting the low-dimensional projection feature set, outputting the abnormal power consumption detection result, and evaluating the output using the model evaluation index include the following steps:

[0045] S31, inputting the low-dimensional projection feature set into the abnormal power consumption detection model to obtain abnormal power consumption detection results;

[0046] S32. Based on the abnormal power consumption detection results, construct a power load abnormality detection confusion matrix;

[0047] S33. Based on the confusion matrix of power load anomaly detection, the recall rate, precision rate, true positive rate and false positive rate are calculated respectively to evaluate the detection effect of the abnormal power consumption detection model.

[0048] The beneficial effects of the present invention are:

[0049] (1) The present invention introduces uniform manifold approximation and projection algorithm to perform dimensionality reduction processing on the massive data of electricity users, which significantly improves the processing speed. At the same time, it can better preserve the local structure and global structure of the original data, which is conducive to better clustering of the data, thereby improving the accuracy of the abnormal electricity consumption detection model.

[0050] (2) The present invention makes innovative improvements to the traditional unsupervised clustering algorithm. The selection of cluster centers in the traditional algorithm requires drawing a decision diagram and manually selecting cluster centers, which requires relying on human experience. It faces the problem of being unable to accurately select the optimal cluster center and is unable to achieve unsupervised selection throughout the entire process. Based on the shortcomings of the traditional unsupervised clustering algorithm, the present invention optimizes and improves the unsupervised clustering algorithm to achieve accurate selection of the optimal cluster center, improve the clustering effect, and thereby improve the performance of the abnormal power consumption detection model.

[0051] (3) The present invention reduces the difficulty of parameter selection. The existing unsupervised learning anomaly detection technology is difficult to select model parameters because abnormal electricity users are unknown in advance and there are no actual abnormal user labels. In addition, the selection of parameters will also be different for different types of electricity users. The present invention sets a dual criterion for abnormal power consumption detection of power loads, and also improves the comprehensive performance of the anomaly detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 is a flow chart of a dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering according to an embodiment of the present invention;

[0054] Figure 2 is a specific implementation diagram of a dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering according to an embodiment of the present invention;

[0055] Figure 3is a PCA low-dimensional feature space distribution diagram according to an embodiment of the present invention;

[0056] Figure 4 is an LDA low-dimensional feature space distribution diagram according to an embodiment of the present invention;

[0057] Figure 5 is a UMAP low-dimensional feature space distribution map according to an embodiment of the present invention;

[0058] Figure 6 is a curve diagram of daily electricity consumption of users in the cluster center of the PCA dimensionality reduction model according to an embodiment of the present invention;

[0059] Figure 7 is a curve diagram of daily electricity consumption of users in the cluster center of the LDA dimensionality reduction model according to an embodiment of the present invention;

[0060] Figure 8 is a curve diagram of daily electricity consumption of users in the cluster center of the UMAP dimensionality reduction model according to an embodiment of the present invention;

[0061] Fig. 9 is a comparison diagram between the detection results of the LDA model according to an embodiment of the present invention and actual abnormal users;

[0062] Fig.10 is a comparison diagram between the detection results of the PCA model according to an embodiment of the present invention and actual abnormal users;

[0063] Fig.11 is a comparison diagram between the detection results of the UMAP model according to an embodiment of the present invention and actual abnormal users;

[0064] Fig.12 is a diagram of abnormal detection results of the first type of users according to an embodiment of the present invention;

[0065] Fig.13 is a diagram of abnormal detection results of the second type of users according to an embodiment of the present invention;

[0066] Fig.14 is a diagram of abnormal detection results of the third type of users according to an embodiment of the present invention;

[0067] Fig.15 is an influence diagram of parameter α on criterion 1 and the AUC of the dual criterion model according to an embodiment of the present invention;

[0068] Fig.16 is an influence diagram of parameter β on the AUC of criterion 1 and the dual criterion model according to an embodiment of the present invention;

[0069] Fig.17 3 is an influence diagram of the parameter ω on the AUC of the criterion 2 and the dual criterion model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0070] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0071] According to an embodiment of the present invention, a dual-criteria abnormal electricity consumption detection method based on dimensionality reduction clustering is provided.

[0072] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1 As shown, according to one embodiment of the present invention, a dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering is provided, and the dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering includes the following steps:

[0073] S1. Based on the original smart meter data, a daily and monthly electricity consumption dataset of electricity users is established, and characteristic indicators are constructed and normalized to obtain a high-dimensional daily and monthly electricity consumption feature set;

[0074] S2. Use uniform manifold approximation and projection algorithm to reduce the dimension of high-dimensional daily and monthly electricity consumption feature set to obtain a low-dimensional projection feature set, and set dual-criteria detection rules based on unsupervised clustering algorithm to build an abnormal electricity consumption detection model;

[0075] S3. Based on the abnormal power consumption detection model, input the low-dimensional projection feature set, output the abnormal power consumption detection result, and use the model evaluation index to evaluate the output.

[0076] In one embodiment, based on the original smart meter data, a daily and monthly electricity consumption data set of electricity users is established, and characteristic indicators are constructed. Normalization to obtain a high-dimensional daily and monthly electricity consumption feature set includes the following steps:

[0077] S11, preprocessing the original smart meter data to generate daily and monthly electricity consumption data sets of electricity users;

[0078] Specifically, the power load is preprocessed. In this embodiment, the original smart meter data released by the Irish Energy Regulatory Commission is called. The original smart meter data contains about 160 million rows of user power consumption information and time period data, which cannot be directly input into the model. Therefore, according to the time period information, the power consumption of each user is accumulated on a daily basis to obtain the daily power consumption data of 6445 power users for 536 days. The data set contains data points with a full day of zero power consumption. Preliminary analysis shows that this is normal in practice, but these data points will cause some Some characteristic values ​​cannot be obtained; therefore, the present invention modifies the data points with a daily electricity consumption of 0 to 0.01, using a very small value to represent the situation where the user has no electricity consumption in a day, and generates processed daily electricity consumption data of 6445 electricity users. In order for the subsequent construction to reflect the electricity consumption characteristics under a long time span, on the basis of the user's daily electricity consumption data set, the daily electricity consumption data of every 30 days are accumulated to obtain the monthly electricity consumption data of all users for 18 months; this step obtains 536 days of daily electricity consumption data and 18 months of electricity consumption data of 6445 Irish electricity users.

[0079] S12. construct characteristic indicators based on the daily and monthly electricity consumption dataset of electricity users;

[0080] Specifically, feature construction. In this embodiment, based on the daily and monthly electricity consumption data of 6445 electricity users generated in step S11, feature construction is performed on the daily and monthly electricity consumption data. Feature construction can mine the deep information of the original load data and improve the accuracy of the anomaly detection model. The present invention constructs four types of feature indicators based on the daily and monthly electricity consumption data of users, namely, morphology, volatility, trend and correlation: ① morphological indicators, including (1) daily average electricity consumption, (2) monthly average electricity consumption, (3) daily electricity consumption rate, (4) monthly electricity consumption rate, that is, the ratio of average electricity consumption to maximum electricity consumption, (5) monthly electricity consumption peak-to-valley difference rate, that is, the ratio of the difference between the maximum and minimum electricity consumption to the maximum electricity consumption, (6) the proportion of electricity consumption in the first quarter to the annual electricity consumption, (7) the proportion of electricity consumption in the second quarter to the annual electricity consumption, (8) the proportion of electricity consumption in the third quarter to the annual electricity consumption, and (9) the proportion of electricity consumption in the fourth quarter to the annual electricity consumption. The proportion of electricity consumption, where the daily and monthly electricity consumption rates reflect the overall change in electricity consumption, the peak-to-valley difference rate reflects the magnitude of the change in electricity consumption, and the quarterly electricity consumption ratio reflects the distribution of electricity consumption; ② Volatility indicators, including (10) daily electricity consumption dispersion coefficient, (11) monthly electricity consumption dispersion coefficient, that is, the ratio of the daily and monthly electricity consumption standard deviation to the daily and monthly electricity consumption mean, (12) the ratio of the daily electricity consumption dispersion coefficient to the industry daily electricity consumption dispersion coefficient, (13) the ratio of the monthly electricity consumption dispersion coefficient to the industry monthly electricity consumption dispersion coefficient (in the present invention, the average value of all users' electricity consumption represents the industry's electricity consumption), (14) the difference between the beginning and the end of the electricity consumption in the previous m months, where the dispersion coefficient of each household's electricity consumption represents the degree of deviation of the user's electricity consumption curve from the average value curve, which can reflect the fluctuation of the user's electricity consumption in more detail. Some daily data with very high electricity consumption will have a greater impact on the dispersion coefficient. The difference between the beginning and the end of the electricity consumption data is based on the monthly electricity consumption data with a long span, which can reflect the overall fluctuation of the user's electricity consumption; ③ trend indicators, including (15) the slope k of the linear fitting of the daily electricity consumption series, (16) the upward trend of the monthly electricity consumption series, and (17) the downward trend of the monthly electricity consumption series; ④ correlation indicators, including (18) the Pearson correlation coefficient between the daily electricity consumption series of each household and the typical daily electricity consumption series (represented by the daily average series of all users in this invention); according to the above method, 18 feature values ​​are extracted.

[0081] S13. Normalize the characteristic indicators to obtain a high-dimensional daily and monthly electricity consumption feature set.

[0082] Specifically, normalization processing, since the magnitudes of the 18 features in step S12 are not the same, in order to balance the impact of each feature on the result, the above features need to be normalized.

[0083] Specifically, the normalization method used in this embodiment is min-max normalization, also known as deviation normalization, and the calculation formula of the normalized processing characteristic index is:

[0084]

[0085] In the formula, x i , X i are the values ​​of a feature of the i-th data point before and after normalization, x min 、x max are the minimum and maximum values ​​of this feature respectively.

[0086] In one embodiment, preprocessing the original smart meter data to generate a daily and monthly electricity consumption data set of electricity users includes the following steps:

[0087] S111. According to the time period information, the power consumption of each user in the original smart meter data is accumulated on a daily basis to obtain daily power consumption data;

[0088] S112, identifying a data point in the daily power consumption data where the power consumption is 0, and replacing the data point with a preset value to generate a user daily power consumption data set;

[0089] S113, based on the user's daily electricity consumption data set, extract and accumulate the electricity consumption data within a preset time span to generate the user's monthly electricity consumption data set, and establish the user's daily and monthly electricity consumption data set in combination with the user's daily electricity consumption data set.

[0090] In one embodiment, the characteristic indicators include a morphological indicator, a volatility indicator, a trend indicator, and a correlation indicator;

[0091] Among them, the morphological indicators include daily and monthly average electricity consumption, daily and monthly electricity consumption rate, monthly electricity consumption peak-to-valley difference rate, and the proportion of electricity consumption in each quarter to the annual electricity consumption;

[0092] The volatility indicators include the daily and monthly electricity consumption dispersion coefficient, the ratio of the daily and monthly electricity consumption dispersion coefficient to the daily and monthly electricity consumption dispersion coefficient of the industry, and the difference between the beginning and the end of electricity consumption;

[0093] The trend indicators include the slope of the linear fitting of the daily power consumption series, the rising trend of the monthly power consumption series, and the falling trend of the monthly power consumption series;

[0094] The correlation indicators include the Pearson correlation coefficient of the daily electricity consumption series of each household and the Pearson correlation coefficient of the typical daily electricity consumption series.

[0095] In one embodiment, the high-dimensional daily and monthly electricity consumption feature set is processed by dimensionality reduction using uniform manifold approximation and projection algorithm to obtain a low-dimensional projection feature set, and based on an unsupervised clustering algorithm, a dual-criteria detection rule is set to construct an abnormal electricity consumption detection model, including the following steps:

[0096] S21. Based on the high-dimensional daily and monthly electricity consumption feature set, construct a high-dimensional topological weight, and perform low-dimensional projection to generate a low-dimensional projection feature set;

[0097] S22. Construct an exponential density distance evaluation index, introduce a silhouette coefficient, and formulate a multi-stage clustering strategy, wherein the multi-stage clustering strategy includes a pre-clustering stage strategy and an optimization stage strategy;

[0098] S23. According to the δ-ρ decision diagram and the actual coordinates of the data points, a dual-criteria detection rule is established, and combined with a multi-stage clustering strategy, an abnormal power consumption detection model is constructed, wherein the dual-criteria detection rule includes a first abnormal point judgment rule and a second abnormal point judgment rule.

[0099] In one embodiment, based on the high-dimensional daily and monthly electricity consumption feature set, a high-dimensional topological weight is constructed, and a low-dimensional projection is performed to generate a low-dimensional projection feature set, including the following steps:

[0100] S211. Construct high-dimensional topological weights based on data points and introduce a weighted adjacency matrix to ensure the consistency of weights between any two data points.

[0101] S212. Construct a cross entropy function and optimize the cross entropy loss;

[0102] S213. Use the stochastic gradient descent method to optimize the cross entropy function, obtain the weights of all low-dimensional data points, and establish a low-dimensional projection feature set.

[0103] Specifically, continuing with step S13, in the feature dimensionality reduction part, the present invention selects the Uniform Manifold Approximation and Projection (UMAP) algorithm, which has a faster calculation speed and can better preserve the local structure and global structure of the original data, and can better improve the accuracy of abnormal electricity users.

[0104] Specifically, the present invention uses uniform manifold approximation and projection algorithm to reduce the dimension of high-dimensional daily and monthly electricity consumption feature set to obtain a low-dimensional projection feature set. The implementation process mainly includes two stages: high-dimensional topological construction and low-dimensional projection. In the high-dimensional topological construction stage, the core content is to create a weighted k-nearest neighbor graph, where V is a vertex set, that is, a set of N high-dimensional data points X = {x1, x2, ..., x N}, E is the edge set, that is, the set of directed edges that can be formed according to k adjacent points, W is the weight function, and its calculation expression is:

[0105]

[0106] Where W ij For point x i With xj The weight function in the high-dimensional space, s ij is x i The distance between x j and x; ρ i is x i The distance between x and the neighboring points (derived from the local connectivity constraint); σ i is the smoothing normalization factor set according to the Riemannian metric.

[0107] Specifically, after calculating W for all data points, in order to maintain the weight consistency between any two data points, the expression is introduced: ij In the formula, A is the weighted adjacency matrix composed of all W

[0108]

[0109] In the formula, A is the weighted adjacency matrix composed of all W ij ; is the Hadamard product of the matrices; B is the weighted adjacency matrix of the final graph G.

[0110] Specifically, after completing the construction of the high-dimensional topology, in order to achieve the low-dimensional projection, it is necessary to construct the weight function in the low dimension, and the expression is:

[0111]

[0112] In the formula, V ij is the weight function of the point x i and x j in the low-dimensional space, d ij is the distance between y N and y i in the low-dimensional data points Y = {y1, y2,..., y j}; a and b are both hyperparameters.

[0113] Specifically, in order to make the reduced dataset as close as possible to the original dataset, it is necessary to optimize the cross-entropy loss between V ij and W ij .

[0114] Specifically, construct the cross-entropy function, and the expression for optimizing the cross-entropy loss is:

[0115]

[0116] In the formula, represents the attractive component that constrains similar data points to form clusters, represents the repulsive component that ensures sufficient spacing between the formed clusters.

[0117] Specifically, in the cross entropy function, the first term in the sum is the attractive component that constrains similar data points to form clusters, and the second term is the repulsive component that ensures that there is sufficient spacing between the formed clusters; in this embodiment, the function f is optimized using the stochastic gradient descent method, and after obtaining the weights of all low-dimensional data points, the construction of the weighted k-nearest neighbor graph in low dimension is completed, and finally the low-dimensional representation of the high-dimensional topological structure is achieved.

[0118] Specifically, in the above embodiment, the present invention reduces 18 features to 3 features through uniform manifold approximation and projection algorithm.

[0119] In one embodiment, an exponential density distance evaluation index is constructed, and a silhouette coefficient is introduced to formulate a multi-stage clustering strategy, including the following steps:

[0120] S221, extracting preliminary clustering of the data set and setting a dynamic range for pre-clustering;

[0121] S222, for each pre-clustering result within the dynamic range, calculate the corresponding silhouette coefficient and identify the optimal number of clusters;

[0122] S223. Construct an exponential density distance evaluation index to identify potential cluster centers.

[0123] It should be noted that the unsupervised clustering algorithm is an important algorithm used in machine learning to explore the intrinsic structure of data, including K-Means clustering algorithm, hierarchical clustering algorithm and density peak clustering algorithm. The present invention improves the density peak clustering algorithm (DPC), clusters the low-dimensional projection feature set obtained in step S213, and divides the low-dimensional projection feature set into several clusters (different clusters represent different types of electricity users). The same type of electricity users have similar electricity consumption patterns. More refined anomaly detection is performed in each type of electricity user, which can eliminate mutual interference between users of different categories to a certain extent; there are mainly two types of algorithms used for abnormal electricity consumption behavior detection: partition-based and density-based. The partition-based clustering divides the user's feature set and then takes the outliers in the subset that are farther away from the center as abnormal points. The density-based clustering considers those points that are far away from high-density points and are in low-density areas as abnormal points.

[0124] Specifically, the traditional density peak clustering algorithm selects the cluster center by manually selecting the local density ρ in the decision graph. i and the nearest neighbor distance δ iIn order to improve the automation level and efficiency of the density peak clustering (DPC) algorithm and overcome its dependence on manually selected cluster centers, the present invention adopts an innovative multi-stage clustering strategy, which makes use of the efficient initial clustering performance of the K-means algorithm and combines the silhouette coefficient as a quantitative evaluation indicator of clustering effectiveness to automatically determine the optimal number of clusters.

[0125] Specifically, 1) pre-clustering stage: using the K-means algorithm as a preliminary clustering tool, the preliminary clustering of the data set is quickly obtained due to its fast convergence characteristics. By setting a dynamic range (in this embodiment, the k value is from 1 to 10), the present invention performs K-means clustering for each preset k value and records the corresponding results.

[0126] 2) Optimization stage: The silhouette coefficient is introduced as an important indicator to measure the quality of clustering. For each clustering result generated by the k value, the present invention calculates the corresponding silhouette coefficient and identifies the k value with the best silhouette coefficient value. This k value reflects the optimal number of clusters for clustering the data set.

[0127] Specifically, the corresponding silhouette coefficients are calculated for each k (1-10) value clustered in the previous stage of pre-clustering, and the k value with the largest silhouette coefficient is identified. This k value is selected as the optimal number of clusters for clustering the data set.

[0128] It should be noted that the silhouette coefficient is a way to evaluate the clustering effect. In the present invention, the silhouette coefficient is used as an indicator to describe the outline clarity of each category after clustering. It specifically includes two factors, cohesion and separation. Among them, cohesion can be understood as reflecting the closeness between a sample point and the elements within the class, and separation can be understood as reflecting the closeness between a sample point and the elements outside the class.

[0129] Specifically, for one of the sample points i, calculate a(i) = average (the distance from point i to all other points in the cluster to which it belongs), calculate b(i) = min (the average distance from point i to all points in a cluster that does not contain it), and then obtain the calculation formula of the silhouette coefficient of sample point i as follows:

[0130]

[0131] Where a(i) represents the average value of the dissimilarity between vector i and other points in the same cluster, and b(i) represents the minimum value of the average dissimilarity between vector i and other clusters.

[0132] Specifically, from the above formula, it can be obtained that the value of the silhouette coefficient is between [-1, 1]. The closer it is to 1, the better the cohesion and separation are. The silhouette coefficients of all points are averaged to obtain the total silhouette coefficient of the clustering result.

[0133] Specifically, a total of 10 silhouette coefficient values ​​are calculated through the above steps, and the k value corresponding to the largest silhouette coefficient is selected. The k value at this time is the optimal number of clusters.

[0134] 3) DPC clustering application stage: The optimal number of clusters obtained in the previous stage is directly applied to the DPC algorithm. This step enables the DPC algorithm to adaptively determine cluster centers and perform efficient data set clustering analysis without human intervention.

[0135] Specifically, in order to overcome the problem of inaccurate cluster center selection caused by manual selection and over-reliance on a single variable in traditional methods, the present invention also introduces an innovative decision function, which achieves more accurate identification of potential cluster center points by constructing an exponential density distance evaluation index. The expression of the exponential density distance evaluation index is:

[0136]

[0137] In the formula, γ(i) is the exponential density distance evaluation index of each sample point, ρ i and δ i are the local density and relative distance of each sample point respectively.

[0138] Specifically, in the above embodiment, firstly, by calculating ρ for each point in the data set i and δ i Then, the product of these two variables is used as the input of the exponential function to calculate γ(i) for each sample point. Finally, the cluster center is determined by arranging the evaluation indicators in descending order and following the principle of maximizing the indicators. This method ensures that the selected cluster centers are not only points with high density, but also points with relatively long distances in the data space, thereby reducing the influence of a single variable on the cluster center selection process. The new indicator provides a comprehensive score for each data point, taking into account both the local density of the point and its relative distance to other points.

[0139] Specifically, to further verify that the improved DPC clustering algorithm is superior to the traditional DPC clustering algorithm, the present invention reduces the data set to 2, 3 and 4 dimensions to verify the effectiveness of the improved clustering method for different data sets, and calculates the silhouette coefficient and DB index of the traditional DPC clustering algorithm and the improved clustering algorithm, respectively. As shown in Table 1, the DB index after clustering data of different dimensions is compared. Under 4-dimensional data, except that the improved DPC clustering algorithm is slightly higher than the traditional DPC clustering algorithm, the others are far lower, proving that the improved algorithm has a larger inter-cluster distance and a better clustering effect.

[0140] Table 1

[0141]

[0142]

[0143] In one embodiment, according to the δ-ρ decision diagram and the actual coordinates of the data points, a dual-criteria detection rule is established, and combined with a multi-stage clustering strategy, the abnormal power consumption detection model is constructed, including the following steps:

[0144] S231, calculating the reference density and reference distance of each cluster based on the δ-ρ decision diagram;

[0145] S232, if the data point is located at the upper left of the δ-ρ decision diagram, it is determined that the first abnormal point criterion rule is satisfied;

[0146] S233, defining the boundary points of each cluster, and calculating the boundary density of each cluster;

[0147] S234, if the density of the data point is less than the boundary density of the cluster to which it belongs, it is determined that the second outlier criterion rule is met;

[0148] S235. Combine the multi-stage clustering strategy to build an abnormal power consumption detection model, and mark the data points that meet both the first abnormal point judgment rule and the second type of abnormal point judgment rule as abnormal points.

[0149] Specifically, a dual-criteria anomaly detection model is constructed. According to the recognition principle of outliers in the DPC clustering algorithm, two criteria for identifying outliers are set based on the δ-ρ decision diagram and the actual coordinates of the data points. The point located at the upper left of the δ-ρ decision diagram is an outlier. Thus, the criterion for the first outlier point is given, that is, criterion 1: if the density ρ of a data point is i Less than the reference density ρ0 and distance δ i If it is greater than the reference distance δ0, then this point is an outlier. The calculation expressions of the reference density and reference distance of each cluster are:

[0150]

[0151] Where: ρ0(c) With δ0 (c) are the base density and base distance of the cth cluster respectively; α and β are empirical constants; N (c) is the total number of data points contained in the cth cluster; M (c) is the set of data points contained in the cth cluster.

[0152] Specifically, the farther the actual coordinates are from the cluster center, the more likely they are outliers. This gives the second outlier criterion, namely, criterion 2: If the density ρ of a data point is i Less than the boundary density ρ of the cluster to which this point belongs b , then this point is an outlier.

[0153] It should be noted that in DPC (Density Peaks Clustering) clustering, the identification of cluster centers is mainly based on the following two core assumptions: first, the preset cluster center points often show higher local density than their surrounding data points; second, the cluster center points are often farther away from those sample points with greater density than them.

[0154] Specifically, the density ρ in criterion 2 i is the value calculated in the clustering process. Suppose the data set X = {x1, x2, …, x n}, x i ={x i1 ,x i2 ,…,x im},i=1,2,…,n,x ij represents the j-th dimension attribute of the i-th sample, j = 1, 2, ..., m. In order to identify the cluster center, for each sample point x i , and calculate their local density ρ i And the distance δ from the sample point to the sample point with higher density i .

[0155] Specifically, the local density ρ of each sample point is calculated in criterion 2 i There are two methods, calculating local density based on truncated kernel and calculating local density based on Gaussian kernel. The expression for calculating local density based on truncated kernel is:

[0156]

[0157] The expression for calculating local density based on Gaussian kernel is:

[0158]

[0159] In the formula, χ(·) represents a piecewise function, which is used to determine the distance d ij Is it less than the cutoff distance? ij Represents point xi With x j The distance between them; n represents the number of sample points; d c Represents the cutoff distance.

[0160] Specifically, in this embodiment, the local density is calculated based on the Gaussian kernel. The above two methods, whether based on the truncated kernel or the Gaussian kernel, calculate the local density ρ i The calculation process is consistent with the preset cutoff distance d c Closely related, c The calculation formula is:

[0161]

[0162] Where, d c ∈D, D={d1,d2,…,d Nd} is the distance set between two sample points, and d1≤d2≤…≤d Nd , is the total number of distances in D, P is generally taken as around 2%.

[0163] Specifically, for a point xi, the calculation expression of the distance δi between it and a sample point with a higher density than itself is:

[0164]

[0165] In the formula, for the sample point x that is not the largest in local density i , δ i Represents point x i The shortest distance to a sample point with a density greater than its own; if point x i is the current maximum density point, then δ i Defined as point x i The distance to the farthest sample point from itself.

[0166] In one embodiment, defining the boundary points of each cluster and calculating the boundary density of each cluster includes the following steps:

[0167] S2331, define the boundary points of each cluster, and set the initial value of the boundary density of all clusters to 0;

[0168] S2332, taking the average density of the two points that meet the boundary point condition as the boundary density of the cluster to which the two points belong;

[0169] S2333, repeat step S2332 until the boundary densities of all clusters are obtained;

[0170] S2334. Select the maximum boundary density in each cluster as the boundary density of the cluster.

[0171] Specifically, the present invention defines d c The two points are the boundary points of these two clusters (there are many groups of boundary points), and the boundary density ρ of each cluster can be obtained from the boundary points b (c) (superscript c represents the cluster number), the process of obtaining is as follows:

[0172] Step I: Let the boundary density of all clusters be b (c) The initial value of is 0;

[0173] Step II: The average density of the two points that meet the boundary point conditions is taken as the boundary density of the cluster to which the two points belong;

[0174] Step III, repeat step II to obtain multiple boundary densities of all clusters;

[0175] Step IV: Take the maximum boundary density in each cluster as the final ρ of this cluster b (c) .

[0176] Specifically, parameters α and β determine the strictness of criterion 1. The smaller α is and the larger β is, the fewer abnormal values ​​the model detects based on criterion 1. Parameter ω determines the strictness of criterion 2. The smaller ω is, the fewer abnormal values ​​the model detects based on criterion 2. The abnormal power consumption detection model constructed based on the dual-criteria detection rule will determine the points that simultaneously meet the above two single criteria as abnormal points.

[0177] Specifically, the abnormal power consumption detection model of the present invention consists of four parts: "feature construction - dimension reduction - clustering - abnormal detection". In the cluster analysis stage, DPC clustering is used to calculate the density ρ corresponding to each household. i and distance δ i At the same time, users are clustered into several categories according to their electricity consumption characteristics.

[0178] Specifically, in the abnormality detection stage, a dual-criteria detection rule is introduced as a detection standard for the abnormal power consumption detection model to detect abnormal power users.

[0179] Specifically, criterion 1, with the help of the density ρ corresponding to each household in the previous step clustering i and distance δ i , the base density ρ0 of each cluster is calculated by the calculation formula of criterion 1 (c) and the reference distance δ0 (c) , α and β are empirical parameters and need to be manually selected; traverse each electricity user, when there is a density ρ corresponding to a certain user i Less than the reference density ρ0 (c) , distance δ iGreater than the reference distance δ0 (c) , then criterion 1 identifies the user as an abnormal user and saves the number corresponding to the user.

[0180] Specifically, criterion 2 also uses the density ρ calculated in the previous clustering step i and distance δ i , the boundary density ρ of each cluster is calculated by the calculation formula of criterion 2 b (c) , ω is an empirical parameter that needs to be manually selected; traverse each electricity user, when there is a density ρ corresponding to a household itself i is smaller than the boundary density ρ of the cluster to which it belongs b (c) , then criterion 2 identifies household 1 as an abnormal user and saves the number corresponding to the household.

[0181] The abnormal electricity users detected by the abnormal electricity consumption detection model refer to users detected as abnormal by both judgment 1 and judgment 2.

[0182] In one embodiment, based on the abnormal power consumption detection model, inputting a low-dimensional projection feature set, outputting abnormal power consumption detection results, and evaluating the output using a model evaluation index include the following steps:

[0183] S31, inputting the low-dimensional projection feature set into the abnormal power consumption detection model to obtain abnormal power consumption detection results;

[0184] S32. Based on the abnormal power consumption detection results, construct a power load abnormality detection confusion matrix;

[0185] S33. Based on the confusion matrix of power load anomaly detection, the recall rate, precision rate, true positive rate and false positive rate are calculated respectively to evaluate the detection effect of the abnormal power consumption detection model.

[0186] Specifically, the effect of the abnormal power consumption detection model is evaluated based on the abnormal power consumption detection result output by the model. The abnormal detection of power load is a binary classification problem with unbalanced categories, and an evaluation index based on accuracy cannot be used. The quality of the abnormal power consumption detection model in the present invention is evaluated by the area under the receiver operating characteristic (ROC) curve (AUC), which is a comprehensive indicator that can simultaneously reflect the detection rate and precision of the classification model. In this embodiment, in order to obtain the AUC, the confusion matrix of the binary classifier is first obtained.

[0187] Specifically, the confusion matrix contains all possible classification results of a binary classifier, as shown in Table 2.

[0188] Table 2 Confusion matrix for power load anomaly detection

[0189]

[0190] In Table 2, the letters T and F represent the correct and incorrect classification results of the classifier, respectively; the letters P and N represent the classifier's predictions as abnormal and normal, respectively; TP and TN represent two correct classification results, and FP and FN represent two incorrect classification results.

[0191] Specifically, based on the confusion matrix, the recall rate R and precision rate P of the abnormal power consumption detection model are calculated. The calculation expression of the recall rate R is:

[0192]

[0193] In the formula, A TP A represents the number of users predicted by the classifier as abnormal and actually abnormal; FN It indicates the number of users predicted by the classifier as normal but actually abnormal; R indicates the ratio of correctly detected abnormal data to the total number of abnormal data, that is, the detection rate. The larger the recall rate R, the better the classifier performance.

[0194] The calculation expression of precision P is:

[0195]

[0196] In the formula, A FP It indicates the number of users predicted by the classifier as abnormal but actually normal; P indicates the ratio of the number of correctly detected abnormal data to the total number of detected abnormal data. The larger P is, the lower the false positive rate is and the better the classifier performance is.

[0197] Specifically, the true positive rate (TPR) and false positive rate (FPR) of the classifier can be calculated according to the confusion matrix, which can reflect the detection rate and false positive rate respectively. Different thresholds correspond to different TPR and FPR values. The ROC curve takes FPR as the horizontal axis and TPR as the vertical axis, reflecting the trade-off between the detection rate and the false positive rate under different thresholds. The calculation expression is:

[0198]

[0199] In the formula, A TN It indicates the number of users who are predicted by the classifier to be normal and are actually normal. The meaning of TPR is the same as the recall rate R. FPR indicates the ratio of users who are predicted by the classifier to be abnormal but are actually normal to all normal users, which can reflect the false positive rate of the classifier.

[0200] Specifically, the value range of the AUC indicator is [0,1], which represents the area enclosed by the ROC curve and the horizontal axis of the coordinate. The larger the AUC value, the closer the ROC curve is to the optimal classification point (0, 1), and the better the classification effect.

[0201] In order to facilitate understanding of the above technical solution of the present invention, the present invention is specifically described as follows:

[0202] like Figure 2 As shown, it is the workflow diagram of the anomaly detection model of the present invention; the present invention first processes the original data set, accumulates the data to obtain the daily and monthly power consumption data set of all electricity users; then, constructs the feature index, normalizes the features, and then uses the UMAP method to reduce the dimension of the feature set to obtain a new feature set, thereby concentrating the high-dimensional daily and monthly power consumption data set with low information density after preprocessing into a new low-dimensional feature set with high information density, so as to improve the calculation efficiency of the model and facilitate the visual analysis of the detection results; finally, the obtained new feature set is input into the anomaly detection model, and after all users are classified by the improved DPC clustering algorithm, they are detected by category according to the outlier criterion, and the user number predicted by the model as abnormal is output. After the detection is completed, the model evaluation index is used to evaluate the detection effect of the model.

[0203] like Figure 3-Figure 5 As shown, it is a user clustering effect diagram of different dimensionality reduction algorithms of the present invention. The present invention draws a three-dimensional space structure diagram of low-dimensional features using different data dimensionality reduction methods. As shown in the figure, each coordinate axis represents a principal component. Here, the technical solution finds that different dimensionality reduction methods produce completely different data performances. The data centers of the PCA and LDA dimensionality reduction methods have changed and the data structure has changed significantly, and the data structure is crowded. In low-dimensional space, PCA and LDA dimensionality reduction can only control the cumulative contribution rate of the data after dimensionality reduction, and cannot guarantee the visualization of the data and the change of the data structure; in contrast, the UMAP low-dimensional space data distribution diagram shows that the low-dimensional features well retain the structural relationship between the data, which is more conducive to the maintenance of the data structure, so that the data that are far away in the high-dimensional space also maintain a long distance in the low-dimensional space (that is, the highest similarity), which solves the problem of data congestion and provides better feature data for subsequent clustering and anomaly detection.

[0204] like Figure 6-Figure 8As shown in the figure, it is a daily electricity consumption curve of the cluster center user of different dimensionality reduction algorithms of the present invention. The cluster center of a cluster can represent the overall characteristics of all samples in the cluster. Therefore, the electricity consumption curve of the cluster center user of each category is used as the typical electricity consumption curve of the corresponding category; the daily electricity consumption data of the cluster center user determined by the UMAP, PCA and LDA models are retrieved. The typical daily electricity consumption curves of the three types of users determined by the PCA model intersect with each other most of the time, which is not conducive to the clustering of the clustering algorithm; the typical daily electricity consumption curve of the first type of user determined by the LDA model and the typical daily electricity consumption curves of the second and third types of users only have a small part of intersection, which is clearer, but the typical daily electricity consumption curves of the second and third types of users intersect with each other most of the time, which is also not conducive to the clustering of the clustering algorithm. The typical daily electricity consumption curves of the three types of users determined by the UMAP model do not intersect most of the time, and the category attributes of the three types of users are relatively clear, which is conducive to the clustering of the clustering algorithm and the abnormal user detection of the model.

[0205] like Figure 9-11 As shown, it is a diagram of the anomaly detection result of the model under different dimensionality reduction algorithms of the present invention. Through image observation, the gray circles are users diagnosed as abnormal by both criteria, the gray dots are actual abnormal users, and the black dots are normal users. It can be found that the UMAP model has more gray gray circles and fewer gray hollow circles than the PCA and LDA models, that is, the UMAP model has a higher detection rate and a lower false positive rate than the PCA and LDA models. Secondly, the present invention performs data analysis again to calculate the AUC values ​​of three different dimensionality reduction algorithms respectively. The AUC values ​​of PCA, LDA and UMAP are 0.8985, 0.8461 and 0.9647 respectively. The AUC value of UMAP is much higher than that of PCA and LDA. In summary, it can be concluded that the UMAP dimensionality reduction model is better than PCA and LDA for detecting abnormal users.

[0206] like Figure 12-14 As shown in FIG. 1 , the model detection result diagram of the three types of users under different criteria of the present invention is shown. On the three-dimensional coordinate diagram, when criterion 1 detects the first type of users, normal users near the cluster center will be mistaken as abnormal (such as Fig.12 As shown); when criterion 2 detects the first type of users, the abnormal users identified are mostly points on the edge of the cluster to which the users belong, and the recognition effect is more accurate, so it is concluded that criterion 2 performs better in identifying the first type of users; as shown Fig.13 , Fig.14As shown in the figure, on the three-dimensional coordinate graph, the criterion for abnormal users detected by criterion 2 is too broad, which leads to the misidentification of normal users near the edge of the cluster as abnormal when detecting the second and third category users, and also misidentifies a few normal users near the center of the cluster as abnormal users. When criterion 1 identifies the second and third category users, most of the abnormal users identified are points on the edge of the cluster to which the users belong, and the recognition effect is better. In summary, the two single criteria have their own advantages and disadvantages in detecting abnormalities for different categories of users. The abnormal users identified by the dual criteria are Figure 5 Users who are marked by two single criteria at the same time have the advantages of both single criteria.

[0207] like Figure 15-17 As shown in the figure, it is an influence diagram of different criterion parameters on the detection model performance of the present invention. In order to verify that the dual criterion has good performance in different parameters of the model, the present invention adopts the control variable method; on the basis of α0, β0, and ω0, one of the parameters is changed, and the other two parameters are kept unchanged, and the AUC index of the model under different parameters is calculated; unreasonable parameters will cause the AUC of the criterion 1 model and the criterion 2 model to be far lower than the optimal value, but the changes in the three parameters have little effect on the AUC of the dual criterion model; although the AUC value of the dual criterion also decreases slightly in the face of unreasonable parameters, it still remains at a better level and is much higher than the AUC value of the single criterion; it can be concluded that the setting of the dual outlier criterion reduces the influence of parameter changes on the detection results, increases the reasonable value range of each parameter, and reduces the difficulty of parameter selection; in addition, the comprehensive performance of the dual criterion model for abnormal detection is also better than that of the single criterion model.

[0208] In summary, with the help of the above-mentioned technical scheme of the present invention, the local and global structures of the power load data can be efficiently retained, and the clustering effect can be effectively improved; the improved DPC clustering algorithm realizes the whole process without supervision, is not affected by human errors, and the clustering effect is far better than the traditional DPC clustering algorithm; the setting of the dual-criteria detection rule makes the detection results less affected by parameter changes, increases the reasonable value range of each parameter, and reduces the difficulty of parameter selection; in addition, the comprehensive performance of the abnormal power consumption detection model is also better than the single-criteria model.

[0209] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering, characterized in that: The dual-criteria abnormal power consumption detection method based on dimensionality reduction clustering includes the following steps: S1. Based on the original smart meter data, a daily and monthly electricity consumption dataset of electricity users is established, and characteristic indicators are constructed and normalized to obtain a high-dimensional daily and monthly electricity consumption feature set; S2. Use uniform manifold approximation and projection algorithm to reduce the dimension of high-dimensional daily and monthly electricity consumption feature set to obtain a low-dimensional projection feature set, and set dual-criteria detection rules based on unsupervised clustering algorithm to build an abnormal electricity consumption detection model; S3. Based on the abnormal power consumption detection model, input the low-dimensional projection feature set, output the abnormal power consumption detection result, and use the model evaluation index to evaluate the output.

2. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 1 is characterized in that: The method of establishing a daily and monthly electricity consumption data set of electricity users based on the original smart meter data, constructing feature indicators, and normalizing to obtain a high-dimensional daily and monthly electricity consumption feature set includes the following steps: S11, preprocessing the original smart meter data to generate daily and monthly electricity consumption data sets of electricity users; S12. construct characteristic indicators based on the daily and monthly electricity consumption dataset of electricity users; S13. Normalize the characteristic indicators to obtain a high-dimensional daily and monthly electricity consumption feature set.

3. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 2 is characterized in that: The preprocessing of the original smart meter data to generate the daily and monthly electricity consumption data sets of the electricity users comprises the following steps: S111. According to the time period information, the power consumption of each user in the original smart meter data is accumulated on a daily basis to obtain daily power consumption data; S112, identifying a data point in the daily power consumption data where the power consumption is 0, and replacing the data point with a preset value to generate a user daily power consumption data set; S113, based on the user's daily electricity consumption data set, extract and accumulate the electricity consumption data within a preset time span to generate the user's monthly electricity consumption data set, and establish the user's daily and monthly electricity consumption data set in combination with the user's daily electricity consumption data set.

4. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 2 is characterized in that: The characteristic indicators include morphological indicators, volatility indicators, trend indicators and correlation indicators; The morphological indicators include daily and monthly average electricity consumption, daily and monthly electricity consumption rate, monthly electricity consumption peak-to-valley difference rate, and the proportion of quarterly electricity consumption to annual electricity consumption; The volatility indicators include the daily and monthly electricity consumption dispersion coefficient, the ratio of the daily and monthly electricity consumption dispersion coefficient to the daily and monthly electricity consumption dispersion coefficient of the industry, and the difference between the beginning and the end of electricity consumption; The trend indicators include the slope of the linear fitting of the daily power consumption sequence, the rising trend of the monthly power consumption sequence and the falling trend of the monthly power consumption sequence; The correlation index includes the Pearson correlation coefficient of the daily electricity consumption series of each household and the Pearson correlation coefficient of the typical daily electricity consumption series.

5. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 1, characterized in that: The method of using uniform manifold approximation and projection algorithm to reduce the dimension of the high-dimensional daily and monthly electricity consumption feature set to obtain a low-dimensional projection feature set, and setting a dual-criteria detection rule based on an unsupervised clustering algorithm to construct an abnormal electricity consumption detection model includes the following steps: S21. Based on the high-dimensional daily and monthly electricity consumption feature set, construct a high-dimensional topological weight, and perform low-dimensional projection to generate a low-dimensional projection feature set; S22, constructing an exponential density distance evaluation index, introducing a silhouette coefficient, and formulating a multi-stage clustering strategy, wherein the multi-stage clustering strategy includes a pre-clustering stage strategy and an optimization stage strategy; S23. According to δ - ρ The decision diagram and the actual coordinates of the data points are used to establish a dual-criteria detection rule, and combined with a multi-stage clustering strategy, an abnormal power consumption detection model is constructed, wherein the dual-criteria detection rule includes a first abnormal point criterion rule and a second abnormal point criterion rule.

6. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 5, characterized in that: The method of constructing a high-dimensional topological weight based on a high-dimensional daily and monthly electricity consumption feature set, and performing low-dimensional projection to generate a low-dimensional projection feature set includes the following steps: S211. Construct high-dimensional topological weights based on data points and introduce a weighted adjacency matrix to ensure the consistency of weights between any two data points. S212. Construct a cross entropy function and optimize the cross entropy loss; S213. Use the stochastic gradient descent method to optimize the cross entropy function, obtain the weights of all low-dimensional data points, and establish a low-dimensional projection feature set.

7. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 5, characterized in that: The construction of the exponential density distance evaluation index, the introduction of the silhouette coefficient, and the formulation of a multi-stage clustering strategy include the following steps: S221, extracting preliminary clustering of the data set and setting a dynamic range for pre-clustering; S222, for each pre-clustering result within the dynamic range, calculate the corresponding silhouette coefficient and identify the optimal number of clusters; S223. Construct an exponential density distance evaluation index to identify potential cluster centers.

8. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 5, characterized in that: The basis δ - ρ The decision diagram and the actual coordinates of the data points are used to establish the dual-criteria detection rules, and combined with the multi-stage clustering strategy, the construction of the abnormal power consumption detection model includes the following steps: S231, based on δ - ρ Decision graph, calculates the baseline density and baseline distance of each cluster; S232, if the data point is located at δ - ρ The upper left corner of the decision diagram is determined to meet the first abnormal point criterion rule; S233, defining the boundary points of each cluster, and calculating the boundary density of each cluster; S234, if the density of the data point is less than the boundary density of the cluster to which it belongs, it is determined that the second outlier criterion rule is met; S235. Combine the multi-stage clustering strategy to build an abnormal power consumption detection model, and mark the data points that meet both the first abnormal point judgment rule and the second type of abnormal point judgment rule as abnormal points.

9. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 8, characterized in that: Defining the boundary points of each cluster and calculating the boundary density of each cluster includes the following steps: S2331, define the boundary points of each cluster, and set the initial value of the boundary density of all clusters to 0; S2332, taking the average density of the two points that meet the boundary point condition as the boundary density of the cluster to which the two points belong; S2333, repeat step S2332 until the boundary densities of all clusters are obtained; S2334. Select the maximum boundary density in each cluster as the boundary density of the cluster.

10. The method for detecting abnormal power consumption based on dual criteria of dimension reduction clustering according to claim 1, characterized in that: The abnormal power consumption detection model is based on the input of the low-dimensional projection feature set, the abnormal power consumption detection result is output, and the output is evaluated by using the model evaluation index, including the following steps: S31, inputting the low-dimensional projection feature set into the abnormal power consumption detection model to obtain abnormal power consumption detection results; S32. Based on the abnormal power consumption detection results, construct a power load abnormality detection confusion matrix; S33. Based on the confusion matrix of power load anomaly detection, the recall rate, precision rate, true positive rate and false positive rate are calculated respectively to evaluate the detection effect of the abnormal power consumption detection model.

Images