Abnormal electricity utilization detection method and system based on multi-scale neighborhood density

Abstract

The invention discloses an abnormal power consumption detection method and system based on multi-scale neighborhood density and a storage medium. The method comprises the following steps: S1, preprocessing original data; s2, carrying out adaptive granular ball representation on the original user sample set; s3, calculating a fuzzy relation matrix among the pellets; s4, calculating the fuzzy density of the pellets; s5, calculating a sample neighborhood density based on the Chebyshev boundary; s6, multi-scale neighborhood abnormal factors of all samples are calculated; s7, judging abnormal users in the power consumption data through threshold comparison; according to the method, the limitation of a current density-based anomaly detection method in adaption to the data characteristics of the electric power Internet of Things is solved, and the capability of depicting the multi-scale neighborhood density by the unsupervised learning model is improved.

Description

Technical Field

[0001] This invention relates to the field of power data analysis technology, specifically to an abnormal power consumption detection method and system based on multi-scale neighborhood density. Background Technology

[0002] With the advancement of smart grid construction, the power system has upgraded from a traditional centralized architecture to a distributed, interconnected Internet of Things (IoT) model. The system's coverage has expanded from power generation to the entire power transmission, distribution, and consumption chain, with data volume growing exponentially. However, at the same time, various abnormal behaviors have become more diverse and covert: for example, some users steal electricity by modifying meters or illegally wiring; some enterprises engage in violations such as exceeding power capacity and improper wiring; and there are even malicious cyberattacks targeting power dispatching systems and smart meters (such as falsifying electricity consumption data and interfering with metering systems). These abnormal behaviors not only directly threaten the safe and stable operation of the power grid, causing local line overload trips at best, but also lead to large-scale power outages at worst, resulting in huge economic losses for power companies. Therefore, monitoring and detecting anomalies in electricity consumption data within the power IoT has become a crucial link in ensuring power grid security and enterprise profitability.

[0003] Current density-based anomaly detection methods still suffer from three major limitations when adapting to the characteristics of power IoT data, hindering further improvements in detection performance: 1) Single-granularity modeling struggles to match the multi-layered structure of electricity consumption data: Electricity consumption data generated by the power IoT exhibits significant multi-layered structural characteristics, while traditional density-based methods mostly employ single-granularity modeling, failing to simultaneously capture the correlation features between data of different granularities. This limitation of single-granularity makes it difficult for the model to fully reveal the inherent structure of electricity consumption data, thus affecting detection efficiency and effectiveness. 2) Precision limitations in high-dimensional data processing: Power IoT data is not only massive in scale but also characterized by high dimensionality and strong uncertainty: Electricity consumption data often contains dozens of feature dimensions such as voltage, current, power, and power factor, making it typical high-dimensional data. Traditional density-based methods face insufficient precision in similarity measurement and distribution modeling when processing this type of data, leading to biases in similarity judgment and ultimately causing misjudgments or missed detections of anomalies. 3) Insufficient fusion of local and global features leads to poor robustness in density estimation: The accuracy of density estimation directly determines the detection performance of density-based methods, and accurate density estimation requires considering both local neighborhood features and global statistical features of the data. Local features capture the microscopic distribution of the data, while global features reflect the macroscopic patterns. However, some existing methods overly rely on local neighborhood features, judging anomalies solely by calculating the density difference between the data object and surrounding samples, making them susceptible to uneven local data distribution. Other methods focus on global statistical features, identifying anomalies only by fitting the overall data distribution, making it difficult to capture subtle local anomalies. This separation between local and global features results in a lack of robustness in the model's density estimation, making it difficult to meet practical needs in complex power consumption scenarios. Summary of the Invention

[0004] To address the aforementioned shortcomings in existing technologies, this invention proposes an abnormal power consumption detection method and system based on multi-scale neighborhood density. The aim is to overcome the limitations of current density-based power Internet of Things anomaly detection methods through technological innovation, and achieve more accurate, efficient, and robust detection results.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: This solution provides a method and system for detecting abnormal power consumption based on multi-scale neighborhood density, including the following steps: S1: Obtain raw data and preprocess it; S2: Adaptive granular representation of the original user sample set based on the processed dataset; S3: Based on the set of particles, calculate the fuzzy relationship matrix between particles; S4: Calculate the particle fuzzy density based on the particle set and the particle fuzzy relationship matrix; S5: Calculate the sample neighborhood density based on Chebyshev boundaries; S6: Calculate the multi-scale neighborhood anomaly factor for all samples; S7: Quantify the degree of user abnormality based on the multi-scale neighborhood anomaly factor, and determine users with abnormal electricity consumption data by threshold comparison; The beneficial effects of this invention are that it realizes the extraction of multi-scale structural information of samples through adaptive modeling based on particle-sphere computation, and uses fuzzy kernel density to estimate the uncertainty of data distribution. Furthermore, it uses fuzzy kernel density and sample local neighborhood density based on Chebyshev boundary to calculate the multi-scale neighborhood density of samples, which effectively improves the accuracy of power data anomaly detection.

[0006] Furthermore, the preprocessing in step S1 includes: In abnormal electricity consumption detection, raw data is imported into an information system (information table), where each row represents a user (i.e., a sample), and each column represents an attribute (i.e., a characteristic) of the user. Attribute values ​​can include numerical (e.g., electricity consumption, voltage), nominal (e.g., user type, wiring method), and hybrid (i.e., including both numerical and nominal values). An information system uses binary tuples... It means that, among them, Representative by A set of users to be detected. Indicates by The data is a set consisting of several attributes. Then, the min-max method is used to normalize the data, adjusting the value range of both numerical and nominal data to the real number interval of 0 to 1. The calculation formula is as follows: , in, For normalization function, Indicates sample In attributes The value on, Indicates all samples in the attribute The minimum value in the set of values ​​above. Indicates all samples in the attribute The maximum value in the set of values ​​above.

[0007] Furthermore, step S2 includes the following sub-steps: Calculate the Euclidean distance between all sample points, select the two samples with the greatest distance as the initial sphere centers, divide all samples into two initial spheres, and calculate the sphere center coordinates of the initial spheres using the sample mean. The formula for calculating the sphere center coordinates is: , in Indicates granules The number of members in Indicates the first One user sample.

[0008] Furthermore, the distribution divergence and splitting threshold of each grain are calculated. The formula for calculating the distribution divergence is: , in Indicates granules The number of members in Indicates the first One user sample, Indicates granules The coordinates of the center of the sphere are given, and the divergence of the resulting spheres after splitting is assumed to be... and The dispersion of the granules before splitting is If the conditions are met Then the granules produced by splitting are replaced with the granules produced before splitting, where , Representing granules and The number of samples included; Furthermore, the granule splitting threshold The calculation formula is: , in This represents the total number of user samples. If the number of sphere members is less than or equal to... Then, no further splitting is needed. The splitting process is repeated until all spheres no longer meet the splitting condition, resulting in the final set of spheres. .

[0009] The beneficial effects of the above-mentioned further scheme are: the influence of dimensions is eliminated by normalization, and the multi-scale structural information of the sample is extracted by adaptive modeling based on particle-sphere calculation, which is conducive to improving the effect of anomaly detection.

[0010] Furthermore, the formula for calculating the fuzzy relationship matrix between particles in step S3 is as follows: in, , They represent granules and balls respectively. and The center of the ball, The bandwidth of the Gaussian kernel function controls the steepness of the similarity function for a set of particles and spheres. The above formula is used to calculate... The fuzzy similarity relationship between any two particles is obtained, thus yielding the particle-sphere fuzzy relationship matrix. .

[0011] Furthermore, the formula for calculating the particle fuzzy density in step S4 is as follows: , in, Indicates granules The number of members in Indicated by granules Fuzzy information particles centered on the center Indicates granules The cardinality of fuzzy information granules, defined in fuzzy rough set theory as a set of objects aggregated through fuzzy relations between objects, utilizing arbitrary fuzzy relations. For all granular objects The partitioning process yields a set of fuzzy particles. ,in, For a granular ball The fuzzy information particles centered on the subject.

[0012] Furthermore, the formula for calculating the fuzzy information granularity is: , in, Indicated by granules Fuzzy information particles centered on the center The th in the particle-sphere fuzzy relation matrix Okay, number The elements of the column.

[0013] The beneficial effects of the above-mentioned further scheme are that the kernel fuzzy similarity relationship modeling method based on the particle-sphere model can effectively alleviate the "curse of dimensionality" problem that traditional distance metrics are prone to in high-dimensional data scenarios while improving efficiency, avoiding a significant decrease in distance discrimination, and further using fuzzy kernel density to improve the ability to characterize the uncertainty of data distribution.

[0014] Furthermore, step S5 includes the following sub-steps: S51: Calculate the user sample set The neighborhood boundary; S52: Calculate the number of neighbors within the neighborhood boundary; S53: Calculate the neighborhood density for each user sample.

[0015] Furthermore, the formula for calculating the neighborhood boundary in step S51 is as follows: , in, The standard deviation coefficient is... Let L2 be the standard deviation vector of the user sample set, and its calculation formula is: , in, The representative sample set The standard deviation of dimension is calculated using the following formula: , in, represent In the The values ​​that can be taken on each feature Representative user sample set In the The mean of each feature.

[0016] Furthermore, in step S52, the user sample Number of neighbors within the neighborhood boundary The calculation formula is: , in, For the neighborhood boundary, It is a relative distance matrix. The total number of samples, This means taking the diagonal elements and forming a vector. Let be an indicator function, if it is the th element in the vector If any element satisfies the condition, return 1; otherwise, return 0.

[0017] Furthermore, in step S53, the user sample The formula for calculating the neighborhood density is: , in, Representative sample The number of neighbors, This represents the maximum number of neighbors for all samples. This represents the minimum number of neighbors for all samples.

[0018] Furthermore, the calculation formula for the multi-scale neighborhood anomaly factor in step S6 is as follows: , in, These are weighting coefficients that control the contributions of the two density fractions. For neighborhood density, for The fuzzy density of the spheres they belong to This is the mapping function between user samples and their corresponding particles, where there is a one-to-one relationship between user samples and particles.

[0019] Furthermore, in step S7, the outlier threshold for abnormal users is set to... If the sample Multi-scale neighborhood anomaly factors ,So Users identified as anomalous are compared to the outlier threshold of all samples in the electricity usage data. By comparing them one by one, the abnormal users in the system can be calculated.

[0020] The beneficial effects of this invention are that it uses particle-sphere computation theory, fuzzy rough set theory and Chebyshev inequality theory to characterize the structural information of the sample set from multiple perspectives, which effectively improves the accuracy of anomaly detection in hybrid power data. In addition, the detection method provided by this invention is an unsupervised algorithm that does not require training data, thus saving manpower costs. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. It should be understood that the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 The diagram illustrates the steps of the abnormal power consumption detection method and system based on multi-scale neighborhood density provided in this embodiment of the invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings. In specific implementation, the present invention will be further explained below with reference to specific user electricity consumption data. Given an information table containing 6 samples and 3 attributes (1 nominal attribute and 2 numerical attributes), as shown on the left side of Table 1.

[0024] Step 1: Data Preprocessing To facilitate subsequent calculations, the input data is first normalized using a min-max method to convert numerical data to a range of 0 to 1, while nominal attributes remain unchanged. The processing results are shown on the right side of Table 1.

[0025] Table 1. Original data and normalization results Step 2: Adaptive Particle Representation Step 2.1: Generate initial spheres Calculate the Euclidean distance between all sample points in 3D space. The two points with the greatest distance are... and Divide the sample points according to their distance. and The granules located and granules Include , granules Include , , , The positions of the centers of the two spheres were calculated. The center of the ball granules The center of the ball Step 2.2: Adaptive splitting of granules Calculate the granule splitting threshold. . Calculate the distribution divergence of the particles. judge Whether or not a split is necessary depends on... With only two members, based on the split threshold No further splitting is needed. Next, we will determine... Do you need to split? (Selecting) The two samples with the longest distance and We use these as new initial points for partitioning. Based on the samples... and We obtain the new sphere from the distance. and The positions of the centers of the two spheres were calculated: Calculate the dispersion divergence of the granules: Calculate the total divergence after splitting: Pre-split distribution divergence: satisfy Therefore, it should be used and replace .at this time The number of members is 1, which is less than the splitting threshold, so no further splitting is needed; [Judgment] Whether further splitting is needed: Based on the distance between samples Divide into new pellets , Furthermore, the dispersion divergence of the granules is calculated: satisfy Therefore, it should be used and replace At this point, none of the granules satisfy the conditions for further splitting, resulting in the final set of granules. The sample mapping relationship is as follows: , , , .

[0026] Step 3: Calculate the fuzzy similarity matrix between particles. Given bandwidth The fuzzy similarity matrix of the particles was calculated. : Step 4: Calculate the particle fuzzy density Step 4.1: Calculate the fuzzy information particle number for each particle. Based on the fuzzy relation matrix generated in step 3, calculate the fuzzy information particle number for each particle: by For example, Similarly, we can obtain , , .

[0027] Step 4.2: Calculate the fuzzy density by For example, Similarly, we can obtain , , .

[0028] Step 5: Calculate the sample neighborhood density based on Chebyshev boundaries Step 5.1: Calculate the standard deviation of each feature dimension and the L2 norm of the dataset's standard deviation vector. First, calculate the mean of the sample set: using features For example, , Further, calculate the standard deviation: Similarly, we can conclude that , .

[0029] Furthermore, the L2 norm of the standard deviation vector is calculated: Furthermore, calculate the neighborhood boundary and take... : Step 5.2: Calculate the number of neighbors within the neighborhood boundary. For each sample Calculate its Euclidean distance with all other samples in the dataset; the statistical distance is less than or equal to... The number of its neighbors (excluding itself).

[0030] The Euclidean distance matrix between sample pairs is calculated as follows: Determine the neighbors of each sample, in order to For example, with , The distance between them is less than ,and , , The distance between them is greater than ,therefore Similarly, the number of neighbors for other samples can be obtained: , , , , .

[0031] Step 5.3: Calculate the neighborhood density anomaly factor for each user sample. by For example, calculate the neighborhood density anomaly factor: Similarly, we can obtain , , , , .

[0032] Step 6: Calculate the multi-scale neighborhood anomaly factor for all samples. by For example, to calculate the multi-scale neighborhood anomaly factor, let... : Similarly, we can obtain , , , .

[0033] Step 7: Determine abnormal users by comparing thresholds. Set an abnormal threshold By comparing each anomaly with the multi-scale neighborhood anomaly factors of each user, a conclusion can be drawn. This is a user with abnormal electricity usage.

[0034] In the embodiments provided by this invention, it should be understood that the disclosed methods can be implemented using electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0035] In the embodiments provided by this invention, it should be understood that the disclosed method can be implemented in other ways. For example, the method embodiments described above are merely illustrative. For instance, the division of the method step units is only a logical functional division, and in actual implementation, there may be other division methods, such as more steps and details, or some steps may be ignored or not executed.

[0036] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0037] This invention provides an abnormal power consumption detection algorithm based on multi-scale neighborhood density. Through technological innovation, it overcomes the limitations of current density-based power Internet of Things anomaly detection methods, achieving more accurate, efficient, and robust detection results.

Claims

1. A method, system, and storage medium for detecting abnormal power consumption based on multi-scale neighborhood density, characterized in that, The method includes: S1: Obtain the raw data and preprocess it. The raw data contains numerical, nominal, or mixed attributes. After preprocessing, the data value range is adjusted to the real number range of 0 to 1. S2: Adaptive granular representation of the original user sample set based on the processed dataset; S3: Based on the set of particles, calculate the fuzzy relationship matrix between particles; S4: Calculate the particle fuzzy density based on the particle set and the particle fuzzy relationship matrix; S5: Calculate the neighborhood density based on Chebyshev boundaries; S6: Calculate the multi-scale neighborhood anomaly factor for all samples; S7: Quantify the degree of user abnormality based on the multi-scale neighborhood anomaly factor, and determine abnormal users in the electricity consumption data by threshold comparison; Step S1 specifically includes: The min-max method is used to normalize the data, adjusting the value range of both numerical and nominal data to the real number interval of 0 to 1. The calculation formula is as follows: ， in, For normalization function, Indicates sample In attributes The value on, Indicates all samples in the attribute The minimum value in the set of values ​​above. Indicates all samples in the attribute The maximum value in the set of values ​​above; Step S2 specifically includes: S21: Calculate the Euclidean distance between all sample points, select the two samples with the greatest distance as the initial sphere centers, divide all samples into two initial spheres, and calculate the sphere center coordinates of the initial spheres using the sample mean. The formula for calculating the sphere center coordinates is: ， in, Indicates granules The number of members in Indicates the first One user sample; S22: Calculate the distribution divergence and splitting threshold for each grain. The formula for calculating the distribution divergence is: ， in, Indicates granules The number of members in Indicates the first One user sample, Indicates granules The coordinates of the center of the sphere are given, and the divergence of the resulting spheres after splitting is assumed to be... and The dispersion of the granules before splitting is If the conditions are met Then the granules produced by splitting are replaced with the granules produced before splitting, where , Representing granules and The number of samples included; The splitting threshold in step S22 The calculation formula is: ， in, This represents the total number of user samples. If the number of sphere members is less than or equal to... Then, no further splitting is needed; repeat the splitting process until all spheres no longer meet the splitting condition, resulting in the final set of spheres. ; Step S3 specifically includes: The fuzzy relationship matrix between particles and spheres is calculated using a Gaussian kernel function. and The formula for calculating the similarity between them is: ， in, , They represent granules and balls respectively. and The center of the ball, The bandwidth of the Gaussian kernel function controls the steepness of the similarity function for a set of particles and spheres. The above formula is used to calculate... The fuzzy similarity relationship between any two particles is obtained, thus yielding the particle-sphere fuzzy relationship matrix. .

2. The abnormal power consumption detection method, system, and storage medium based on multi-scale neighborhood density according to claim 1, characterized in that, The calculation of the spherical fuzzy density includes: Calculate the fuzzy information cardinality of each sphere. The fuzzy information cardinality of a sphere is defined as follows: ， in, Indicated by granules Fuzzy information particles centered on the center The th in the particle-sphere fuzzy relation matrix Okay, number Column elements; Furthermore, the fuzzy density of each sphere is calculated, regarding... The spherical fuzzy density is defined as: ， in, Indicates granules The number of members in Indicated by granules Fuzzy information particles centered on the center Indicates granules The number of fuzzy information particles.

3. The abnormal power consumption detection method, system, and storage medium based on multi-scale neighborhood density according to claim 1, characterized in that, Step S5 includes: Calculate the user sample set The neighborhood boundary of is calculated using the following formula: ， in, The standard deviation coefficient is... Let L2 be the standard deviation vector of the user sample set, and its calculation formula is: ， in, The representative sample set The standard deviation of dimension is calculated using the following formula: ， in, represent In the The values ​​of each feature Representative user sample set In the The mean of each feature; Calculate the number of neighbors within the neighborhood boundary for any user sample. The number of its neighbors within its neighborhood boundary The calculation formula is: ， in, For the neighborhood boundary, It is a relative distance matrix. The total number of samples, This means taking the diagonal elements and forming a vector. Let be an indicator function, if it is the th element in the vector If all elements satisfy the condition, return 1; otherwise, return 0. Calculate the neighborhood density for each user sample. Its neighborhood density The calculation formula is: ， in, Representative sample The number of neighbors, This represents the maximum number of neighbors for all samples. This represents the minimum number of neighbors for all samples.

4. The abnormal power consumption detection method, system, and storage medium based on multi-scale neighborhood density according to claim 1, characterized in that, The calculation formula for the multi-scale neighborhood anomaly factor in step S6 is as follows: ， in, These are weighting coefficients that control the contributions of the two density fractions. For neighborhood density, for The fuzzy density of the spheres they belong to This is the mapping function between user samples and their corresponding particles, where there is a one-to-one relationship between user samples and particles.

Images