Daily load curve clustering method based on improved k-means algorithm

Abstract

The invention provides a daily load curve clustering method based on an improved k-means algorithm, and the method comprises the steps: selecting an initial clustering center in a heuristic random sowing mode in a daily load curve clustering process, and preventing the randomness of the initial clustering center from affecting the clustering quality; and based on an elbow rule, comprehensively evaluating and determining an optimal clustering number by utilizing an error sum of squares index SSE and a clustering evaluation index DBI under different clustering numbers. According to the method, the problems that the clustering number of a k-means clustering algorithm is difficult to select, and random initialization clustering center selection is prone to causing slow convergence speed and falling into local optimum are solved, a high level is shown in the aspect of clustering quality, reasonable load classification can be provided, and full mastering of load characteristics of users is facilitated.

Description

Technical Field

[0001] The invention relates to the field of power system load forecasting, and in particular to a daily load curve clustering method based on an improved k-means algorithm. Background Art

[0002] In recent years, the demand for electricity has grown rapidly, and a large number of diversified loads such as electric vehicles have been invested, resulting in tight electricity loads in some areas. It is necessary to have a deep understanding of the load characteristics under different time and space conditions of the entire network, explore the internal connections between load data, grasp the laws and future trends of load changes, and analyze the user's electricity consumption habits hidden in the data for the rational use of electricity data. Exploring effective data mining algorithms, segmenting users, and quickly and accurately mining a large amount of valuable information such as electricity consumption behavior and electricity consumption for users of different electricity categories, guiding users to be passive and lagging in energy-saving transformation and other work, will help power companies achieve a macro, real-time, and accurate grasp of the overall situation of the power grid, improve the overall understanding of the power grid, and provide a basis for the next step of the power grid optimization plan, which is of great significance in power grid planning.

[0003] Clustering the load curves of power users through clustering technology in data mining is one of the important means of load characteristic analysis. Clustering algorithms are applied to load characteristic analysis, and commonly used ones include partition-based clustering algorithms, hierarchical-based clustering algorithms, density-based clustering algorithms, model-based clustering algorithms, and spectrum-based clustering algorithms. Among them, the k-means clustering algorithm in the partition-based clustering algorithm has a simple calculation process and fast convergence speed, so it is the most widely used.

[0004] However, it is difficult to select the number of clusters for the k-means clustering algorithm, and the random selection of initial cluster centers can easily lead to slow convergence and fall into local optimal problems. Summary of the invention

[0005] In view of the problems existing in the above-mentioned background technology, the present invention proposes a daily load curve clustering method based on an improved k-means algorithm. The method of the present invention selects the initial cluster center by a heuristic random seeding method to avoid the randomness of the initial cluster center affecting the clustering quality; based on the elbow rule, the error square sum index SSE under different cluster numbers and the clustering evaluation index DBI are used for comprehensive evaluation to determine the optimal number of clusters. The method of the present invention solves the problem that the number of clusters of the k-means clustering algorithm is difficult to select, and the random selection of the initialization cluster center is prone to slow convergence and falling into the local optimum. The present invention shows a high level in clustering quality, can provide a more reasonable load classification, and is helpful to fully grasp the load characteristics of users.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is:

[0007] A daily load curve clustering method based on an improved k-means algorithm comprises the following steps:

[0008] Step 1: Preprocess the daily load curve data;

[0009] Step 2: Randomly determine the number of clusters and use the improved k-means algorithm to cluster the daily load curve: First, determine the initial cluster center based on the heuristic random seeding method; then, use the k-means algorithm to cluster the daily load curve;

[0010] Step 3: Keep changing the number of clusters and repeat step 2 to calculate DBI and SSE under different numbers of clusters, determine the optimal number of clusters, and store the clustering results.

[0011] In the above technical solution, further, the step 1 is specifically:

[0012] (1) Data Collection

[0013] Obtain the initial load curve set X0 consisting of n daily load curves = {x 10 ,x 20 ,…x m0 …,x n0}, whose dimension is n×q, n represents the number of curves, q represents the number of data points for each curve, and x m0 is the q-dimensional vector of the load data of the day to be clustered at the corresponding sampling time. The common actual sampling frequency is 15 minutes, corresponding to q is 96.

[0014] (2) Data cleaning

[0015] First, check the load data of each load curve in the set X0 to find abnormal data such as missing data, load mutation or negative value. If the proportion of missing data and abnormal data of a load curve reaches a certain percentage, the curve will be regarded as an invalid curve and will be eliminated. The total number of curves of valid samples after eliminating invalid samples is recorded as N.

[0016] Assume that the load data x collected at time t of the mth load curve is m,t If it is abnormal data, the smoothing correction formula is used to correct the abnormal data, and the corrected data value x m,t 'for:

[0017]

[0018] Where: f1 and b1 represent the step lengths of forward and backward acquisition respectively, which are determined according to specific requirements and are generally 5 to 10.

[0019] (3) Data normalization

[0020] Loads of different orders of magnitude with the same characteristics are classified into one category. In order to eliminate the influence of dimension and increase credibility, the max-min normalization method is used to process load data of different orders of magnitude. The formula is as follows:

[0021]

[0022] In the formula, x m,t " is the load value of the mth sample at the tth time after normalization. It can be seen that the load values ​​of each sample after normalization are distributed between [0,1].

[0023] After data preprocessing, the daily load curve data set X = {x1, x2, …, x m ,…,x N}.

[0024] Furthermore, the step 2 is specifically as follows:

[0025] The improved k-means algorithm uses a heuristic random seeding method to find the cluster centers of k-means clustering. The basic principle and operation mode of the improved k-means algorithm is to assign different probabilities to each sample, so that points farther away from the existing cluster centers are more likely to be selected as the initial cluster centers to ensure that the distances between the initial cluster centers are as far as possible.

[0026] First, assuming that the number of clusters is k, the initial cluster center is determined based on the heuristic random seeding method. The specific method is:

[0027] (1) Initialize the first cluster center: randomly and uniformly select a data sample as the first cluster center;

[0028] (2) Select other cluster centers: For the sample x in the data set m , calculate its distance from the selected cluster center c i The shortest Euclidean distance d(x m ,c i ):

[0029]

[0030] Assign each sample to the nearest cluster center according to the distance; sample x m The probability of being selected as the next cluster center is P m for

[0031]

[0032] That is, when selecting each subsequent center, the probability of each sample being selected is proportional to its distance to the nearest center that has been selected, so that data points farther away are more likely to become the next cluster center, ensuring that the distance between clusters is as far as possible.

[0033] (3) Repeat step (2) until the pre-set k cluster centers are selected.

[0034] This initialization method can make the initial cluster center distribution more extensive, which helps to avoid the problem of the K-means algorithm falling into the local optimal solution, thereby improving the stability and accuracy of the clustering results.

[0035] Then, the k-means algorithm is used to cluster the daily load curve. The k-means algorithm is an iterative clustering method based on partitioning, which aims to divide all data points into k clusters and assign each data point to the cluster corresponding to the cluster center closest to it according to the similarity, so that the square error criterion function between the data points in the cluster is stabilized at the minimum value. The specific method is:

[0036] The samples are divided using Euclidean distance as the evaluation index of sample similarity. Through iterative calculation, the cluster center of each cluster is updated to the average value of all data points in the cluster until the cluster center position no longer changes or the predetermined number of iterations is reached. During the iteration process, the square error criterion function between data points in the cluster must be stabilized at the minimum value.

[0037] Among them, the square error criterion function is the error square sum function (SSE, Sum of Squared Errors), and the calculation formula is as follows:

[0038]

[0039]

[0040] Where: c i is the cluster center of class Ri; m is the sample number; N is the number of samples; k is the number of clusters; d mi Is whether the mth sample belongs to the i-th category.

[0041] Furthermore, the step 3 is specifically as follows:

[0042] Set different numbers of clusters and obtain the clustering results based on step 2; calculate the error square sum index SSE and clustering evaluation index DBI under different numbers of clusters to comprehensively evaluate and determine the optimal number of clusters.

[0043] The Elbow Method is a common heuristic method used in cluster analysis to determine the optimal number of clusters. It helps determine the optimal number of clusters by calculating the sum of squared errors (SSE) under different numbers of clusters. Plot a graph of the number of clusters and the corresponding SSE. The optimal number of clusters is defined as the point where the "elbow" appears in the graph, that is, the inflection point where the impact of the number of clusters on the performance index gradually weakens. The elbow point means that adding categories will not bring more returns, and stop adding categories, which means that this is the optimal number of clusters.

[0044] The elbow rule is simple and intuitive, and is widely used in cluster analysis. However, in some cases, the elbow point is not clear, difficult to observe directly, and is subjective. Therefore, the present invention simultaneously uses the cluster evaluation index DBI (Davies-Bouldin Index) to measure the intra-cluster density and inter-cluster separation of clusters, and comprehensively evaluates the number of clusters.

[0045] By calculating the ratio of the intra-cluster distance to the inter-cluster distance I DBI To measure the closeness and separation of clusters, the calculation formula is as follows:

[0046]

[0047] In the formula, σ i , σ j Respectively represent the average distances of the elements in the i-th and j-th categories to the cluster centers of the corresponding categories. j ,c i ) represents the Euclidean distance from the cluster center of the i-th class to the j-th class. DBI The smaller it is, the smaller the distance within the class is, and the larger the distance between classes is, which means that the clustering results are close within the same category and the different categories are far apart, and the clustering effect is better.

[0048] Analyze and compare the SSE and DBI indicators under different cluster numbers k, select the optimal cluster number k, and store the clustering results.

[0049] The beneficial effects of the present invention are:

[0050] The daily load curve clustering method based on the improved k-means algorithm of the present invention selects the initial cluster center by using a heuristic random seeding method to avoid the randomness of the initial cluster center affecting the cluster quality; based on the elbow rule, the error square sum SSE under different cluster numbers and the cluster evaluation index DBI are used to comprehensively evaluate and determine the optimal cluster number. The method of the present invention shows a high level in clustering quality, can provide a more reasonable load classification, and is helpful to fully grasp the load characteristics of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention is further described below in conjunction with the accompanying drawings and embodiments;

[0052] Figure 1 It is a flow chart of the clustering algorithm of the present invention;

[0053] Figure 2 are the SSE and DBI values ​​under different numbers of clusters in the embodiment of the present invention;

[0054] Figure 3 It is the clustering result of the daily load curve of the embodiment of the present invention. DETAILED DESCRIPTION

[0055] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] like Figure 1 The figure shows a daily load curve clustering method based on an improved k-means algorithm of the present invention, the method comprising the following steps:

[0058] Step 1: Preprocess the daily load curve data;

[0059] Step 2: Randomly determine the number of clusters and use the improved k-means algorithm to cluster the daily load curve: First, determine the initial cluster center based on the heuristic random seeding method; then, use the k-means algorithm to cluster the daily load curve;

[0060] Step 3: Keep changing the number of clusters and repeat step 2 to calculate DBI and SSE under different numbers of clusters, determine the optimal number of clusters, and store the clustering results.

[0061] The step 1 is specifically as follows:

[0062] (1) Data Collection

[0063] Obtain the initial load curve set X0 consisting of n daily load curves = {x 10 ,x 20 ,…x m0 …,x n0}, whose dimension is n×q, n represents the number of curves, q represents the number of data points for each curve, and x m0 is the q-dimensional vector of the load data of the day to be clustered at the corresponding sampling time. The common actual sampling frequency is 15 minutes, corresponding to q is 96.

[0064] (2) Data cleaning

[0065] First, check the load data of each load curve in the set X0 to find abnormal data such as missing data, load mutation or negative value. If the proportion of missing data and abnormal data of a load curve reaches a certain percentage, the curve will be regarded as an invalid curve and will be eliminated. The total number of curves of valid samples after eliminating invalid samples is recorded as N.

[0066] Assume that the load data x collected at time t of the mth load curve is m,t If it is abnormal data, the smoothing correction formula is used to correct the abnormal data, and the corrected data value x m,t 'for:

[0067]

[0068] Where: f1 and b1 represent the step lengths of forward and backward acquisition respectively, which are determined according to specific requirements and are generally 5 to 10.

[0069] (3) Data normalization

[0070] Loads of different orders of magnitude with the same characteristics are classified into one category. In order to eliminate the influence of dimension and increase credibility, the max-min normalization method is used to process load data of different orders of magnitude. The formula is as follows:

[0071]

[0072] In the formula, x m,t " is the load value of the mth sample at the tth time after normalization. It can be seen that the load values ​​of each sample after normalization are distributed between [0,1].

[0073] After data preprocessing, the daily load curve data set X = {x1, x2, …, x m ,…,x N}.

[0074] The step 2 is specifically as follows:

[0075] First, assuming that the number of clusters is k, the initial cluster center is determined based on the heuristic random seeding method. The specific method is:

[0076] (1) Initialize the first cluster center: randomly and uniformly select a data sample as the first cluster center;

[0077] (2) Select other cluster centers: For the sample x in the data set m , calculate its distance from the selected cluster center c i The shortest Euclidean distance d(x m ,c i ):

[0078]

[0079] Assign each sample to the nearest cluster center according to the distance; sample x m The probability of being selected as the next cluster center is P m for

[0080]

[0081] That is, when selecting each subsequent center, the probability of each sample being selected is proportional to its distance to the nearest center that has been selected, so that data points farther away are more likely to become the next cluster center, ensuring that the distance between clusters is as far as possible.

[0082] (3) Repeat step (2) until the pre-set k cluster centers are selected.

[0083] This initialization method can make the initial cluster center distribution more extensive, which helps to avoid the problem of the K-means algorithm falling into the local optimal solution, thereby improving the stability and accuracy of the clustering results.

[0084] Then, the k-means algorithm is used to cluster the daily load curve, and all data points are divided into k clusters. The specific method is as follows:

[0085] The samples are divided using Euclidean distance as the evaluation index of sample similarity. Through iterative calculation, the cluster center of each cluster is updated to the average value of all data points in the cluster until the cluster center position no longer changes or the predetermined number of iterations is reached. During the iteration process, the square error criterion function between data points in the cluster must be stabilized at the minimum value.

[0086] Among them, the square error criterion function is the error square sum function (SSE, Sum of Squared Errors), and the calculation formula is as follows:

[0087]

[0088]

[0089] Where: c i is the cluster center of class Ri; m is the sample number; N is the number of samples; k is the number of clusters; d mi Is whether the mth sample belongs to the i-th category.

[0090] The step 3 is specifically as follows:

[0091] Set different numbers of clusters and obtain the clustering results based on step 2; calculate the error square sum index SSE and clustering evaluation index DBI under different numbers of clusters to comprehensively evaluate and determine the optimal number of clusters.

[0092] The Elbow Method is a common heuristic method used in cluster analysis to determine the optimal number of clusters. It helps determine the optimal number of clusters by calculating the sum of squared errors (SSE) under different numbers of clusters. Plot a graph of the number of clusters and the corresponding SSE. The optimal number of clusters is defined as the point where the "elbow" appears in the graph, that is, the inflection point where the impact of the number of clusters on the performance index gradually weakens. The elbow point means that adding categories will not bring more returns, and stop adding categories, which means that this is the optimal number of clusters.

[0093] The elbow rule is simple and intuitive, and is widely used in cluster analysis. However, in some cases, the elbow point is not clear, difficult to observe directly, and is subjective. Therefore, the present invention simultaneously uses the cluster evaluation index DBI (Davies-Bouldin Index) to measure the intra-cluster density and inter-cluster separation of clusters, and comprehensively evaluates the number of clusters.

[0094] By calculating the ratio of the intra-cluster distance to the inter-cluster distance I DBI To measure the closeness and separation of clusters, the calculation formula is as follows:

[0095]

[0096] In the formula, σ i , σ j Respectively represent the average distances of the elements in the i-th and j-th categories to the cluster centers of the corresponding categories. j ,c i ) represents the Euclidean distance from the cluster center of the i-th class to the j-th class. DBI The smaller it is, the smaller the distance within the class is, and the larger the distance between classes is, which means that the clustering results are close within the same category and the different categories are far apart, and the clustering effect is better.

[0097] Analyze and compare the SSE and DBI indicators under different cluster numbers k, select the optimal cluster number k, and store the clustering results.

[0098] This specific example is a total of 14,826 load curves of 10kv special transformers and public transformers in Sichuan Tianfu Power Grid on a certain working day in March 2022. The collection frequency is 15 minutes, with a total of 96 measurement points. After data cleaning, this example contains a total of 14,321 valid daily load curves.

[0099] After constructing the example data set, cluster analysis is performed according to the method proposed in the present invention. Based on the elbow rule, the DBI index is selected to measure the clustering effect. The results are shown in Table 1. Figure 2 It can be seen that the error sum of squares SSE changes smoothly with the number of clusters, and the elbow point is not significant. At the same time, DBI reaches a minimum value when k=6, and the SSE change tends to be flat at this time, so it is believed that the number of clusters 6 is the best number of clusters.

[0100] Table 1 Clustering effect evaluation table of different cluster numbers

[0101]

[0102] Take k=6 and cluster the data set samples to get the following results: Figure 3 shown.

[0103] exist Figure 3 In the data, six different load curves were observed, showing obvious load characteristics and good clustering effect. The first type of load curve showed a bimodal shape, maintaining a high load from 10 am to 10 pm, with a slight drop at noon, mainly representing commercial load. The second type of load showed a stable trend and a high load, mainly large industrial load, and the night load slightly rebounded, and there may be behaviors of taking advantage of low valley prices for night production. The third type of load was mainly evening peak load, with a small peak from 8 am to 12 pm, mainly representing residential load. The fourth type of load showed a stable trend and a low load, mainly small industrial load. The fifth type of load maintained a high load from 9 am to 5 pm, with a slight drop at noon, mainly non-industrial load. Finally, the sixth type of load was mainly nighttime electricity consumption, representing urban administrative loads, such as street lighting.

[0104] In order to test the robustness of the method of the present invention, a uniformly distributed disturbance is introduced on the basis of the above example, and its ratio is ±r (r = 5%, 10%, ..., 40%). This step simulates the intraday load fluctuation of the user load curve caused by random factors in the actual sampling process, so as to evaluate the adaptability of the method of the present invention to the intraday load fluctuation that may exist in the real environment.

[0105] To illustrate the impact of different degrees of disturbance on the clustering results, the Adjusted Rand Index (ARI) is used to quantify the consistency of the clustering results under different disturbances and the clustering results without disturbance. The value range of the Adjusted Rand Index is [-1,1], where -1 means that the two results are completely different, 0 means that the two results are similar to random, and 1 means that the two results are exactly the same. ARI The calculation formula is:

[0106]

[0107] Where: RI is the Rand coefficient, which is equal to the ratio of the number of samples with consistent clustering results under interference and without interference to the total number of samples. It is easy to know that the range of RI is [0,1]. E(RI) is the expected value of RI.

[0108] The larger the adjusted Rand coefficient is, the higher the consistency of the clustering results under interference and without interference is, which means the better the robustness is. Clustering is performed under different interferences, and the consistency test of the clustering results is performed, and Table 2 is obtained.

[0109] Table 2 Clustering consistency test under different disturbances

[0110]

[0111] It can be seen from Table 2 that when the superimposed interference ratio is less than 10%, the clustering result is very consistent with the clustering result when there is no interference, that is, the algorithm of the present invention maintains very good stability and accuracy under small interference conditions. When the signal-to-noise ratio reaches 40%, the accuracy of the algorithm of the present invention fluctuates greatly, and the clustering accuracy rate will drop significantly. Overall, the algorithm used in the present invention has a certain anti-interference ability and good algorithm robustness.

[0112] The present invention proposes a daily load curve clustering method based on an improved k-means algorithm, which uses a heuristic random seeding method to select the initial cluster center, and uses the error square sum SSE under different cluster numbers and the cluster evaluation index DBI to comprehensively evaluate and determine the optimal cluster number, so as to avoid the randomness of the initial cluster center affecting the clustering quality. Taking the load data of a certain day in Tianfu area of ​​Sichuan as an example, the number of clusters is selected as 6, and the daily load curve is clustered and analyzed. The clustering results show that the morphology of the 6 types of load curves obtained is significantly different, showing obvious load characteristics. It can be seen that the improved algorithm of the present invention shows a high level in clustering quality, can provide a more reasonable load classification, and is helpful to fully grasp the load characteristics of users, and has practical application value.

Claims

1. A daily load curve clustering method based on an improved k-means algorithm, characterized in that: The steps include: Step 1: Preprocess the daily load curve data; Step 2: Randomly determine the number of clusters and use the improved k-means algorithm to cluster the daily load curve: First, determine the initial cluster center based on the heuristic random seeding method; Then, the k-means algorithm is used to cluster the daily load curves; Step 3: Keep changing the number of clusters and repeat step 2. Determine the optimal number of clusters based on the clustering evaluation index DBI and the error square sum index SSE under different cluster numbers, and store the clustering results.

2. The daily load curve clustering method based on the improved k-means algorithm according to claim 1 is characterized in that: The step 2 is specifically as follows: First, assuming that the number of clusters is k, the initial cluster center is determined based on the heuristic random seeding method. The specific method is: (1) Initialize the first cluster center: randomly and uniformly select a data sample as the first cluster center; (2) Select other cluster centers: For the data set X = {x1, x2, ..., x m ,…,x N Sample x in} m , calculate its distance from the selected cluster center c i The shortest Euclidean distance d(x m ,c i ): Assign each sample to the nearest cluster center according to the distance; sample x m The probability of being selected as the next cluster center is P m for (3) Repeat step (2) until the pre-set k cluster centers are selected; Then, the k-means algorithm is used to cluster the daily load curve and divide all samples into k clusters; the specific method is: The samples are divided using Euclidean distance as the evaluation index of sample similarity. Through iterative calculation, the cluster center of each cluster is updated to the average value of all data points in the cluster until the cluster center position no longer changes or the predetermined number of iterations is reached; During the iteration process, the error sum of squares (SSE) between data points in the cluster needs to be stabilized at the minimum value.

3. The daily load curve clustering method based on the improved k-means algorithm according to claim 1 is characterized in that: The step 3 is specifically as follows: Set different numbers of clusters and obtain clustering results based on step 2; calculate the error square sum index SSE under different numbers of clusters and the clustering evaluation index DBI under different numbers of clusters; analyze and compare the SSE and DBI indicators under different numbers of clusters k, select the optimal number of clusters k, and store the clustering results.

4. The daily load curve clustering method based on the improved k-means algorithm according to claim 2 or 3, characterized in that: The calculation formula of the error square sum index SSE is: Where: c i is the cluster center of class Ri; m is the sample number; N is the number of samples; k is the number of clusters; d mi Is whether the mth sample belongs to the i-th category.

5. The daily load curve clustering method based on the improved k-means algorithm according to claim 3 is characterized in that: The calculation formula of the clustering evaluation index DBI is as follows: In the formula, I DBI represents the ratio of the intra-cluster distance to the inter-cluster distance; σ i , σ j Respectively represent the average distances from the elements in the i-th and j-th categories to the cluster centers of the corresponding categories; d(c j ,c i ) represents the Euclidean distance from the cluster center of the i-th class to the j-th class.

Images