District user behavior analysis algorithm based on improved DWDC-k-means secondary clustering

Abstract

The invention discloses a transformer area user behavior analysis algorithm based on improved dynamic weighted density clustering-k-means (DWDC-k-means) secondary clustering. The method comprises the following steps: firstly, performing data preprocessing, including bad data cleaning, vacancy value filling and the like, on an obtained power grid area measured data set; secondly, putting forward and calculating a clustering initial value return index based on a user aggregation degree and an error sum of squares to select an optimal clustering k value, and performing primary clustering based on DWDC to select an initial clustering center; then comprehensively considering load numerical value distance and curve form change characteristics, fusing weighted Euclidean distance and differential Pearson's correlation distance as an optimized similarity distance, and carrying out k-means secondary clustering; data are input and a comparison algorithm is added for verification, and a result shows that the improved DWDC-k-means secondary clustering algorithm disclosed by the invention has a better improvement effect on a given clustering performance index compared with the comparison algorithm, and can extract a typical user power consumption mode more accurately; and power grid managers are helped to provide personalized services and fine management for users.

Description

Technical Field

[0001] The invention belongs to the field of power data detection, and in particular relates to density clustering and k-means clustering algorithms. Background Art

[0002] With the development of artificial intelligence technology and the deepening of intelligent power grid reform, user-side data continues to accumulate, and the modern power system has ushered in the era of big data. Using data mining technology to effectively divide the massive power user curves and mine typical behavior patterns in a large number of disordered loads will help power grid personnel to deeply analyze user patterns, establish user portraits, and provide refined management and personalized services for different users.

[0003] Load curve clustering is an important method for studying user behavior in substations. The most widely used clustering algorithms include k-means clustering (k-means), density-based spatial clustering of applications with noise (DBSCN), fuzzy C means clustering (FCM), Gaussian mixture models clustering (GMM), etc. Among them, k-means clustering is an unsupervised learning algorithm with simple principle and low complexity, which can be effectively used in the study of user load curve clustering in substations. However, there are also inherent problems such as low efficiency in selecting initial values ​​for clustering and large errors in similarity judgment. In view of the existing problems, the initial value and similarity feature selection can be optimized and the user load data set can be processed by combining the secondary clustering method, and load data preprocessing methods can be added to achieve a better level. Summary of the invention

[0004] Purpose of the invention: The present invention proposes an improved DWDC-k-means secondary clustering algorithm for analyzing user behavior in a substation.

[0005] Technical solution: The present invention proposes an improved DWDC-k-means secondary clustering user behavior analysis algorithm, which includes the following steps:

[0006] (1) Preprocess the data set, propose and calculate the initial return index to confirm the initial k value.

[0007] (2) Perform a clustering based on dynamic weighted density clustering (DWDC) to select the best initial center.

[0008] (3) The weighted Euclidean distance and differential Pearson correlation distance are combined to optimize the similarity feature distance for k-means secondary clustering.

[0009] The step (1) comprises the following steps:

[0010] (11) Users with missing values ​​exceeding 10% are eliminated, and the rest are completed using the Lagrange interpolation method, and the data set is updated. The formula is as follows:

[0011]

[0012]

[0013] Where: x, x i are the serial numbers of missing data points and normal points, respectively, and the corresponding values ​​are y i .L m (x) is the result after interpolation, l i (x) is the Lagrange basis function. Here, we take 4 points before and after the missing value for calculation.

[0014] (12) Calculate the initial return index r based on the user aggregation degree and the error square sum k , the formula is as follows:

[0015] r k =r SSE (k)-r d (k)

[0016] Where: r SSE (k), r d (k) are the ratio of the squared error reduction rate and the distance between clusters at the time when the initial value of clustering k is selected, and the formulas are as follows:

[0017]

[0018]

[0019] Where: SSE(k) is the sum of squared errors for different k, r d1 (k), r d2 (k) are the sum of squares of intra-cluster and inter-cluster distances under different k values, r d (k) Describes the degree of aggregation of the data set. The smaller the value, the better the classification effect.

[0020] The specific content of the primary clustering based on dynamic weighted density clustering DWDC described in step (2) is as follows:

[0021] (21) Calculate the density parameters of all points And take the maximum point as the first cluster center, the formula is as follows:

[0022]

[0023] Where: is the Euclidean distance, d mean is the average distance of the data set, and the formula is as follows:

[0024]

[0025]

[0026] Where: is the number of pairs of load curves in the data set, KNN(X i ) is the set of k sequences that are closest to the sequence.

[0027] (22) Confirm the cluster where the first center is located and dynamically update the data set based on the unclustered data, and calculate the weighted density parameter of each point in the current data set The formula and steps are as follows:

[0028]

[0029] Where: X p is the average distance within the cluster centered, ω is the weighted density parameter weight, and the formulas are as follows:

[0030]

[0031]

[0032] Where: X p , X q The average distance between clusters of the cluster centered is as follows:

[0033]

[0034] (23) The point with the maximum weighted density parameter is taken as the next cluster center. Based on the center, the next cluster is confirmed and the data set is dynamically updated. The cluster centers are selected in turn until the value is equal to the k value confirmed in step (1).

[0035] The k-means secondary clustering method of step (3) by combining weighted Euclidean distance and differential Pearson correlation distance to optimize similarity feature distance includes the following steps:

[0036] (31) From the load sequence X p , X q Extract the first-order difference vector from the vector and calculate the difference Pearson correlation distance based on it The formula is as follows:

[0037]

[0038] Where: Zp , Z q is the load sequence X p , X q The first-order difference vector of It's Z p , Z q The Pearson coefficients are as follows:

[0039] z pi =x p(i+1) -x p(i)

[0040] z qi =x q(i+1) -x q(i)

[0041]

[0042] In the formula: pi , z qi X p , X q The value of the i-th dimension of the difference sequence, The sequence X p , X q The average of the data points.

[0043] (32) The Euclidean distance is integrated to construct a new similarity distance, and the formula is as follows:

[0044]

[0045] Where: α, β are distance weights, ω is the constraint The weight coefficient of the order of magnitude is as follows:

[0046]

[0047] (33) Based on the fused similarity distance, k-means secondary clustering is performed, and the clustering results are output when the clustering termination condition is reached.

[0048] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are: 1. An initial value return index is proposed to calculate the initial k value of clustering, which can select the clustering k value more intuitively and accurately compared with the elbow rule to calculate SSE; 2. A primary clustering based on DWDC is performed to improve the efficiency of selecting the initial center of the cluster and thus improve the effectiveness of the overall clustering model; 3. The fusion multi-scale feature distance of the weighted Euclidean distance and the differential Pearson correlation distance is optimized to optimize the secondary clustering k-means to improve clustering accuracy, improve the effect and efficiency of load clustering, and more accurately extract user behavior patterns, which is better than the comparative algorithm in terms of given clustering indicator performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the overall flow chart of the algorithm model of the present invention;

[0050] Figure 2 Schematic diagram of filling missing data for Lagrangian interpolation;

[0051] Figure 3 To confirm the initial value result chart based on the initial value return indicator and comparison method;

[0052] Figure 4 It is a schematic diagram of density parameters and distance;

[0053] Figure 5 Clustering results, four typical user pattern curves; DETAILED DESCRIPTION

[0056] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0057] The data used in this experiment are measured data from a substation in the Southern Power Grid. The data collection frequency is 15 minutes, that is, 96 data points a day, including 312 users in the substation, and the collection period is one month. The collection indicators are user voltage, current, active power, etc. The user active power in the data set is analyzed by an improved DWDC-k-means secondary clustering substation user behavior analysis algorithm of the present invention. The overall flow chart of the algorithm model of the present invention is as follows: Figure 1 The specific steps are as follows:

[0058] 1. Preprocess the data set, propose and calculate the initial return index to confirm the initial k value.

[0059] (11) Users with more than 10% missing values ​​are eliminated, and the rest are supplemented using the Lagrange interpolation method and the data set is updated. Among them, 18 users have more than 10% missing data points and are considered to be abnormal data and eliminated. For users with fewer missing values, the Lagrange interpolation method is used to supplement them. The supplement diagram is shown in the figure. Figure 2 shown.

[0060] (22) Calculate the initial return index r based on the user aggregation degree and the sum of squared errors k , the formula is as follows:

[0061] r k =r SSE (k)-r d (k)

[0062] Where: r SSE (k), r d(k) are the ratio of the squared error reduction rate and the distance between clusters at the time when the initial value of clustering k is selected, and the formulas are as follows:

[0063]

[0064]

[0065] Where: SSE(k) is the sum of squared errors for different k, r d1 (k), r d2 (k) are the sum of squares of intra-cluster and inter-cluster distances under different k values, r d (k) Describes the degree of aggregation of the data set. The smaller the value, the better the classification effect.

[0066] (23) Example results of selecting the initial k value of clustering based on the initial return index and the SSE and r under different k values d (k) Figure 3 As shown by Figure 3 It can be seen that as the number of clusters k increases, SSE continues to decrease, r d (k) also has a minimum value, but the inflection point is not obvious when the elbow method is used to determine the k value in the SSE graph. d (k) The minimum value is also relatively close, and it is easy to make errors when selecting. For the k value, the parameter r is returned. k , which can better identify the k value at the maximum value, and the optimal number of clusters k = 4.

[0067] 2. Perform a clustering based on dynamic weighted density clustering (DWDC) to select the best initial center.

[0068] (21) Calculate the density parameters of all points And take the largest point as the first cluster center, the density parameter and distance diagram are as follows Figure 4 As shown, the formula is as follows:

[0069]

[0070] Where: is the Euclidean distance, d meam is the average distance of the data set, and the formula is as follows:

[0071]

[0072]

[0073] Where: is the number of pairs of load curves in the data set, KNN(X i ) is the set of k sequences that are closest to the sequence.

[0074] (22) Confirm the cluster where the first center is located and dynamically update the data set based on the unclustered data, and calculate the weighted density parameter of each point in the current data set The formula and steps are as follows:

[0075]

[0076] Where: X p is the average distance within the cluster centered, ω is the weighted density parameter weight, and the formulas are as follows:

[0077]

[0078]

[0079] Where: X p , X q The average distance between clusters of the cluster centered is as follows:

[0080]

[0081] (23) The point with the maximum weighted density parameter is taken as the next cluster center. Based on the center, the next cluster is confirmed and the data set is dynamically updated. The cluster centers are selected in turn until the value is equal to the k value confirmed in step (1).

[0082] 3. The weighted Euclidean distance and differential Pearson correlation distance are combined to optimize the similarity feature distance for k-means secondary clustering, and typical clusters are obtained and analyzed.

[0083] (31) From the load sequence X p , X q Extract the first-order difference vector from the vector and calculate the difference Pearson correlation distance based on it The formula is as follows:

[0084]

[0085] Where: Z p , Z q is the load sequence X p , X q The first-order difference vector of It's Z p , Z q The Pearson coefficients are as follows:

[0086] z pi =x p(i+1) -x p(i)

[0087] z qi =zq(i+1) -x q(i)

[0088]

[0089] In the formula: pi , z qi X p , X q The value of the i-th dimension of the difference sequence, The sequence X p , X q The average of the data points.

[0090] (32) The Euclidean distance is integrated to construct a new similarity distance, and the formula is as follows:

[0091]

[0092] Where: α, β are distance weights, ω is the constraint The weight coefficient of the order of magnitude is as follows:

[0093]

[0094] (32) Based on the fusion similarity distance, k-means secondary clustering is performed, and the clustering results are output when the clustering termination condition is reached. The clustering results are as follows Figure 5 As shown in Table 1, there are 116 households in the first category, 46 households in the second category, 27 households in the third category, and 89 households in the fourth category. The characteristic analysis of the four typical user patterns is shown in Table 1. Table 1 Analysis of characteristics of four typical user modes

[0095] 4. Add a comparison algorithm and use the given clustering index to perform an indicator evaluation and analysis on the overall algorithm model.

[0096] (41) Silhouette coefficient SC

[0097]

[0098] Where: They are the average distance within a cluster and the average distance between clusters respectively. The SC value range is [-1, 1]. The larger the value, the better the clustering algorithm effect.

[0099] (42) Davidson Bowden Index (DBI)

[0100]

[0101] From the above formula, we can see that I DBI The smaller it is, the better the clustering effect is.

[0102] (43) Intra-cluster compactness index (ratio of intra-cluster spacing to inter-cluster spacing)

[0103]

[0104] Where: r d1 (k), r d2 (k) are the sum of squares of intra-cluster and inter-cluster distances under different k values, r d (k) Describes the degree of aggregation of the data set. The smaller the value, the better the classification effect.

[0105] (44) The traditional k-means algorithm, fuzzy C-means FCM algorithm, Dk-means algorithm based on fusion density optimization initial value, and k-means algorithm based on fusion multi-feature similarity were added for comparison. The comparison index results are shown in Table 2. The traditional k-means algorithm and fuzzy C-means algorithm performed similarly. In the improvement of k-means and fuzzy C-means clustering single clustering, the initial value optimization of fusion density and fusion of multi-feature similarity were added respectively. The indicators SC, DBI and r were improved. d (k) has been improved to a certain extent; except for being better than single k-means and FCM clustering, there is no obvious difference in the indicators of the two single improved algorithms, and they have similar clustering effects. The algorithm DWDC-k-means proposed in this paper further reduces SC and r on the basis of single improved k-means and FCM. d (k) It also improves DBI, with the highest improvement of 21.6% to 27.6% in each indicator, which can better analyze user patterns. Table 2 Analysis of the index results of the algorithm proposed by the present invention and the comparative algorithm

Claims

1. An improved DWDC-k-means secondary clustering algorithm for user behavior analysis in substations, characterized in that: The following steps are involved: (1) Preprocess the data set, propose and calculate the initial return index to confirm the initial k value. (2) Perform a clustering based on dynamic weighted density clustering (DWDC) to select the best initial center. (3) The weighted Euclidean distance and differential Pearson correlation distance are combined to optimize the similarity feature distance for k-means secondary clustering.

2. According to claim 1, the improved DWDC-k-means secondary clustering user behavior analysis algorithm is characterized in that: The data set preprocessing described in step (1) and the method and steps for calculating the initial return index and confirming the initial k value are as follows: (21) Users with more than 10% missing values ​​are eliminated, and the rest are completed using the Lagrange interpolation method, and the data set is updated. (22) Calculate the initial return index r based on the user aggregation degree and the sum of squared errors k , the formula is as follows: r k =r SSE (k)-r d (k) Where: r SSE (k), r d (k) are the ratio of the squared error reduction rate and the distance between clusters at the time when the initial value of clustering k is selected, and the formulas are as follows: Where: SSE(k) is the sum of squared errors for different k, r d1 (k), r d2 (k) are the sum of squares of intra-cluster and inter-cluster distances under different k values, r d (k) Describes the degree of aggregation of the data set. The smaller the value, the better the classification effect.

3. According to the improved DWDC-k-means secondary clustering algorithm for user behavior analysis in substations according to claim 1, it is characterized in that: The method and steps of primary clustering based on dynamic weighted density clustering (DWDC) described in step (1) are as follows: (31) Calculate the density parameters of all points And take the maximum point as the first cluster center, the formula is as follows: Where: is the Euclidean distance, d mean is the average distance of the data set, and the formula is as follows: Where: is the number of pairs of load curves in the data set, KNN(X i ) is the set of k sequences that are closest to the sequence. (32) Confirm the cluster where the first center is located and dynamically update the data set based on the unclustered data, and calculate the weighted density parameter of each point in the current data set The formula and steps are as follows: Where: X p is the average distance within the cluster centered, ω is the weighted density parameter weight, and the formulas are as follows: Where: X p , X q The average distance between clusters of the cluster centered is as follows: (33) The point with the maximum weighted density parameter is taken as the next cluster center. Based on the center, the next cluster is confirmed and the data set is dynamically updated. The cluster centers are selected in turn until they are equal to the k value confirmed in step (1).

4. According to the improved DWDC-k-means secondary clustering algorithm for user behavior analysis in substations according to claim 1, it is characterized in that: The steps of fusing the weighted Euclidean distance and the differential Pearson correlation distance to optimize the similarity feature distance for k-means secondary clustering in step (1) are as follows: (41) From the load sequence X p , X q Extract the first-order difference vector from the vector and calculate the difference Pearson correlation distance based on it The formula is as follows: Where: Z p , Z q is the load sequence X p , X q The first-order difference vector of It's Z p , Z q The Pearson coefficients are as follows: Z pi =x p(i+1) -x p(i) z qi =x q(i+1) -x q(i) In the formula: Z pi , Z qi X p , X q The value of the i-th dimension of the difference sequence, The sequence X p , X q The average of the data points. (42) The Euclidean distance is integrated to construct a new similarity distance, and the formula is as follows: Where: α, β are distance weights, ω is the constraint The weight coefficient of the order of magnitude is as follows: (43) Based on the fused similarity distance, k-means secondary clustering is performed, and the clustering results are output when the clustering termination condition is reached.

Images