A load type identification method based on deep embedding clustering

Abstract

The application provides a load type identification method based on deep embedding clustering, and belongs to the field of power system load characteristic analysis and data mining. The method comprises data preprocessing, data dimension reduction, load clustering and load classification. The collected daily load curve data is preprocessed to ensure the integrity and accuracy of the data; a one-dimensional residual convolution autoencoder is used to extract features from the input data, realize data dimension reduction, and solve the problem of too high data dimension; a clustering algorithm is used to divide the low-dimensional features and construct a reliable sample database to alleviate the influence of noise samples; and a convolutional neural network (CNN) classification model is trained using the reliable sample database to obtain the identification result of the load type. The application solves the problems of information loss caused by load data dimension reduction and unsatisfactory load type identification effect caused by noise samples in the prior art.

Description

Technical Field

[0001] This invention belongs to the field of power system load characteristic analysis and data mining, specifically involving a load type identification method based on deep embedding clustering. Background Technology

[0002] With the deployment of electricity consumption information collection devices and the development of data acquisition technology, power system load data resources have accumulated rapidly. However, the acquired load data is often unlabeled, making it difficult to distinguish specific load types. Faced with diverse load types and massive datasets, manually labeling and classifying all load nodes using manual methods is not only extremely time-consuming and resource-intensive but also slow. Therefore, how to utilize the collected measured electricity load data to classify and categorize samples based on the internal distribution patterns of the data has become an important research topic in the field of load characteristic analysis and data mining.

[0003] Traditional load classification methods rely on statistical synthesis, which is resource-intensive and time-consuming. Currently, artificial intelligence-based methods are increasingly used, applying clustering to classify unlabeled load curve data. This yields classification results based on differences in load characteristics, significantly improving efficiency. Electricity load curve clustering essentially utilizes unsupervised algorithms to group unlabeled load data into different clusters based on curve similarity, extracting their group characteristics. Electricity load clustering methods include direct and indirect clustering. Direct clustering methods, such as K-means and fuzzy C-means and their improved algorithms, typically use Euclidean distance as a similarity measure, directly clustering in the original space. However, as smart meter data becomes increasingly detailed, the dimensionality of the data increases, limiting the computational efficiency of direct clustering methods and ultimately reducing clustering performance. Indirect clustering enhances clustering capabilities by combining clustering analysis with dimensionality reduction techniques. It can uncover the inherent pattern information of power load data with lower computational overhead. However, using traditional dimensionality reduction methods such as principal component analysis and linear discriminant analysis to reduce the dimensionality of load data has high computational complexity on large-scale datasets. Furthermore, the separation of feature extraction and clustering tasks cannot guarantee that the extracted features are suitable for the clustering task, which may reduce the clustering quality.

[0004] To address the aforementioned technical challenges, deep learning technology is applied to the power grid. Unsupervised deep network models are used to nonlinearly map samples into a deep feature space before clustering, effectively handling the complex features of power data. Among these, the autoencoder-based deep embedding clustering method addresses the problem of excessive dimensionality through nonlinear dimensionality reduction. It utilizes neural networks to extract latent features from samples, which are then combined with clustering algorithms to partition the samples. This effectively extracts deep temporal features from load curves, ensuring the clustering quality of high-dimensional load data. Furthermore, it integrates feature extraction and clustering processes to prevent fine-tuning from distorting the embedding space. However, the model's recognition performance is less than ideal when encountering difficult-to-distinguish samples or a large number of noisy samples.

[0005] Therefore, it is necessary to design a load type identification method based on deep embedding clustering to solve the above-mentioned technical problems. Summary of the Invention

[0006] This invention provides a load type identification method based on deep embedding clustering, which improves the network feature extraction capability during data dimensionality reduction while effectively mitigating the impact of noise samples on clustering, thereby improving the accuracy of load type identification. This addresses the problems of information loss due to load data dimensionality reduction and the unsatisfactory load type identification effect caused by noise samples in existing technologies.

[0007] The method of this invention is roughly divided into two parts:

[0008] Firstly, a deep embedding clustering network framework is designed to extract low-dimensional features and segment samples. To address the issue of excessively high data dimensionality, an autoencoder is used for non-linear dimensionality reduction. Simultaneously, convolutional layers are introduced to replace the fully connected layers of the traditional autoencoder, enabling more effective extraction of deep features from the input data. A residual learning mechanism is introduced to improve the training process and enhance the network's feature extraction capabilities. This is then combined with a clustering algorithm to segment the reduced low-dimensional features. A joint loss function is designed to update the network model, and pseudo-labels are assigned to each sample.

[0009] Secondly, based on the clustering results, a portion of reliable samples are selected to train the load type recognition model: In order to effectively mitigate the impact of noisy samples on clustering and thus enable the trained model to achieve better recognition results, after dimensionality reduction and clustering, reliable samples with pseudo-labels are selected based on the distance between the samples and each cluster center, and pseudo-label samples with low confidence are filtered out to construct a reliable sample dataset. This dataset is then used to train a load type recognition model with better classification performance.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A load type identification method based on deep embedding clustering includes the following steps:

[0012] Step 1: Data Acquisition: The dataset is based on daily load curves of industrial, commercial, residential, and power generation loads measured at the substation.

[0013] Step 2: Data preprocessing: The daily load curve data is cleaned using methods such as missing value interpolation, outlier smoothing, and data normalization to obtain the preprocessed dataset.

[0014] Step 3: Construct a one-dimensional residual convolutional autoencoder model: The model consists of a convolutional autoencoder and residual blocks. The convolutional autoencoder comprises an encoder and a decoder. The encoder consists of convolutional layers, pooling layers, flattening layers, and embedding layers. The decoder structure is a mirror image of the encoder structure. The input daily load curve data first passes through a structure of "convolutional layer 1 - max pooling layer 1 - convolutional layer 2 - max pooling layer 2 - convolutional layer 3" to extract data features. Then, the flattening layer expands the features learned by the encoding part, and finally, the embedding layer reduces the learned features to [the desired level]. In the low-dimensional space, the low-dimensional features of the data are obtained, completing the encoding process. The decoding process first passes through a fully connected layer corresponding to the flattening layer, then through a reshaping layer to restore the data, and through the "deconvolution layer 1-upsampling layer 1-deconvolution layer 2-upsampling layer 2-deconvolution layer 3" structure corresponding to the encoder, the reconstructed daily load curve is obtained, completing the decoding process. The identity mapping of the residual block is achieved by the input data passing through the "convolution layer 1-max pooling layer 1" structure of the encoder and then jumping to the output of the deconvolution layer 2 of the decoder, completing the residual learning process.

[0015] The residual learning output is:

[0016] F'(x)=F(x)+x conv1

[0017] Where, x conv1 F(x) represents the latent features extracted from input x through the "convolutional layer 1 - max pooling layer 1" structure, and F(x) represents the output of input x through deconvolutional layer 2.

[0018] Step 4: Train the network model: Input the preprocessed dataset from Step 2 into the network model built in Step 3 using a mini-batch method for pre-training to obtain the dimensionality-reduced low-dimensional features. The reconstruction loss uses the mean squared error as the network's loss function, and the Adam algorithm is used to optimize the network and update its parameters, completing the pre-training of the network model.

[0019] The reconstruction loss function uses the mean squared difference formula L. r Represented as:

[0020]

[0021] Where n is the number of samples, x i Let y be the i-th original input data. i This is the i-th reconstructed data in the output.

[0022] Step 5: Cluster and update the network model: Feed the low-dimensional features obtained in Step 4 into the clustering layer, use K-means to divide the samples, obtain the initial cluster centers, and use the Student's T-distribution to measure the probability q of sample i belonging to class j. ij , can be represented as:

[0023]

[0024] Among them, z i Indicates sample x i The low-dimensional features extracted by the encoder, μ j Let τ represent the cluster center of the j-th class, and τ represent the degrees of freedom of the student's T-distribution.

[0025] Define an auxiliary cluster center-based high-confidence target probability distribution p ij A higher probability is assigned to clusters closer to the cluster center, and a lower probability is assigned to clusters farther away from the cluster center.

[0026] Target probability distribution p ij It can be represented as:

[0027]

[0028] Using KL divergence as the clustering loss function L c This brings the original distribution closer to the target distribution.

[0029] Clustering loss function L c Represented as:

[0030]

[0031] Among them, D KL (P||Q) represents the information entropy between the original probability distribution Q calculated by the clustering layer and the auxiliary target probability distribution P, i.e., the KL divergence.

[0032] The reconstruction loss and clustering loss are weighted and summed to form the joint network loss function L. The Adam algorithm is used to minimize the objective function L, the network model is updated, and pseudo-labels are assigned to samples based on the clustering results.

[0033] The joint loss function is expressed as:

[0034] L = L r +γL c

[0035] Where γ controls the degree of twisting the embedding layer, L c This represents the clustering loss.

[0036] Step Six: Construct a Trusted Sample Database: Using the pseudo-labels of all samples obtained from the clustering layer in Step Five, select the sample point closest to the cluster center as the center of that cluster. Select samples in each cluster that are relatively close to the center as trusted samples, thus constructing a trusted sample database. If sample x... i If the distance from the class center is less than the threshold λ, the sample will be selected as a reliable sample; otherwise, the sample will be judged as an unreliable sample.

[0037] Step 7: Train the load type identification model: Use the reliable sample database built in Step 6 to train the CNN classification model to obtain the load type identification model. Input the new load curve data into the model to identify the load type, and update the reliable sample database according to the identification results to optimize the model.

[0038] The beneficial effects of this invention are:

[0039] The present invention provides a load type identification method based on deep embedding clustering. Compared with existing technologies, it employs a one-dimensional residual convolutional autoencoder to extract low-dimensional features from massive high-dimensional data to achieve dimensionality reduction of data features. This method retains more important information to describe the key features of the original sequence, ensuring the clustering quality of high-dimensional load data. The introduction of a residual learning mechanism improves the gradient problem in the deep network training process, thereby controlling reconstruction errors and improving the network's feature extraction capabilities. By selecting samples that are close to the center point in each class from all samples to construct a reliable sample database, a load type identification model with better classification performance is trained to train the CNN classification model. This effectively mitigates the impact of noisy samples on the model and yields more accurate load type identification results. Attached Figure Description

[0040] Figure 1 This is an overall framework diagram of the present invention;

[0041] Figure 2 This is a schematic diagram of the one-dimensional residual convolutional autoencoder structure of the present invention;

[0042] Figure 3 This is a schematic diagram of the residual module of the present invention;

[0043] Figure 4 This is a diagram showing the overall structure of the load type identification model of the present invention. Detailed Implementation

[0044] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0045] The overall framework of this embodiment is as follows: Figure 1 As shown, the overall structure of the load type identification model is as follows: Figure 4 As shown,

[0046] The overall approach mainly includes: a preprocessing module, a dimensionality reduction module, a clustering module, and a classification module;

[0047] The preprocessing module is used to fill in missing values ​​in the dataset, smooth out outliers, and handle data volume differences to obtain a preprocessed dataset.

[0048] The dimensionality reduction module is used to reduce the dimensionality of the preprocessed dataset through a one-dimensional residual convolutional autoencoder model, thereby extracting low-dimensional features from high-dimensional data.

[0049] The clustering module is used to divide the extracted low-dimensional features using a clustering algorithm and assign pseudocodes to the samples to obtain a reliable sample database.

[0050] The classification module is used to train a CNN classification model using a constructed database of reliable samples to obtain the load type identification results.

[0051] The specific steps are as follows:

[0052] Step 1: Data Acquisition: Daily load curve data from n actual measured loads of industrial, commercial, residential, and power generation types from a 10kV dedicated and public transformer in a certain city are selected. The sampling frequency is 15 minutes / time, i.e., 96 sampling points per day, for a duration of m days. The matrix formed by the load data is denoted as X∈R. n×96m .

[0053] Step 2: Data Preprocessing: To ensure data integrity and accuracy, samples with identical rows or more than half of their values ​​being 0 are removed. Missing values ​​are handled using average interpolation, and outlier peaks are handled using moving average. To address the issue of magnitude differences between data points and improve the convergence speed during neural network training, max-min normalization is used to normalize each feature component of the data to the range of 0 to 1. The resulting preprocessed dataset matrix is ​​denoted as Y∈R. s×96m (s≤n).

[0054] Step 3: Constructing a one-dimensional residual convolutional autoencoder model: A schematic diagram of the one-dimensional residual convolutional autoencoder structure of this invention is shown below. Figure 2As shown, the model consists of a convolutional autoencoder and residual blocks. The convolutional autoencoder comprises an encoder and a decoder. The encoder consists of convolutional layers, pooling layers, flattening layers, and embedding layers. The decoder structure is a mirror image of the encoder structure. Compared to fully connected layers, convolutional layers can extract the depth features of the input data more effectively. Pooling layers and upsampling layers can reduce the stride of the convolutional layer's filter to 1 to obtain more detailed features. The input data first passes through a structure of "convolutional layer 1 - max pooling layer 1 - convolutional layer 2 - max pooling layer 2 - convolutional layer 3" to achieve dimensionality reduction of the high-dimensional input data. In order to optimize the dimensionality-reduced features and the clustering process together, the flattening layer is used to unfold the features learned in the encoding part. Finally, the embedding layer reduces the learned features to a low-dimensional space to obtain the low-dimensional feature vector of the data, completing the encoding process. By reducing the dimensionality of high-dimensional data features, the storage space of the data can be reduced, and the calculation time of Euclidean distance between vectors can be reduced, thus improving the efficiency of the algorithm. The decoding process first passes through a fully connected layer corresponding to the flattening layer, then through a reshaping layer to restore the data, and finally through a structure corresponding to the encoder: "Deconvolutional Layer 1 - Upsampling Layer 1 - Deconvolutional Layer 2 - Upsampling Layer 2 - Deconvolutional Layer 3" to obtain the reconstructed load curve, thus completing the decoding process. The convolutional kernel size is 3, the activation functions of the three convolutional layers and deconvolutional layers 1 and 2 are ReLU, and the activation function of deconvolutional layer 3 is Sigmoid.

[0055] The ReLU activation function is:

[0056]

[0057] The Sigmoid activation function is:

[0058]

[0059] Residual modules such as Figure 3 As shown, the identity mapping of the residual block is achieved by the input data being passed through the "convolutional layer 1 - max pooling layer 1" structure of the encoder and then jumping to the output of the deconvolutional layer 2 of the decoder to complete the residual learning process. The parameters learned by the convolutional layer 1 determine the quality of the entire feature extraction. Therefore, this jump connection can effectively improve the performance of the feature extractor and improve the gradient propagation problem in deep networks.

[0060] The residual learning output is:

[0061] F'(x)=F(x)+x conv1

[0062] Where, x conv1 F(x) represents the latent features extracted from input x through the "convolutional layer 1 - max pooling layer 1" structure, and F(x) represents the output of input x through deconvolutional layer 2.

[0063] Step 4: Train the network model: Input the preprocessed dataset Y from Step 2 into the network model built in Step 3 for pre-training using a mini-batch method. This results in a one-dimensional residual convolutional encoder outputting low-dimensional features, and a one-dimensional residual convolutional decoder outputting reconstructed daily load data. The reconstruction loss uses the mean squared error as the network's loss function to measure the distance between the original data and the reconstructed data. The Adam algorithm is used to optimize the network and update its parameters to obtain the optimal autoencoder parameters, thus completing the pre-training of the network model.

[0064] The reconstruction loss function uses the mean squared difference formula L. r express:

[0065]

[0066] Where n is the number of samples, x i Let y be the i-th original input data. i For the i-th reconstructed data, the smaller the mean squared error, the closer the reconstructed data is to the original data, the higher the network accuracy, and thus the more accurate the time series features after dimensionality reduction.

[0067] Step 5: Clustering and updating the network model: To combine the clustering layer with the encoding layer of the autoencoder, the low-dimensional feature vectors obtained after dimensionality reduction are fed into the clustering layer. K-means is used to partition the samples to obtain initial cluster centers. Student's T-distribution is used to measure the probability q of sample i belonging to class j. ij It is used to fine-tune cluster centers.

[0068] The probability q that sample i belongs to class j ij It can be represented as:

[0069]

[0070] Among them, z i Indicates sample x i Features extracted by the coding layer, μ j Let τ represent the cluster center of the j-th class, and let τ represent the degrees of freedom of the student's T-distribution. In this invention, the value of τ is set to 1.

[0071] To improve the quality of load clustering, optimize cluster centers, standardize the loss contribution of each cluster center, and enhance the high-confidence data for load curve allocation, an auxiliary target probability distribution p is defined. ij A higher probability is assigned to clusters closer to the cluster center, and a lower probability is assigned to clusters farther away from the cluster center.

[0072] Target probability distribution p ij It can be represented as:

[0073]

[0074] To make the probability distribution q of the soft partitioning degree of the clustering layer ij With auxiliary target distribution p ij Consistent, using KL divergence as the clustering loss function L c This brings the original distribution closer to the target distribution.

[0075] Clustering loss function L c Represented as:

[0076]

[0077] Among them, D KL (P||Q) represents the information entropy between the original probability distribution Q calculated by the clustering layer and the auxiliary target probability distribution P, i.e., the KL divergence.

[0078] The reconstruction loss and clustering loss are weighted and summed to form the joint loss function L. The Adam algorithm is used to minimize the objective function L, the network model is updated, and pseudo-labels are assigned to samples based on the clustering results.

[0079] The joint loss function is expressed as:

[0080] L = L r +γL c

[0081] Where γ controls the degree of twisting the embedding layer, L c Clustering loss. Reconstruction loss function L r Responsible for the meaningful representation of low-dimensional features for the input data; clustering loss function L c It is responsible for the network's ability to cluster data, making the distance within a cluster closer and the distance between clusters more sparse.

[0082] Step 6: Construct a reliable sample database: Using the pseudo-labels of all samples obtained from the clustering layer, select the sample point closest to the cluster center as the center of the cluster, and select the samples that are close to the center in each cluster as reliable samples to construct a reliable sample database. By filtering out pseudo-label samples with low confidence, the overall reliability of the samples can be improved, and the impact of noisy samples on clustering can be mitigated.

[0083] Sample x i When the distance of a point from the class center is less than the threshold λ, then sample x... i Will be selected as a reliable sample (v i =1), otherwise the sample will be judged as an unreliable sample (v i =0). In this invention, λ is taken as 0.65.

[0084] The formula for calculating distance D is:

[0085]

[0086] Wherein, φ(x) i ) represents the sample x extracted by the encoder. i Low-dimensional features, c j It is the center of class j.

[0087] To address the possibility of missing samples in a particular class when selecting reliable samples, the class center is selected by averaging the features of all reliable samples in that class and choosing the sample point closest to the center point. This ensures that each class contains at least one reliable sample point. Therefore, the class center c... j The calculation method is as follows:

[0088] c j =argmin i ||φ(x i )-μ j || 2

[0089] Where, μ j This represents the cluster center of the j-th cluster obtained by K-means clustering.

[0090] Step 7: Train the load type identification model: Using the reliable sample database built in Step 6, divide it into training data and test data in a 7:3 ratio to participate in the training of the CNN classification model. The resulting model is the load type identification model. Input the new load curve data into the model to identify the load type, and update the reliable sample database based on the identification results to optimize the model.

[0091] After verification, the load type identification method proposed in this invention, which uses the above steps to train a one-dimensional residual convolutional autoencoder using the original dataset, can effectively extract low-dimensional features of high-dimensional load curves. By training a CNN classification model using selected reliable samples, the load type identification result corresponding to the new sample can be obtained.

Claims

1. A load type identification method based on deep embedding clustering, characterized in that, Includes the following steps: Step 1: Data Acquisition: The dataset is based on daily load curves measured at the substation, including load types of industrial, commercial, residential, and power generation. Step 2, Data Preprocessing: The daily load curve data is cleaned using methods such as missing value interpolation, outlier smoothing, and data normalization to obtain the preprocessed dataset; Step 3: Construct a one-dimensional residual convolutional autoencoder model: The model consists of a convolutional autoencoder and residual blocks. The convolutional autoencoder comprises an encoder and a decoder. The encoder consists of convolutional layers, pooling layers, flattening layers, and embedding layers. The decoder structure is a mirror image of the encoder structure. The input daily load curve data first passes through a structure of "convolutional layer 1 - max pooling layer 1 - convolutional layer 2 - max pooling layer 2 - convolutional layer 3" to extract data features. Then, the flattening layer expands the features learned by the encoding part, and finally, the embedding layer reduces the learned features to [the desired level]. In the low-dimensional space, the low-dimensional features of the data are obtained, completing the encoding process. The decoding process first passes through a fully connected layer corresponding to the flattening layer, then through a reshaping layer to restore the data, and through the "deconvolution layer 1-upsampling layer 1-deconvolution layer 2-upsampling layer 2-deconvolution layer 3" structure corresponding to the encoder, the reconstructed daily load curve is obtained, completing the decoding process. The identity mapping of the residual block is achieved by the input data passing through the "convolution layer 1-max pooling layer 1" structure of the encoder and then jumping to the output of the deconvolution layer 2 of the decoder, completing the residual learning process. The residual learning output is: F'(x)=F(x)+x conv1 Where, x conv1 F(x) represents the latent features extracted from input x through the "convolutional layer 1 - max pooling layer 1" structure, and F(x) represents the output of input x through deconvolutional layer 2. Step 4: Train the network model: Input the preprocessed dataset from Step 2 into the network model built in Step 3 for pre-training using a mini-batch method to obtain the dimensionality-reduced low-dimensional features; the reconstruction loss uses the mean squared error as the network loss function, and the Adam algorithm is used to optimize the network and update the parameters, thus completing the pre-training of the network model. The reconstruction loss function uses the mean squared difference formula L. r Represented as: Where n is the number of samples, x i Let y be the i-th original input data. i This is the i-th reconstructed data in the output; Step 5: Cluster and update the network model: Feed the low-dimensional features obtained in Step 4 into the clustering layer, use K-means to divide the samples, obtain the initial cluster centers, and use the Student's T-distribution to measure the probability q of sample i belonging to class j. ij , is represented as: Among them, z i Indicates sample x i The low-dimensional features extracted by the encoder, μ j Let represent the cluster center of the j-th class, and τ represent the degrees of freedom of the student's T-distribution; Define an auxiliary cluster center-based high-confidence target probability distribution p ij , is represented as: Using KL divergence as the clustering loss function L c This brings the original distribution closer to the target distribution; Clustering loss function L c Represented as: Among them, D KL (P||Q) represents the information entropy between the original probability distribution Q calculated by the clustering layer and the auxiliary target probability distribution P, i.e., the KL divergence; The reconstruction loss and clustering loss are weighted and summed to form the joint network loss function L. The Adam algorithm is used to minimize the objective function L, the network model is updated, and pseudo-labels are assigned to samples based on the clustering results. The joint loss function is expressed as: L=L r +γL c Where γ controls the degree of twisting the embedding layer, L c Clustering loss; Step Six: Construct a Trusted Sample Database: Using the pseudo-labels of all samples obtained from the clustering layer in Step Five, select the sample point closest to the cluster center as the center of that cluster, and select samples in each cluster that are relatively close to the center as trusted samples to construct a trusted sample database; if sample x i If the distance from the class center is less than the threshold λ, the sample will be selected as a reliable sample; otherwise, the sample will be judged as an unreliable sample. Step 7: Train the load type identification model: Use the reliable sample database built in Step 6 to train the CNN classification model to obtain the load type identification model. Input the new load curve data into the model to identify the load type, and update the reliable sample database according to the identification results to optimize the model.

Images