Industrial system operation mode description method based on dynamic clustering algorithm DeepDPM

Abstract

The invention belongs to the technical field of data processing and analysis, and particularly relates to an industrial system operation mode description method based on a dynamic clustering algorithm DeepDPM. The method comprises the following steps: firstly, collecting actual operation data of a system; preprocessing the data, and dividing a training set and a verification set according to a fixed proportion; and then establishing an AE model, pre-training the AE model by using the training set, storing the trained AE model and the weight, and verifying the effect of the trained AE model by using the verification set. Inputting the preprocessed data into the trained AE model for feature extraction; then, the extracted feature data are input into a DeepDPM model to be clustered; and finally, carrying out visual display on a clustering result by using a dimension reduction visualization algorithm: t-distributed random neighborhood embedding (t-SNE for short). According to the invention, the efficiency and accuracy of operation mode prediction in an industrial system are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing and analysis, and more specifically, relates to a method for characterizing an industrial system operation mode based on a dynamic clustering algorithm DeepDPM. Background Art

[0002] With the continued rise in global demand for sustainable operations and the rapid development of artificial intelligence technology, industrial systems are undergoing a profound transformation. This transformation is not only reflected in the continued increase in the number of various distributed resources connected to the industrial system, but also in how they fundamentally reshape the industrial resource allocation mechanism. In particular, when these high-performance devices and intelligent systems are integrated into the industrial system at a high proportion, their inherent dynamics (such as real-time response and flexible scheduling) require operations managers to adopt more agile and flexible strategies to ensure the stability and operational efficiency of the industrial system.

[0003] The operation mode of traditional industrial systems mainly relies on in-depth analysis of historical data and experience summary, and predicts operation trends by refining some typical operation modes. However, with the widespread application of intelligent and automated equipment, traditional methods are difficult to adapt to the operation mode of modern new industrial systems. These new operation modes need to flexibly respond to changing and diverse scenarios and more equipment access to optimize overall stability and production efficiency.

[0004] With the increasing popularity of intelligent equipment, it is particularly important to accurately capture the typical operating mode of current industrial systems. This requires us to conduct in-depth mining of high-dimensional data. By using advanced clustering algorithms to effectively classify data, we can identify typical operating modes. For example, in intelligent transportation systems, cluster analysis can reveal peak-time scheduling modes, off-peak energy-saving modes, and emergency response modes; in energy management systems, cluster analysis can reveal peak power management modes, valley power optimization modes, and fault recovery modes. These clustering results represent different operating modes and provide strong data support for the optimization and upgrading of industrial systems. The problem that follows is that these data are characterized by high complexity and large data volume. The operating mode of modern industrial systems no longer follows the traditional empirical model, and it is difficult to manually mark and judge them. These factors determine that the cluster analysis of industrial system operation data requires the use of unsupervised clustering methods. However, some common unsupervised clustering methods such as K-Means and DBSCAN are very sensitive to parameters, that is, the number of clusters K needs to be predefined. If the parameters are not selected properly, the clustering effect may be very poor. For example, K-Means is sensitive to the selection of the initial cluster center point. Secondly, traditional unsupervised learning methods usually require model selection, training the model multiple times to try different K values, and then selecting the "best" K value. This leads to high computational overhead and is often not feasible in deep learning. Finally, these unsupervised clustering algorithms themselves are not combined with deep learning, do not use the architecture of neural networks or deep learning models, and therefore lack the ability to express complex features. For complex high-dimensional or nonlinear data such as those in industrial systems, they cannot extract the deep features of the data and have limited ability to express complex data, resulting in relatively poor clustering effects in high-dimensional space. The above limitations indicate that further expansion and optimization of existing methods is needed to improve the efficiency and accuracy of industrial system operation mode prediction.

[0005] Chinese patent document CN115275995A discloses a method and system for determining a typical operation mode of an electric power system, comprising the following steps: step 1: obtaining a sample data set of new energy output and performing normalization processing; step 2: performing typical day clustering calculation using an improved fuzzy C-means algorithm based on the normalized sample data set, the number of clusters and the initial cluster center; wherein the similarity calculation in the improved fuzzy C-means algorithm uses a combination of Euclidean distance and dynamic time bending distance to construct a Euclidean dynamic bending distance, and the initial cluster center is selected based on the peak density idea and the Euclidean dynamic bending distance; the number of clusters in the initial state uses a preset minimum number of clusters; step 3: calculating the silhouette coefficient of the clustering result, if the current number of clusters is not greater than the preset maximum number of clusters, the number of clusters is increased by 1, and the process goes to step 2; otherwise, the number of clusters with the largest silhouette coefficient and the cluster center are output as the clustering result of the typical day; step 4: selecting the operation modes with the smallest and largest stability margins as the typical operation modes of the power grid through the calculation of the stability margin of the electric power system. However, this method relies on the set minimum and maximum number of clusters for multiple calculations, and lacks dynamic adjustment capabilities, which may lead to clustering errors. Secondly, this patented method does not use a deep learning structure, but only extracts features through a combination of Euclidean distance and dynamic time warp distance, and has limited expressive power for high-dimensional complex data.

[0006] In view of this, the present invention designs an industrial system operation mode characterization method based on the dynamic clustering algorithm DeepDPM to solve the problem of improving the efficiency and accuracy of operation mode prediction in industrial systems. Summary of the invention

[0007] The present invention aims to overcome at least one defect of the above-mentioned prior art and provide an industrial system operation mode characterization method based on the dynamic clustering algorithm DeepDPM to solve the problem of improving the efficiency and accuracy of operation mode prediction in the industrial system.

[0008] The detailed technical scheme of the present invention is as follows:

[0009] A method for characterizing an industrial system operation mode based on a dynamic clustering algorithm DeepDPM, the method comprising:

[0010] S1. Collect the operation data of the industrial system, pre-process the data and divide it into training set and validation set according to a fixed ratio;

[0011] S2. Establish an AE model, use the training set to pre-train the AE, save the AE model and weights after training, and then use the validation set to verify the effect of the trained AE model. Then input the pre-processed data into the trained AE model for feature extraction;

[0012] S3, input the extracted feature data into the DeepDPM model for clustering;

[0013] S4. Finally, a dimensionality reduction visualization algorithm is used: t-distributed stochastic neighbor embedding, abbreviated as t-SNE, to visualize the clustering results.

[0014] Preferably according to the present invention, the data preprocessing refers to filling in missing data and aligning all data, with the aim of ensuring the integrity and consistency of the data, thereby improving the accuracy of analysis and modeling.

[0015] Preferably, the detailed steps of step S2 are as follows:

[0016] S21. Establish an AE model: First, set the first layer of the AE model as the input layer. Next, set the number of hidden layers of the encoder part to 3, and the number of neurons in each hidden layer of the encoder part is 500, 500, and 2000 respectively; set the dimension of the middle latent space layer to 50; since the structure of AE is symmetrical, set the decoder part with 3 hidden layers as well, and the number of neurons in each hidden layer of the decoder part is 2000, 500, and 500 respectively; finally, set the output layer;

[0017] S22. First, use the training set to pre-train the AE: Compared with training starting from random initialization, the model convergence speed after pre-training is usually faster, which can reduce the calculation time, and pre-training can better capture the underlying structure of the input data, making the extracted features more accurate and having better distinguishing ability in subsequent clustering tasks. Input the training set into the built AE model, set the parameters learning_rate=0.002, batch_size=256;

[0018] The mean square error (MSE) loss function is used during training. MSE is used to measure the difference between the reconstructed input and the original input of AE. The smaller the difference, the closer the reconstructed data is to the original data, and the better the training effect. The formula is as follows:

[0019]

[0020] In formula (1), y i is the input data, is the data output after the AE model, and n is the total number of samples.

[0021] In the training of neural networks, optimizers are often used. Their main function is to adjust weights and bias terms during the training process to minimize the loss function. Then, the adaptive moment estimation optimizer Adam is selected. The adaptive moment estimation optimizer Adam uses the first-order moment estimation and second-order moment estimation of the gradient to adaptively adjust the learning rate and converge faster, especially in high-dimensional and sparse data. Finally, the weights and bias parameters of the AE model are saved after the training is completed.

[0022] S23. Use the validation set to verify the pre-trained AE model, and use the same MSE loss function as the pre-training as the evaluation indicator. If the final loss value of the validation set is close to the final loss value of the training set, it means that the performance of the model on the training set and the validation set is relatively consistent, indicating that the AE model has good generalization ability;

[0023] S24, feature extraction using AE model: The main feature extraction only uses the three main parts of input layer, encoder part and latent space layer. All data samples are input into the trained AE model, pass through the encoder part, and finally the latent space layer outputs the extracted feature data;

[0024] Preferably, according to the present invention, the specific steps of S3 are as follows:

[0025] S31. Calculate soft cluster assignment: The feature data is first passed to the clustering network, which is a multi-layer perceptron MLP with one hidden layer and 50 fixed neurons. The last layer has K neurons, corresponding to K clusters. The clustering network generates K soft cluster assignment probability vectors for each sample, and then the soft cluster assignment probability vectors of all samples form a matrix R, where each row represents the soft cluster assignment probability vector of a sample, and each column represents the probability of belonging to a cluster:

[0026]

[0027] In formula (2), the input is the set of all N samples x, and the output is the K soft clustering assignments for each sample, expressed as Each r i is a K-dimensional vector, indicating the soft clustering assignment of the i-th sample; N represents the total number of samples; r i,k represents the probability that the i-th sample is assigned to the K-th cluster, called “responsibility”, and all r i,k The sum of is 1;

[0028] S32. Calculate hard allocation: Calculate hard allocation to explicitly allocate samples to a specific cluster: Based on the soft cluster allocation result R, calculate the hard allocation z of each sample:

[0029]

[0030] In formula (3), z i Indicates the actual cluster label assigned to the i-th sample, i.e. z i = argmax k r i,k ;

[0031] S33. After obtaining the hard assignment, a sub-clustering network is created for each cluster. The sub-clustering network is a multi-layer perceptron MLP with one hidden layer, 50 neurons fixed, and 2 neurons in the last layer, corresponding to 2 sub-clusters; the sub-clustering network is for each sample v belonging to cluster k. i Generate 2 soft clustering probability vectors, and then form a matrix with the soft clustering probability vectors of all samples belonging to the kth cluster Each row represents the soft clustering assignment probability vector of a sample, and each column represents the probability of belonging to a cluster:

[0032]

[0033] In formula (4), the input is the sample set of each cluster k The output sub-cluster soft assignment is expressed as Each sample v i The sum of all sub-cluster probabilities of is 1; where, is a matrix of size N×2, consisting of the soft cluster assignment probability vectors of all samples in the kth cluster; represents the soft clustering assignment of the i-th sample, Represents sample v i The probability of being assigned to the jth cluster, v k represents all samples in the kth cluster;

[0034] S34. Dynamic adjustment during training: The DeepDPM model will dynamically adjust the number of clusters K during training, propose split or merge operations, and accept or reject these operations based on a certain probability.

[0035] Preferably, according to the present invention, the dynamic adjustment in the training process is as follows:

[0036] In the split operation, the algorithm will propose a splitting scheme to split each cluster into two sub-clusters at regular intervals. The acceptance probability of the splitting scheme is given by a Hastings ratio H. S The ratio compares the marginal likelihood of the data under the two sub-clusters with the marginal likelihood under the original cluster, with probability min(1,H S ), H S The formula is as follows:

[0037]

[0038] In formula (5), α is a constant used to control the frequency of the splitting operation; Γ is the gamma function; N k Represents the number of samples in cluster k; N k,j represents the number of samples in subcluster j, j∈{1,2}; X k represents the set of all samples in cluster k; X k,j represents the set of all samples in sub-cluster j; f x is the marginal likelihood function; λ represents the NIW hyperparameter; the numerator calculates the two sub-clusters X after the split k,1 and X k,2 The denominator calculates the clustering X before the split k The marginal likelihood function of ;

[0039] If H S If H is greater than 1, the split operation is accepted because the split model can better explain the data. S When less than 1, the split will be H S proportion is accepted;

[0040] In the merge operation, the algorithm merges each cluster with its three nearest neighbor clusters. The merge cannot be processed in parallel but is performed step by step. The acceptance probability of the merge is given by a Hastings ratio H. m Determine that the ratio is the Hastings ratio H of the split S The reciprocal of is as follows:

[0041] H m =1 / H s (6)

[0042] If the merge proposal is accepted, the two clusters will be merged and initialized into a new sub-clustering network; the last layer of units of the merged cluster will be removed, and the weights of the last layer of units of the merged cluster and the previous hidden layer will also be removed; the parameters and weights of the newly formed sub-clustering network will be initialized using weighted maximum a posteriori MAP estimation, which also indicates that when processing the merged cluster, the parameters in the clustering model learned before the cluster merge will be considered.

[0043] Preferably, according to the present invention, the loss function used in the clustering network is a newly defined loss function used to measure the clustering network f cl The difference between the generated soft assignment result and the E-step probability is given by the following formula:

[0044]

[0045] In formula (7), r i is the soft clustering assignment of the i-th sample; It is the probability that the i-th sample belongs to each cluster K calculated by the E-step of the EM-GMM algorithm;

[0046] The E-step is a step in the standard EM-GMM algorithm, called the expectation step, which mainly uses the current parameter estimate to calculate the probability that each sample belongs to each Gaussian distribution. The purpose of using the E-step here is to calculate the probability of each sample x i The probability of belonging to each cluster K The calculation formula is as follows:

[0047]

[0048] where π k is the prior probability, i.e., weight, which indicates the proportion of the cluster in all samples; μ k and∑ k are the mean vector and covariance matrix respectively; Ν(x i ;μ k ,Σ k ) represents the probability density function; the numerator calculates the sample x i The denominator is a normalization factor for all categories k′. The sum of the probability density functions of all clusters k′ is calculated to ensure that the sum of all probabilities is 1. In general, E-step is based on the current clustering parameter (π k ,μ k ,Σ k ) Calculate the probability that each sample belongs to each cluster;

[0049] The purpose of the loss function is to encourage the clustering network f cl Generate the soft allocation calculated in the E-step Similar results. This loss function is based on KL divergence. By calculating the soft distribution r of the clustering network output i Soft assignment with the E-step calculation of the EM algorithm The goal is to hope that the output of the network can reflect more accurate category assignments, thereby improving the clustering effect.

[0050] The sub-clustering network uses an isometric loss to measure the sample v i The distance between the sub-cluster center and the sub-cluster center to which it belongs is expressed as follows:

[0051]

[0052] In formula (9), express norm, which is the square of the Euclidean distance; Represents sample v i The probability of belonging to center j in cluster k, the higher the value, the better the sample v i The more likely it is to belong to center j. i represents the i-th sample. Represents the center of the j-th sub-cluster of cluster k, that is, the mean.

[0053] The overall loss function The purpose is to evaluate the proximity of samples to their corresponding cluster centers through a combination of weights and distances, hoping to minimize the distance from each sample to the cluster center of its class, while considering the weights or importance of different samples in different clusters, so as to improve the aggregation of samples in different categories of the model.

[0054] Preferably, in step S4, the basic idea of ​​the t-SNE algorithm is to embed the data points in the high-dimensional space into the low-dimensional space while trying to maintain the local similarity relationship of the data points in the high-dimensional space:

[0055] The t-SNE algorithm defines the similarity of different operation modes in high-dimensional space and low-dimensional space to represent the distribution of operation modes: in high-dimensional space, any two high-dimensional operation mode vectors p i ′ and p′ j The similarity S between ij It is expressed by the conditional probability of Gaussian distribution:

[0056]

[0057] In formula (10), σ i represents the variance of the data, and M represents the total number of data points;

[0058] In low-dimensional space, any two low-dimensional operation mode vectors y i and j The similarity q between ij It is expressed by the conditional probability of t distribution:

[0059]

[0060] In formula (11), y m and n represents the pattern vectors in the low-dimensional space, which are used to calculate q ij Other low-dimensional data points are used to normalize the denominator, in order to measure the relative distance between all modes;

[0061] t-SNE minimizes the difference between high-dimensional space and low-dimensional space to find the best low-dimensional representation. This difference is measured by KL divergence, and then the gradient descent method is used to minimize KL divergence and update the position of data points in the low-dimensional space to make the distribution of high-dimensional and low-dimensional operation modes as close as possible to achieve the purpose of dimensionality reduction. The KL divergence formula is as follows:

[0062]

[0063] In formula (12), S ij represents the similarity between any two high-dimensional operation mode vectors, q ij Represents the similarity between any two low-dimensional operation mode vectors;

[0064] After multiple iterations, the algorithm converges and finally obtains a low-dimensional representation for data visualization. Low-dimensional data is usually two-dimensional or three-dimensional. Here, three-dimensional data is selected and a scatter plot is used to visualize the clustering of operating modes.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention adopts a dynamic clustering method that combines deep neural networks and Dirichlet process Gaussian mixture models (DPGMM) to extract key nonlinear features from high-dimensional data through the AE neural network structure, which is more advantageous than traditional linear dimensionality reduction methods, such as principal component analysis PCA. It can discover implicit structures and patterns in complex data, making clustering models more accurate when processing complex data. DeepDPM is based on a split / merge framework and a dynamic architecture, and flexibly adjusts the number of clusters K throughout the learning process, thereby avoiding the problem of misclassification that may be caused by a fixed K value. In addition, this method overcomes the limitation of the previous need to specify the number of clusters K in advance, and provides greater flexibility and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of the AE automatic encoder model described in the present invention.

[0068] Figure 2 This is the DeepDPM structural framework diagram described in the present invention.

[0069] Figure 3 This is a clustering result diagram of power system operation modes under the background of a penetration rate of less than 15% in an embodiment of the present invention.

[0070] Figure 4 This is a clustering result diagram of power system operation modes under the background of 15%-30% penetration rate in an embodiment of the present invention.

[0071] Figure 5This is a clustering result diagram of power system operation modes under the background of more than 30% penetration rate in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0073] Embodiment 1,

[0074] This embodiment takes the power system in the industrial system as an example and provides a method for characterizing the operation mode of the power system based on the dynamic clustering algorithm DeepDPM. The method includes:

[0075] S1. Collect the power system operation data throughout the year. The data is measured every hour. The data is preprocessed and divided into training set and validation set according to a fixed ratio under each penetration rate background;

[0076] S2. Establish an AE model, use the training set to pre-train the AE, save the AE model and weights after training, and then use the validation set to verify the effect of the trained AE model. Then input the pre-processed data into the saved AE model for feature extraction;

[0077] S3, input the extracted feature data into the DeepDPM model for clustering;

[0078] S4. Finally, a dimensionality reduction visualization algorithm, t-SNE (t-distributed stochastic neighbor embedding), is used to visualize the clustering results.

[0079] In the step S1, the new energy penetration rate at each moment is first calculated, and then divided into three categories according to the penetration rate level: 15% or less is low penetration, 15%-30% is medium penetration, and 30% or more is high penetration. Next, data preprocessing operations are performed separately in each category, mainly including missing data filling and all data alignment, in order to ensure the integrity and consistency of the data, thereby improving the accuracy of analysis and modeling. The ratio of training set and validation set in each category is 2:1, that is, two-thirds of the data is used as a training set to train the model, and the remaining one-third is used as a validation set to evaluate the generalization ability and performance of the model. It not only ensures that there is enough data to support the learning process of the model, but also retains a certain proportion of data for objectively evaluating the effect of the model, which helps to prevent the occurrence of overfitting; the training set and the validation set are respectively composed of a power system operation mode P matrix:

[0080] P=(x b ,x c ,x l ,x t ) T (1)

[0081] In formula (1), x b Represents the busbar data, x c Represents the compensator data, x l Represents load data, x t Represents transformer data. The content of operation mode P is not unique and can be customized according to actual needs.

[0082] The detailed steps of step S2 are as follows:

[0083] S21. Establish AE model: Figure 1 As shown in the figure, first, the first layer of the AE model is set as the input layer, then the number of hidden layers of the encoder part is set to 3, and the number of neurons in each hidden layer of the encoder part is 500, 500, and 2000 respectively; the dimension of the middle potential space layer is set to 50, that is, the feature dimension extracted by AE is 50; since the structure of AE is symmetrical, the decoder part is also set to have 3 hidden layers, and the number of neurons in each hidden layer of the decoder part is 2000, 500, and 500 respectively; finally, the output layer is set; the entire AE architecture has 9 layers;

[0084] S22. First, use the training set to pre-train the AE: Compared with training starting from random initialization, the model convergence speed after pre-training is usually faster, which can reduce the calculation time. And pre-training can better capture the underlying structure of the input data, making the extracted features more accurate and having better distinguishing ability in subsequent clustering tasks. Input the training set into the built AE model, set the parameters learning_rate=0.002, batch_size=256, and set different epochs for different data at each penetration rate;

[0085] The mean square error (MSE) loss function is used during training. MSE is used to measure the difference between the reconstructed input and the original input of AE. The smaller the difference, the closer the reconstructed data is to the original data, and the better the training effect. The formula is as follows:

[0086]

[0087] In formula (2), y i is the input data, is the data output after the AE model, and n is the total number of samples.

[0088] In the training of neural networks, optimizers are often used. Their main function is to adjust weights and bias terms during the training process to minimize the loss function. Then, the adaptive moment estimation optimizer Adam is selected. The adaptive moment estimation optimizer Adam uses the first-order moment estimation and second-order moment estimation of the gradient to adaptively adjust the learning rate and converge faster, especially in high-dimensional and sparse data. Finally, the weights and bias parameters of the AE model are saved after the training is completed.

[0089] S23. Use the validation set to verify the pre-trained AE model, and use the same MSE loss function as the pre-training as the evaluation indicator. If the final loss value of the validation set is close to the final loss value of the training set, it means that the performance of the model on the training set and the validation set is relatively consistent, indicating that the AE model has good generalization ability;

[0090] S24, feature extraction using AE model: The main feature extraction only uses the three main parts of input layer, encoder part and latent space layer. All data samples under each permeability are input into the trained AE model, and then the latent space layer outputs the extracted feature data after passing through the encoder part. The feature dimension is unified as 50.

[0091] In step S3, the DeepDPM deep learning model is used. In the clustering process, DeepDPM adopts a split / merge framework to automatically determine the number of clusters K through a dynamic architecture and adaptive process. The split / merge method is not only able to adjust the value of K, but also to make some large-step adjustments, thereby helping the model get rid of some not-so-good local optimal solutions. The entire DeepDPM structure framework process is as follows Figure 2 As shown in the figure, it is generally divided into two parts: cluster network and sub-cluster network.

[0092] The forward propagation step of the DeepDPM model during training is also the process of generating cluster labels as follows:

[0093] 1. The feature data is first passed to the clustering network, which is used to generate a clustering network for each sample x. i Generate K soft cluster assignments:

[0094]

[0095] The input is a set of all N samples x, and the output is each sample x i The K soft cluster assignments of where r i,k Represents sample x i The probability of being assigned to the Kth cluster is called “responsibility”. i,kThe sum of is 1. R is a matrix of size N×K, which represents the soft clustering assignment results of all samples, and each row represents the soft assignment probability vector of a sample. i represents the soft clustering assignment of the i-th sample. Each r i is a K-dimensional vector. N represents the total number of samples.

[0096] The clustering network is essentially a multilayer perceptron, or MLP, with one hidden layer and 50 fixed neurons, and the last layer has K neurons, corresponding to K clusters. It forms a matrix with the soft clustering probability vectors of all samples, where each row represents the soft clustering probability vector of a sample, and each column represents the probability of belonging to a cluster.

[0097] 2. Calculate the hard assignment. The step of calculating the hard assignment is to explicitly assign the sample to a specific cluster. Based on the soft assignment R, calculate the hard assignment of each sample:

[0098] z i Represents sample x i The cluster number assigned, i.e. z i = argmax k r i,k , z i is x i The assignment of points to clusters is the actual cluster label for each sample.

[0099] Here is a brief explanation of the two concepts of soft assignment and hard assignment. Hard assignment is a strict assignment method in which each sample can only be assigned to a unique cluster category, that is, the belonging of each sample is clear, either belonging to a certain category or not. For example, K-Means is a hard clustering algorithm. In contrast, soft assignment is more flexible, allowing each sample to be assigned to multiple cluster categories at the same time, but the degree of assignment varies. The attribution of samples is ambiguous, and the probability value is used to represent the possibility of belonging to each cluster. The larger the probability value, the more likely the sample is to belong to the corresponding category. For example, the Gaussian mixture model GMM is a soft clustering algorithm. To summarize, in the DeepDPM method, soft assignment calculates the probability weight of each sample relative to each cluster, and hard assignment selects the cluster with the largest probability as the final assignment result.

[0100] 3. After obtaining the hard assignment, a subclustering network subclusteringnet is created for each cluster. The process is similar to the clustering network. This network has a subclusteringnet for each sample v belonging to cluster k. i Generate 2 soft cluster assignments:

[0101]

[0102] The input is a set of samples for each cluster k Output the sub-cluster soft assignment for each cluster k, expressed as Each sample v i The sum of all sub-cluster probabilities of is 1. Among them, is a matrix of size N×2, representing the soft cluster assignment probability vector of all samples in the kth cluster. represents the soft clustering assignment probability vector of the i-th sample, represents the probability that the i-th sample is assigned to the j-th cluster. k represents all samples in the kth cluster.

[0103] The sub-clustering network is also an MLP architecture with one hidden layer and 50 fixed neurons. The difference from the clustering network is that the last layer has 2 neurons, corresponding to 2 sub-clusters. It forms a matrix with the soft clustering probability vectors of all samples belonging to the kth cluster. Each row represents the soft clustering probability vector of a sample, and each column (two columns in total) represents the probability of belonging to a cluster.

[0104] 4. Dynamic adjustment during training: The algorithm will dynamically adjust the number of clusters K during training, propose split or merge operations, and accept or reject these operations based on a certain probability.

[0105] The algorithm proposes a splitting scheme to split each cluster into two sub-clusters at regular intervals. The acceptance probability of the splitting scheme is given by a Hastings ratio H. S The ratio is a tool for model comparison in Markov chains. It is mainly used to evaluate the relative merits of different models, especially in Bayesian statistics and Markov chain Monte Carlo MCMC methods. The ratio compares the marginal likelihood of the data under the two sub-clusters with the marginal likelihood under the original cluster, with probability min(1,H S ). The formula is as follows:

[0106]

[0107] Where α is a constant used to control the frequency of the splitting operation; Γ is the gamma function; N k Represents the number of samples in cluster k; N k,j represents the number of samples in sub-cluster j; X k represents the set of all samples in cluster k; X k,j represents the set of all samples in sub-cluster j; f x is the marginal likelihood function; λ represents the NIW hyperparameter. The numerator calculates the two sub-clusters X after the split k,1 and Xk,2 The denominator calculates the clustering X before the split k The marginal likelihood function of .

[0108] If H S If H is greater than 1, the split operation is accepted because the split model can better explain the data. S When it is less than 1, the proposal will be H S The proportion of is accepted. This mechanism ensures the stability and rationality of the algorithm to avoid the algorithm falling into the situation of local optimal solution. In this process, the marginal likelihood function is used to evaluate the distribution of data in the current cluster: if cluster k is split into two sub-clusters, then what is the distribution of the combined data of these two sub-clusters. The marginal likelihood function helps us determine whether splitting a cluster into two sub-clusters can more effectively explain the data, thereby guiding clustering decisions.

[0109] In the merge operation, the algorithm considers merging each cluster with its three nearest neighbor clusters. When merging, it must ensure that no merge proposals are accepted at the same time, such as merging clusters k1 and k2, and merging clusters k2 and k3, which would cause the three clusters k1, k2, and k3 to be mistakenly merged into one. Therefore, merge proposals cannot be processed in parallel, but must be done step by step. The acceptance probability of a merge proposal is given by a Hastings ratio H m Determine, this ratio is the Hastings ratio H of the splitting scheme S The reciprocal of . The formula is as follows:

[0110] H m =1 / H s (7)

[0111] If a merge proposal is accepted, the two clusters will be merged and a new sub-clustering network will be initialized. The last layer of units of a merged cluster will be removed, and its weight with the previous hidden layer will also be removed, and the network structure will be adjusted. The parameters and weights of the newly formed cluster will be initialized using the weighted maximum a posteriori MAP estimate. This also means that when processing the merged clusters, the parameters of the clustering model learned before the cluster merge, such as weights, means, covariances, etc., will be considered. These parameters are learned from the previous clusters during the process of training the model through the dynamic architecture. During the merging process, these learned parameters can be used through weighted MAP estimation, so that the newly formed clusters can better reflect the distribution characteristics of the data, thereby improving the performance and accuracy of the model.

[0112] The loss function used in the clustering process: A new loss function is defined to measure the clustering network f clThe difference between the generated soft assignment result and the E-step probability. The formula is as follows:

[0113]

[0114] where r i is the soft clustering assignment of the i-th sample; The E-step of the EM-GMM algorithm calculates the probability that the i-th sample belongs to each cluster K. The E-step is a step in the standard EM-GMM algorithm, also known as the expectation step, which mainly uses the current parameter estimate to calculate the probability that each sample belongs to each Gaussian distribution. The purpose of using the E-step here is to calculate the probability that each sample x i The probability of belonging to each cluster K The calculation formula is as follows:

[0115]

[0116] where π k is the prior probability, i.e., weight, which indicates the proportion of the cluster in all samples; μ k and∑ k are the mean vector and covariance matrix respectively; Ν(x i ;μ k ,Σ k ) represents the probability density function; the numerator calculates the sample x i The denominator is a normalization factor for all categories k′. The sum of the probability density functions of all clusters k′ is calculated to ensure that the sum of all probabilities is 1. In general, E-step is based on the current clustering parameter (π k ,μ k ,Σ k ) calculates the probability that each sample belongs to each cluster.

[0117] The purpose of the loss function is to encourage the clustering network f cl Generate the soft allocation calculated in the E-step Similar results. This loss function is based on KL divergence. By calculating the soft distribution r of the clustering network output i Soft assignment with the E-step calculation of the EM algorithm The goal is to hope that the output of the network can reflect more accurate category assignments, thereby improving the clustering effect.

[0118] The sub-clustering network and the clustering network use different methods for loss calculation. The equal distance loss is used in the sub-clustering network, and the formula is as follows:

[0119]

[0120] This formula measures the sample v i The distance between the center of the cluster to which it belongs. The norm is the square of the Euclidean distance. It indicates the probability that sample i belongs to center j in category k. The higher the value, the better the sample v i The more likely it is to belong to center j. i represents the i-th sample. Represents the center of the j-th cluster of the k-th class, that is, the mean.

[0121] The overall loss function The purpose is to evaluate the proximity of sample points to their corresponding cluster centers through a combination of weights and distances, in the hope of minimizing the distance from each sample to the cluster center of its class, while considering the weights or importance of different samples in different clusters, so as to improve the aggregation of samples in different categories of the model.

[0122] In step S4, the final clustering result is visualized using the t-SNE algorithm to intuitively show the changing rules of the power system operation mode under different permeability backgrounds. The t-SNE algorithm is an effective and cutting-edge nonlinear dimensionality reduction algorithm for visualization, which has been successfully applied to many fields, such as medical image visualization for disease diagnosis. Its basic idea is to embed samples in high-dimensional space into low-dimensional space while trying to maintain the local similarity relationship of samples in the high-dimensional space.

[0123] The t-SNE algorithm defines the similarity of different operation modes in high-dimensional space and low-dimensional space to represent the distribution of operation modes. In high-dimensional space, any two high-dimensional operation mode vectors p i ′ and p′ j The similarity S between ij It is expressed by the conditional probability of Gaussian distribution:

[0124]

[0125] σ i represents the variance of the data, and M represents the total number of data points.

[0126] In low-dimensional space, any two low-dimensional operation mode vectors y i and j The similarity q between ij It is expressed by the conditional probability of t distribution:

[0127]

[0128] t-SNE minimizes the difference between high-dimensional space and low-dimensional space to find the best low-dimensional representation. This difference is measured by KL divergence. Then the gradient descent method is used to minimize the KL divergence and update the position of the data point in the low-dimensional space so that the distribution of the high-dimensional and low-dimensional operation modes is as close as possible to achieve the purpose of dimensionality reduction. The KL divergence formula is as follows:

[0129]

[0130] In formula (13), S ij represents the similarity between any two high-dimensional operation mode vectors, q ij Represents the similarity between any two low-dimensional operation mode vectors;

[0131] After multiple iterations of the algorithm convergence, the final low-dimensional representation can be used for data visualization. Low-dimensional data is usually two-dimensional or three-dimensional. Here, three-dimensional is selected and a scatter plot is used to visualize the clustering of power system operation modes. For details, see Figure 3-Figure 5 , obtained Figure 3-Figure 5 Figure 1 shows the clustering results of the power system operation modes with penetration rates below 15%, between 15% and 30%, and above 30%, respectively. 3D Cluster Visualization represents 3D cluster visualization. Feature 1, Feature 2, and Feature 3 are the dimensional positions in the 3D space after t-SNE dimensionality reduction. t-SNE maps data points in high-dimensional space to three-dimensional space, so that similar data points in high-dimensional space also remain close in three-dimensional space. Feature 1 represents the x-axis coordinate of the data after dimensionality reduction, which is used to show the position of one dimension of the data in the three-dimensional space. Feature 2 represents the y-axis coordinate of the data after dimensionality reduction, which is used to show the position of the second dimension of the data in the three-dimensional space. Feature 3 represents the z-axis coordinate of the data after dimensionality reduction, which is used to show the position of the third dimension of the data in the three-dimensional space. It can be seen from the figure that the samples of the same cluster are close in the low-dimensional space, while the samples of different clusters are separated, which shows that the method used can effectively map the high-dimensional data distribution to the low-dimensional space. Furthermore, the number of operating modes will increase as the penetration of renewable energy sources increases, and so will decentralization.

[0132] The dynamic clustering method DeepDPM proposed in the present invention combines a deep neural network with a Dirichlet process Gaussian mixture model DPGMM. It adopts a split / merge framework and a dynamic architecture, and can dynamically adjust the number of clusters K during the learning process. Ultimately, each cluster represents a specific operating mode. First, the autoencoder AE in the deep learning model is applied to learn the potential representation of the data, and the features are input into the subsequent clustering model for clustering. DPGMM is a Bayesian non-parametric model, a specific case of the Dirichlet process mixture model DPM, and can also be said to be a specific implementation of DPM, which is used for clustering when the number of clusters K of data points is unknown. Unlike the traditional Gaussian mixture model GMM, DPGMM can adaptively infer the number of clusters K and will not be affected by the wrong K value, thereby avoiding the problem of needing to pre-specify the number of clusters, overcoming the limitations of existing parameter methods, and improving the efficiency and accuracy of clustering.

Claims

1. A method for characterizing the operation mode of an industrial system based on a dynamic clustering algorithm DeepDPM, characterized in that: The method comprises: S1. Collect the operation data of the industrial system, pre-process the data and divide it into training set and validation set according to a fixed ratio; S2. Establish an AE model, use the training set to pre-train the AE, save the AE model and weights after training, use the validation set to verify the effect of the trained AE model, and then input the pre-processed data into the trained AE model for feature extraction; S3, input the extracted feature data into the DeepDPM model for clustering; S4. Finally, a dimensionality reduction visualization algorithm is used: t-distributed stochastic neighbor embedding, abbreviated as t-SNE, to visualize the clustering results.

2. According to claim 1, a method for characterizing the operation mode of an industrial system based on the dynamic clustering algorithm DeepDPM, characterized in that: The data preprocessing refers to filling in missing data and aligning all data.

3. The method for characterizing the operation mode of an industrial system based on the dynamic clustering algorithm DeepDPM according to claim 1 is characterized in that: The detailed steps of step S2 are as follows: S21. Establish an AE model: Set the first layer of the AE model as the input layer. Next, set the number of hidden layers of the encoder part to 3, and the number of neurons in each hidden layer of the encoder part is 500, 500, and 2000 respectively; set the dimension of the middle latent space layer to 50; set the decoder part with 3 hidden layers, and the number of neurons in each hidden layer of the decoder part is 2000, 500, and 500 respectively; finally set the output layer; S22. First, use the training set to pre-train the AE: input the training set into the constructed AE model, and set the parameters learning_rate=0.002 and batch_size=256; The mean square error MSE loss function is used during training, and its formula is as follows: In formula (1), y i is the input data, is the data output after the AE model, and n is the total number of samples; Then, the adaptive moment estimation optimizer Adam is selected. The adaptive moment estimation optimizer Adam uses the first-order moment estimation and second-order moment estimation of the gradient to adaptively adjust the learning rate and converge faster. Finally, the weights and bias parameters of the AE model after training are saved. S23. Use the validation set to verify the pre-trained AE model, and use the same MSE loss function as the pre-training as the evaluation indicator. If the final loss value of the validation set is close to the final loss value of the training set, it means that the performance of the model on the training set and the validation set is relatively consistent, indicating that the AE model has good generalization ability; S24. Use AE model for feature extraction: The main feature extraction only uses three main parts: input layer, encoder part and latent space layer. All data samples are input into the trained AE model, pass through the encoder part, and finally the latent space layer outputs the extracted feature data.

4. The method for characterizing the operation mode of an industrial system based on the dynamic clustering algorithm DeepDPM according to claim 1 is characterized in that: The specific steps of S3 are as follows: S31. Calculate soft cluster assignment: The feature data is first passed to the clustering network, which is a multi-layer perceptron MLP with one hidden layer and 50 fixed neurons. The last layer has K neurons, corresponding to K clusters. The clustering network generates K soft cluster assignment probability vectors for each sample, and then the soft cluster assignment probability vectors of all samples form a matrix R, where each row represents the soft cluster assignment probability vector of a sample, and each column represents the probability of belonging to a cluster: In formula (2), the input is the set of all N samples x, and the output is the K soft clustering assignments for each sample, expressed as Each r i is a K-dimensional vector, indicating the soft clustering assignment of the i-th sample; N represents the total number of samples; r i,k represents the probability that the i-th sample is assigned to the K-th cluster, called "responsibility", and all r i,k The sum of is 1; S32. Calculate hard allocation: Calculate hard allocation to explicitly allocate samples to a specific cluster: Based on the soft cluster allocation result R, calculate the hard allocation z of each sample: In formula (3), z i Indicates the actual cluster label assigned to the i-th sample, i.e. z i = argmax k r i,k ; S33. After obtaining the hard assignment, a sub-clustering network is created for each cluster. The sub-clustering network is a multi-layer perceptron MLP with one hidden layer, 50 neurons fixed, and 2 neurons in the last layer, corresponding to 2 sub-clusters; the sub-clustering network is for each sample v belonging to cluster k. i Generate 2 soft clustering probability vectors, and then form a matrix with the soft clustering probability vectors of all samples belonging to the kth cluster Each row represents the soft clustering assignment probability vector of a sample, and each column represents the probability of belonging to a cluster: In formula (4), the input is the sample set v of each cluster k k =(v i ) i:zi＝k , the output sub-cluster soft assignment is expressed as Each sample v i The sum of all sub-cluster probabilities of is 1; where, is a matrix of size N×2, consisting of the soft cluster assignment probability vectors of all samples in the kth cluster; represents the soft clustering assignment of the i-th sample, Represents sample v i The probability of being assigned to the jth cluster, v k represents all samples in the kth cluster; S34. Dynamic adjustment during training: The DeepDPM model will dynamically adjust the number of clusters K during training, propose split or merge operations, and accept or reject these operations based on a certain probability.

5. The method for characterizing the operation mode of an industrial system based on the dynamic clustering algorithm DeepDPM according to claim 4 is characterized in that: The dynamic adjustment during the training process is as follows: In the split operation, the algorithm will propose a splitting scheme to split each cluster into two sub-clusters at regular intervals. The acceptance probability of the splitting scheme is given by a Hastings ratio H. S The ratio compares the marginal likelihood of the data under the two sub-clusters with the marginal likelihood under the original cluster, with probability min(1,H S ), H S The formula is as follows: In formula (5), α is a constant used to control the frequency of the splitting operation; Γ is the gamma function; N k Represents the number of samples in cluster k; N k,j represents the number of samples in subcluster j, j∈{1,2}; X k represents the set of all samples in cluster k; X k,j represents the set of all samples in sub-cluster j; f x is the marginal likelihood function; λ represents the NIW hyperparameter; the numerator calculates the two sub-clusters X after the split k,1 and X k,2 The denominator calculates the clustering X before the split k The marginal likelihood function of ; If H S is greater than 1, the split operation is accepted; when H S When less than 1, the split will be H S proportion is accepted; In the merge operation, the algorithm merges each cluster with its three nearest neighbor clusters. The merge cannot be processed in parallel but is performed step by step. The acceptance probability of the merge is given by a Hastings ratio H. m Determine that the ratio is the Hastings ratio H of the split S The reciprocal of is as follows: H m =1 / H s (6) If the merge proposal is accepted, the two clusters will be merged and initialized into a new sub-clustering network; the last layer units of the merged cluster will be removed, and the weights of the last layer units of the merged cluster and the previous hidden layer will also be removed; the parameters and weights of the newly formed sub-clustering network will be initialized using weighted maximum a posteriori MAP estimation.

6. The method for characterizing the operation mode of an industrial system based on the dynamic clustering algorithm DeepDPM according to claim 4 is characterized in that: The loss function used in the clustering network is a newly defined loss function used to measure the clustering network f cl The difference between the generated soft assignment result and the E-step probability is given by the following formula: In formula (7), r i is the soft clustering assignment of the i-th sample; It is the probability that the i-th sample belongs to each cluster K calculated by the E-step of the EM-GMM algorithm; The E-step is a step in the standard EM-GMM algorithm, called the expectation step. The E-step is used to calculate each sample x i The probability of belonging to each cluster K The calculation formula is as follows: where π k is the prior probability, i.e., weight, which indicates the proportion of the cluster in all samples; μ k and∑ k are the mean vector and covariance matrix respectively; Ν(x i ;μ k ,Σ k ) represents the probability density function; the numerator calculates the sample x i The denominator is a normalization factor for all categories k′. The sum of the probability density functions of all clusters k′ is calculated to ensure that the sum of all probabilities is 1. In general, E-step is based on the current clustering parameter (π k ,μ k ,Σ k ) Calculate the probability that each sample belongs to each cluster; The sub-clustering network uses an isometric loss to measure the sample v i The distance between the sub-cluster center and the sub-cluster center to which it belongs is expressed as follows: In formula (9), l2 represents the l2 norm, that is, the square of the Euclidean distance; Represents sample v i The probability of belonging to center j in cluster k, the higher the value, the better the sample v i The more likely it is to belong to center j; v i represents the i-th sample; Represents the center of the j-th sub-cluster of cluster k, that is, the mean.

7. The method for characterizing the operation mode of an industrial system based on the dynamic clustering algorithm DeepDPM according to claim 1 is characterized in that: In step S4, the basic idea of ​​the t-SNE algorithm is to embed the data points in the high-dimensional space into the low-dimensional space while trying to maintain the local similarity relationship of the data points in the high-dimensional space: The t-SNE algorithm defines the similarity of different operation modes in high-dimensional space and low-dimensional space to represent the distribution of operation modes: in high-dimensional space, any two high-dimensional operation mode vectors p′ i and p′ j The similarity S between ij It is expressed by the conditional probability of Gaussian distribution: In formula (10), σ i represents the variance of the data, and M represents the total number of data points; In low-dimensional space, any two low-dimensional operation mode vectors y i and j The similarity q between ij It is expressed by the conditional probability of t distribution: In formula (11), y m and n represents the mode vector in the low-dimensional space, which is used to calculate q ij Other low-dimensional data points used to normalize the denominator when ; t-SNE minimizes the difference between high-dimensional space and low-dimensional space to find the best low-dimensional representation. This difference is measured by KL divergence, and then the gradient descent method is used to minimize KL divergence and update the position of data points in the low-dimensional space to make the distribution of high-dimensional and low-dimensional operation modes as close as possible to achieve the purpose of dimensionality reduction. The KL divergence formula is as follows: In formula (12), S ij represents the similarity between any two high-dimensional operation mode vectors, q ij Represents the similarity between any two low-dimensional operation mode vectors; The final low-dimensional representation is used for data visualization, and a scatter plot is used to visualize the clustering of operating modes.

Images