A Feature Extraction and Selection Method for Multi-Source Heterogeneous Data in a Power System

Abstract

The present invention belongs to the cross - field of power systems and artificial intelligence, and is a method for feature extraction and selection of multi - source heterogeneous data in a power system. The method includes using the multi - source heterogeneous data in the power system as input data to construct a training data set; designing neural networks with different structures for each group of multi - source heterogeneous data, training the auto - encoder model using a layer - by - layer training algorithm to obtain a trained stacked auto - encoder model, and extracting the encoded features of each group of multi - source heterogeneous data through the trained stacked auto - encoder model; constructing a fusion - layer network to fine - tune the parameters of the entire stacked auto - encoder model; performing sparsification processing on the obtained homogeneous features, calculating the weights of each feature dimension, and screening out the features with higher weights. The present invention can comprehensively mine data features, and at the same time can reflect the semantics of actual data. The selected features can support task requirements, greatly improving the completion degree of actual tasks.

Description

Technical Field

[0001] The present invention belongs to the cross - field of power systems and artificial intelligence, and particularly relates to a method for feature extraction and selection of multi - source heterogeneous data in power systems. Background Art

[0002] With the rapid development of information technologies such as computers, networks, and databases, and their widespread application in various fields of society, the informatization process of all walks of life in society has been accelerated, especially in the power industry. In the power industry, with the in - depth implementation of the smart grid construction and the extensive use of intelligent sensing devices, the amount of data shows an explosive growth trend, and the power industry has entered the big data era. The State Grid has many business systems, including enterprise management systems such as various information systems like ERP, MES, and CRM. These information systems have different development cycles and different developers, with diverse product architecture, different coding data structures, different front - end functions, and different underlying databases. The data from various sensing devices and various information systems form multi - source heterogeneous power data, which is not conducive to information sharing and the exploration of potential data value.

[0003] The hidden value of multi - source heterogeneous data can be obtained through data analysis means. Due to its heterogeneity, people need to perform feature extraction to represent it in a form available for computational analysis. Common feature extraction methods include manually designing extraction rules, linear mapping, non - linear mapping, etc. Manually designing extraction rules is to design rules according to the characteristics of the data structure for data transformation processing. Linear mapping includes principal component analysis and linear discriminant analysis, etc., which map high - dimensional data to a low - dimensional space. These methods all have certain limitations, such as local feature information, high computational complexity, and inability to reflect semantic information, etc.

[0004] During the operation of the power system, there are many task requirements, such as equipment fault diagnosis, fault prediction, health status assessment, etc. The completion of these tasks requires the support of power data. How to mine the key information of the data from the vast amount of multi - source heterogeneous data and use it to meet the requirements is a major difficulty. Existing feature extraction technologies such as manual extraction and linear mapping have certain limitations, such as one - sided feature extraction and complex calculation. Summary of the Invention

[0005] To solve the technical problems existing in the prior art, the present invention provides a method for feature extraction and selection of multi - source heterogeneous data in power systems, which can more comprehensively mine data features and at the same time can reflect the semantics of actual data. The selected features can support task requirements and greatly improve the completion degree of actual tasks.

[0006] The present invention can be achieved by adopting the following technical solutions:

[0007] A method for feature extraction and selection of multi-source heterogeneous data in a power system, the method comprising:

[0008] S1. Using the multi-source heterogeneous data in the power system as input data to construct a training data set;

[0009] S2. Designing neural networks with different structures for each group of multi-source heterogeneous data, training the autoencoder model using a layer-by-layer training algorithm to obtain a trained stacked autoencoder model, and extracting the encoded features of each group of multi-source heterogeneous data through the trained stacked autoencoder model;

[0010] S3. Using the encoded features of each group of multi-source heterogeneous data as the input data of the stacked autoencoder model, constructing a fusion layer network, eliminating the heterogeneity of the encoded features of the multi-source heterogeneous data to obtain a homogeneous feature representation, and fine-tuning the parameters of the entire stacked autoencoder model;

[0011] S4. Sparsifying the obtained homogeneous features, calculating the weights of each feature dimension, and screening out the features with higher weights.

[0012] In the preferred technical solution, the step S2 specifically includes the steps of:

[0013] Constructing n heterogeneous stacked autoencoders according to the input n groups of multi-source heterogeneous data;

[0014] When training each hidden layer of the nth stacked autoencoder, for the current input heterogeneous data, performing a non-linear transformation through a weight matrix and an activation function in the hidden layer to obtain an output hidden representation;

[0015] Decoding the hidden representation, reconstructing and obtaining a reconstructed output through the transformation of the weight matrix and the activation function, and using the gradient descent method to solve the error between the original input and the reconstructed output; when the error is 0, using the reconstructed output as the original input of the next layer for retraining to obtain a stacked autoencoder model.

[0016] The specific steps of the step S3 are as follows:

[0017] Using a feedforward neural network as the fusion layer network, performing feature fusion on multiple groups of multi-source heterogeneous data in the fusion layer network, and connecting the feedforward neural network to the stacked autoencoder network of each group of multi-source heterogeneous data;

[0018] The fusion layer network is externally connected to a softmax classifier to calculate the label class probability of the input vector;

[0019] Using the gradient descent method to fine-tune the parameters of the stacked autoencoder model.

[0020] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0021] The present invention provides a method for feature extraction and selection of multi-source heterogeneous data in a power system. By using deep learning methods to construct an artificial neural network, different-structured neural networks are designed for each group of multi-source heterogeneous data, and a stacked autoencoder model is trained using a layer-by-layer training algorithm to achieve feature extraction and selection of multi-source heterogeneous data in the power system. It can comprehensively mine data features, reflect the semantics of actual data at the same time, and the selected features can support task requirements, greatly improving the completion degree of actual tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0023] Figure 1 is a flowchart of the method for feature extraction and selection of multi-source heterogeneous data in the power system in the embodiment of the present invention;

[0024] Figure 2 is a structural block diagram of the stacked autoencoder in the embodiment of the present invention;

[0025] Figure 3 is a flowchart of the stacked autoencoder algorithm program in the embodiment of the present invention;

[0026] Figure 4 is a structural diagram of the fusion layer in the example of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The following will further describe the technical solutions of the present invention in detail in combination with the drawings and embodiments. Obviously, the described embodiments are some but not all of the embodiments of the present invention. The embodiments of the present invention are not limited thereto. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0028] Embodiment 1:

[0029] As Figure 1 shown, the specific implementation of the present invention is a method for feature extraction and selection of multi-source heterogeneous data in a power system, including the following steps:

[0030] S1. Introduce multi-source heterogeneous data in the power system as input data to construct a training dataset. The multi-source heterogeneous data includes voltage, current, active power output, switch status, etc. detected by each sensor, and also includes audio data and image data brought by the audio-visual monitoring system, as well as text records of system operation.

[0031] The input data is specifically defined as:

[0032] X = [x1, x2, …, x n

[0033] x n = {data1, data2, …, data m}

[0034] where X represents the collection of multi-source heterogeneous data, x n represents the nth group of multi-source heterogeneous data, and data m represents the data of the mth dimension of the nth group of multi-source heterogeneous data. Each group of multi-source heterogeneous data has different dimensions due to heterogeneity.

[0035] S2. Design neural networks with different structures for each group of multi-source heterogeneous data through a stacked autoencoder model, train the autoencoder model using a layer-by-layer training algorithm to obtain a trained stacked autoencoder model, and extract the encoded features of each group of multi-source heterogeneous data through the trained stacked autoencoder model.

[0036] An autoencoder is a neural network model whose function is to perform representation learning on the input information by taking the input information as the learning target. A stacked autoencoder is to stack multiple autoencoders and take the output of the hidden layer of each autoencoder as the input of the second autoencoder to increase the representation ability of the model. As Figure 2 shown, the stacked autoencoder model consists of multiple layers of neural networks, that is, the input x undergoes non-linear transformations through 3 hidden layers h 1 , h 2 , h 3 and finally obtains the reconstructed output layer h 4 of the 4th layer, whose meaning and structure are the same as those of the input x. It should be noted that the neural network structure here is only an example and does not represent that all neural network structures of autoencoders are like this. As Figure 3 shown, it is the algorithm program flow chart of the stacked autoencoder.

[0037] Step S2 specifically includes the steps:

[0038] ​Based on the input of n groups of multi-source heterogeneous data, n heterogeneous stacked autoencoders are constructed. That is, the number of hidden layers and neuron nodes is different. The number of hidden layers in the neural network of the nth group of heterogeneous data is defined as m n , the ith hidden layer is denoted as The connection weight between the ith hidden layer and the previous layer is denoted as

[0039] When training each hidden layer of the nth stacked autoencoder, for the current input heterogeneous data x n , it undergoes a non-linear transformation via the weight matrix W1 and the activation function f(·) in the hidden layer to obtain the output hidden representation:

[0040] h = f(W1x n + c)

[0041] where h is the output hidden representation, c is the bias term, and the activation function f(·) is the sigmoid function.

[0042] The hidden representation is decoded, and the reconstructed output is obtained through the transformation of the weight matrix and the activation function. The gradient descent method is used to solve the error between the original input and the reconstructed output; when the error is 0, the reconstructed output is used as the original input of the next layer for retraining to obtain the stacked autoencoder model.

[0043] Specifically, the hidden representation is decoded, and it is reconstructed through the weight matrix W2 and the activation function to obtain the reconstructed output:

[0044]

[0045] where is the reconstructed output of the autoencoder, and b is the bias term.

[0046] The error between the original input and the reconstructed output is as follows:

[0047]

[0048] where x is the original input, is the reconstructed output of the autoencoder.

[0049] This error is the optimization objective, and the gradient descent method is used to solve it. After the partial derivatives are solved, the weights and biases can be updated:

[0050]

[0051]

[0052] where lr is the learning rate. Through continuous updates of the weights and biases, the reconstruction error is made to be 0, i.e., the optimization is completed.

[0053] After the optimization is completed, the reconstructed output of this layer can be used as the original input of the next layer for retraining, and finally a stacked autoencoder model is obtained. By inputting each group of multi-source heterogeneous data into the corresponding stacked autoencoder model, the corresponding feature encoding output can be obtained for use in subsequent computing tasks. The feature expressions of multi-source heterogeneous data are extracted through this model.

[0054] S3. Use the encoded features of each group of multi-source heterogeneous data as the input data of the stacked autoencoder model, construct a fusion layer network, eliminate the heterogeneity of the encoded features of multi-source heterogeneous data to obtain isomorphic feature expressions, and fine-tune the parameters of the entire stacked autoencoder model.

[0055] Step S3 specifically includes the following steps:

[0056] S31. Use a feedforward neural network as the fusion layer network. In the fusion layer network, perform feature fusion on multiple groups of multi-source heterogeneous data. This feedforward neural network is connected to the stacked autoencoder network of each group of multi-source heterogeneous data, and the weight is T n , and the weights are shared to eliminate the heterogeneity and strong correlation of the extracted features. As Figure 4 shown, the structure diagram of the fusion layer and the stacked autoencoder.

[0057] S32. The fusion layer network is externally connected to a softmax classifier to calculate the label category probability of the input vector h n . The sofxmax classifier can calculate the scores of each label category and map all the scores into a probability value, that is, the classification probability. Define the neuron at the top layer of the stacked autoencoder of the nth group of multi-source heterogeneous data as h n , and the label information of the fusion layer is p. The loss function can be defined as follows:

[0058]

[0059] where n represents the number of groups of multi-source heterogeneous data, M represents the number of training samples, b is the bias term, y (i) represents the label of the sample x (i) , Y represents the probability event in the conditional probability density function, and Y = y (i) means that the probability event at this time is y (i) . represents the top-layer output of the nth sub-network for the input x (i) . For a k-classification task, the probability that the input vector h n belongs to the label category i is:

[0060]

[0061] Among them, b i , b l represents the bias vector, represents the l-th row vector of the weight matrix T, T i represents the i-th row vector of the weight matrix T.

[0062] S33. Use the gradient descent method to fine-tune the parameters of the stacked autoencoder model, and the entire network adopts supervised fine-tuning. Preferably, iterate and adjust the parameters of each stacked encoder in turn. Each time, adjust one of the models and fix the parameters of the other model networks until the parameters of all stacked autoencoder models are adjusted.

[0063] According to the model loss function obtained in the previous step, the optimization objective is as follows:

[0064]

[0065] Solve using the gradient descent method. After the partial derivatives are solved, the parameters of the model can be updated, that is, the weights W1, W2 and biases b, c shown in the above steps.

[0066] S4. Use the structured sparsity method to sparsify the obtained homogeneous features, calculate the weights of each feature dimension, and select the features with higher weights.

[0067] First, define the data of the homogeneous features, and then perform formula expression and calculation according to the defined variables.

[0068] Specifically, define the homogeneous feature expression obtained in step 3 containing a p-dimensional feature vector, is the label, X = (x 1 , x 2 , …, x n ) represents the input training data matrix, Y = (y 1 , y 2 , …, y n ) represents the label matrix; set the p-dimensional feature vector to be divided into k feature groups, K j represents the number of feature dimensions of the j-th group; β l = (β l1 , β l2 , …, β lj ) represents the weight coefficient vector for the l-th category, β lj represents the sub-coefficient vector corresponding to the j-th group.

[0069] Through the objective function feature selection for the l-th category, obtain the features with non-zero weights, the objective function

[0070]

[0071] Among them, is the loss function, is the regularization term, and the loss function is the same as the loss function in step 3.2:

[0072]

[0073] The regularization term is described as follows:

[0074]

[0075] Among them, λ1 and λ2 are regularization term coefficients, and the hyperparameter ω j is the weight of the jth feature group, and y (i) represents the label of the sample input x (i) . T is the weight matrix, and b is the bias term. This regularization term includes two parts: the L1 norm as the penalty term and the L2 norm for the sparsification effect. Through the regularization term, the feature vectors in the same group and the feature vectors in each group can simultaneously produce a sparsification effect, making the weights of certain predetermined feature dimensions zero, so as to obtain the features with non-zero weights after feature selection.

[0076] It should be noted that the features here are the encoded feature data obtained after being extracted by the stacked autoencoder model, which can be used for the computing tasks of the computer and cannot correspond to the feature attributes in the original data such as voltage and current. The weight β l is the weight coefficient of each feature in the process of optimizing the objective function, and it is continuously updated during the optimization process after initialization.

[0077] After obtaining the selected features, the features can be input into the task model to verify the effect according to the actual task requirements. In this embodiment, taking the fault classification task as an example, the multi-source heterogeneous data of the power system such as switch status, active and reactive power output, text records, images, etc. are used to implement feature extraction and selection of the data through the above steps; the selected features are input into the sofxmax classifier for calculation, and the probability of equipment failure can be obtained, which can greatly improve the accuracy of the task.

[0078] It should be understood that the parts not elaborated in this specification all belong to the prior art.

[0079] The above embodiments are the preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and shall be included in the protection scope of the present invention.

Claims

1. A method for feature extraction and selection of multi-source heterogeneous data in a power system, characterized in that It includes the following steps: S1. Introduce multi-source heterogeneous data in the power system as input data and construct a training data set; S2. Design neural networks with different structures for each group of multi-source heterogeneous data through a stacked autoencoder model, train the autoencoder model using a layer-by-layer training algorithm to obtain a trained stacked autoencoder model, and extract the encoded features of each group of multi-source heterogeneous data through the trained stacked autoencoder model; S3. Use the encoded features of each group of multi-source heterogeneous data as the input data of the stacked autoencoder model, construct a fusion layer network, eliminate the heterogeneity of the encoded features of the multi-source heterogeneous data to obtain a homogeneous feature representation, and fine-tune the parameters of the entire stacked autoencoder model; The specific steps of step S3 are as follows: S31. Use a feedforward neural network as the fusion layer network, perform feature fusion on multiple groups of multi-source heterogeneous data in the fusion layer network, and connect the feedforward neural network to the stacked autoencoder network of each group of multi-source heterogeneous data; S32. The fusion layer network is externally connected to a softmax classifier to calculate the label category probability of the input vector; S33. Use the gradient descent method to fine-tune the parameters of the stacked autoencoder model; The use of the gradient descent method to fine-tune the parameters of the stacked autoencoder model includes the steps of: iteratively adjusting the parameters of each autoencoder model in turn, adjusting one autoencoder model each time, and fixing the parameters of other autoencoder models until the parameters of all autoencoder models are adjusted; S4. Use a structured sparse method to sparsify the obtained homogeneous features, calculate the weights of each feature dimension, and select the features with higher weights to complete the feature extraction and selection of multi-source heterogeneous data; Step S4 includes: Define the data with isomorphic features and define the expression of isomorphic features , containing a p-dimensional feature vector, is the label, represents the input training data matrix, represents the label matrix; divide the p-dimensional feature vector into k feature groups, represents the j number of feature dimensions of the th group; represents the weight coefficient vector for the th category, j represents the sub-coefficient vector corresponding to the Through the objective function For the feature selection of the th category, features with non-zero weights are obtained. The objective function : ； Among them, is the loss function, is the regularization term; ； ； Among them, and are the coefficients of the regular terms, hyperparameters is the weight of the j-th feature group, represents the label of the sample input, T is the weight matrix, b is the bias term.

2. The feature extraction and selection method for multi-source heterogeneous data of a power system according to claim 1, wherein The multi-source heterogeneous data includes the voltage, current, active power output, and switch status detected by each sensor, and also includes audio data, image data in the audio-visual monitoring system, and text records of the operation of the power system.

3. The feature extraction and selection method for multi-source heterogeneous data of a power system according to claim 1, characterized in that The stacked autoencoder is a stack of multiple autoencoders, and the output of the hidden layer of each autoencoder is used as the input of another connected autoencoder.

4. A method for feature extraction and selection of multi-source heterogeneous data in a power system according to claim 3, characterized in that, The specific steps of step S2 specifically include the steps of: According to the input n groups of multi-source heterogeneous data, construct n heterogeneous stacked autoencoders; When training each hidden layer of the nth stacked autoencoder, for the current input heterogeneous data, perform a non-linear transformation through a weight matrix and an activation function in the hidden layer to obtain an output hidden representation; Decode the hidden representation, reconstruct the output through the transformation of the weight matrix and the activation function, and use the gradient descent method to solve the error between the original input and the reconstructed output; when the error is 0, use the reconstructed output as the original input of the next layer for training again to obtain a trained stacked autoencoder model; Input each group of multi-source heterogeneous data into the trained stacked autoencoder model to obtain the corresponding feature coding output.

5. A method for feature extraction and selection of multi-source heterogeneous data in a power system, characterized in that, The output hidden representation is: ； where h is the output hidden representation, represents the nth group of multi-source heterogeneous data, c is the bias term, is the weight matrix, is the sigmoid function; The reconstructed output is: ； where, is the reconstruction output of the autoencoder, b is the bias term, is the weight matrix and is the activation function.

Images