A power equipment fault detection model based on the attention mechanism combined with GRU

Abstract

The present invention discloses an electric power equipment fault detection model based on an attention mechanism combined with GRU. The electric power equipment fault detection model includes a classification neural network model. The training data of the classification neural network model comes from a preprocessing model. The preprocessing model converts the input unbalanced electric power equipment data into balanced data and performs embedding representation, and outputs intermediate data: a historical state sequence based on the representation of the electric power equipment, an embedded representation of the label data, and an embedded representation of the portrait features of the electric power equipment; then the temporal and spatial features of the electric power equipment are extracted from the historical state sequence through a GRU module; the state sequence features are extracted from the output of the GRU module through an attention mechanism module; the environmental information of the electric power equipment is extracted from the embedded representation of the portrait features of the electric power equipment through a graph attention mechanism module; and the state sequence features, the embedded representation of the label data and the environmental information are aligned and fused as the training data input of the classification neural network.

Description

Technical Field

[0001] The present invention belongs to the technical field of power equipment fault detection, relates to a power grid fault detection method, and in particular to a power equipment fault detection model based on an attention mechanism combined with a GRU. Background Art

[0002] In an era of rapid technological advancement and continuous economic optimization, power generation faces significant challenges. With the growing number of electricity users and businesses, especially in regions with a high proportion of industrial development, the demand for power supply is increasing. When power supply equipment fails in these areas, industrial equipment can be permanently disabled, leading to a series of serious consequences. Therefore, automated fault detection for power equipment plays a vital role in power supply systems. Existing traditional power equipment fault detection methods are complex and inefficient. Furthermore, existing models fail to fully utilize unbalanced power fault data, significantly reducing their practical value.

[0003] Therefore, how to propose a training model and method for a power equipment fault detection model to achieve better performance in optimization, identification, and classification tasks and improve the detection accuracy of the model is a technical problem that needs to be solved urgently by technical personnel in this field. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of the present invention is to propose a power equipment fault detection model based on the attention mechanism set GRU with reasonable design, sufficient data utilization and high detection accuracy.

[0005] To achieve the above objectives, the present invention provides an electric power equipment fault detection model based on an attention mechanism combined with GRU, wherein the electric power equipment fault detection model includes a classification neural network model, the training data of the classification neural network model comes from a preprocessing model, and the preprocessing model includes an upsampling module, a word embedding representation learning module, a GRU module, an attention mechanism module, and a graph attention mechanism module.

[0006] The up-sampling module is used to convert the input unbalanced power equipment data into balanced data;

[0007] The word embedding representation learning module is used to embed the balance data and output an embedding representation based on the historical state sequence of the power equipment representation, the label data embedding representation, and the power equipment portrait feature;

[0008] The GRU module is used to extract the temporal and spatial characteristics of the power equipment from the historical state sequence based on the power equipment representation output by the word embedding representation learning module;

[0009] The attention mechanism module is used to extract state sequence features from the temporal and spatial features of the device;

[0010] The graph attention mechanism module is used to extract the environmental information of the power equipment from the embedded representation of the power equipment portrait feature;

[0011] The state sequence features, the label data embedding representation and the environmental information are aligned and fused as training data input for the classification neural network.

[0012] Furthermore, the upsampling algorithm adopted by the upsampling module is the SC-SMOTE upsampling algorithm.

[0013] Furthermore, the SC-SMOTE upsampling algorithm specifically includes:

[0014] Step 21: Traverse the input power equipment data set to determine the majority class seed samples and the minority class seed samples;

[0015] Step 22: Based on the seed sample information, upsample the majority class and the minority class simultaneously, and calculate the number of samples generated by the seed sample of each minority class;

[0016] Step 23: After obtaining the number of samples generated by each minority class seed sample, perform linear interpolation to obtain the final new sample, merge the newly generated sample with the original seed sample to generate a balanced sample data set;

[0017] Step 24: Embed the data in the generated balanced sample dataset.

[0018] Furthermore, the step 21 includes: traversing the data of the power equipment dataset and using the KNN algorithm to determine the neighboring sample set D of the sample x. n , in the neighbor set D n In the example, the samples of the same category as the sample x are set D same , the sample set of different categories from sample x is called D other ; Compare D same The sample size and D other The number of samples is calculated according to the formula:

[0019]

[0020] Determine whether the sample x is a seed sample and add the seed sample label S to the original dataset.

[0021] Furthermore, the calculation of the number of generated samples in step 22 includes the following formula:

[0022] label_diff j =Nmaj -N j

[0023]

[0024]

[0025] Among them, label_diff j Represents the majority class and minority class C in the original data set j The sample size gap; N j Indicates that it belongs to category C j The number of samples; D s_j Indicates that it belongs to category C j A collection of seed samples; Rs j Represents the majority class seed sample and category C j The proportion of seed samples; N gj Represents each category C j The average number of new samples generated by the seed sample; label_diff j / |D s_j |Indicates category C j The number of seed samples that need to be generated to balance the number of original data.

[0026] Furthermore, the step 23 includes the following steps: after obtaining the number of samples generated by the seed sample, the K-means algorithm is used to update the coordinates of the cluster center at each iteration while dividing the sample according to the Euclidean distance between the cluster center and the sampled sample; wherein the hyperparameter k of the K-means algorithm is c Indicates the number of clusters, hyperparameter k c The value of depends on the ratio of the number of majority classes to the number of minority classes in the dataset, and the formula is expressed as:

[0027]

[0028] After clustering the data set using the K-means algorithm, each sample is labeled with the cluster label C, and the data set is updated as follows:

[0029]

[0030] Furthermore, the data processing method of the newly generated samples in step 23 is: filtering out samples of the same category from each cluster to form a sample set D c , each sample contains a feature set F = {f1,f2,…,f p}, and then, depending on the feature type, execute: when the feature is a discrete feature, during the data generation process, field selection is performed based on the probability distribution of the occurrence of different fields; when the feature is a continuous feature, during the data generation process, data is randomly selected in the [min, max] interval of the feature value as the generated value, where max and min are the maximum and minimum values of the feature value, respectively.

[0031] Furthermore, the method of obtaining the final new sample by linear interpolation in step 23 is:

[0032] For each seed sample x i , the corresponding category y i , the cluster c i , there is a corresponding number of new samples N that need to be generated gi , each time a new sample is generated, according to N gi And the distribution of each feature of the cluster FD[c i ][y i ], first generate auxiliary samples x temp , and then perform linear interpolation to obtain the final generated sample x new ;

[0033] Among them, the auxiliary sample x temp The construction needs to meet three rules:

[0034] Temporary sample x temp With the sample x i Belong to the same category label y i ;

[0035] Temporary sample x temp With the sample x i Belong to the same cluster c i ;

[0036] Temporary sample x temp With the sample x i Have the same features, but the eigenvalues of each feature are based on the cluster c i The characteristic distribution F[c i ][y i ] Random sampling is obtained;

[0037] Get a temporary sample x temp Then, the new sample x is obtained by linear interpolation. new :

[0038] x temp =[f1,f2,…,f p ],f p =Random(FD[c i ][y i][p])

[0039] X new =x+Random(0,1)×(x temp -x)

[0040] Cycle N of seed samples gj After the sample generation operation, a set of generated samples based on the seed sample is obtained. These generated samples belong to the same category as the seed sample. When each seed sample completes the sample generation, the generated sample set D is obtained. g Merge with the original dataset D to get the final balanced dataset D balance .

[0041] Furthermore, the data output of the classification neural network model is expressed as:

[0042]

[0043] Among them, y pred ∈{0,1,2},O b is the vector representation of the state sequence feature; e′ p is the state feature vector representation of the power equipment; e a is the vector representation of the predicted target, W deep ,b deep are the output layer parameters.

[0044] Furthermore, the vector representation of the state sequence feature is b Obtained by calculation through the GRU module and the attention mechanism module;

[0045] The data processing process and results of the GRU module are expressed as follows:

[0046] r t =σ(W r i t +U r h t-1 +b r )

[0047] z t =σ(W z i t +U z h t-1 +b z )

[0048]

[0049]

[0050] Among them, σ represents the Sigmoid function, ⊙ represents the Hadamard product, where n hid represents the hidden layer size of the GRU network, k represents the size of the vector after embedding, i t Represents the input of GRU, which represents the tth vector representation in the historical state sequence, i.e. t =e b [t]; the output value h of the GRU module t represents the tth hidden state, which is a potential expression of the past state of the power equipment;

[0051] The data processing process and results of the attention mechanism module are expressed as follows:

[0052]

[0053]

[0054] Among them, a t represents the attention score calculated by the attention distribution of the attention mechanism module; F[.] represents the attention scoring function.

[0055] Furthermore, the state characteristic vector of the power equipment is represented by e′ p Obtained through the calculation of the graph attention mechanism module; the data processing result of the graph attention mechanism module is expressed as:

[0056] e′ p =[e p ,e′ p1 ,e′ p2 ,…,e′ pn ,X]

[0057] Among them, e p The embedding vector of the power equipment portrait feature; e′ p1 ,e′ p2 ,…,e′ pn The embedding vector is e p Embedding vectors of related power equipment nodes of the power equipment; X is the coefficient set.

[0058] The present invention achieves the following beneficial effects:

[0059] 1. The present invention uses a time series feature capture technology based on the attention mechanism combined with GRU. In order to solve the problem that the time series feature information of the input data cannot be fully utilized, the present invention uses GRU combined with the attention mechanism to improve the feature extraction ability of the input samples in the time series and spatial structure, thereby capturing more equipment status features, and then inputting the obtained power equipment features into the neural network to predict the fault conditions of the power equipment. On the one hand, this effectively solves the problem of insufficient utilization of the input power equipment sample information. The difference between the present invention and the previous power equipment fault detection method is that the GRU network is used to extract the temporal and spatial feature information of the input sample data at the same time, and the attention mechanism is used to perceive the state sequence features of the input samples. By capturing the input sample features in multiple dimensions, high-quality feature input is provided for the classification of the downstream neural network model, and ultimately the model can better identify and detect the status of the input power grid equipment.

[0060] 2. When extracting features from a single power device node, the present invention not only considers the node's own state feature temporal and spatial information, but also considers the impact of information from surrounding power devices on the node. This invention uses a graph attention mechanism to capture information about surrounding devices, integrating environmental information with the node's surroundings. This enhances the feature dimension of the device itself, providing more feature information for downstream neural network classification tasks, and thus improving the model's ability to detect faulty devices.

[0061] 3. The present invention adopts data upsampling technology based on SC-SMOTE, which can effectively solve the problems caused by sample imbalance and effectively alleviate the situation of sample imbalance in power equipment.

[0062] 4. The present invention adopts SC-SMOTE and a GRU network based on the attention mechanism to achieve automatic and accurate detection of power equipment faults in power grid power supply scenarios by extracting multi-dimensional features of the input power equipment samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of the SC-SMOTE-based upsampling technology of the present invention for obtaining balanced sample data;

[0064] Figure 2 This is a schematic diagram of the processing flow of the present invention for acquiring temporal and spatial features of input samples based on the attention mechanism combined with GRU;

[0065] Figure 3 This is a diagram of a related information capture network framework for obtaining auxiliary information about the environment in which the current device node is located based on the graph attention mechanism of the present invention;

[0066] Figure 4 It is the overall framework diagram of the present invention. DETAILED DESCRIPTION

[0067] To further illustrate various embodiments, the present invention provides accompanying drawings. These drawings form part of the present disclosure and are primarily used to illustrate the embodiments and, in conjunction with the relevant description in the specification, to explain the operating principles of the embodiments. By referring to these drawings, those skilled in the art will be able to understand other possible implementations and the advantages of the present invention.

[0068] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0069] The present invention proposes a power equipment fault detection model based on an attention mechanism combined with a GRU. The power equipment fault detection model includes a classification neural network model, the training data of the classification neural network model comes from a preprocessing model, and the preprocessing model includes an upsampling module, a word embedding representation learning module, a GRU module, an attention mechanism module, and a graph attention mechanism module. The upsampling module is used to convert the input unbalanced power equipment data into balanced data; the word embedding representation learning module is used to embed the balanced data and output a historical state sequence based on the power equipment representation, a label data embedding representation, and an embedding representation of the state of the power equipment and its corresponding related equipment (power equipment portrait features); the GRU module is used to extract the time and space features of the power equipment from the historical state sequence based on the power equipment representation output by the word embedding representation learning module; the attention mechanism module is used to extract the state sequence features from the time and space features of the equipment; the graph attention mechanism module is used to extract the environmental information of the power equipment from the embedding representation of the power equipment portrait features; the state sequence features, the label data embedding representation, and the environmental information are aligned and fused as the training data input of the classification neural network.

[0070] Specifically, the construction and training of the power equipment fault detection model based on the attention mechanism combined with GRU includes the following steps:

[0071] Step 1. Input a power equipment dataset, which contains the physical status of the power equipment and corresponding labels. The physical status of the power equipment includes various status data such as the content of gas components in transformer oil, partial discharge of the transformer, temperature of the equipment contact surface, moisture condition of internal components, etc. The label of the power equipment is the fault type of the power equipment, such as transformer, including: insulation degradation, abnormal vibration, etc.

[0072] Step 2: Use upsampling algorithms such as SC-SMOTE to convert the input unbalanced power equipment data into balanced data, and then perform embedding representation;

[0073] The specific steps of step 2 include:

[0074] The power equipment entity states and labels contained in the power equipment dataset in step 1 are used as inputs to the SC-SMOTE upsampling algorithm. These inputs are used as seed data to generate new samples based on the corresponding feature distribution matrix. Then, linear interpolation is performed to obtain the final new samples. The newly generated samples and the original seed samples are merged together to generate a balanced sample dataset. These data are then embedded and represented as follows:

[0075] (1) Obtain a data set of power equipment, which contains the entity status of the equipment and the corresponding labels;

[0076] (2) Traverse the data set of power equipment and use the KNN algorithm (K-Nearest Neighbor) to determine the nearest neighbor sample set D of sample x n , in the neighbor set D n In the example, there are samples of different categories, and the samples of the same category as sample x are set D same , the sample set of different categories from sample x is called D other Compare D same The sample size and D other The number of samples is determined according to Formula 2.1 to determine whether the sample x is a seed sample, and a seed sample label S is added to the original dataset.

[0077]

[0078] (3) Based on the seed sample information, both the majority class and the minority class are upsampled simultaneously. For the majority class in the seed sample, the sampling rate is 100%, that is, one new sample is generated from one majority class seed sample. The sampling rate of the minority class seed sample is determined based on the sample ratio of the original data and the sample ratio of the seed sample.

[0079] In order to make up for the sample gap of the original data set, it is necessary to calculate the sample size gap between the majority class and the minority class in the original data set label_diff j :

[0080] label_diff j =N maj -N j (2.2)

[0081] Among them, N maj Represents the number of majority class samples; N j Indicates that it belongs to category C j The number of samples.

[0082] In the seed sample set, the majority class seeds are more than the minority class seeds, and the sampling rate of the majority class seeds is 100%. In order to make up for the number gap of seed samples, it is necessary to calculate the ratio R of the majority class seeds to the minority class seeds. sj :

[0083]

[0084] Among them, D s_maj represents the seed sample set of the majority class; D s_j Indicates that it belongs to category C j A collection of seed samples.

[0085] Calculate each category C j The average number of new samples N generated by the seed sample gj :

[0086]

[0087] Among them, R sj Indicates the number of samples that need to be generated for each seed sample to balance the number of seed samples. label_diff j / |D s_j | represents the number of samples that need to be generated for each seed sample to balance the number of original data.

[0088] (4) After obtaining the number of samples generated by each minority class seed sample, the K-means algorithm (K-means clustering algorithm) is used to update the cluster center coordinates at each iteration while dividing the samples according to the Euclidean distance between the cluster center and the sampled samples. The hyperparameter k of the K-means algorithm is c Indicates the number of clusters. In the SC-SMOTE algorithm, the hyperparameter k c The value of depends on the ratio of the number of majority classes to the number of minority classes in the dataset and is expressed as:

[0089]

[0090] After clustering the data set according to the general K-means algorithm, each sample is labeled with the cluster label C, and the data set is updated as follows:

[0091]

[0092] (5) Filter out samples of the same category from each cluster to form a sample set D c , each sample contains a feature set F = {f1,f2,…,f p}, and then perform corresponding processing according to the different feature types.

[0093] For discrete features, such as "abnormal noise" and "abnormal vibration of machinery and equipment," the selection of discrete features cannot be randomly selected from all fields. Instead, they must be determined based on the frequency of occurrence of different fields to ensure that the feature distribution of the generated samples and the final balanced dataset does not change.

[0094] For continuous features, such as "temperature data of the device itself", etc. During the data generation process, it is necessary to consider the value of the continuous feature in the [min, max] interval, so it is necessary to calculate the maximum and minimum values of the feature value, and randomly select data in the [min, max] interval as the generated value when generating data. c There are p features of L different categories in a cluster, and the calculation dimension is (K c ×L×p×2) feature distribution matrix FD.

[0095] (6) For each seed sample x i , the corresponding category y i , the cluster c i , there is a corresponding number of new samples N that need to be generated gi , each time a new sample is generated, according to N gi And the distribution of each feature of the cluster FD[c i ][y i ], first generate auxiliary samples x temp , and then perform linear interpolation to obtain the final generated sample x new .

[0096] The SC-SMOTE algorithm first constructs an auxiliary sample x based on the feature distribution temp . Auxiliary sample x temp Three rules need to be met:

[0097] Temporary sample x temp With the sample x i Belong to the same category label y i ;

[0098] Temporary sample x temp With the sample x i Belong to the same cluster c i ;

[0099] Temporary sample x temp With the sample x i Have the same features, but the eigenvalues of each feature are based on the cluster c i The characteristic distribution F[c i ][y i ] Random sampling is obtained;

[0100] Get a temporary sample x tempAfter that, the new sample x can be obtained by linear interpolation. new :

[0101] x temp =[f1,f2,…,f p ],f p =Random(FD[c i ][y i ][p]) (2.6)

[0102] X new =x+Random(0,1)×(x temp -x) (2.7)

[0103] Cycle N of seed samples gj After the sample generation operation, a set of generated samples based on the seed sample is obtained. These generated samples belong to the same category as the seed sample. When each seed sample completes the sample generation, the generated sample set D g Merge with the original dataset D to get the final balanced dataset D balance After the balanced dataset, the ratio of majority class to minority class returns to normal, and the overall number of samples is also expanded.

[0104] (7) For the samples finally obtained, the data format is defined as M×N, where M is the number of samples, which represents the description of different power equipment. N is the number of features, including equipment temperature, equipment image features, equipment parameter features, and context features. In the feature processing process, it is generally customary to discretize continuous features. Discrete features will make the data matrix extremely sparse after encoding. If they are not effectively processed, the number of parameters in the subsequent modeling process will surge. The main function of the data embedding layer is to compress the sparse vector after one-hot encoding. The dimension of the data vector after the embedding layer will be significantly reduced, and the feature information will be mainly expressed in numerical form. Assume that the feature vector after one-hot encoding is represented as [x1; x2; ...; x n ], where n is the number of feature domains, x i is the one-hot code representation of feature field i. The size of the embedding layer matrix V is n×k, where k is the size of the embedding layer vector.

[0105] After the embedding layer, the sparse vector will be encoded into a dense vector of equal length. Let the output of the embedding layer be E, as shown in Formula 2.8.

[0106] E=[e1,e2,…,e n ]=[v1x1,v2x2,…,v n x n ] (2.8)

[0107] Among them, e i Represents the feature domain vector. For single-valued features, each x i There is only one 1 in the feature domain vector. For multiple features, e i With multiple vectors, the embedded representation of the dataset is finally completed.

[0108] Step 3: Based on the sample embedding representation obtained in step 2, define a preprocessing model based on the attention mechanism combined with the GRU network. The preprocessing model includes a module for capturing state sequence features of a single device in both time and space dimensions and a module for obtaining auxiliary information about surrounding related devices for a single device.

[0109] The specific steps of step 3 include:

[0110] (1) Define the state trend capture layer (GRU module). After the behavior sequence data is embedded, the GRU network is used to model the order of the behavior sequence, as shown in Formula 2.18.

[0111]

[0112] Among them, σ represents the Sigmoid function, ⊙ represents the Hadamard product, where n hid represents the hidden layer size of the GRU network, k represents the size of the vector after embedding, i t Represents the input of GRU, which represents the tth vector representation in the behavior sequence, i.e. t =e b [t], the network output value h t Represents the tth hidden state, which is a potential expression of the past state of the power equipment. The main function of the state trend capture layer is to provide the temporal characteristics of the interest representation.

[0113] (2) Define the key state perception layer (attention mechanism module). The attention mechanism is used to obtain the association between the current power equipment and the power equipment status at different time points in the historical sequence. This process is measured by similarity and can be considered as a process of perceptual evolution, as shown in Formula 2.19.

[0114]

[0115] Among them, e a is the target vector, F[.] represents the attention scoring function, and the bilinear method is used here to calculate the attention distribution. By multiplying the hidden state of the power equipment at different positions with the attention score, the state sequence feature representation at this time can be obtained: bThe key state perception layer is mainly used to provide local characteristics for trend representation, and to capture the state sequence characteristics of a single device in both time and space dimensions.

[0116] (3) Define the graph attention mechanism layer (GraphAttention layer, graph attention mechanism module) to ensure that when extracting features, i.e., modeling the behavior sequence, we also need to make full use of the modeling method of environmental features. The embedding vector of a power equipment portrait feature is e p , the embedding vector of the related power equipment node is e′ p1 ,e′ p2 ,…,e′ pn , and then use the graph attention mechanism to generate a new embedding vector e′ for each power equipment node p , the formula is shown in 2.20.

[0117] e′ p =[e p ,e′ p1 ,e′ p2 ,…,e′ pn ,X] (2.20)

[0118] in, X is a coefficient set. Graphattention is performed on the feature vector of each power device to obtain the output vector e′ p , complete the acquisition of auxiliary information of surrounding related devices for a single device.

[0119] (4) The embedding representation of the features obtained in (2) and (3) and the embedding representation of the labels are fused as the input of the classification neural network of the power equipment fault detection model. The final result is shown in Formula 2.21.

[0120]

[0121] Among them, y pred ∈{0,1,2},e a is the vector representation of the predicted target, W deep ,b deep The output layer parameters can be stacked to better capture the combination of high-order features. Finally, the construction of the power equipment fault detection model is completed.

[0122] Step 4: First, the embedded representation generated in step 2 is used as the input of the preprocessing model obtained in step 3, and then the output of the preprocessing model is used as the input of the power equipment fault detection model, and finally the power equipment fault detection model is trained and generated.

[0123] The specific steps of step 4 include:

[0124] (1) The network architecture adopted in the present invention is based on the attention mechanism and the GRU network. First, based on the embedding layer in step 2, the corresponding embedding representation is generated for the constructed balanced sample; secondly, we will define a GRU module based on the attention mechanism. On the one hand, this module combines the attention mechanism and GRU to capture the key temporal and spatial features of the device nodes (defined as: state sequence features), and on the other hand, it combines the graph attention mechanism to enable a single power device node to obtain auxiliary information of related power devices, and fuse the generated features through vector connections. Finally, we use the attention mechanism defined in step 3 combined with the GRU network to learn the ability to capture state sequence features on the one hand, and to achieve fault detection of power equipment through feature learning on the other hand. Its network architecture is as follows: Figure 4 shown.

[0125] (2) Set the number of training iterations epochs and start training when epochs equals 1.

[0126] (3) Obtain the dataset sample embedding representation from step 2), then input the data embedding representation in batches into the attention mechanism combined with the GRU network to obtain the prediction of the input embedding representation.

[0127] (4) Calculate the loss function of the estimated value and the actual label value to minimize it.

[0128] (5) Repeat the steps in (3) and (4) within the value range defined by epochs, and finally train a power equipment fault detection model based on the attention mechanism combined with GRU for data preprocessing.

[0129] In the application systems of classification and detection problems, the main focus is on the ability to extract features, and whether the features can be fully mined and utilized. The innovation of the present invention is mainly based on GRU combined with the attention mechanism. On the one hand, it can obtain temporal and spatial features, and on the other hand, it can use the attention mechanism to utilize and mine state sequence features, and at the same time eliminate feature noise to a certain extent; in addition, in addition to considering the extraction of state features of the power equipment itself, the present invention also uses the graph attention mechanism to obtain auxiliary information that can be generated by related equipment. Based on the above two aspects, the features of the power equipment can be fully extracted and utilized. This allows the network model to better learn features and use these features to achieve more accurate power equipment fault detection. In response to the above problems, many methods will choose to make some improvements in the direction of deepening the network depth, multimodal fusion, etc. The method proposed in this article is different from the previous ones. It mainly combines the attention mechanism and GRU technology to realize multi-dimensional feature mining of input data, obtain more feature information, and improve the fault detection capability of the network model.

[0130] The design of the method in the present invention is based on the attention mechanism combined with the GRU network to fully mine the features of power equipment, so as to better serve the classification and detection tasks of the downstream neural network. The attention mechanism is combined with the GRU module. We use the multi-head attention mechanism and the GRU network model. First, the GRU network is used to extract the time and space features of the power equipment, and then the attention mechanism is used to capture the state sequence features to complete the mining of the power equipment's own features; in addition, in the graph attention mechanism model, we use the graph attention mechanism to fully mine the auxiliary information of the relevant equipment nodes and complete the full mining of environmental information; then this information is combined with the embedded representation of the label for attention alignment and fusion, generating a vector of the same size as the state encoding of the input downstream neural network as the input of the downstream neural network, and the output is trained against the label to finally generate a power equipment fault detection model.

[0131] Based on the above improvements, the power equipment fault detection model based on the attention mechanism combined with GRU proposed in this invention is realized. This method can effectively improve the accuracy of power equipment fault detection.

[0132] The working principle of the present invention is:

[0133] The present invention first performs SC-SMOTE upsampling on power grid power equipment samples to generate balanced sample data, then uses the embedding layer to embed the input samples, and then uses the generated embedding representation as the attention mechanism combined with the GRU network model input. The input power equipment features are first fully mined using the attention mechanism combined with the GRU module and the graph attention mechanism model respectively, and then the features generated by each module are fused based on the attention mechanism, and finally used as the input of the downstream classification neural network. The output is adversarial training with the defined label, and finally the model training is completed to generate a model that can accurately detect power equipment faults.

[0134] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of the present invention also fall within the scope of protection of the present invention.

Claims

1. A power equipment fault detection model based on attention mechanism combined with GRU, characterized by: The power equipment fault detection model includes a classification neural network model, the training data of the classification neural network model comes from a preprocessing model, and the preprocessing model includes an upsampling module, a word embedding representation learning module, a GRU module, an attention mechanism module and a graph attention mechanism module. The up-sampling module is used to convert the input unbalanced power equipment data into balanced data; The word embedding representation learning module is used to embed the balance data and output an embedding representation based on the historical state sequence of the power equipment representation, the label data embedding representation, and the power equipment portrait feature; The GRU module is used to extract the temporal and spatial characteristics of the power equipment from the historical state sequence based on the power equipment representation output by the word embedding representation learning module; The attention mechanism module is used to extract state sequence features from the temporal and spatial features of the device; The graph attention mechanism module is used to extract the environmental information of the power equipment from the embedded representation of the power equipment portrait feature; The state sequence features, the label data embedding representation and the environment information are aligned and fused as training data input for the classification neural network; The upsampling algorithm used by the upsampling module is the SC-SMOTE upsampling algorithm; The SC-SMOTE upsampling algorithm specifically includes: Step 21: Traverse the input power equipment data set to determine the majority class seed samples and the minority class seed samples; Step 22: Based on the seed sample information, upsample the majority class and the minority class simultaneously, and calculate the number of samples generated by the seed sample of each minority class; Step 23: After obtaining the number of samples generated by each minority class seed sample, perform linear interpolation to obtain the final new sample, merge the newly generated sample with the original seed sample to generate a balanced sample data set; Step 24: Embed the data in the generated balanced sample dataset; The data processing result of the classification neural network model is expressed as: Among them, y pred ∈{0,1,2},O b is the vector representation of the state sequence feature; e p ′ is the state feature vector representation of the power equipment; e a is the vector representation of the predicted target, W deep ,b deep are the output layer parameters.

2. The power equipment fault detection model according to claim 1, characterized in that: The step 21 includes: traversing the data of the power equipment data set, and using the KNN algorithm to determine the neighboring sample set D of the sample x. n , in the neighbor set D n In the example, the samples of the same category as the sample x are set D same , the sample set of different categories from sample x is called D other ; Compare D same The sample size and D other The number of samples is calculated according to the formula: Determine whether the sample x is a seed sample and add the seed sample label S to the original dataset.

3. The power equipment fault detection model according to claim 1, wherein: The calculation of the number of generated samples in step 22 includes the following formula: label_diff j =N maj -N j Among them, label_diff j Represents the majority class and minority class C in the original data set j The sample size gap; N j Indicates that it belongs to category C j The number of samples; D s_j Indicates that it belongs to category C j The seed sample set of R sj Represents the majority class seed sample and category C j The proportion of seed samples; N gj Represents each category C j The average number of new samples generated by the seed sample; label_diff j / |D s_j |Indicates category C j The number of seed samples that need to be generated to balance the number of original data.

4. The power equipment fault detection model according to claim 1, wherein: The step 23 includes the following steps: after obtaining the number of samples generated by the seed sample, using the K-means algorithm, according to the Euclidean distance between the cluster center and the sampled sample, the cluster center coordinates are updated at each iteration of the sample division; wherein the hyperparameter k of the K-means algorithm is c Indicates the number of clusters, hyperparameter k c The value of depends on the ratio of the number of majority classes to the number of minority classes in the dataset, and the formula is expressed as: After clustering the data set using the K-means algorithm, each sample is labeled with the cluster label C, and the data set is updated as follows:

5. The power equipment fault detection model according to claim 4, characterized in that: The data processing method of the newly generated samples in step 23 is: filter out samples of the same category from each cluster to form a sample set D c , each sample contains a feature set F = {f1,f2,…,f p }, and then, depending on the feature type, execute: when the feature is a discrete feature, during the data generation process, field selection is performed based on the probability distribution of the occurrence of different fields; when the feature is a continuous feature, during the data generation process, data is randomly selected in the [min, max] interval of the feature value as the generated value, where max and min are the maximum and minimum values of the feature value, respectively.

6. The power equipment fault detection model according to claim 4, characterized in that: The method of obtaining the final new sample by linear interpolation in step 23 is: For each seed sample x i , the corresponding category y i , the cluster c i , there is a corresponding number of new samples N that need to be generated gi , each time a new sample is generated, according to N gi And the distribution of each feature of the cluster FD[c i ][y i ], first generate auxiliary samples x temp , and then perform linear interpolation to obtain the final generated sample x new ; Among them, the auxiliary sample x temp The construction needs to meet three rules: Temporary sample x temp With the sample x i Belong to the same category label y i ; Temporary sample x temp With the sample x i Belong to the same cluster c i ; Temporary sample x temp With the sample x i Have the same features, but the eigenvalues of each feature are based on the cluster c i The characteristic distribution F[c i ][y i ] Random sampling is obtained; Get a temporary sample x temp Then, the new sample x is obtained by linear interpolation. new : x temp =[f1,f2,…,f p ],f p =Random(FD[c i ][y i ][p]) X new =x+Random(0,1)×(x temp -x) Cycle N of seed samples hj After the sample generation operation, a set of generated samples based on the seed sample is obtained. These generated samples belong to the same category as the seed sample. When each seed sample completes the sample generation, the generated sample set D is obtained. g Merge with the original dataset D to get the final balanced dataset D balance .

7. The power equipment fault detection model according to claim 6, characterized in that: The vector representation of the state sequence feature is O b Obtained by calculation through the GRU module and the attention mechanism module; The data processing process and results of the GRU module are expressed as follows: r t =σ(W r i t +U r h t-1 +b r ) z t =σ(W z i t +U z h t-1 +b z ) Among them, σ represents the Sigmoid function, ⊙ represents the Hadamard product, where n hid represents the hidden layer size of the GRU network, k represents the size of the vector after embedding, i t Represents the input of GRU, which represents the tth vector representation in the historical state sequence, i.e. t =e b [t]; the output value h of the GRU module t represents the tth hidden state, which is a potential expression of the past state of the power equipment; The data processing process and results of the attention mechanism module are expressed as follows: Among them, a t represents the attention score calculated by the attention distribution of the attention mechanism module; F[.] represents the attention scoring function.

8. The power equipment fault detection model according to claim 6, wherein: The state characteristic vector of the power equipment is represented by e p ′ Obtained through the calculation of the graph attention mechanism module; the data processing result of the graph attention mechanism module is expressed as: And ′ p =[and p ,And ′ p1 ,And ′ p2 ,…,And ′ pn ,X] Among them, e p is the embedding vector of the power equipment portrait feature; e ′ p1 ,e ′ p2 ,…,e ′ pn The embedding vector is e p Embedding vectors of related power equipment nodes of the power equipment; X is the coefficient set; The graph attention mechanism is used to process the feature vector of each power device to obtain the output vector e ′ p .

Images