Partial discharge fault type classification method based on GAN adversarial network and CNN-BiLSTM-Attention mechanism

Abstract

The invention discloses a partial discharge fault type classification method based on a GAN adversarial network and a CNN-BiLSTM-Attention mechanism, and belongs to the technical field of fault diagnosis, and the method comprises the following steps: S1, carrying out the batch normalization processing of a collected high-frequency current signal, and carrying out the single pulse extraction of a processed waveform signal; s2, performing time-frequency analysis on the processed signal by using a wavelet transform method, constructing a data time-frequency spectrogram, and introducing an optimized GAN generative adversarial network to expand an unbalanced data set; and S3, converting the time-frequency spectrogram into a time-frequency matrix, and inputting the time-frequency matrix into a CNN-BiLSTM-Attention attention mechanism model. According to the method, the diagnosis model is adopted, the type of the partial discharge fault is accurately judged, meanwhile, the method can be applied to an unbalanced data model through data expansion processing, and it is indicated that the proposed model has good generalization ability.

Description

Technical Field

[0001] The present invention relates to the field of fault diagnosis technology, and in particular to a partial discharge fault type classification method based on a GAN adversarial network and a CNN-BiLSTM-Attention mechanism. Background Art

[0002] The main ways to improve the accuracy of partial discharge fault detection can be summarized into the following three aspects: (1) In terms of fault mechanism, by adopting different phenomenon generation equipment, data acquisition methods, data processing technology, and advanced experimental equipment to obtain data on physical phenomena such as sound waves, light, and current generated by partial discharge. (2) There is a strong correlation between PD patterns and defects. These associations can be used to extract different feature information from partial discharge information. Since the signal itself contains certain information, the Mantach team directly put the waveform signal into the model for classification. In addition, image processing tools such as texture analysis algorithms and wavelet image decomposition algorithms can be used, or signal processing tools such as Fourier series analysis, Hilbert Huang transform, wavelet transform, and fast Fourier transform can be used to extract information from time domain information. (3) After determining the data type of the fault sample, in order to improve pattern recognition classification, research and develop novel and ideal pattern recognition classification methods to improve the accuracy of PD pattern recognition. In previous studies, commonly used pattern recognition methods include back propagation neural network (BPNN), support vector machine (SVM), and artificial neural network (NN). Recently, convolutional neural networks (CNNs) have been widely used in the field of PD pattern recognition due to their powerful capabilities in processing images and extracting local features.

[0003] CNN has certain limitations when extracting global features of image data. LSTM (Long Short-Term Memory Network) is a special recurrent neural network that can model sequence data and has advantages in mining time series feature information of partial discharge information. Combining the advantages of convolutional neural network (CNN) and long short-term memory network (LSTM), a CNN-LSTM model is constructed. Experiments have shown that the model has obvious advantages in mining local spatial features and time series features of PD spectra. BiLSTM is an extension of LSTM. It considers the previous and next information of the input sequence at the same time, and has better performance in processing the global features of images.

[0004] In many experiments and practical application scenarios, the collected data samples often show unbalanced characteristics, especially in the field of fault detection, where fault data is usually more scarce than normal data. This data imbalance problem seriously restricts the recognition performance and accuracy of fault diagnosis models such as CNN. Therefore, experts and scholars have conducted a lot of research on how to find this balance between fault samples. Summary of the invention

[0005] The purpose of this invention is to provide a fault identification system based on the GAN adversarial network and the CNN-BiLSTM-Attention framework to achieve accurate identification of the fault type of partial discharge. At the same time, the addition of the GAN generative adversarial network can overcome the errors caused by the data imbalance problem. The results show that the proposed model can achieve high recognition accuracy and strong generalization ability.

[0006] To achieve the above object, the present invention provides a fault diagnosis method based on GAN adversarial network and CNN-BiLSTM-Attention mechanism, comprising the following steps:

[0007] S1. The pulse current method is used to collect discharge signals of different partial discharge fault types. In order to describe the fault characteristics, discharge location and law in the actual working state of the power transformer as much as possible, this paper designs four typical insulation defects of transformer partial discharge. The oil-immersed transformer is taken as the research object. The four discharge models are metal tip discharge, oil-paper air gap discharge, metal surface discharge and oil bubble discharge.

[0008] S2. Since the noise level is not simply defined by a signal statistic, a higher threshold is set for the noise by dividing by the number of fault types, thereby increasing the robustness of the normalization process. The expression for information processing is:

[0009]

[0010] In order to reduce the difficulty of model training and more conveniently represent the characteristics of each partial discharge defect, pulse extraction is performed on the original waveform signal.

[0011] S3. Wavelet transform inherits and develops the localization idea of ​​short-time Fourier transform, while overcoming the disadvantage that the window size does not change with frequency. At the same time, wavelet transform can provide higher resolution, especially when processing non-stationary signals, and can well represent the details and transient characteristics of the signal.

[0012] After the time-frequency spectrum is obtained by wavelet transform, the GAN adversarial network is used to enhance the data. Compared with the objective function of WassersteinGAN, GW-GAN adds a new gradient penalty term, where is the penalty coefficient, which represents random sampling along the straight line between the real distribution and the generated distribution. GW-GAN can generate higher quality data samples.

[0013] S4. In order to reduce or even avoid the loss caused by gradient vanishing, the ReLu activation function is introduced into the CNN model. Then, the batch normalization (BN) is used to replace the traditional normalization between the Conv2D convolution layer and the activation layer. At the end of the model, the regularization technology is used to alleviate the impact of overfitting. A unique simplified network is sampled and trained in each training case.

[0014] S5. BiLSTM is an improvement of LSTM, including forward LSTM and backward LSTM, usually represented by and, which can take into account the features of two directions in the sequence at the same time.

[0015] Then the output feature information is input into BiLSTM to obtain a bidirectional time feature vector;

[0016] S6, repeat steps S4 and S5 multiple times to calculate and update the training parameters in the CNN model until the termination condition is met;

[0017] S7. By introducing the attention mechanism to selectively focus on the model, the model can better capture the key information in the input sequence, thereby improving the performance of the model on various tasks. The attention mechanism is located after BiLSTM. This selective attention method enables the model to better capture the key information in the input sequence, thereby improving the performance of the model on various tasks. In the proposed model, the attention mechanism receives the features output by BiLSTM, and then uses the following attention function to calculate the similarity score between each feature and the regression value. The calculation process is as follows:

[0018] e t，i = score(H i ,y t )=V T tanh(U[H i ;y t ])

[0019] Where V and U are the parameter values ​​learned during the training process, and the similarity score is quantized using the softmax function. t,i Normalized to get a t,i The probability distribution at H i importance.

[0020]

[0021] Finally take a t,i H i The weight of H i Perform multiplication to obtain weighted features The calculation process is as follows

[0022]

[0023] Finally, the final weighted feature vector is obtained

[0024] S8. Convert data features in the fully connected layer and perform classification tasks, outputting the model classification results.

[0025] Compared with the prior art, the advantages of the present invention are:

[0026] 1. A CNN convolutional neural network model with multiple convolution and pooling layers is proposed. Experiments show that the features extracted by CNN can help the final model perform better than other methods.

[0027] 2. In order to better extract the information features in the signal waveform, the wavelet transform is introduced to mine the information features in the time domain and frequency domain. The wavelet transform can capture the time-varying characteristics of the signal. Through the transformation, the specific positions of different frequency components in the signal can be understood, which is especially important for the analysis of non-stationary signals.

[0028] 3. For the problem of data sample imbalance, an enhanced GAN adversarial network is proposed. Compared with the traditional data enhancement method, the data generated in this paper is the closest to the characteristics of real data. Experimental verification shows that the GAN used in this paper significantly improves the recognition accuracy of the model and has good generalization ability.

[0029] 4. In order to overcome the limitations of CNN in extracting global feature information, the bidirectional BiLSTM was introduced. The model performed better than CNN+LSTM and its recognition of the four defects was also higher than that of CNN, LSTM and other models.

[0030] 5. After the BiLSTM module outputs the eigenvalue, in order to enhance the importance of important weight information, the attention mechanism is introduced to optimize the weight. By performing feature weighting on the received vector, a new feature vector is generated to improve the efficiency and interpretability of the model in processing complex problems.

[0031] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a system flow chart based on the GAN adversarial network and CNN-BiLSTM-Attention mechanism framework of the present invention;

[0033] Figure 2 Flowchart for data preprocessing and wavelet analysis;

[0034] Figure 3 This is the principle diagram of the adversarial network;

[0035] Figure 4 This is the framework diagram of the convolutional network model;

[0036] Figure 5 This is the model flow chart of the CNN-BiLSTM-Attention mechanism framework;

[0037] Figure 6 (a) is the curve of the accuracy of the training set and the validation set as the number of iterations increases. Figure 6 (b) is the change curve of the loss function of the training set and the validation set;

[0038] Figure 7 (a) is the T-SNE visualization result without data enhancement after adding the attention mechanism. Figure 7 (b) Visualization results after adding GAN to generate adversarial network;

[0039] Figure 8 Confusion matrix for the model trained on the test set;

[0040] Fig. 9 The ROC curve is used to judge the overall performance of the model;

[0041] Fig.10 Comparison of the accuracy of several fault diagnosis models. DETAILED DESCRIPTION

[0042] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific implementation disclosed below.

[0043] Explanation of the embodiment:

[0044] See also Figure 1 This embodiment describes a bearing fault detection method based on discrete wavelet transform and CNN-BiLSTM, and the method includes:

[0045] Step S1: First, the pulse current method is used to collect discharge signals of different partial discharge fault types; for the proposed method, a transformer partial discharge test bench is first built to collect high-frequency current signals. In order to obtain high-frequency current signals, a high-frequency current sensor is placed at the neutral point grounding of the transformer model for signal collection. The collected data is shown in Table 1.

[0046] Table 1 Discharge test data information

[0047]

[0048] In order to more conveniently represent the characteristics of each partial discharge defect and reduce the difficulty and complexity of subsequent model training, pulse extraction is performed on the original waveform signal.

[0049] Then the collected signals are batch normalized. The data of the PD model is generated at a pulse frequency of 40Mhz. In order to facilitate the calculation of the pulse signal of the partial discharge phenomenon occurring after the sample point N0, all M signals x1, x2, ..., x m The measurement interval is controlled at n * ∈[1; N0] to determine the parameters and level of the noise. Since the noise level is usually not simply defined by a certain statistic of the signal, a higher threshold is set for the noise by multiplying the denominator by 4, thereby increasing the robustness of the normalization process. In this way, even if the noise level has certain fluctuations or uncertainties, the normalized signal can still remain relatively stable, which is conducive to subsequent pulse extraction. The entire preprocessing process is as follows Figure 2 shown.

[0050] Then, the extracted signal is subjected to wavelet transform, and the GAN adversarial network is used to enhance the data, such as Figure 3 As shown in Figure 2, the generative adversarial network enhances the collected imbalanced data. Compared with the objective function of Wasserstein GAN, the adversarial network in the model adds a new gradient penalty term, which can generate higher quality data samples.

[0051] After data enhancement, the time-frequency spectrum is converted into a time-frequency matrix and input into the CNN model for feature extraction. This paper designs an improved CNN model to classify the collected data and identify faults. The CNN model consists of 8 convolutional blocks and 2 fully connected blocks. The first layer contains a convolutional layer, a batch normalization layer, and an activation layer, and each of the remaining convolutional blocks has one more maximum pooling layer than the first layer. The filter size of the first two layers of the model is (5×5×16), the filter size of the third and fourth layers is (3×3×64), the filter size of the fifth and sixth layers is (3×3×64), and the filter size of the seventh and eighth layers is (3×3×128). The maximum pooling layer is used in the model to downsample the features, and the window size is 2×2.

[0052] The output feature information is then input into BiLSTM to obtain a bidirectional temporal feature vector. During the model training process, these weight matrices and bias matrices are randomly initialized and then learned through the optimization algorithm. BiLSTM includes forward and reverse It can take into account the characteristics of both directions in the sequence at the same time. At time t, the state of the hidden layer changes from the positive direction and reverse express. and The calculation formula is as follows:

[0053]

[0054]

[0055]

[0056] In the formula, x t represents the input data of the network at time t, h t Represents the LSTM hidden layer state at time t, and [] represents the connection of vectors. The BiLSTM module in the model continues to extract the time features in the signal after the CNN convolutional network model, and obtains the bidirectional time feature vector [H0, H1, ..., H n ].

[0057] Repeat steps S4 and S5 multiple times to calculate and update the training parameters in the CNN model until the termination condition is met; then put the output features into the attention mechanism. The attention mechanism receives the features [H0, H1, ..., H n ], and then use the following attention function to calculate each feature H i and the regression value y t The similarity score between t,i . Then, the data features are converted in the fully connected layer and the classification task is performed, and the classification result of the model is output.

[0058] At this point, the optimization design of the CNN-BiLSTM-Attention mechanism framework is completed.

[0059] Experiment and result analysis:

[0060] 1. Model evaluation indicators combining CNN and BiLSTM

[0061] In the experiment, four features were extracted to train the model, and the deep learning algorithms CNN, LSTM, CNN+LSTM were compared and tested together with this model. The test model used accuracy, precision, recall rate and f1-measure performance as evaluation indicators. The experimental results are shown in Table 2. It can be seen from the table that the original CNN has the worst diagnostic performance among the compared models. BiLSTM is an improved LSTM, and its performance is also higher than the latter. CNN+BiLSTM has the highest performance.

[0062] Table 2 Effectiveness evaluation and comparison of CNN+BiLSTM

[0063]

[0064] 2. Evaluation of the attention mechanism

[0065] Following the standard practice of data set segmentation, the data set is divided into a training set and a test set. The training set consists of 70% of the data, and the test set accounts for 30%. The CNN+BiLSTM+Attention model and the CNN+BiLSTM model without the attention mechanism are used for comparative diagnosis. As can be seen from Table 3, since the attention mechanism optimizes some weights, feature selection can be performed during training, and the performance and accuracy of the model are improved compared to the original. Cross entropy is often used to evaluate the performance of multi-classification problem models, so we use cross entropy as the loss function.

[0066] Table 3 Effectiveness evaluation of attention mechanism

[0067]

[0068] 3. Overall evaluation and visualization of the model after adding the adversarial network to balance the data

[0069] In order to verify the effect of the adversarial network, this paper introduces cosine similarity (CS), Euclidean distance (ED) and Pearson correlation coefficient (PCC) to evaluate the quality of the generated data samples. The principle of the first two parameters is to measure the similarity between two vectors or parameters. The smaller the value, the more similar the data. PCC describes the correlation between distributions, and the larger the value, the closer the connection. Table 4 describes the comparison of the data generated by GW-GAN in this paper with other models. The ED and CS values ​​of this experimental model are 3.011 and 0.075 respectively, which are the smaller of the two in the compared models. The PCC value is also the highest among peer models.

[0070] Table 4 Statistical comparison of parameters of several adversarial networks

[0071]

[0072]

[0073] As can be seen from Table 5, among the six methods, the model CBLAG proposed in this paper has the highest overall accuracy, reaching 98.8%, followed by MACGAN. Using this method to train 200 times on the training set and test set of this paper, the average recognition rate of the four models finally obtained reached 83.6%. The smallest is BPNN. Using the same method for testing, the recognition rate is only 64.8%. Therefore, the model in this paper has the best recognition rate among the concentrated models compared.

[0074] Table 5 Detailed results of the comparison of the accuracy of several fault diagnosis models

[0075]

[0076] V. Conclusion

[0077] In order to improve the diagnostic accuracy of the partial discharge fault model and reduce the losses caused by the fault, a new CNN-BiLSTM-Attention-GAN (CBLAG) comprehensive model is proposed. For the proposed method, a transformer partial discharge test bench is first built to collect high-frequency current signals. By preprocessing the received signals, including batch normalization, high-pass filtering and noise processing, it is verified that these operations increase the generalization ability of the model. After that, data augmentation technology is introduced to solve the problem of data imbalance. Then, in order to better extract the features in the signal, the processed waveform data is constructed based on the optimized wavelet time-frequency analysis. The spectrum diagram. All the sample data obtained are put into the proposed CBLAG model for training. After training and verification, the final recognition accuracy is as high as 98.8%. It is higher than the SVM, MobileNet, ResNet and BPNN models used for comparison.

[0078] Before the introduction of the attention mechanism, the accuracy of the model had no obvious advantage. The problem was that the importance of weights was not strictly evaluated during the training process. After the introduction of the attention mechanism, the accuracy increased by another 1.2%. In terms of data enhancement, the recognition rate of the model was the highest under the condition of balanced data samples.

[0079] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A partial discharge fault type classification method based on GAN adversarial network and CNN-BiLSTM-Attention mechanism, characterized by The steps include: S1: The pulse current method is used to collect discharge signals of different partial discharge fault types; S2: performing batch normalization on the collected signals, and then extracting single pulses from the signals; S3: Perform wavelet transform on the extracted signal to obtain a time-frequency spectrum, and use a GAN adversarial network to enhance the data; S4: Convert the time-frequency spectrum into a time-frequency matrix and input it into the CNN model for feature extraction; S5: Then input the output feature information into BiLSTM to obtain a bidirectional time feature vector; S6: Repeat steps S4 and S5 multiple times to calculate and update the training parameters in the CNN model until the termination condition is met; S7: The attention mechanism is responsible for capturing the key information in the sequence and then forming a weighted feature vector; S8: Convert data features and perform classification tasks in the fully connected layer, and output the model classification results.

2. A fault detection method based on wavelet transform and CNN-BiLSTM-Attention mechanism as claimed in claim 1, characterized in that: In S2: In the normalization process, a higher threshold is set for noise by multiplying the denominator by 4, thereby increasing the robustness of the normalization process. The calculation formula is: where x i represents the i-th metric of M signals.

3. A fault detection method based on wavelet transform and CNN-BiLSTM-Attention mechanism as claimed in claim 1, characterized in that: In S3: (a) The wavelet transform is used to perform time-frequency analysis on the pulse diagram. The definition is to shift the window function by a displacement of τ and make an inner product with the original signal under the varying scale ɑ. It can be equivalently expressed in the frequency domain as: In the formula, ψ(t) is the basic wavelet, F(ω), ψ(ω) are the Fourier transforms of F(t) and ψ(t), respectively. It can be seen from the formula that when α>1, the window function will be expanded, resulting in a larger window width. When α<1, the window function will be compressed, resulting in a smaller window width. (b) The gradient penalty function is introduced to improve Wasserstein GAN, which not only avoids the problems that may be caused by weight clipping, but also provides a more stable way to train the model. The objective function of the improved wassersteinGAN with gradient penalty can be expressed as: Where μ is the penalty coefficient, represents the real distribution p r With the generated distribution p z Random sampling of straight lines between .

4. The fault detection method based on wavelet transform and CNN-BiLSTM-Attention mechanism as claimed in claim 1, characterized in that: In S4: The architecture of the CNN model includes 8 convolutional blocks and 2 fully connected blocks. In order to reduce or even avoid the loss caused by gradient vanishing, the ReLu activation function is introduced in the model. A unique simplified network is sampled and trained in each training case.

5. The fault detection method based on wavelet transform and CNN-BiLSTM-Attention mechanism as claimed in claim 1, characterized in that: In S5: At time t, the state of the hidden layer changes from positive to and reverse express. and The calculation formula is as follows: In the formula, x t represents the input data of the network at time t, h t Represents the LSTM hidden layer state at time t, and [] represents the connection of vectors. The BiLSTM module in the model continues to extract the time features in the signal after the CNN convolutional network model, and obtains the bidirectional time feature vector [H0, H1, ..., H n ].

6. A fault detection method based on wavelet transform and CNN-BiLSTM-Attention mechanism as claimed in claim 1, characterized in that: In S7: The attention mechanism receives the features output by BiLSTM and calculates each feature H using the following attention function i and the regression value y t The similarity score between t,i , the calculation process is as follows: e t,i =score(H i ,y t )=V T tanh(U[H i ;y t ]) Where V and U are the parameter values ​​learned during the training process, and the similarity score is quantized using the softmax function. t,i Normalized to get a t,i The probability distribution at H i importance. Finally take a t,i is the weight of Hi, and H i Perform multiplication to obtain weighted features Finally, the final weighted feature vector is obtained 7. The fault detection method based on wavelet transform and CNN-BiLSTM-Attention mechanism as claimed in claim 1, characterized in that: In S8: The fully connected layer consists of two layers of neural networks with 192 and 4 neurons respectively, and the activation function is ReLu.

Images