Partial discharge signal defect type identification method, device and product

Abstract

The invention provides a partial discharge signal defect type identification method, a partial discharge signal defect type identification device and a product, and the identification method identifies the partial discharge signal defect type based on a Transform model architecture which is improved in a mode of replacing position coding with a moving time slice, adding a frequency domain block and preposing a layer normalization structure, a Transformer model architecture comprises n layers of encoders, each encoder comprises a self-attention block and a feedforward neural network, and the identification method comprises the following steps: S1, preprocessing a time sequence of a partial discharge signal to obtain a first matrix; s2, inputting the first matrix into a frequency domain block for frequency domain transformation to obtain a second matrix, and performing feature extraction on the second matrix to obtain a third matrix; s3, inputting the third matrix into a Transform encoder to obtain a fourth matrix; and S4, inputting the fourth matrix into a full connection layer to obtain a classification result. By means of the technical scheme, efficient feature extraction and accurate classification of the partial discharge signals are achieved, and the accuracy and efficiency of partial discharge signal defect recognition are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of electrical equipment detection, and in particular to a method, device and product for identifying a partial discharge signal defect type. Background Art

[0002] In power systems, partial discharge (PD) is a key indicator for evaluating the insulation performance of electrical equipment. PD signals contain rich information. Different types of insulation defects will lead to different discharge signal characteristics. Therefore, it is of great significance to judge the type of insulation defects by classifying and identifying these signals for defect location, defect severity assessment and countermeasure formulation. Traditional PD detection technology relies on manually extracted signal features and conventional machine learning algorithms, such as support vector machines (SVM) for pattern recognition, but these methods have problems such as information loss in the feature extraction process and limited ability to recognize complex signal patterns. In recent years, with the development of deep learning technology, especially the powerful ability of Transformer architecture in sequence data processing, researchers have begun to explore the use of end-to-end deep learning models to automatically extract features from PD signals and perform classification and recognition directly to improve the accuracy and efficiency of recognition. However, existing deep learning models still face challenges such as complex signal preprocessing and limited model generalization when processing PD signals. Summary of the invention

[0003] The embodiments of the present invention provide a method, device and product for identifying the defect type of a partial discharge signal, so as to achieve efficient feature extraction and accurate classification of partial discharge signals, and further improve the accuracy and efficiency of partial discharge signal defect identification.

[0004] In order to achieve the above-mentioned purpose, on the one hand, a method for identifying the defect type of a partial discharge signal is provided. The method is based on a Transformer model architecture with an improved normalized structure of a front layer, and adds a predetermined frequency domain block, and identifies the defect type of a partial discharge signal by combining time domain features with frequency domain features, wherein the Transformer model architecture includes: an n-layer encoder, the encoder includes: a self-attention block and a feedforward neural network, and the identification method includes:

[0005] S1, preprocessing the time series of the collected partial discharge signal; wherein the preprocessing includes:

[0006] The time series is normalized, and then a moving time window moving slice process is performed according to a predetermined time window size and a predetermined step length to obtain a plurality of time window slices; wherein there is a predetermined overlapping portion between the plurality of time window slices, so that each time window slice contains the time information of the time window slices adjacent to it before and after; the predetermined time window size, the predetermined step length and the predetermined overlapping portion are adjusted and the values ​​are confirmed according to the parameter amount of the Transformer model calculated by the following formula:

[0007]

[0008] Params Total represents the number of parameters of the Transformer model, d model represents the time window size, l represents the step size;

[0009] Combining the plurality of time window slices into a first matrix;

[0010] S2, inputting the first matrix into a predetermined frequency domain block for frequency domain transformation and feature extraction, including:

[0011] Normalizing the first matrix layer by layer, and calculating it by fast Fourier transform to obtain a frequency domain signal;

[0012] The frequency domain signal is weighted according to a preset learnable weight vector corresponding to the time window slice length, the frequency domain signal is converted into a time domain sequence by inverse fast Fourier transform, and the time domain sequence is added and fused with the first matrix by residual connection to obtain a second matrix;

[0013] The second matrix is ​​layer normalized, and features are extracted from the second matrix through a multilayer perceptron to obtain a feature vector; the feature vector is added and fused with the second matrix through a residual connection to obtain a third matrix, wherein the third matrix includes: time domain features and frequency domain features extracted from the second matrix;

[0014] S3, inputting the third matrix into the front-layer normalized Transformer encoder for encoding to obtain a fourth matrix; wherein the encoding step includes:

[0015] The third matrix of the time domain sequence is encoded by the multi-head attention mechanism of the self-attention block and the feedforward neural network,

[0016]

[0017] Wherein, l = [1, n], H' represents the matrix obtained after the third matrix is ​​processed by the self-attention block, and H lrepresents the matrix output by the encoder of the lth layer, LayerNorm represents layer normalization, MultiHeadAttn represents a multi-head self-attention mechanism, and FFN represents a feedforward neural network; wherein the calculations of the self-attention block and the feedforward neural network are first layer normalized and then residually connected;

[0018] S4, inputting the fourth matrix into a predetermined fully connected layer to obtain a classification result of the partial discharge signal defect type, which includes:

[0019]

[0020] in, represents the classification result, softmax represents the normalized exponential function, W n represents the weight matrix, H n represents the fourth matrix, b n Represents the bias vector.

[0021] Preferably, in the method for identifying partial discharge signal defect types, the partial discharge signal defect types include: tip corona discharge, internal air gap discharge, floating potential discharge and / or surface discharge.

[0022] Preferably, in the method for identifying the defect type of partial discharge signal, the Transformer model architecture is a Transformer architecture that only includes an encoder.

[0023] Preferably, in the method for identifying the defect type of partial discharge signal, a preset learnable weight vector having the same length as the time window slice is applied to the frequency domain signal in the form of weight sharing.

[0024] Preferably, in the method for identifying the defect type of partial discharge signal, during the training process of the model, a preset learnable weight vector equal to the length of the time window slice for extracting frequency domain signal features is updated according to the calculated loss function through a back propagation algorithm.

[0025] Preferably, in the method for identifying the defect type of a partial discharge signal, an initial value of a preset learnable weight vector equal to the length of a time window slice is randomly extracted from a probability distribution with a standard deviation of 0 to 0.02.

[0026] Preferably, in the method for identifying the defect type of partial discharge signal, the first matrix and the second matrix are subjected to layer normalization processing before being respectively input into the frequency domain feature weighting and the multilayer perceptron in the predetermined frequency domain block, and residual connection is performed after the frequency domain feature weighting and the multilayer perceptron output. Preferably, in the method for identifying the defect type of partial discharge signal, the third matrix is ​​subjected to layer normalization processing before being input into the self-attention block or the feedforward neural network, and residual connection is performed after the self-attention block or the feedforward neural network output.

[0027] On the other hand, an embodiment of the present invention provides a device for identifying a local discharge signal defect type, which includes a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement any of the methods for identifying a local discharge signal defect type as described above.

[0028] On the other hand, an embodiment of the present invention provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the method for identifying a partial discharge signal defect type as described above is implemented.

[0029] The above technical solution has the following technical effects:

[0030] The technical solution of the embodiment of the present invention divides the continuous partial discharge signal into multiple time window slices by using the parameters obtained through qualitative and quantitative analysis, and combines the time window slices into a first matrix, thereby reasonably balancing the hardware resources, model performance and computing efficiency; the frequency domain signal of the first matrix is ​​calculated by using the fast Fourier transform (FFT), and the frequency domain signal is weighted by a preset learnable weight vector, and then the frequency domain signal is converted back to a time domain sequence by using the inverse fast Fourier transform (IFFT), and then the features of the time domain sequence are extracted by a multilayer perceptron (MLP). Before the data enters the frequency domain signal weighting and multi-layer perceptron parts, they are all processed by layer normalization, and the outputs are added to the corresponding inputs through residual connections to finally form a third matrix; the Transformer model with an improved pre-layer normalization structure is adopted, and the third matrix is ​​deeply encoded through the self-attention block and feedforward neural network in the n-layer encoder. Each layer includes a multi-head self-attention mechanism and a feedforward neural network, and both are added with pre-layer normalization and residual connection to capture the complex dependencies within the sequence, and generate the encoded fourth matrix. The matrix integrates multi-level time and frequency information, significantly improving the richness and accuracy of feature expression; the fourth matrix is ​​input into the predetermined fully connected layer, and the final classification result of the defect type of the partial discharge signal is obtained through the softmax function, which not only effectively captures the complex patterns and dynamic changes in the partial discharge signal, but also significantly improves the accuracy and robustness of the classification by introducing the front and back position information and multi-scale feature extraction;

[0031] In a further embodiment, adjacent sequence segments are partially overlapped so that the positional relationship information of the sequence data is contained in each segment, so that the model can use the relative position information without additional position encoding, thereby enhancing the modeling capability of the time series, more comprehensively capturing the dynamic changes and contextual dependencies in the partial discharge signal, and ultimately achieving more reliable and efficient partial discharge signal defect classification;

[0032] In a further embodiment, a preset learnable weight vector is introduced to dynamically weight different frequency components. The weight is applied to the frequency domain signals of all time window slices in the form of weight sharing, realizing the unified weighted processing of the frequency domain features of the partial discharge signal, which not only reduces the model parameters, but also gives higher importance to the key frequencies, so that the model can learn the frequency domain features applicable to all time window slices, and enhance the model's ability to capture complex time-frequency dynamics; during the training process, the back propagation algorithm is used to optimize and update these weights according to the loss function, ensuring that the model can automatically adjust the weights to minimize the prediction error; the initial value of the weight is randomly drawn from a normal distribution with a standard deviation of 0 to 0.02, which helps to prevent the gradient problem in the early stage of training and provides a good starting point for the model, thereby improving the efficiency of model training and the final classification performance; the calculation of each part of the data in the frequency domain block is firstly optimized by layer normalization to optimize the data distribution, and the processed data is added to the original input data through residual connection, forming a shortcut for information transmission to alleviate the gradient vanishing problem in deep feature extraction and improve the stability and generalization ability of the network.

[0033] In a further embodiment, for the self-attention block and the fully connected layer in the Transformer encoder, layer normalization is performed before calculation to make the input distribution more stable, and a residual connection is added at the end, which helps the gradient to be better transferred during the training process, helps the model get rid of the warm-up stage in the early stage of training, and makes the early stage of training more stable. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic flow chart of a method for identifying a defect type of a partial discharge signal according to an embodiment of the present invention;

[0035] Figure 2 A flowchart of a method for identifying a partial discharge signal defect type according to an embodiment of the present invention;

[0036] Figure 3 It is a schematic diagram of time window moving slice data in a method for identifying defect types of partial discharge signals according to an embodiment of the present invention;

[0037] Figure 4It is a schematic diagram of the structure of a device for identifying defect types of partial discharge signals according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] To further illustrate the various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, a person of ordinary skill in the art should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0039] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0040] Embodiment 1:

[0041] In order to achieve efficient feature extraction and accurate classification of partial discharge signals and further improve the accuracy and efficiency of partial discharge signal defect identification, an embodiment of the present invention provides a method for identifying the defect type of a partial discharge signal. Figure 1 FIG. 1 is a flow chart of a method for identifying a defect type of a partial discharge signal according to an embodiment of the present invention. Figure 1 As shown, the method is based on the Transformer model architecture improved by the normalized structure of the front layer, and adds a predetermined frequency domain block, and identifies the defect type of the partial discharge signal by combining the time domain features and the frequency domain features. The Transformer model architecture includes: an n-layer encoder, the encoder includes: a self-attention block and a feedforward neural network, and the recognition method includes:

[0042] S1, preprocessing the time series of the collected partial discharge signal; wherein the preprocessing includes:

[0043] The time series is normalized, and then a moving time window moving slice process is performed according to a predetermined time window size and a predetermined step length to obtain a plurality of time window slices; wherein, there is a predetermined overlap between the plurality of time window slices, so that each time window slice contains the time information of the time window slices adjacent to it before and after; the predetermined time window size, the predetermined step length and the predetermined overlap are adjusted and the values ​​are confirmed according to the parameter amount of the Transformer model calculated by the following formula:

[0044]

[0045] Params Total Represents the number of parameters of the Transformer model, d model represents the time window size, l represents the step size;

[0046] Combining a number of time window slices into a first matrix;

[0047] S2, inputting the first matrix into a predetermined frequency domain block for frequency domain transformation and feature extraction, including:

[0048] The first matrix is ​​layer-normalized and calculated by fast Fourier transform to obtain a frequency domain signal;

[0049] The frequency domain signal is weighted according to a preset learnable weight equal to the length of the time window slice, the frequency domain signal is converted into a time domain sequence by inverse fast Fourier transform, and the time domain sequence is added and fused with the first matrix by residual connection to obtain a second matrix;

[0050] The second matrix is ​​layer-normalized, and features are extracted from the second matrix through a multi-layer perceptron to obtain a feature vector; the feature vector is added and fused with the second matrix through a residual connection to obtain a third matrix, which includes: time domain features and frequency domain features extracted from the second matrix;

[0051] S3, inputting the third matrix into the front-layer normalized Transformer encoder for encoding to obtain a fourth matrix; wherein the encoding step includes:

[0052] The third matrix of the time domain sequence is encoded through the multi-head attention mechanism of the self-attention block and the feedforward neural network.

[0053]

[0054] Where l = [1, n], H' represents the matrix obtained after the third matrix is ​​processed by the self-attention block, and H l represents the matrix output by the encoder of the lth layer, LayerNorm represents layer normalization, MultiHeadAttn represents the multi-head self-attention mechanism, and FFN represents the feedforward neural network; the calculations of the self-attention block and the feedforward neural network are first layer normalized and then residually connected;

[0055] S4, inputting the fourth matrix into a predetermined fully connected layer to obtain a classification result of the partial discharge signal defect type, which includes:

[0056]

[0057] in, represents the classification result, softmax represents the normalized exponential function, W n represents the weight matrix, H n represents the fourth matrix, b n Represents the bias vector.

[0058] Embodiment 2:

[0059] In order to solve the problem of defect type classification of partial discharge ultrasonic time series signals, an embodiment of the present invention proposes a method for identifying defect types of partial discharge signals. The identification method is based on an improved Transformer partial discharge time series classification model.

[0060] The original Transformer structure is designed to place layer normalization after the residual network composed of multi-head self-attention. This structure is called Post-Layer Normalization (Post-LN). When calculating the gradient, such a structure cannot be directly transferred to the next sublayer through the residual connection. When the model layer is very deep, it will cause the gradient norm to grow, and the use of a large learning rate in the early stage of training will make the training process unstable. Therefore, it is necessary to preheat the learning rate at the beginning of training so that the model can better adapt to the data in the early stage of training and improve the stability and performance of training. However, the hyperparameter optimization and slow training process caused by preheating the learning rate are costly in large-scale model training. Therefore, the embodiment of the present invention modifies the model into a pre-layer normalization (Pre-Layer Normalization, Pre-LN) structure to achieve a process that does not require preheating the learning rate and can keep the gradient norm of each layer approximately unchanged.

[0061] The automatic pattern recognition of partial discharge signals is usually used in online monitoring systems. The sample data obtained is large in size, but the samples that actually contain discharge signals are limited. In addition, when the same type of discharge occurs in different devices, there are certain differences in their signals. The overfitting and stability problems of the model are more prominent, which is also the reason why the current recognition methods have a high misjudgment rate. Based on this consideration, the Pre-LN structure is used to improve the conventional Transformer architecture.

[0062] In addition, the inventors found that in the processing of local discharge signal data, since the discharge signal is weak and the background interference is strong, the introduction of position coding will have a serious impact on the original time domain signal sequence, resulting in poor model training effect, and even lower recognition accuracy than the model without position coding. Therefore, the local discharge time domain data is processed by moving time window slicing to replace the position coding link in the Transformer model architecture, thereby retaining the position information; and the frequency domain features of the time series are extracted by using frequency domain blocks, and the inter-layer structure of the Transformer model is adjusted to solve the learning rate preheating problem in the initial stage of model training; preferably, the Transformer model architecture is a Transformer architecture that only includes an encoder (Encoder-Only).

[0063] Figure 2FIG. 1 is a flow chart of a method for identifying a type of partial discharge signal defect according to an embodiment of the present invention. Figure 2 As shown, the method is based on the Transformer model architecture improved by the front layer normalization structure to identify the type of partial discharge signal defects, wherein the Transformer model architecture includes: n stacked Transformer encoders, the encoder includes: a self-attention block and a feedforward neural network; the identification method includes:

[0064] (1) Preprocessing the time series of the collected partial discharge signal;

[0065] Specifically, the time series is normalized, and then a moving time window slice process is performed according to a predetermined time window size and a predetermined step length to obtain a plurality of time window slices, and the plurality of time window slices are combined into a first matrix; wherein there are overlapping parts between the plurality of time window slices, so that each time window slice contains time information of the time window slices adjacent to the front and back thereof;

[0066] In a specific embodiment, Figure 3 FIG. 1 is a schematic diagram of time window moving slice data in a method for identifying a defect type of a partial discharge signal according to an embodiment of the present invention. Figure 3 As shown, each time window slice contains part of the time signal of the adjacent time window, so that the model can utilize and capture the previous and next time information without position encoding;

[0067] In a specific embodiment, for a time series that is continuously collected and segmented according to 20ms, the predetermined time window size is 45 time nodes, the predetermined step size is 15 time nodes, and there is a 2 / 3 overlap between two time window slices; the sliced ​​time window slices are combined into a matrix, specifically, each time domain data is sliced ​​into a 104*45 matrix to replace word embedding;

[0068] Among them, for the moving time window slicing data method, by defining the time window size d model The step size, i.e., the step length l, directly affects the size of the input matrix, which further affects the model's parameter scale, computational complexity, and memory consumption. Therefore, according to the model structure, the calculation formula for the total number of model parameters can be obtained:

[0069]

[0070] In the above formula, d hidden is the dimension of the hidden layer of the multilayer perceptron (MLP), n is the number of layers of the encoder, and d ffn is the dimension of the hidden layer of the feedforward neural network in the encoder, L is the length of the local discharge time domain signal before slicing, doutput is the number of classifications. These parameters are known when designing the model, so the above formula can be further simplified by substituting specific values ​​into it, as shown below:

[0071]

[0072] After analysis, it can be seen that in the model parameter estimation, the window size d model The change of plays a dominant role. When the window size d model As the value increases, the parameter quantity will show a rapid growth trend, mainly due to the fact that d model The reduction of the step size l (i.e., the increase of the overlap of the time windows) will also lead to a further increase in the number of parameters, and also increase the computational complexity of the self-attention mechanism O(n 2 )(where n is the number of fragments).

[0073] Therefore, the selection of the time window size and step size requires a trade-off between model performance, computing resources, and memory consumption. Larger time windows and smaller step sizes can improve the model's ability to capture temporal dependencies, but may also lead to a sharp increase in computational complexity and memory consumption, especially in the processing of long sequence data; smaller time windows and larger step sizes can reduce model parameters, but the reduction in the number of parameters will lead to a decrease in the model's capacity, making it difficult to capture the complex features of the input data, especially long temporal dependencies or highly nonlinear patterns, thereby affecting the model's generalization ability. Large step sizes will also reduce the overlap between adjacent segments, resulting in the loss of local front-and-back position information.

[0074] Through quantitative and qualitative analysis, a compromise parameter configuration is selected in this embodiment, and the window size d model =45 and step size l = 15, so that there is about 2 / 3 overlap between adjacent time window segments. In actual selection, the selection of time window size and step size needs to be adjusted according to the characteristics and requirements of the specific task, and combined with the constraints of hardware resources to ensure the optimal balance between model performance and computational efficiency.

[0075] (2) inputting the first matrix into a predetermined frequency domain block for frequency domain transformation and feature extraction;

[0076] Introducing frequency domain information into the model input process enables the Transformer model to simultaneously analyze time and frequency domain features to distinguish different types of partial discharges, which can improve the performance and robustness of the model and reduce the risk of overfitting caused by extracting time domain information alone.

[0077] Specifically, the first matrix is ​​layer-normalized and calculated by fast Fourier transform to obtain a frequency domain signal;

[0078] The frequency domain signal is weighted according to a preset learnable weight equal to the length of the time window slice, the frequency domain signal is converted into a time domain sequence by inverse fast Fourier transform, and the time domain sequence is added and fused with the first matrix by residual connection to obtain a second matrix;

[0079] The second matrix is ​​layer-normalized, and features are extracted from the second matrix through a multi-layer perceptron to obtain a feature vector; the feature vector is added and fused with the second matrix through a residual connection to obtain a third matrix, which includes: time domain features and frequency domain features extracted from the second matrix;

[0080] Preferably, a preset learnable weight vector equal to the length of the time window slice is applied to the frequency domain signal in the form of weight sharing;

[0081] Preferably, during the training process of the model, a preset learnable weight vector equal to the length of the time window slice for extracting frequency domain signal features is updated according to the calculated loss function through a back propagation algorithm; preferably, the first matrix and the second matrix are layer normalized before being respectively input into the frequency domain feature weighting and the multilayer perceptron in the frequency domain block, and residual connection is performed after the frequency domain feature weighting or the output of the multilayer perceptron.

[0082] In a specific embodiment, the number of model parameters is reduced by narrowing the range of initial weights. A smaller initial weight value helps prevent gradient explosion, thereby improving the stability of the model during training, helping the network to start the learning process more smoothly, and further improving the stability and speed of model convergence; preferably, the initial value of a preset learnable weight vector is randomly extracted from a probability distribution with a standard deviation of 0 to 0.02; in a specific implementation, the standard deviation of the learnable weight vector is 0.02 when initialized.

[0083] (3) Input the third matrix into the front-layer normalized Transformer encoder for encoding to obtain a fourth matrix;

[0084] Specifically, the third matrix of the time domain sequence is encoded through the multi-head attention mechanism of the self-attention block and the feedforward neural network.

[0085]

[0086] Where l = [1, n], H' represents the matrix obtained after the third matrix is ​​processed by the self-attention block, and H l represents the matrix output by the encoder of the lth layer, LayerNorm represents layer normalization, MultiHeadAttn represents the multi-head self-attention mechanism, FFN represents the feedforward neural network, and the calculations of the self-attention block and the feedforward neural network are first layer normalized and then residual connections are added;

[0087] Preferably, the third matrix is ​​subjected to layer normalization processing before being input into the self-attention block or the feed-forward neural network, and a residual connection is performed after being output from the self-attention block or the feed-forward neural network;

[0088] In a specific embodiment, the third matrix H after frequency domain block processing is 0 The n-layer Transformer encoder is passed to the final output H n .

[0089] (4) inputting the fourth matrix into a predetermined fully connected layer to obtain a classification result of the partial discharge signal defect type;

[0090] Specifically,

[0091]

[0092] in, represents the classification result, softmax represents the normalized exponential function, W n represents the weight matrix, H n represents the fourth matrix, b n represents the bias vector;

[0093] Preferably, the local discharge signal defect types include: tip corona discharge, internal air gap discharge, floating potential discharge and / or surface discharge.

[0094] In a specific embodiment, for the improved Transformer model designed in this embodiment, an ablation experiment is used to verify the contribution of each model component to the overall recognition effect and performance index of partial discharge classification. In the experiment, the Transformer encoder of Post-LN is used as the benchmark network, and the position encoding (PE), time window moving slice and frequency domain block are ablated on the produced data set. Table 1 shows the various indicators in the ablation experiment, where the bold font is the optimal result of the indicator. Among them, the accuracy ACC focuses on measuring the classification effect of the model on the overall sample, while the average precision mean mAP can reflect the classification performance of the model in different categories, and more comprehensively evaluate the accuracy and recall rate.

[0095]

[0096] Table 1 Evaluation of ablation experimental model

[0097] As shown in Table 1, compared with the baseline model, the ACC and mAP of the model after adding position encoding to the baseline model decreased by 1.2% and 6.5% respectively. After replacing the position encoding with the time window moving slice, the ACC and mAP increased by 15.4% and 13.2% respectively compared with the baseline model, which is consistent with expectations. This is because although the addition of position encoding adds position information to each segment, the time domain signal is severely distorted. In addition, the subsequent self-attention block may not be able to infer and summarize the complex position information behind the mathematical formula. While maintaining the waveform of the time domain signal, the data of the time window moving slice uses the overlapping part between the two segments before and after, so that the sliced ​​time segment carries the position information related to the front and back, so that the periodic changes unique to the local discharge time domain signal, the interval between the two discharge pulse clusters, and the two-level symmetry and other time domain waveform features will not be lost due to the parallel calculation of the similarity of all time segments in the self-attention block.

[0098] The addition of the frequency domain block increased the ACC and mAP of the baseline model by 15.3% and 12.7% respectively, proving that the frequency domain block can effectively improve the model's adaptability and classification accuracy for partial discharge time domain signals by weighted extraction of frequency domain information features.

[0099] Compared with the Post-LN baseline model, the ACC of the Pre-LN Transformer model increased by 8.7% and the mAP increased by 9.1%, which is in line with expectations. The Pre-LN structure performs layer normalization before calculating each sub-layer, making the input distribution more stable, which helps the gradient to be better transferred during the training process and makes the initial training more stable. Finally, the ACC and mAP of the Transformer model with the pre-layer normalization structure of the embodiment of the present invention reached the optimal 97.9% and 99.6%,

[0100] In a specific embodiment, in order to further verify the superiority of the embodiment of the present invention in the classification of partial discharge time domain signals, classic sequence signal classification models such as LSTM, 1D-CNN, SAE, RNN, GRU, etc. were selected for comparison with the Transformer model of the front layer normalization structure of the embodiment of the invention. Among them, the first four models have been widely used in power equipment status diagnosis and prediction tasks, and the GRU proposed to alleviate the problem of LSTM gradient disappearance or explosion also appears as an insertion module in various time series tasks. The training method of each network is consistent with the Transformer model of the front layer normalization structure of the embodiment of the present invention. The comparison results are shown in Table 2.

[0101]

[0102] Table 2 Comparison of different method models

[0103] Table 2 compares the accuracy, memory consumption and storage space evaluation indicators of the models horizontally. The accuracy reflects the classification performance of the model, and the memory consumption and storage space reflect the ease of deployment of the model in the future. The model of the embodiment of the present invention is 3.01% higher in accuracy than 1D-CNN, while other methods fail to learn effective time domain waveform features on the local discharge time domain data, and the overall component effect is poor; the memory consumption of the model of the embodiment of the present invention is the highest among all models, while the storage space of the parameters is the smallest among all models, and the minimum parameter amount is only 1.4% of the largest model. The main reason is that the self-attention mechanism enables the Transformer model to capture the global dependencies of different positions in the input sequence, thereby improving the expressiveness and learning ability of the model. However, the computational complexity and spatial complexity of multi-head attention are quadratic of the sequence length, which will lead to an increase in memory consumption.

[0104] In summary, the embodiment of the present invention adopts the method of moving time window slice data to replace the position encoding link of Transformer, adds frequency domain blocks to strengthen Transformer's processing of the power frequency correlation feature information of the local discharge signal, and improves the Transformer encoder with the pre-layer normalization structure. And through ablation experiments, the necessity and improvement efficiency of each improved module are analyzed, and compared with five classic sequence data classification models. The results show that the recognition method proposed in the embodiment of the present invention enhances the recognition ability of Transformer for the characteristics of local discharge signals, and the improvement has achieved significant results. The recognition accuracy of the verification case is 97.9%, which is better than the other five classic sequence data models for comparison.

[0105] Embodiment three:

[0106] The present invention also provides a device for identifying the type of partial discharge signal defects, such as Figure 4 As shown, the device includes a processor 401, a memory 402, a bus 403, and a computer program stored in the memory 402 and executable on the processor 401. The processor 401 includes one or more processing cores. The memory 402 is connected to the processor 401 via the bus 403. The memory 402 is used to store program instructions. When the processor 401 executes the computer program, the steps in the above method embodiment of the first embodiment of the present invention are implemented.

[0107] Further, as an executable solution, the device for identifying the type of partial discharge signal defect may be a computer unit, which may be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The computer unit may include, but is not limited to, a processor and a memory. Those skilled in the art may understand that the composition structure of the above-mentioned computer unit is only an example of a computer unit and does not constitute a limitation on the computer unit, and may include more or less components than the above-mentioned, or a combination of certain components, or different components. For example, the computer unit may also include input and output devices, network access devices, buses, etc., which are not limited in the embodiments of the present invention.

[0108] Further, as an executable solution, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, and various interfaces and lines are used to connect the various parts of the entire computer unit.

[0109] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer unit by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0110] Embodiment 4:

[0111] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described above are implemented.

[0112] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. A method for identifying the defect type of a partial discharge signal, characterized in that: Based on the improved Transformer model architecture of the front layer normalization structure, a predetermined frequency domain block is added to identify the defect type of partial discharge signal by combining time domain features and frequency domain features, wherein the Transformer model architecture includes: an n-layer encoder, the encoder includes: a self-attention block and a feedforward neural network, and the identification method includes: S1, preprocessing the time series of the collected partial discharge signal; wherein the preprocessing includes: The time series is normalized, and then a moving time window moving slice process is performed according to a predetermined time window size and a predetermined step length to obtain a plurality of time window slices; wherein there is a predetermined overlapping portion between the plurality of time window slices, so that each time window slice contains the time information of the time window slices adjacent to it before and after; the predetermined time window size, the predetermined step length and the predetermined overlapping portion are adjusted and the values ​​are confirmed according to the parameter amount of the Transformer model calculated by the following formula: Params Total represents the number of parameters of the Transformer model, d model represents the time window size, l represents the step size; Combining the plurality of time window slices into a first matrix; S2, inputting the first matrix into a predetermined frequency domain block for frequency domain transformation and feature extraction, including: Normalizing the first matrix layer by layer, and calculating it by fast Fourier transform to obtain a frequency domain signal; The frequency domain signal is weighted according to a preset learnable weight vector equal to the length of the time window slice, the frequency domain signal is converted into a time domain sequence by inverse fast Fourier transform, and the time domain sequence is added and fused with the first matrix by residual connection to obtain a second matrix; The second matrix is ​​layer normalized, and features are extracted from the second matrix through a multilayer perceptron to obtain a feature vector; the feature vector is added and fused with the second matrix through a residual connection to obtain a third matrix, wherein the third matrix includes: time domain features and frequency domain features extracted from the second matrix; S3, inputting the third matrix into the front-layer normalized Transformer encoder for encoding to obtain a fourth matrix; wherein the encoding step includes: The third matrix of the time domain sequence is encoded by the multi-head attention mechanism of the self-attention block and the feedforward neural network, Wherein, l = [1, n], H' represents the matrix obtained after the third matrix is ​​processed by the self-attention block, and H l represents the matrix output by the encoder of the lth layer, LayerNorm represents layer normalization, MultiHeadAttn represents a multi-head self-attention mechanism, and FFN represents a feedforward neural network; wherein the calculations of the self-attention block and the feedforward neural network are first layer normalized and then residually connected; S4, inputting the fourth matrix into a predetermined fully connected layer to obtain a classification result of the partial discharge signal defect type, which includes: in, represents the classification result, softmax represents the normalized exponential function, W n represents the weight matrix, H n represents the fourth matrix, b n Represents the bias vector.

2. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: The local discharge signal defect types include: tip corona discharge, internal air gap discharge, floating potential discharge and / or surface discharge.

3. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: The Transformer model architecture is a Transformer architecture that only includes an encoder.

4. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: A preset learnable weight vector equal to the length of the time window slice is applied to the frequency domain signal in the form of weight sharing.

5. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: During the training process of the model, the preset learnable weight vector equal to the length of the time window slice for extracting frequency domain signal features is updated according to the calculated loss function through the back propagation algorithm.

6. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: The initial value of the learnable weight vector equal to the length of the time window slice is randomly extracted from a probability distribution with a standard deviation of 0 to 0.

02.

7. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: The first matrix and the second matrix are subjected to layer normalization processing before being respectively input into the frequency domain feature weighting and the multilayer perceptron in the predetermined frequency domain block, and residual connection is performed after the frequency domain feature weighting and the multilayer perceptron output.

8. The method for identifying the defect type of partial discharge signal according to claim 1, characterized in that: The third matrix is ​​layer normalized before being input into the self-attention block or the feed-forward neural network, and a residual connection is performed after the output of the self-attention block or the feed-forward neural network.

9. A device for identifying the defect type of a partial discharge signal, characterized in that: The method comprises a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement the method for identifying the defect type of a partial discharge signal according to any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying the defect type of a partial discharge signal according to any one of claims 1 to 8 is implemented.

Images