A Transformer-based power equipment fault detection model

Abstract

This invention discloses a Transformer-based power equipment fault detection model. The training data for the Transformer model comes from a preprocessing model, which includes an upsampling module and a word embedding representation learning module. The upsampling module converts the input unbalanced power equipment data into balanced data. The word embedding representation learning module embeds the balanced data and outputs an embedding representation based on the historical state sequence and label data of the power equipment. The Transformer model employs a multi-head attention mechanism, using the embedding representation of the historical state sequence and label data as input to train and generate the power equipment fault detection model. This invention uses the Transformer framework and, based on the time series information and time location information of the input data, uses a multi-head attention mechanism to perform multi-dimensional feature mining, improving the feature input quality of the power equipment fault detection model and thus enhancing the model's accuracy in power equipment fault detection.

Description

Technical Field

[0001] This invention belongs to the field of power grid equipment fault detection technology, and relates to power grid equipment fault detection models, especially a power equipment fault detection model based on Transformer. Background Technology

[0002] With increasing power demand and the growing complexity of power supply systems, the requirements for the accuracy and real-time performance of power system equipment fault detection are also rising. Currently, most fault detection algorithms for power system equipment employ traditional subjective detection, digital detection, and some deep learning algorithms. These methods, on the one hand, lack an understanding of the insufficient fault sample data in power system equipment, and on the other hand, ignore the temporal location information of corresponding features in the time series when modeling the state of power supply system equipment.

[0003] Therefore, how to propose a power equipment fault detection model that can better optimize, predict, and classify tasks, and improve the model's detection accuracy is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of this invention is to provide a power equipment fault detection model that is reasonably designed and has high detection accuracy.

[0005] To achieve the above objectives, this invention provides a power equipment fault detection model based on Transformer. The power equipment fault detection model includes a Transformer model, and the training data for the Transformer model comes from a preprocessing model. The preprocessing model includes an upsampling module and a word embedding representation learning module. The upsampling module is used to convert the input unbalanced power equipment data into balanced data. The word embedding representation learning module is used to embed the balanced data and output an embedding representation based on the historical state sequence and label data of the power equipment. The Transformer model employs a multi-head attention mechanism, using the embedding representation of the historical state sequence and label data as input to train and generate the power equipment fault detection model.

[0006] Furthermore, the upsampling module employs the Borderline-SMOTE algorithm. The data processing procedure of the upsampling module includes: using the power equipment entity status and labels contained in the power grid power equipment dataset as input to the Borderline-SMOTE upsampling algorithm, employing the nearest neighbor algorithm to calculate the K nearest neighbors of each minority class sample, randomly selecting N samples from the K nearest neighbors for random linear interpolation to construct new minority class samples, then merging the newly generated minority class samples with the original samples to generate a balanced sample dataset, and finally performing embedding representation based on the newly constructed dataset.

[0007] Furthermore, the data processing procedure of the upsampling module specifically includes:

[0008] Step 21: Obtain the power grid equipment dataset, which includes the equipment entity status and corresponding labels;

[0009] Step 22: Traverse the data in the power grid equipment dataset and use the nearest neighbor algorithm to determine the set of nearest neighbors D of sample x. n In the nearest neighbor sample set D n In this context, samples of the same category as sample x are grouped into set D. same The set of samples of different categories from sample x is called D. other ;

[0010] Step 23: Calculate the difference in the number of samples between the majority and minority classes in the original power grid equipment dataset, and generate samples based on this difference;

[0011] Step 24: After obtaining the number of samples generated from the seed samples, the K-means algorithm updates the coordinates of the cluster centers in each iteration by dividing the samples based on the Euclidean distance between the cluster centers and the sampled samples.

[0012] Step 25: Select samples of the same category from each cluster to form a sample set D. c Then, based on the different feature types, appropriate processing is performed;

[0013] Step 26: For each seed sample x i The corresponding category y i The cluster c it belongs to i There exists a corresponding number N of new samples that need to be generated. gi Each time a new sample is generated, based on N gi and the distribution of each feature of the cluster FD[c i ][y i First, generate auxiliary sample x. temp Then, linear interpolation is performed to obtain the final generated sample x. new ; Perform N cycles on the seed samplegj After the sample generation operation, a set of generated samples based on the seed sample is obtained. These generated samples belong to the same category as the seed sample. After each seed sample has completed sample generation, the resulting set of generated samples D is... g By merging it with the original dataset D, we obtain the final balanced dataset D. balance ;

[0014] Step 27: Perform embedding representation on the finally obtained samples. The data form of the embedded samples is defined as M×N, where M is the number of samples used to represent the description of different power equipment; N is the number of features.

[0015] Furthermore, the minority class samples in step 23 are divided into three categories: when all k-nearest neighbors of a minority class sample belong to the majority class, it is defined as "noise"; when more than half of all k-nearest neighbors of a minority class sample belong to the majority class, it is defined as "danger"; when more than half of all k-nearest neighbors of a minority class sample belong to the minority class, it is defined as "safe"; then, a sample is randomly selected from the samples in the "danger" state as a seed sample.

[0016] Furthermore, the seed sample in step 23 is obtained using the following formula:

[0017] label_diff j =N maj -N j

[0018]

[0019]

[0020] Where, N maj N represents the number of samples for the majority class seed. j This indicates that it belongs to category C. j The number of samples; D s_maj D represents the set of seed samples belonging to the majority class; s_j This indicates that it belongs to category C. j Seed sample set; R sj N represents the number of samples that need to be generated for each seed sample to balance the difference in the number of seed samples; gj Represent each category C j The average number of new samples generated from the seed samples.

[0021] Furthermore, in step 24, the hyperparameter k of the K-means algorithm... c Represented as:

[0022]

[0023] Where, N maj N represents the number of majority class samples in the dataset; min This indicates the number of minority class samples in the dataset.

[0024] Furthermore, the feature types in step 25 are divided into discrete features and continuous features.

[0025] For discrete features, the selection of discrete features is determined based on the frequency of occurrence of different fields;

[0026] For continuous features, calculate the maximum value (max) and minimum value (min) of the feature, and give the value interval [min, max].

[0027] Furthermore, the auxiliary sample x in step 26 temp Constructed based on feature distribution, satisfying three rules:

[0028] Temporary sample x temp With sample x i Labels belonging to the same category y i ;

[0029] Temporary sample x temp With sample x i Belonging to the same cluster c i ;

[0030] Temporary sample x temp With sample x i They have the same characteristics, but the eigenvalues ​​of each characteristic are determined according to the cluster c. i Feature distribution FD[c i ][y i Obtained by random sampling;

[0031] The sample x new The calculation formula is:

[0032] x temp = [f1, f2, ..., f p ], f p =Random(FD[c i ][y i [p])

[0033] x new =x + Random(0,1)×(x temp -x)

[0034] Where FD is the feature distribution matrix, and x represents the minority class sample.

[0035] Furthermore, the Transformer model includes:

[0036] The input data processing module is used to perform position encoding on the embedded representation of the input power equipment state sequence information;

[0037] The Encoder module is used to encode the input power equipment state sequence information according to the time order, and then use a multi-head attention mechanism to achieve self-attention information fusion.

[0038] The Decoder module is used to decode the encoding generated by the Encoder module using a multi-head attention mechanism combined with a mask;

[0039] The output of the Decoder module is processed by Linear and softmax to output the prediction result.

[0040] Furthermore, the power equipment state sequence information is represented in the following form:

[0041] X b = [b1, b2, ..., b t ]

[0042] in, N is the number of samples, t is the number of power equipment states in the time series, and b i It represents the i-th state in the sequence, and is a one-hot encoded state.

[0043] For sequence information X b The embedding layer encodes it as a low-dimensional dense representation:

[0044] e b = [e1, e2, ..., e t ]

[0045] in, k represents the size of the embedding layer.

[0046] Advantages and beneficial effects of the present invention:

[0047] (1) This invention uses Transformer technology to extract effective features from input data at different times based on the time and location information contained in the input data, and then uses them as input to the Transformer model. This effectively solves the problem of insufficient utilization of the temporal features of the time and location information of the input data.

[0048] This invention uses the Transformer framework to perform multi-dimensional feature mining based on the time series information and time location information of the input data, thereby improving the feature input quality of the power equipment fault detection model and thus improving the accuracy of the model in detecting power equipment faults. Attached Figure Description

[0049] Figure 1 This is a flowchart of the Borderline-SMOTE-based upsampling process of the present invention;

[0050] Figure 2 This is a schematic diagram of the power equipment fault detection model processing flow based on Transformer technology of the present invention;

[0051] Figure 3 This is a general framework diagram of power equipment fault detection according to the present invention;

[0052] Figure 4 This is a verification framework diagram of the power equipment fault detection model of the present invention. Detailed Implementation

[0053] To further illustrate the various embodiments, the present invention provides accompanying drawings. These drawings are part of the disclosure of the present invention, primarily used to illustrate the embodiments, and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these drawings, those skilled in the art should be able to understand other possible implementations and the advantages of the present invention.

[0054] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments.

[0055] like Figures 1 to 4 As shown, this invention provides a power equipment fault detection model based on Transformer. The power equipment fault detection model includes a Transformer model, and the training data of the Transformer model comes from a preprocessing model. The preprocessing model includes an upsampling module and a word embedding representation learning module. The upsampling module is used to convert the input unbalanced power equipment data into balanced data. The word embedding representation learning module is used to embed the balanced data and output the embedding representation based on the historical state sequence and label data of the power equipment. The Transformer model is trained to generate the power equipment fault detection model using the embedding representation of the historical state sequence and label data as input.

[0056] The training of the power equipment fault detection model includes the following steps:

[0057] Step 1: Input the power grid equipment dataset, which includes the physical status and labels of the power grid or power equipment. The physical status of the power equipment includes: the gas composition content in the transformer oil, the partial discharge status of the transformer, the temperature of the equipment contact surfaces, the moisture status of internal components, and other status information. This status information is accompanied by time information, forming a historical status sequence of the power equipment. The labels correspond to power equipment such as transformer fault types, which include: insulation degradation, abnormal vibration, etc. The severity of the identified faults can also be classified.

[0058] Step 2: Use the Borderline-SMOTE upsampling algorithm to convert the input unbalanced power equipment dataset into a balanced dataset, and then perform embedding representation.

[0059] The specific steps of step 2 include:

[0060] Using the power equipment entity states and labels from the power grid equipment dataset in Step 1 as input to the Borderline-SMOTE upsampling algorithm, the nearest neighbor algorithm is employed to calculate the K nearest neighbors for each minority class sample. N samples are randomly selected from these K nearest neighbors and subjected to random linear interpolation to construct new minority class samples. These newly generated minority class samples are then merged with the original samples to generate a balanced sample dataset. Finally, embedding representation is performed based on the newly constructed dataset.

[0061] (1) Obtain a power grid equipment dataset, which includes the status of equipment entities and their corresponding labels;

[0062] (2) Traverse the data in the power grid equipment dataset and use the nearest neighbor algorithm (KNN algorithm) to determine the set of nearest neighbor samples D for sample x. n In the nearest neighbor set D n In the sample x, there are samples of different categories, and samples of the same category as sample x are grouped into a set D. same The set of samples of different categories from sample x is called D. other .

[0063] (3) Calculate the difference in the number of samples in the majority and minority classes in the original dataset:

[0064] label_diff j =N maj -N j

[0065] Where, N maj N represents the number of samples belonging to the majority class. j This indicates that it belongs to category C. j Sample size:

[0066] All minority class samples are divided into three categories: when all k-nearest neighbors of a minority class sample belong to the majority class, it is defined as "noise"; when more than half of all k-nearest neighbors of a minority class sample belong to the majority class, it is defined as "danger"; when more than half of all k-nearest neighbors of a minority class sample belong to the minority class, it is defined as "safe". Then, samples in the "danger" state are randomly selected as seed samples.

[0067] In the seed sample set, the majority class seeds outnumber the minority class seeds, and the sampling rate of the majority class seeds is 100%. To compensate for the difference in the number of seed samples, it is necessary to calculate the ratio R between the majority class seeds and the minority class seeds. sj :

[0068]

[0069] Among them, D s_maj D represents the set of seed samples belonging to the majority class; s_j This indicates that it belongs to category C. j The seed sample set.

[0070] N gj Represent each category C j The average number of new samples generated from the seed samples:

[0071]

[0072] Among them, label_diff j / |D s_j | indicates the number of samples that need to be generated for each seed sample in order to balance the difference in the number of original data.

[0073] (4) After obtaining the number of samples generated from the seed samples, the K-means algorithm updates the cluster center coordinates in each iteration based on the Euclidean distance between the cluster centers and the sampled samples. The hyperparameter k of the K-means algorithm... c The hyperparameter k represents the number of clusters in the Borderline-SMOTE algorithm. c The value depends on the ratio of the majority class to the minority class in the dataset:

[0074]

[0075] After clustering the dataset using the standard K-means algorithm, each sample is labeled with its cluster label C, and the dataset is updated as follows:

[0076]

[0077] (5) Select samples of the same category from each cluster to form a sample set D. c Each sample contains a feature set F = {f1, f2, ..., f...} p Then, based on the different feature types, appropriate processing is performed.

[0078] For discrete features, such as "abnormal noise" or "abnormal vibration of machinery and equipment," the selection of discrete features cannot be randomized from all fields. It needs to be determined based on the frequency of occurrence of different fields to ensure that the feature distribution of the generated samples and the final balanced dataset remains unchanged.

[0079] For continuous features, such as "equipment temperature data," the data generation process needs to consider values ​​within the [min, max] interval. Therefore, it's necessary to calculate the maximum and minimum values ​​of the feature, and then randomly select data from this [min, max] interval as the generated values. For K... c There are p features of L different categories in _ ... c The characteristic distribution matrix FD is (×L×p×2).

[0080] (6) For each seed sample x i The corresponding category y i The cluster c it belongs to i There exists a corresponding number N of new samples that need to be generated. gi Each time a new sample is generated, based on N gi and the distribution of each feature of the cluster FD[c i ][y i First, generate auxiliary sample x. temp Then, linear interpolation is performed to obtain the final generated sample x. new .

[0081] The Borderline-SMOTE algorithm first constructs an auxiliary sample x based on the feature distribution. temp Auxiliary sample x temp Three rules need to be met:

[0082] Temporary sample x temp With sample x i Labels belonging to the same category y i ;

[0083] Temporary sample x temp With sample x i Belonging to the same cluster c i ;

[0084] Temporary sample x temp With sample xi They have the same characteristics, but the eigenvalues ​​of each characteristic are determined according to the cluster c. i Feature distribution FD[c i ][y i Obtained by random sampling.

[0085] Obtain temporary sample x temp Then, a new sample x can be obtained through linear interpolation. new :

[0086] x temp = [f1, f2, ..., f p ], f p =Random(FD[c i ][y i [p])

[0087] x new =x + Random(0,1)×(x temp -x)

[0088] Repeat N times on the seed sample gj After the sample generation operation, a set of generated samples based on the seed sample is obtained. These generated samples belong to the same category as the seed sample. Once the sample generation for each seed sample is complete, the resulting set of generated samples D is... g By merging it with the original dataset D, we obtain the final balanced dataset D. balance After balancing, the ratio of majority to minority classes in the dataset returned to normal, and the overall sample size was also increased.

[0089] (7) For the final obtained samples, the data format is defined as M×N, where M is the number of samples, representing the description of different power equipment. N is the number of features, including equipment temperature, equipment image features, equipment parameter features, and context features, etc. The final obtained samples are embedded and represented. In the feature processing process, it is generally customary to discretize continuous features. Discrete features, after encoding, will make the data matrix extremely sparse. If they are not effectively processed, the number of parameters in the subsequent modeling process will increase dramatically. The main function of the data embedding layer is to compress and represent the sparse vector after one-hot encoding. After the data vector is embedded, the dimension will be significantly reduced, and the feature information will be mainly represented in numerical form. Assume that the feature vector after one-hot encoding is represented as [x1; x2; ...; x n ], where n is the number of feature domains, x i Let V be the one-hot code representation of feature domain i. The size of the embedding layer matrix V is n×k, where k is the size of the embedding layer vector.

[0090] After the embedding layer, the sparse vector will be encoded into a dense vector of equal length. Let the output of the embedding layer be E, as shown below.

[0091] E = [e1, e2, ..., e n ] = [v1x1, v2x2, ..., v n x n ]

[0092] Among them, e i This represents the feature domain vector. For single-valued features, each x... i If there is one and only one bit that is 1, the feature domain vector represents the feature vector. For multiple features, e in this case... i It has multiple vectors. This ultimately completes the embedding representation of the dataset.

[0093] Step 3: Based on the sample embedding representation obtained in Step 2, define a Transformer-based power equipment fault detection model.

[0094] The specific steps of step 3 include:

[0095] (1) Define the input data processing module. First, there's the data input. In the Transformer network, the input to the model is the power equipment state sequence information, represented by X. b In other words, behavioral sequence information is generally represented using a list of power equipment states at a certain time series, and its representation is as follows.

[0096] X b = [b1, b2, ..., b t ]

[0097] in, N is the number of samples, t is the number of power equipment states in the time series, and b i It represents the i-th state in the sequence, and is in the form of one-hot encoding.

[0098] After obtaining the one-hot encoded input, we densify the high-dimensional sparse vector representation for the sequence information X. b The embedding layer encodes it as a low-dimensional dense representation, as shown below.

[0099] e b = [e1, e2, ..., e t ]

[0100] in, k represents the size of the embedding layer. Then, using the temporal location information, the embedded representation of the input power equipment state sequence information is positionally encoded.

[0101] (2) Define the Encoder module.

[0102] Then, the Transformer performs multi-head attention learning to fully acquire contextual and environmental feature information. After position-wise Feed Forward, each sub-layer is connected through residuals, which facilitates rapid network updates.

[0103] (3) Define the Decoder module. This module also performs positional encoding on the information input from the Encoder, and then performs multi-head attention learning to fully acquire the feature information of the context and environment. Then it goes through position-wise Feed Forward. Here, each sub-layer is connected through residuals. However, a Masked Multi-head attention module needs to be added during decoding because this is a prediction process and some information is actually unknown and needs to be hidden.

[0104] (4) Finally, the output of the Decoder module is processed by Linear and softmax to output the probability. Then, the label is used for supervised learning, and the parameters are updated by backpropagation.

[0105] Step 4: First, the embedded representation generated in Step 2 is used as the input to the power equipment fault detection model obtained in Step 3, and finally the power equipment fault detection model based on the Transformer network is trained and generated.

[0106] The specific steps of step 4 include:

[0107] (1) Implementation of the Transformer-based network framework: First, based on the embedding layer in step 2, corresponding embedding representations are generated for the constructed balanced samples. Second, the temporal position information of the input data is encoded. Then, an Encoder module of the Transformer network is defined. This module first encodes the input sequence data according to the temporal order, and then uses a multi-head attention mechanism to achieve self-attention information fusion, which can effectively capture the temporal, positional, and related information of the input data. Next, a Decoder module of the Transformer network is defined. This module decodes the encoding generated by the Encoder module. Like the Encoder module, it uses a multi-head attention mechanism and normalization operation. The difference is that a mask mode is added to this module to cover up the part of the information that should be predicted, so that the model can be trained better. Finally, the output of the Decoder is processed by Linear and softmax to output the prediction result, i.e., the classification of the detection result. Its network architecture is as follows: Figure 3 As shown.

[0108] (2) Set the number of training iterations epochs, and start training from epochs equal to 1.

[0109] (3) Obtain the embedding representation of the dataset samples from step 2), and then input the embedding representation of the data in batches into the Transformer-based power equipment fault detection model to obtain the prediction of the input data.

[0110] (4) Calculate the loss function that minimizes the estimated value and the actual label value.

[0111] (5) Within the range of values ​​defined by epochs, repeat the steps in (3) and (4) to finally train a power equipment fault detection model based on Transformer.

[0112] In power equipment fault detection systems, the primary focus is on feature extraction capabilities from input samples. The innovation of this invention lies in its use of Transformer technology for feature mining of input data. This allows for the acquisition of both temporal and spatial features of the input data, while simultaneously encoding temporal and location information. Furthermore, an attention mechanism can be used to utilize and mine key features. This model employs multi-head attention mechanisms in both the Encoder and Decoder modules, enabling feature mining of input data across multiple dimensions. Based on these two aspects, the features of the input sample data can be fully mined and utilized, allowing the network to acquire more useful information and achieve more accurate power equipment fault detection. Many methods address these issues by improving upon multimodal fusion. The method proposed in this paper differs from previous approaches by combining Transformer technology to achieve multi-dimensional feature mining of input data and encoding location information, thereby obtaining more feature information and enhancing the network model's fault detection capabilities.

[0113] The method in this invention is designed to fully mine the features of power equipment using a Transformer network, better serving the classification and detection tasks of downstream neural networks. In the Transformer input processing module, we use sine and cosine functions to encode the input information in time and location. The encoded data is then input into a predefined Encoder module, where a multi-head attention mechanism is used to extract multi-dimensional features. The output data from the Encoder module is then used as the input data for the Decoder module, where a multi-head attention mechanism combined with a mask is used for decoding. Finally, the data is decoded according to the dimension of the target label, and the result is fed into a classification network for supervised training, ultimately generating a power equipment fault detection model.

[0114] Based on the above improvements, the Transformer-based power equipment fault detection model proposed in this invention has been realized. This method can effectively improve the accuracy of power equipment fault detection.

[0115] The working principle of this invention is:

[0116] This invention first performs Borderline-SMOTE upsampling on power grid equipment samples to generate balanced sample data. Then, it uses an embedding layer to embed the input samples. Next, it encodes the generated embedding representation using a position encoding module. The position-encoded input data embedding representation and the corresponding label embedding representation are used together as input to a Transformer-based model for model training, generating a model that can accurately detect power equipment faults.

[0117] It should be emphasized that the embodiments described in this invention are illustrative rather than limiting. Therefore, this invention includes, but is not limited to, the embodiments described in the specific implementation. Any other implementations derived by those skilled in the art based on the technical solutions of this invention are also within the scope of protection of this invention.

Claims

1. A power equipment fault detection model based on Transformer, characterized in that, The power equipment fault detection model includes a Transformer model. The training data for the Transformer model comes from a preprocessing model, which includes an upsampling module and a word embedding representation learning module. The upsampling module is used to convert the input unbalanced power equipment data into balanced data; The word embedding representation learning module is used to embed the balance data and output the embedding representation based on the historical state sequence and label data of the power equipment representation; The Transformer model employs a multi-head attention mechanism, using the historical state sequence and the embedded representation of the label data as input to train and generate a power equipment fault detection model. The upsampling module employs the Borderline-SMOTE algorithm. The data processing procedure of the upsampling module includes: using the power equipment entity status and labels contained in the power grid power equipment dataset as input to the Borderline-SMOTE upsampling algorithm, employing the nearest neighbor algorithm to calculate the K nearest neighbors of each minority class sample, randomly selecting N samples from the K nearest neighbors for random linear interpolation to construct new minority class samples, then merging the newly generated minority class samples with the original samples to generate a balanced sample dataset, and finally performing embedding representation based on the newly constructed dataset. The data processing procedure of the upsampling module specifically includes: Step 21: Obtain the power grid equipment dataset, which includes the equipment entity status and corresponding labels; Step 22: Traverse the data in the power grid equipment dataset and use the nearest neighbor algorithm to determine the set of nearest neighbors D of sample x. n In the nearest neighbor sample set D n In this context, samples of the same category as sample x are grouped into set D. same The set of samples of different categories from sample x is called D. other ; Step 23: Calculate the difference in the number of samples between the majority and minority classes in the original power grid equipment dataset, and generate samples based on this difference; Step 24: After obtaining the number of samples generated from the seed samples, the K-means algorithm updates the coordinates of the cluster centers in each iteration by dividing the samples based on the Euclidean distance between the cluster centers and the sampled samples. Step 25: Select samples of the same category from each cluster to form a sample set D. c Then, based on the different feature types, appropriate processing is performed; Step 26: For each seed sample x i The corresponding category y i The cluster c it belongs to i There exists a corresponding number N of new samples that need to be generated. gi Each time a new sample is generated, based on N gi and the distribution of each feature of the cluster FD[c i ][y i First, generate auxiliary sample x. temp Then, linear interpolation is performed to obtain the final generated sample x. new ; Perform N cycles on the seed sample gj After the sample generation operation, a set of generated samples based on the seed sample is obtained. These generated samples belong to the same category as the seed sample. After each seed sample has completed sample generation, the resulting set of generated samples D is... g By merging it with the original dataset D, we obtain the final balanced dataset D. balance ; Step 27: Perform embedding representation on the finally obtained samples. The data form of the embedded samples is defined as M×N, where M is the number of samples used to represent the description of different power equipment; N is the number of features. The Transformer model includes: The input data processing module is used to perform position encoding on the embedded representation of the input power equipment state sequence information; The Encoder module is used to encode the input power equipment state sequence information according to the time order, and then use a multi-head attention mechanism to achieve self-attention information fusion. The Decoder module is used to decode the encoding generated by the Encoder module using a multi-head attention mechanism combined with a mask; The output of the Decoder module is processed by Linear and softmax to output the prediction result.

2. The power equipment fault detection model as described in claim 1, characterized in that, In step 23, the minority class samples are divided into three categories: when all k-nearest neighbors of a minority class sample belong to the majority class, it is defined as "noise"; when more than half of all k-nearest neighbors of a minority class sample belong to the majority class, it is defined as "danger"; when more than half of all k-nearest neighbors of a minority class sample belong to the minority class, it is defined as "safe". Then, a sample is randomly selected from the samples in the "danger" state as a seed sample.

3. The power equipment fault detection model as described in claim 1, characterized in that, The seed sample in step 23 is obtained using the following formula: label_diff j =N maj -N j Where, N maj N represents the number of samples for the majority class seed. j This indicates that it belongs to category C. j The number of samples; D s_maj D represents the set of seed samples belonging to the majority class; s_j This indicates that it belongs to category C. j Seed sample set; R sj N represents the number of samples that need to be generated for each seed sample to balance the difference in the number of seed samples; gj Represent each category C j The average number of new samples generated from the seed samples.

4. The power equipment fault detection model as described in claim 1, characterized in that, The hyperparameter k of the K-means algorithm in step 24 c Represented as: Where, N maj N represents the number of majority class samples in the dataset; min This indicates the number of minority class samples in the dataset.

5. The power equipment fault detection model as described in claim 1, characterized in that, The feature types in step 25 are divided into discrete features and continuous features. For discrete features, the selection of discrete features is determined based on the frequency of occurrence of different fields; For continuous features, calculate the maximum value (max) and minimum value (min) of the feature, and give the value interval [min, max].

6. The power equipment fault detection model as described in claim 1, characterized in that, The auxiliary sample x in step 26 temp Constructed based on feature distribution, satisfying three rules: Temporary sample x temp With sample x i Labels belonging to the same category y i ; Temporary sample x temp With sample x i Belonging to the same cluster c i ; Temporary sample x temp With sample x i They have the same characteristics, but the eigenvalues ​​of each characteristic are determined according to the cluster c. i Feature distribution FD[c i ][y i Obtained by random sampling; The sample x new The calculation formula is: x temp =[f1,f2,…,f p ],f p =Random(FD[c i ][y i ][p]) x new =x+Random(0,1)×(x temp -x) Where FD is the feature distribution matrix, and x represents the minority class sample.

7. The power equipment fault detection model as described in claim 1, characterized in that, The power equipment state sequence information is represented in the following form: X b =[b1,b2,…,b t ] in, N is the number of samples, t is the number of power equipment states in the time series, and b i It represents the i-th state in the sequence, and is a one-hot encoded state. For sequence information X b The embedding layer encodes it as a low-dimensional dense representation: the b =[e1,e2,…,e t ] in, k represents the size of the embedding layer.

Images