Intelligent decision-making method and system fusing deep clustering and Transform model

By integrating the intelligent decision-making method of deep clustering and Transformer model, traditional grid fault diagnosis methods are solved, and the problem of complex fault diagnosis methods are difficult to cope with large-scale data processing is achieved, efficient and intelligent fault identification and response are achieved, and the safety of grid operation is improved. and stability.

CN118940100APending Publication Date: 2024-11-12GUIZHOU POWER GRID CO LTD

Patent Information

Application Number
CN202410868260.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-01
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Traditional grid fault diagnosis methods rely on manual experience and predefined rule databases, making it difficult to deal with complex fault types, lack of automation and intelligence, resulting in a long response time for fault processing. As the scale of the power grid expands, the amount of data has increased exponentially. Methods are difficult to efficiently process and analyze large-scale failure data.

Method used

Intelligent decision-making method that integrates deep clustering and Transformer models, collects and preprocesses grid fault data, uses deep neural networks for feature extraction and nonlinear encoding, combines deep clustering algorithm for automatic classification, and captures sequence data through Transformer models. The long-range dependency and location information of the , and finally traverse the predefined rule base according to the fault type to output the response policy.

Benefits of technology

It significantly improves the accuracy and prediction accuracy of fault type identification, provides a more intelligent and efficient fault handling solution, can flexibly adapt to complex and changeable fault scenarios, and improves the safety and stability of power grid operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940100A_ABST
    Figure CN118940100A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent decision-making method and system fusing deep clustering and a Transform model, and the method comprises the steps: collecting power grid fault operation data related to a fault, and carrying out the preprocessing of the collected power grid fault operation data; performing feature extraction and nonlinear coding on the preprocessed power grid fault operation data by using a deep neural network; performing clustering analysis on results of feature extraction and nonlinear coding by adopting a deep clustering algorithm, and performing automatic classification on fault data to obtain preliminary fault type features; the preliminary fault type features and the feature extraction and nonlinear coding results are combined to serve as input of a Transform model, and a final predicted fault type is obtained; and traversing a predefined rule base according to the fault type, and outputting a corresponding response strategy. Through a self-attention mechanism and position coding, the long-range dependency relationship and position information in the sequence data can be captured, and the prediction accuracy of the fault type and the generalization ability of the model are remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of power grid technology, and specifically to an intelligent decision-making method and system integrating deep clustering and Transformer models. Background Art

[0002] As the demand for electricity in modern society continues to grow, the power system has become an indispensable part of infrastructure. The power system is responsible for transmitting the electricity generated by power plants to various end users, supporting the normal operation of industry, commerce and personal life. However, with the expansion of the scale of the power network and the increase in power load, the power system is also facing more and more challenges.

[0003] Circuit failure is a common and serious problem in the operation of power systems. These failures may be caused by various factors, such as aging equipment, bad weather, human negligence, etc. Once a circuit failure occurs, it may cause power supply interruption, which in turn affects the normal operation of all walks of life. What is more serious is that circuit failure may cause equipment damage or even cause safety accidents, resulting in serious economic losses and casualties.

[0004] Therefore, it is crucial to diagnose circuit faults accurately and quickly and take appropriate response strategies in a timely manner. Traditionally, the diagnosis and response of circuit faults mainly rely on manual experience and pre-established rule bases. However, with the continuous expansion of the scale and complexity of power systems, traditional methods have been unable to meet the needs of fast, accurate and automated. Therefore, it has become an urgent need to introduce advanced technologies and intelligent methods to diagnose and respond to circuit faults. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is: the traditional method mainly relies on manual experience and predefined rule bases, which is difficult to cope with complex fault types in the power grid, lacks automation and intelligent means, and has a long fault handling response time. As the scale of the power grid expands, the amount of data grows exponentially, and the existing methods are difficult to efficiently process and analyze large-scale fault data.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: an intelligent decision-making method integrating deep clustering and Transformer model, comprising:

[0008] Collecting power grid fault operation data related to the fault, and preprocessing the collected power grid fault operation data;

[0009] Use deep neural networks to perform feature extraction and nonlinear encoding on preprocessed power grid fault operation data;

[0010] The results of feature extraction and nonlinear coding are clustered using a deep clustering algorithm to automatically classify fault data and obtain preliminary fault type features;

[0011] The preliminary fault type features are combined with the results of feature extraction and nonlinear coding as the input of the Transformer model to obtain the final predicted fault type;

[0012] Traverse the predefined rule base according to the fault type and output the corresponding response strategy.

[0013] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, the platform parameters include task information and resource information.

[0014] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, wherein: the collected fault-related power grid fault operation data includes voltage, current, power factor, power consumption, frequency, transformer temperature, oil level, winding temperature, load current, load voltage, fault current, fault type, relay action time, fault waveform, event time, temperature, humidity, wind speed and rainfall;

[0015] The preprocessing of the collected power grid fault operation data includes data cleaning, missing value processing, data standardization and data smoothing.

[0016] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, the structure of the deep neural network is as follows:

[0017] Feature extraction includes, input layer: receiving preprocessed time series data;

[0018] 1D convolution layer: used to capture local features of time series data. The 1D convolution layer includes multiple layers, each of which uses a different kernel size and number.

[0019] Pooling layer: Each 1D convolutional layer is followed by a pooling layer, where the pooling layer uses maximum pooling to reduce the dimension and computation of features;

[0020] Output layer: used to output the feature vector after being processed by the pooling layer;

[0021] Non-linear coding includes, input layer: extracting the feature vector obtained by input feature;

[0022] Fully connected layer: includes multiple layers of fully connected neural network, each layer is connected to all neurons in the previous layer, and nonlinear changes and feature combinations are performed; ReLU activation function is added after each fully connected layer to introduce nonlinear elements;

[0023] Output layer: The result of the last full-face layer output is the feature vector after nonlinear encoding.

[0024] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, the training method for the deep neural network is as follows:

[0025] Collect historical fault operation data and pre-process the historical fault operation data;

[0026] Divide the preprocessed historical fault operation data into a training set and a validation set;

[0027] Xavier initialization is used to initialize the parameters of each layer of feature extraction and nonlinear coding;

[0028] For the feature extraction stage, the mean square error loss function is used;

[0029] For nonlinear coding, the cross entropy loss function is used;

[0030] The deep neural network is trained through the training set, and the parameters are optimized using the stochastic gradient descent method;

[0031] In each training iteration, the data is fed into the deep neural network, the loss function is calculated, and the model parameters are updated according to the gradient of the loss function;

[0032] Repeated iterative training of the deep neural network until a predefined stopping condition is reached;

[0033] The performance of the trained deep neural network model is evaluated through the validation set, and the structure or hyperparameters of the deep neural network are adjusted according to the performance on the validation set.

[0034] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, wherein: the result of feature extraction and nonlinear coding is clustered by using a deep clustering algorithm, the fault data is automatically classified, and the preliminary fault types are obtained, including: the feature deep clustering algorithm adopts a deep embedding clustering algorithm, the deep embedding clustering algorithm is clustered to obtain the cluster label to which each fault data sample belongs, and the cluster label is used as the preliminary fault type feature;

[0035] The specific steps of clustering analysis using the deep embedding clustering algorithm are as follows:

[0036] Initialize the cluster center, convert the data into feature representation by the result of feature body and nonlinear encoding conversion, and standardize the feature representation; randomly initialize K cluster centers on the result after standardization;

[0037] Feature representation obtained through feature extraction and nonlinear coding Standardize Z to get the standardized feature representation Z n orm;

[0038] Initialize K cluster centers:

[0039] Calculating the membership degree includes, for each sample, determining the cluster to which it belongs by calculating its membership probability to each cluster center;

[0040] For each sample z i , calculate its membership degree q i j to each cluster center μ j Probability:

[0041]

[0042] Among them, Z represents the feature representation set obtained after feature extraction and nonlinear encoding, z i represents the feature vector of the i-th sample, n represents the total number of samples, Z norm represents the set of standardized feature representations, μ represents the set of cluster centers, and μ j represents the jth cluster center, K represents the number of cluster centers, q ij Represents sample z i To the cluster center μ j The membership probability of i -μ j ∥ represents sample z i and cluster center μ j The Euclidean distance between them, α represents the hyperparameter controlling the membership distribution, and the numerator Represents sample z i To the cluster center μ j The relative similarity of It is the sum of the relative similarities of all samples to all cluster centers;

[0043] Optimizing the cluster centers includes using KL divergence as the objective function and updating the cluster centers by gradient descent to minimize the difference between the membership distribution and the true distribution;

[0044] The KL divergence calculation formula is as follows:

[0045]

[0046] Among them, KL(P∥Q) represents the KL divergence between two distributions P and Q, P represents the true probability distribution, Q represents the probability distribution predicted by the model, Σ i and Σ j It means to sum all samples i and all cluster centers j, i represents the index of the sample, there are n samples in total, j represents the index of the cluster center, there are K cluster centers in total, p ij represents the true membership probability of sample i to cluster center j, q ij Represents the predicted membership probability of sample i to cluster center j;

[0047] Iterative convergence includes repeatedly calculating membership and optimizing cluster centers until the cluster centers converge.

[0048] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, wherein: the preliminary fault type features are combined with the results of feature extraction and nonlinear coding as the input of the Transformer model to obtain the final predicted fault type, including combining the preliminary fault type features with the results of feature extraction and nonlinear coding, and inputting the combined features into the Transformer model;

[0049] Let the result of feature extraction and nonlinear encoding be the feature vector X feat , whose dimension is n f eat×1, where n feat Represents the dimension of the feature extraction and nonlinear coding results; the preliminary fault feature vector X type and the eigenvector X feat Connect and get the merged eigenvector eigenvector X merged ; The dimension of the merged feature vector is (n f eat+n type )+1, where n t ype represents the dimension of preliminary fault characteristics;

[0050] The Transformer model captures relevant information in the sequence through self-attention mechanism and position encoding to produce the final prediction result;

[0051] For a given input sequence, the representation of query, key and value is calculated; the similarity between query and key is calculated using dot product attention, and the similarity matrix is ​​converted into attention weights through softmax function, and the weighted representation is obtained by multiplying the attention weights and the value;

[0052] The dot product attention calculation formula is expressed as:

[0053]

[0054] Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d k represents the dimension of the key, QK T represents the dot product operation of the query matrix and the transpose of the key matrix, represents the dot product result divided by the square root of the key dimension, softmax represents the conversion of the similarity matrix into attention weights, and V represents the multiplication of the attention weights with the value matrix to obtain the weighted representation;

[0055] The position encoding formula is expressed as:

[0056] Sine and cosine functions are used to generate position codes, which are added to the input feature representation to obtain the input representation with position information.

[0057]

[0058] Among them, pos represents the position of the position code, i represents the dimension index of the position code, and d model A hidden dimension representing superstition;

[0059] The post-processing formula is expressed as:

[0060] The attention aggregation representation generated by the attention mechanism is added to the position encoding to obtain a comprehensive representation containing the internal correlation and position information of the sequence; the comprehensive representation is converted into a probability distribution of different fault types through a softmax function, and the fault type with the highest probability is selected as the final prediction result;

[0061] The softmax function formula is expressed as:

[0062]

[0063] Among them, P(y i |x) means that given the input x, the model predicts category y i The probability of represents the word count function, the i-th element z in the model output vector is indexed, and K represents the total number of categories.

[0064] As a preferred solution of the intelligent decision-making method integrating deep clustering and Transformer model described in the present invention, the steps of traversing the predefined rule base according to the fault type and outputting the corresponding response strategy include: the Transformer model captures the relevant information in the sequence through the self-attention mechanism and position encoding, and generates the final prediction result are as follows:

[0065] The self-attention mechanism involves, for a given input sequence, computing representations of queries, keys, and values;

[0066] Dot product attention is used to calculate the similarity between the query and the key, and the similarity matrix is ​​converted into attention weights through the softmax function. The attention weights are multiplied by the value to get the weighted representation;

[0067] The weighted values ​​are weighted summed to obtain the attention aggregation representation of each position;

[0068] Position encoding includes using sine and cosine functions to generate position encoding, adding the generated position encoding to the input feature representation to obtain an input representation with position information;

[0069] Post-processing includes adding the attention aggregation representation generated by the attention mechanism to the position encoding to obtain a comprehensive representation that contains the internal correlation and position information of the sequence;

[0070] The comprehensive representation is converted into a probability distribution of different fault types through a softmax function, and the fault type with the highest probability is selected as the final prediction result.

[0071] Intelligent decision-making system integrating deep clustering and Transformer model:

[0072] A data collection module collects power grid fault operation data related to the fault and pre-processes the collected power grid fault operation data;

[0073] The feature extraction module uses a deep neural network to perform feature extraction and nonlinear encoding on the preprocessed power grid fault operation data;

[0074] The automatic classification module uses the deep clustering algorithm to perform cluster analysis based on the results of feature extraction and nonlinear coding, automatically classifies the fault data, and obtains preliminary fault type characteristics;

[0075] The fault prediction module combines the preliminary fault type features with the results of feature extraction and nonlinear coding as the input of the Transformer model to obtain the final predicted fault type;

[0076] The result output module traverses the predefined rule base according to the fault type and outputs the corresponding response strategy.

[0077] A computer device comprises: a memory and a processor; the memory stores a computer program, wherein the processor implements the steps of any one of the methods of the present invention when executing the computer program.

[0078] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any one of the methods of the present invention.

[0079] Beneficial effects of the present invention: The intelligent decision-making method that integrates deep clustering and Transformer models provided by the present invention utilizes multi-layer 1D convolutional layers and pooling layers, which can capture the local features of time series data, and further enriches the data characterization capabilities through nonlinear coding, making the fault type identification more accurate. By initializing cluster centers, calculating membership probabilities, optimizing cluster centers and other steps, high-dimensional and nonlinear data are effectively processed, and more accurate preliminary fault type features are obtained. Through the self-attention mechanism and position encoding, the long-range dependencies and position information in the sequence data can be captured, which significantly improves the prediction accuracy of the fault type and the generalization ability of the model. It intelligently traverses the predefined rule base and outputs the corresponding response strategy, which can flexibly adapt to various complex and changeable fault scenarios and provide a more intelligent and efficient fault handling solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0081] Figure 1 An overall flow chart of an intelligent decision-making method integrating deep clustering and Transformer model provided for the first embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. The embodiments of the present invention and all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0083] Example 1

[0084] Reference Figure 1 , is an embodiment of the present invention, and provides an intelligent decision-making method integrating deep clustering and Transformer model, comprising:

[0085] S1: Collecting power grid fault operation data related to the fault, and preprocessing the collected power grid fault operation data.

[0086] Data collected includes voltage, current, power factor, power consumption, frequency, transformer temperature, oil level, winding temperature, load current, load voltage, fault current, fault type, relay operation time, fault waveform, event time, temperature, humidity, wind speed and rainfall.

[0087] Data preprocessing:

[0088] Data cleaning: remove outliers, noise and other interfering factors to ensure the reliability and consistency of data.

[0089] Missing value processing: Use interpolation, mean filling and other methods to process missing data to ensure data integrity.

[0090] Data standardization: Standardize or normalize the data so that data with different parameters are at the same scale, which is conducive to subsequent feature extraction and model training.

[0091] S2: Use deep neural network to perform feature extraction and nonlinear encoding on the preprocessed power grid fault operation data.

[0092] Feature extraction includes, input layer: receiving preprocessed time series data.

[0093] 1D convolution layer: used to capture local features of time series data. The 1D convolution layer includes multiple 1D convolution layers, and each 1D convolution layer uses a different kernel size and number.

[0094] Pooling layer: Each 1D convolutional layer is followed by a pooling layer, where the pooling layer uses maximum pooling to reduce the dimension and computation of features.

[0095] Output layer: used to output the feature vector processed by the pooling layer.

[0096] Non-linear coding includes: input layer: feature vector obtained by extracting input features.

[0097] Fully connected layer: includes multiple layers of fully connected neural network, each layer is connected to all neurons in the previous layer, and nonlinear changes and feature combinations are performed; ReLU activation function is added after each fully connected layer to introduce nonlinear elements.

[0098] Output layer: The result of the last full-face layer output is the feature vector after nonlinear encoding.

[0099] The training method for the deep neural network is as follows:

[0100] Collect historical fault operation data and pre-process the historical fault operation data;

[0101] Divide the preprocessed historical fault operation data into a training set and a validation set;

[0102] Xavier initialization is used to initialize the parameters of each layer of feature extraction and nonlinear coding;

[0103] For the feature extraction stage, the mean square error loss function is used;

[0104] For nonlinear coding, the cross entropy loss function is used;

[0105] The deep neural network is trained through the training set, and the parameters are optimized using the stochastic gradient descent method;

[0106] In each training iteration, the data is fed into the deep neural network, the loss function is calculated, and the model parameters are updated according to the gradient of the loss function;

[0107] Repeated iterative training of the deep neural network until a predefined stopping condition is reached;

[0108] The performance of the trained deep neural network model is evaluated through the validation set, and the structure or hyperparameters of the deep neural network are adjusted according to the performance on the validation set.

[0109] S3: Based on the results of feature extraction and nonlinear coding, a deep clustering algorithm is used to perform cluster analysis, automatically classify the fault data, and obtain preliminary fault type characteristics.

[0110] The feature deep clustering algorithm uses a deep embedding clustering algorithm to perform clustering analysis, obtain the cluster label to which each fault data sample belongs, and use the cluster label as the preliminary fault type feature;

[0111] The specific steps of clustering analysis using the deep embedding clustering algorithm are as follows:

[0112] Initialize the cluster center, convert the data into feature representation by the result of feature body and nonlinear encoding conversion, and standardize the feature representation; randomly initialize K cluster centers on the result after standardization;

[0113] Feature representation obtained through feature extraction and nonlinear coding Standardize Z to get the standardized feature representation Zn orm;

[0114] Initialize K cluster centers:

[0115] Calculating the membership degree includes, for each sample, determining the cluster to which it belongs by calculating its membership probability to each cluster center;

[0116] For each sample z i , calculate its membership degree q i j to each cluster center μ j Probability:

[0117]

[0118] Among them, Z represents the feature representation set obtained after feature extraction and nonlinear encoding, z i represents the feature vector of the i-th sample, n represents the total number of samples, Z norm represents the set of standardized feature representations, μ represents the set of cluster centers, and μ j represents the jth cluster center, K represents the number of cluster centers, q ij Represents sample z i To the cluster center μ j The membership probability of i -μ j ∥ represents sample z i and cluster center μ j The Euclidean distance between them, α represents the hyperparameter controlling the membership distribution, and the numerator Represents sample z i To the cluster center μ j The relative similarity of It is the sum of the relative similarities of all samples to all cluster centers;

[0119] Optimizing the cluster centers includes using KL divergence as the objective function and updating the cluster centers by gradient descent to minimize the difference between the membership distribution and the true distribution;

[0120] The KL divergence calculation formula is as follows:

[0121]

[0122] Among them, KL(P∥Q) represents the KL divergence between two distributions P and Q, P represents the true probability distribution, Q represents the probability distribution predicted by the model, Σ i and∑ jIt means to sum all samples i and all cluster centers j, i represents the index of the sample, there are n samples in total, j represents the index of the cluster center, there are K cluster centers in total, p ij represents the true membership probability of sample i to cluster center j, q ij Represents the predicted membership probability of sample i to cluster center j;

[0123] Iterative convergence includes repeatedly calculating membership and optimizing cluster centers until the cluster centers converge.

[0124] S4: The preliminary fault type features are combined with the results of feature extraction and nonlinear encoding as the input of the Transformer model to obtain the final predicted fault type.

[0125] The preliminary fault type features are merged with the results of feature extraction and nonlinear encoding, and the merged features are input into the Transformer model.

[0126] Let the result of feature extraction and nonlinear encoding be the feature vector X feat , whose dimension is n f eat×1, where n feat Represents the dimension of the feature extraction and nonlinear coding results; the preliminary fault feature vector X type and the eigenvector X feat Connect and get the merged eigenvector eigenvector X merged ; The dimension of the merged feature vector is (n f eat+n type )+1, where n t ype represents the dimension of preliminary fault characteristics.

[0127] The Transformer model captures relevant information in the sequence through self-attention mechanism and position encoding to produce the final prediction result.

[0128] For a given input sequence, the representation of query, key and value is calculated; the similarity between query and key is calculated using dot product attention, and then the similarity matrix is ​​converted into attention weights through the softmax function, and the attention weights are multiplied by the value to obtain the weighted representation.

[0129] The dot product attention calculation formula is expressed as:

[0130]

[0131] Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, and d k represents the dimension of the key, QKT represents the dot product operation of the query matrix and the transpose of the key matrix, It represents the dot product result divided by the square root of the dimension of the key, softmax represents the conversion of the similarity matrix into attention weights, and V represents the multiplication of the attention weights with the value matrix to obtain the weighted representation.

[0132] The position encoding formula is expressed as:

[0133] Sine and cosine functions are used to generate position codes, which are added to the input feature representation to obtain the input representation with position information.

[0134]

[0135] Among them, pos represents the position of the position code, i represents the dimension index of the position code, and d model Hidden dimension representing superstition.

[0136] The post-processing formula is expressed as:

[0137] The attention aggregation representation generated by the attention mechanism is added to the position encoding to obtain a comprehensive representation containing the internal correlation and position information of the sequence; the comprehensive representation is converted into a probability distribution of different fault types through the softmax function, and the fault type with the highest probability is selected as the final prediction result.

[0138] The softmax function formula is expressed as:

[0139]

[0140] Among them, P(y i |x) means that given the input x, the model predicts category y i The probability of represents the word count function, the i-th element z in the model output vector is indexed, and K represents the total number of categories.

[0141] S5: Traverse the predefined rule base according to the fault type and output the corresponding response strategy.

[0142] Rule base preparation:

[0143] Collect and organize rules related to various fault types and build a rule base. These rules can be derived from the knowledge of domain experts, historical data analysis, or operational experience.

[0144] Each rule should include the fault type, triggering conditions, and corresponding response strategies. Ensure that the rules in the rule base are complete, accurate, and clear.

[0145] Fault type matching:

[0146] According to the final predicted fault type obtained in S4, the corresponding rules are matched from the rule base.

[0147] Fuzzy matching, exact matching or similarity matching methods can be used to cope with the uncertainty and diversity that may exist in actual scenarios.

[0148] Response strategy generation:

[0149] Once the corresponding rule is matched, the response strategy corresponding to the rule is extracted.

[0150] Response strategies may include but are not limited to: device restart, alarm notification, fault isolation, backup device switching, etc.

[0151] In summary, through the design of the deep neural network structure, the 1D convolution layer and pooling layer in the feature extraction stage of this embodiment can effectively capture the local features in the time series data, and reduce the dimension through maximum pooling, thereby improving the expressiveness and computational efficiency of the features; through multi-layer fully connected layers and ReLU activation functions, more nonlinear transformations can be introduced, thereby improving the expressiveness and generalization ability of the model.

[0152] Xavier initialization and loss function selection: Xavier initialization can make the network weights have appropriate initial values, which helps to accelerate convergence and prevent gradient vanishing or explosion; the mean square error loss function and cross entropy loss function are used, which are suitable for feature extraction and nonlinear coding stages respectively, which helps to improve the model's fitting ability and prediction performance.

[0153] Stochastic gradient descent and validation set evaluation: Using stochastic gradient descent for parameter optimization and combining it with the validation set to evaluate model performance can effectively adjust the model structure and hyperparameters and improve the generalization ability and stability of the model.

[0154] Through feature representation and standardization, the original data can be converted into feature representation, which reduces the dimension while retaining key information, which is conducive to subsequent clustering analysis.

[0155] By using KL divergence as the objective function and iteratively optimizing the cluster centers, the accuracy and stability of clustering can be effectively improved, ensuring the effectiveness and reliability of the clustering results.

[0156] Through the self-attention mechanism, the long-range dependencies within the sequence can be captured, which improves the model's modeling and characterization capabilities for the input sequence and helps improve the accuracy of fault type prediction.

[0157] The introduction of position encoding can integrate the position information in the sequence into the feature representation, which helps the model better understand the sequential relationship of sequence data and improves the model's processing and generalization capabilities for sequence data.

[0158] Through the application of the above technical solutions, the accuracy, efficiency and automation level of power grid fault diagnosis and response can be improved, effectively solving the needs that are difficult to meet with traditional methods, thereby improving the safety and stability of power grid operation.

[0159] On the other hand, this embodiment also provides an intelligent decision-making system integrating deep clustering and Transformer model, which includes:

[0160] The data collection module collects the power grid fault operation data related to the fault and pre-processes the collected power grid fault operation data.

[0161] The feature extraction module uses a deep neural network to perform feature extraction and nonlinear encoding on the preprocessed power grid fault operation data.

[0162] In the automatic classification module, the results of feature extraction and nonlinear coding are clustered using a deep clustering algorithm to automatically classify the fault data and obtain preliminary fault type characteristics.

[0163] The fault prediction module combines the preliminary fault type features with the results of feature extraction and nonlinear coding as the input of the Transformer model to obtain the final predicted fault type.

[0164] The result output module traverses the predefined rule base according to the fault type and outputs the corresponding response strategy.

[0165] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. In this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0166] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with such instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with such instruction execution systems, devices or apparatuses.

[0167] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0168] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0169] Example 2

[0170] Six months of grid fault operation data, including voltage, current, frequency, transformer temperature, fault current, etc., were collected from a provincial power grid dispatching center. Data were collected using smart meters, substation monitoring systems, relay protection devices, and fault recorders. The collected data was preprocessed, including data cleaning, missing value filling, and standardization. During the data cleaning process, outliers and noise were removed; missing values ​​were filled using linear interpolation; and standardization scaled the data to a range between 0 and 1 to facilitate subsequent feature extraction and clustering analysis.

[0171] A deep neural network is used to extract features and perform nonlinear encoding on the preprocessed power grid fault operation data. A deep neural network model containing multiple 1D convolutional layers and pooling layers is designed. The input layer receives the preprocessed time series data, captures local features through the 1D convolutional layer, and the pooling layer is used to reduce the dimension and calculation amount of the features. The output layer generates feature vectors. Then, nonlinear encoding is performed through a multi-layer fully connected neural network. Each layer of the fully connected network is followed by a ReLU activation function to output the feature vector after nonlinear encoding.

[0172] The results of feature extraction and nonlinear coding are clustered using a deep embedding clustering algorithm. First, the cluster center is initialized and the probability of membership of each sample to the cluster center is calculated. Then, the KL divergence is used as the objective function and the cluster center is optimized by the gradient descent method. After multiple iterations, the cluster center converges and the cluster label of each fault data sample is obtained as the preliminary fault type feature.

[0173] The preliminary fault type features are combined with the results of feature extraction and nonlinear encoding as the input of the Transformer model. The Transformer model captures relevant information in the sequence through self-attention mechanism and position encoding to generate the final prediction results. The specific steps include calculating the representation of the query, key and value, using dot product attention to calculate the similarity between the query and the key, and then converting the similarity matrix into attention weights through the softmax function, and multiplying the attention weights and the values ​​to obtain the weighted representation.

[0174] According to the fault type, the predefined rule base is traversed and the corresponding response strategy is output. The rule base is predefined by power system experts and includes strategies such as equipment restart, alarm notification, fault isolation and backup equipment switching. The fault type predicted by the model is matched with the rule base to automatically generate the optimal response strategy. The experimental results are shown in Table 1.

[0175] Table 1 Experimental data table

[0176]

[0177] It can be seen from the experimental data table that the intelligent decision-making method integrating deep clustering and Transformer model has shown significant advantages in power grid fault diagnosis and response. First, in the feature extraction and nonlinear coding stage, the 1D convolution layer and pooling layer of the deep neural network effectively capture the local features of the time series data, improving the richness and accuracy of the feature representation. After being processed by the deep neural network, the voltage, current, fault current and other features in the experimental data can better reflect the actual situation of the fault.

[0178] In the deep clustering analysis stage, the deep embedding clustering algorithm was used to automatically classify the fault data and obtain relatively accurate preliminary fault type characteristics. The experimental data showed that test objects 1, 3, and 5 were correctly classified as overload faults, test objects 2 and 6 were classified as short circuit faults, and test object 4 was classified as ground fault. The accuracy of cluster labels provides a solid foundation for subsequent fault type prediction.

[0179] In the prediction stage of the Transformer model, through the self-attention mechanism and position encoding, the model can capture the sequence correlation and position information in the input data, significantly improving the prediction accuracy of the fault type. The experimental results show that the model's prediction results for different fault types are highly consistent with the actual fault types, proving the powerful ability of the Transformer model in processing power grid fault data.

[0180] Finally, in the response strategy output stage, by traversing the predefined rule base, the model can automatically generate the optimal response strategy according to the predicted fault type. Compared with the traditional method, the method of the present invention has obvious advantages in the intelligence and flexibility of the response strategy. After test objects 1, 3 and 5 were identified as overload faults, the system automatically generated strategies for equipment restart and load adjustment; after test objects 2 and 6 were identified as short-circuit faults, the system generated strategies for fault isolation and alarm notification; after test object 4 was identified as a ground fault, the system generated strategies for fault location and equipment maintenance.

[0181] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An intelligent decision-making method integrating deep clustering and Transformer model, characterized in that: include: Collecting power grid fault operation data related to the fault, and preprocessing the collected power grid fault operation data; Use deep neural networks to perform feature extraction and nonlinear encoding on preprocessed power grid fault operation data; The results of feature extraction and nonlinear coding are clustered using a deep clustering algorithm to automatically classify fault data and obtain preliminary fault type features; The preliminary fault type features are combined with the results of feature extraction and nonlinear coding as the input of the Transformer model to obtain the final predicted fault type; Traverse the predefined rule base according to the fault type and output the corresponding response strategy.

2. The intelligent decision-making method integrating deep clustering and Transformer model as claimed in claim 1, characterized in that: The collected fault-related grid fault operation data include voltage, current, power factor, power consumption, frequency, transformer temperature, oil level, winding temperature, load current, load voltage, fault current, fault type, relay action time, fault waveform, event time, temperature, humidity, wind speed and rainfall; The preprocessing of the collected power grid fault operation data includes data cleaning, missing value processing, data standardization and data smoothing.

3. The intelligent decision-making method integrating deep clustering and Transformer model as claimed in claim 2, characterized in that: The structure of the deep neural network is as follows: Feature extraction includes, input layer: receiving preprocessed time series data; 1D convolution layer: used to capture local features of time series data. The 1D convolution layer includes multiple layers, each of which uses a different kernel size and number. Pooling layer: Each 1D convolutional layer is followed by a pooling layer, where the pooling layer uses maximum pooling to reduce the dimension and computation of features; Output layer: used to output the feature vector after being processed by the pooling layer; Non-linear coding includes, input layer: extracting the feature vector obtained by input feature; Fully connected layer: includes multiple layers of fully connected neural network, each layer is connected to all neurons in the previous layer, and nonlinear changes and feature combinations are performed; ReLU activation function is added after each fully connected layer to introduce nonlinear elements; Output layer: The result of the last full-face layer output is the feature vector after nonlinear encoding.

4. The intelligent decision-making method integrating deep clustering and Transformer model as claimed in claim 3, characterized in that: The training method for the deep neural network is as follows: Collect historical fault operation data and pre-process the historical fault operation data; Divide the preprocessed historical fault operation data into a training set and a validation set; Xavier initialization is used to initialize the parameters of each layer of feature extraction and nonlinear coding; For the feature extraction stage, the mean square error loss function is used; For nonlinear coding, the cross entropy loss function is used; The deep neural network is trained through the training set, and the parameters are optimized using the stochastic gradient descent method; In each training iteration, the data is fed into the deep neural network, the loss function is calculated, and the model parameters are updated according to the gradient of the loss function; Repeated iterative training of the deep neural network until a predefined stopping condition is reached; The performance of the trained deep neural network model is evaluated through the validation set, and the structure or hyperparameters of the deep neural network are adjusted according to the performance on the validation set.

5. The intelligent decision-making method integrating deep clustering and Transformer model as claimed in claim 4, characterized in that: The result of feature extraction and nonlinear coding is clustered by deep clustering algorithm, and the fault data is automatically classified to obtain preliminary fault types, including: the feature deep clustering algorithm uses deep embedding clustering algorithm, and the deep embedding clustering algorithm is used for clustering analysis to obtain the cluster label to which each fault data sample belongs, and the cluster label is used as the preliminary fault type feature; The specific steps of clustering analysis using the deep embedding clustering algorithm are as follows: Initialize the cluster center, convert the data into feature representation by the result of feature body and nonlinear encoding conversion, and standardize the feature representation; randomly initialize K cluster centers on the result after standardization; Feature representation obtained through feature extraction and nonlinear coding Standardize Z to get the standardized feature representation Z n orm; Initialize K cluster centers: Calculating the membership degree includes, for each sample, determining the cluster to which it belongs by calculating its membership probability to each cluster center; For each sample z i , calculate its membership degree q i j to each cluster center μ j Probability: Among them, Z represents the feature representation set obtained after feature extraction and nonlinear encoding, z i represents the feature vector of the i-th sample, n represents the total number of samples, Z norm represents the set of standardized feature representations, μ represents the set of cluster centers, and μ j represents the jth cluster center, K represents the number of cluster centers, q ij Represents sample z i To the cluster center μ j The membership probability of ||z i -μ j || represents sample z i and cluster center μ j The Euclidean distance between them, α represents the hyperparameter controlling the membership distribution, and the numerator Represents sample z i To the cluster center μ j The relative similarity of It is the sum of the relative similarities of all samples to all cluster centers; Optimizing the cluster centers includes using KL divergence as the objective function and updating the cluster centers by gradient descent to minimize the difference between the membership distribution and the true distribution; The KL divergence calculation formula is as follows: Among them, KL(P||Q) represents the KL divergence between two distributions P and Q, P represents the true probability distribution, Q represents the probability distribution predicted by the model, ∑ i and∑ j It means to sum all samples i and all cluster centers j, i represents the index of the sample, there are n samples in total, j represents the index of the cluster center, there are K cluster centers in total, p ij represents the true membership probability of sample i to cluster center j, q ij Represents the predicted membership probability of sample i to cluster center j; Iterative convergence includes repeatedly calculating membership and optimizing cluster centers until the cluster centers converge.

6. The intelligent decision-making method integrating deep clustering and Transformer model as claimed in claim 5, characterized in that: The step of combining the preliminary fault type features with the results of feature extraction and nonlinear coding as input of the Transformer model to obtain the final predicted fault type includes combining the preliminary fault type features with the results of feature extraction and nonlinear coding, and inputting the combined features into the Transformer model; Let the result of feature extraction and nonlinear encoding be the feature vector X feat , whose dimension is n f eat×1, where n feat Represents the dimension of the feature extraction and nonlinear coding results; the preliminary fault feature vector X type and the eigenvector X feat Connect and get the merged eigenvector eigenvector X merged ; The dimension of the merged feature vector is (n f eat+n type )+1, where n t ype represents the dimension of preliminary fault characteristics; The Transformer model captures relevant information in the sequence through self-attention mechanism and position encoding to produce the final prediction result; For a given input sequence, compute the representation of query, key, and value; Dot product attention is used to calculate the similarity between the query and the key, and the similarity matrix is ​​converted into attention weights through the softmax function. The attention weights are multiplied by the value to get the weighted representation; The dot product attention calculation formula is expressed as: Among them, Q represents the query matrix, K represents the key matrix, V represents the value matrix, dk represents the dimension of the key, and QKT represents the dot product operation of the transpose of the query matrix and the key matrix. represents the dot product result divided by the square root of the key dimension, softmax represents the conversion of the similarity matrix into attention weights, and V represents the multiplication of the attention weights and the value matrix to obtain the weighted representation; The position encoding formula is expressed as: Sine and cosine functions are used to generate position codes, which are added to the input feature representation to obtain the input representation with position information. Among them, pos represents the position of the position code, i represents the dimension index of the position code, and d model A hidden dimension representing superstition; The post-processing formula is expressed as: The attention aggregation representation generated by the attention mechanism is added to the position encoding to obtain a comprehensive representation containing the internal correlation and position information of the sequence; the comprehensive representation is converted into a probability distribution of different fault types through a softmax function, and the fault type with the highest probability is selected as the final prediction result; The softmax function formula is expressed as: Among them, P(y i |x) means that given the input x, the model predicts category y i The probability of represents the word count function, the i-th element z in the model output vector is indexed, and K represents the total number of categories.

7. The intelligent decision-making method integrating deep clustering and Transformer model as claimed in claim 6, characterized in that: According to the fault type, the predefined rule base is traversed and the corresponding response strategy is output. The Transformer model captures the relevant information in the sequence through the self-attention mechanism and position encoding. The steps to generate the final prediction result are as follows: The self-attention mechanism involves, for a given input sequence, computing representations of queries, keys, and values; Dot product attention is used to calculate the similarity between the query and the key, and the similarity matrix is ​​converted into attention weights through the softmax function. The attention weights are multiplied by the value to get the weighted representation; The weighted values ​​are weighted summed to obtain the attention aggregation representation of each position; Position encoding includes using sine and cosine functions to generate position encoding, adding the generated position encoding to the input feature representation to obtain an input representation with position information; Post-processing includes adding the attention aggregation representation generated by the attention mechanism to the position encoding to obtain a comprehensive representation that contains the internal correlation and position information of the sequence; The comprehensive representation is converted into a probability distribution of different fault types through a softmax function, and the fault type with the highest probability is selected as the final prediction result.

8. An intelligent decision-making system that integrates deep clustering and Transformer model using the method according to any one of claims 1 to 7, characterized in that: A data collection module collects power grid fault operation data related to the fault and pre-processes the collected power grid fault operation data; The feature extraction module uses a deep neural network to perform feature extraction and nonlinear encoding on the preprocessed power grid fault operation data; The automatic classification module uses the deep clustering algorithm to perform cluster analysis based on the results of feature extraction and nonlinear coding, automatically classifies the fault data, and obtains preliminary fault type characteristics; The fault prediction module combines the preliminary fault type features with the results of feature extraction and nonlinear coding as the input of the Transformer model to obtain the final predicted fault type; The result output module traverses the predefined rule base according to the fault type and outputs the corresponding response strategy.

9. A computer device comprising: Memory and processor; The memory stores a computer program, characterized in that when the processor executes the computer program, the steps of the intelligent decision-making method integrating deep clustering and Transformer model are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of an intelligent decision-making method integrating deep clustering and Transformer model are implemented.

Citation Information

Patent Citations

  • Power transformer fault diagnosis method based on deep learning

    CN115329908A

  • Fault detection method based on feature extraction and Transform model

    CN117312821A

  • Arc fault detection using machine learning

    US20230053431A1

Cited By

  • Ophthalmic disease intelligent pre-diagnosis system and method based on big data recognition

    CN119361133A

  • Communication base station monitoring method and system based on artificial intelligence

    CN119545403A

  • Power network asset key feature extraction method, device, equipment and medium

    CN119577414A

  • A method, apparatus, equipment, and medium for extracting key features of power network assets.

    CN119577414B

  • Fault data analysis method and device of smoke alarm and smoke alarm

    CN119649572A