A training method and a detection method of a time series prediction model, and a device and equipment

Abstract

The application discloses a kind of training method of time series prediction model, detection method, device and equipment, belong to the field of power transformer intelligent operation and maintenance technique, the training method includes: obtaining the transformer multi-source operation data containing gas data and non-gas auxiliary data, constructs original feature set and carries out dimension reduction processing, obtains characteristic vector;Characteristic vector is input into preset time series prediction model iterative training to convergence, obtains trained time series prediction model;Wherein each iteration is carried out feature extraction and time series coding by multi-scale convolution network, gated recurrent unit, gated residual connection and encoder, and the integrated feature vector is obtained by cross attention fusion, and the model parameters are updated according to the prediction difference based on integrated feature vector classification prediction.It can solve the problem that existing model is difficult to consider transformer fault data short-term mutation characteristics and long-term trend dependence by implementing the present application, and improve the accuracy of transformer fault detection.

Description

Technical Field

[0001] This invention relates to the field of power transformer fault detection technology, and in particular to a training method, detection method, device and equipment for a time series prediction model. Background Technology

[0002] Power transformers are core equipment in power systems, responsible for power transmission and distribution. Their safe and stable operation directly impacts the reliability of the power grid. With the large-scale integration of new energy sources and loads with high randomness and intermittency, such as distributed photovoltaic, wind power, and electric vehicle charging stations, into distribution networks, the operating conditions of power systems are becoming increasingly complex and variable, placing higher demands on the reliability and safety of transformers. A transformer failure can lead to widespread power outages, causing significant economic losses and social impact. Therefore, achieving early and accurate detection of transformer faults is crucial for ensuring the safe operation of the power system.

[0003] Traditional transformer fault detection primarily relies on manual inspections and periodic preventative tests, determining fault types by analyzing the composition and content of dissolved gases in transformer oil, such as the IEC three-ratio method and the Rogers ratio method. These methods heavily depend on human experience, have long detection cycles, struggle to capture sudden faults in real time, and are insufficient for high-precision fault identification when facing complex and variable operating conditions, relying solely on simple ratio rules from a single gas data source. With the rapid development of artificial intelligence, deep learning-based transformer fault detection methods have become a research hotspot. By constructing deep neural network models to automatically extract and classify fault features, they have improved the automation level and accuracy of fault detection to some extent.

[0004] However, existing deep learning-based transformer fault detection methods still face the following key challenges. First, transformer fault data exhibits a significant coexistence of short-term and long-term dependencies—sudden faults (such as partial discharge and arc discharge) correspond to instantaneous changes in dissolved gas concentration in the oil, showing a short-term dependency, while long-term equipment aging (such as slow degradation of insulation materials) corresponds to long-term, slow trends in gas concentration, showing a long-term dependency. Most existing methods employ a single convolutional neural network or a single recurrent neural network structure. Convolutional neural networks are limited by their local receptive field, making it difficult to capture long-distance temporal dependencies, while single recurrent neural networks suffer from gradient vanishing and long-term memory decay when sequences are long. Neither approach can simultaneously handle temporal feature extraction at both scales, resulting in limited ability to identify complex fault modes. Secondly, most existing methods rely solely on analytical data of dissolved gases in transformer oil for fault diagnosis, while severely underutilizing non-gas-related auxiliary data such as load current, top oil temperature, winding temperature, and ambient temperature and humidity. This auxiliary data contains crucial information about the transformer's operating conditions and external environment, and is essential for improving the robustness and accuracy of diagnosis under complex conditions. However, existing models lack effective mechanisms for fusing such multi-source heterogeneous data. Furthermore, existing methods typically employ simple vector concatenation or fixed gating methods to fuse local and temporal features during the feature fusion stage, failing to perform dynamic, content-based intelligent interactive fusion based on the actual content of the features, resulting in insufficient discriminative power of the fused features. Summary of the Invention

[0005] This application provides a training method, detection method, apparatus, and equipment for a time series prediction model, which solves the problem that existing deep learning models cannot simultaneously take into account the short-term mutation characteristics and long-term trend dependence of transformer fault data.

[0006] Firstly, this application provides a method for training a time series prediction model, including: Multi-source operating data of the transformer, including gas data and non-gas auxiliary data, is acquired to construct an original feature set, and the original feature set is then subjected to dimensionality reduction processing to obtain a feature vector. The feature vector is input into a preset temporal prediction model for iterative training until a preset convergence condition is met, thereby obtaining the trained temporal prediction model. In each iteration, a parallel multi-scale convolutional network extracts multi-scale local features from the input feature vector. These multi-scale local features are then sequentially passed through a gated recurrent unit and a gated residual connection before being input into an encoder to obtain temporal features. Cross-attention fusion is performed on the multi-scale local features and the temporal features to obtain a comprehensive feature vector. Classification prediction is then performed based on this comprehensive feature vector, and a fault type prediction result is output. The parameters of the multi-scale convolutional network, the gated recurrent unit, and the encoder are updated according to the difference between the fault type prediction result and the actual fault label.

[0007] This application uses multi-source operating data of transformers as input, which provides a more comprehensive basis for fault diagnosis compared to a single gas data source. Through the cooperation of multi-scale convolutional networks, gated recurrent units, and encoders, the model can simultaneously capture the short-term mutation features and long-term trend features of fault data, solving the problem that existing deep learning models cannot simultaneously take into account the temporal dependencies of two scales. Through the cross-attention fusion mechanism, local features and temporal features can be interactively fused based on content, improving the effectiveness of feature fusion. By updating the model parameters in reverse based on the difference between classification prediction and true label during iterative training, end-to-end closed-loop optimization of the model training process is achieved.

[0008] Further, the step of inputting the multi-scale local features into the encoder after sequentially passing them through a gated recurrent unit and a gated residual connection includes: performing short-term temporal encoding on the multi-scale local features by the gated recurrent unit to obtain the output sequence of the gated recurrent unit; aligning the output sequence of the gated recurrent unit with the input sequence of the gated recurrent unit in terms of dimensions; calculating a gate signal; weighting and fusing the output sequence of the gated recurrent unit with the input sequence of the gated recurrent unit using the gate signal; and inputting the fused sequence into the encoder.

[0009] This application first performs short-term temporal encoding on multi-scale local features through a gated recurrent unit, and then dynamically controls the information fusion ratio between the output of the gated recurrent unit and the original input through a learnable gating signal. This enables the model to adaptively decide how much short-term encoded information and how much long-term information of the original input to retain based on the actual characteristics of the current temporal data. This solves the problem that traditional gated recurrent units are difficult to handle long sequence dependencies, while retaining their efficiency advantage in short-term modeling.

[0010] Further, the step of performing cross-attention fusion on the multi-scale local features and the temporal features to obtain a comprehensive feature vector includes: performing global average pooling on the multi-scale local features and the temporal features respectively to obtain local feature vectors and temporal feature vectors; using the local feature vectors as queries and the temporal feature vectors as keys and values, performing attention calculation after projection through a learnable weight matrix to obtain gated local features; concatenating the gated local features with the temporal feature vectors to obtain a fusion vector; and adaptively weighting the fusion vector to obtain the comprehensive feature vector.

[0011] This application achieves content-based intelligent feature interaction fusion by replacing the traditional simple splicing or fixed gating method. It also performs adaptive weighting on the fusion vector before output classification, which can suppress the redundancy and noise that may be caused by simple splicing, highlight key feature patterns that are more relevant to fault type discrimination, and improve the discrimination ability of the comprehensive feature vector and the accuracy of the classification results.

[0012] Furthermore, the step of extracting multi-scale local features from the input feature vector using a parallel multi-scale convolutional network includes: extracting features from the feature vector at a first scale, a second scale, and a third scale, respectively. The first scale is used to extract short-term local features, the second scale is used to extract medium-term local features, and the third scale is used to extract long-term local features. The local features at the three scales are then merged along the channel dimension.

[0013] This application uses three different scales of multi-scale convolutional networks for feature extraction, which can capture short-term local mutation features, medium-term local change features and long-term local trend features in the input data respectively, so that the model can obtain local information at multiple time scales simultaneously during the feature extraction stage.

[0014] Furthermore, before inputting the multi-scale local features into the gated recurrent unit, the method further includes: applying channel attention mechanism and spatial attention mechanism sequentially to the multi-scale local features for weighted filtering, and inputting the weighted filtered multi-scale local features into the gated recurrent unit.

[0015] This application enables the model to adaptively evaluate and enhance features more relevant to fault detection, filter irrelevant information, and improve the input quality of subsequent temporal coding by performing attention screening on multi-scale local features in both channel and spatial dimensions before entering temporal coding.

[0016] Further, the process of acquiring multi-source operating data of the transformer, including gas data and non-gas auxiliary data, constructing an original feature set, and performing dimensionality reduction processing on the original feature set to obtain a feature vector includes: collecting multi-source operating data of the transformer, wherein the gas data includes concentration data of various gases dissolved in the oil, and the non-gas auxiliary data includes load current, voltage fluctuation rate, top oil temperature, winding temperature, ambient temperature, and ambient humidity, with the concentration of each gas and each non-gas auxiliary data serving as one-dimensional features to form a first feature set; according to the gas ratio method, the concentration ratio of each pair of gases in the various gases constitutes a one-dimensional feature, and the obtained gas ratio features are incorporated into the first feature set to obtain the original feature set; after standardizing each feature of the original feature set, the comprehensive contribution of each feature is determined based on the eigenvalue decomposition result of the covariance matrix, and features of a preset number of dimensions are selected as the feature vector after sorting according to the comprehensive contribution.

[0017] This application acquires multi-source information from both the internal chemical state and external operating conditions of the transformer by simultaneously collecting gas data and non-gas auxiliary data. By introducing the gas ratio method to expand features and combining it with principal component analysis for dimensionality reduction, it can mine fault feature information from a more comprehensive perspective, while reducing the input complexity of the model, avoiding interference from low-contribution or noisy features on the fault diagnosis model, and retaining the physical meaning of the original features while reducing dimensionality.

[0018] Furthermore, before inputting the feature vector into the preset time series prediction model for iterative training, the method further includes: using a preset heuristic optimization algorithm to globally optimize the hyperparameters of the preset time series prediction model to determine the optimal hyperparameter combination; and using the optimal hyperparameter combination to configure the preset time series prediction model.

[0019] This application overcomes the limitations of traditional manual experience and grid search methods, which are prone to getting stuck in local optima, by globally optimizing the key hyperparameters of the preset time series prediction model, thus ensuring that the performance of the preset time series prediction model is fully utilized.

[0020] This application also provides a transformer fault detection method, comprising: acquiring multi-source operating data of the transformer under test; inputting the multi-source operating data into a time-series prediction model obtained according to the above training method; and determining the transformer fault type according to the output of the time-series prediction model, wherein the fault type includes normal, medium-low temperature overheating, high temperature overheating, partial discharge, low-energy discharge, and high-energy discharge.

[0021] This application applies the trained time-series prediction model to actual transformer fault detection. It can utilize the multi-scale local feature extraction capability, short-term mutation and long-term trend synchronous capture capability, and content-based intelligent feature fusion capability learned by the time-series prediction model during the training phase to quickly and accurately determine the above six fault types of the transformer under test.

[0022] This application also provides a training device for a time-series prediction model, comprising: a data acquisition module for acquiring multi-source operating data of a transformer including gas data and non-gas auxiliary data, constructing an original feature set, and performing dimensionality reduction processing on the original feature set to obtain a feature vector; and a model training module for inputting the feature vector into a preset time-series prediction model for iterative training until a preset convergence condition is met to obtain the trained time-series prediction model; wherein, in each iteration, multi-scale local features are extracted from the input feature vector through a parallel multi-scale convolutional network, and the multi-scale local features are sequentially passed through a gated recurrent unit and a gated residual connection before being input into an encoder to obtain time-series features; cross-attention fusion is performed on the multi-scale local features and the time-series features to obtain a comprehensive feature vector, and classification prediction is performed based on the comprehensive feature vector to output a fault type prediction result; and the parameters of the multi-scale convolutional network, the gated recurrent unit, and the encoder are updated according to the difference between the fault type prediction result and the actual fault label.

[0023] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the training method of the aforementioned timing prediction model or the aforementioned transformer fault detection method. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating a training method for a time-series prediction model provided in an embodiment of the present invention. Figure 2 This is provided by an embodiment of the present invention. Figure 1 A flowchart illustrating the sub-steps of step 101; Figure 3 This is provided by an embodiment of the present invention. Figure 1 A flowchart illustrating the sub-steps of step 102; Figure 4 This is a schematic diagram of the structure of a time series prediction model provided in an embodiment of the present invention; Figure 5 This is a schematic flowchart of a transformer fault detection method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a training device for a time series prediction model provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] In the description of this application, the terms "first," "second," and "third" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or order. The term "multiple" refers to two or more. "And / or" refers to all three relationships between related objects.

[0027] This embodiment provides a training method for a time series prediction model, which can be used to train a time series prediction model for transformer fault detection.

[0028] Figure 1 This demonstrates the overall flow of the training method in this embodiment. Step 101 involves acquiring multi-source runtime data, constructing an original feature set, and reducing dimensionality to obtain feature vectors; Step 102 involves inputting the feature vectors into a preset time-series prediction model for iterative training to obtain the trained time-series prediction model. The sub-steps of 101 and 102 are respectively described in... Figure 2 and Figure 3 Further developments are underway.

[0029] Reference Figure 1 The training method includes steps 101 to 102, each of which is detailed below: 101. Obtain multi-source operating data of the transformer, including gas data and non-gas auxiliary data, construct an original feature set, and perform dimensionality reduction processing on the original feature set to obtain a feature vector.

[0030] 102. The feature vector is input into a preset temporal prediction model for iterative training until a preset convergence condition is met, thereby obtaining the trained temporal prediction model. In each iteration, a parallel multi-scale convolutional network extracts multi-scale local features from the input feature vector. These multi-scale local features are then sequentially passed through a gated recurrent unit and a gated residual connection before being input into an encoder to obtain temporal features. Cross-attention fusion is performed on the multi-scale local features and the temporal features to obtain a comprehensive feature vector. Classification prediction is then performed based on this comprehensive feature vector, and a fault type prediction result is output. The parameters of the multi-scale convolutional network, the gated recurrent unit, and the encoder are updated according to the difference between the fault type prediction result and the actual fault label.

[0031] In this embodiment, step 101 specifically includes: Figure 2 The three sub-steps of step 101 are shown: 1011 Collect multi-source operation data of the transformer to form the first feature set; 1012 According to the gas ratio method, the concentration ratio of each two gases constitutes a one-dimensional feature, which is then incorporated into the first feature set to obtain the original feature set; 1013 Standardize the original feature set, determine the comprehensive contribution of each dimension based on the eigenvalue decomposition of the covariance matrix, and select a preset number of dimensions as feature vectors after sorting by contribution.

[0032] Reference Figure 2 Step 101 specifically includes the following sub-steps: 1011. Collect multi-source operating data of the transformer.

[0033] The gas data includes concentration data of various gases dissolved in the oil, and the non-gas auxiliary data includes load current, voltage fluctuation rate, top oil temperature, winding temperature, ambient temperature, and ambient humidity. The concentration of each gas and each type of non-gas auxiliary data are each considered as a one-dimensional feature, constituting the first feature set.

[0034] As a specific example, the various gases are hydrogen ( ), methane ( ), ethane ( ), ethylene ( ), carbon monoxide ( ) and carbon dioxide ( The data consists of six gases, and the non-gas auxiliary data consists of these six gases, constituting a first feature set containing 12 dimensions. It is understood that in other embodiments, the types and quantities of gases, and the types and quantities of non-gas auxiliary data, can be adjusted according to actual needs.

[0035] 1012. According to the gas ratio method, the concentration ratio of each two gases in a variety of gases constitutes a one-dimensional feature. The obtained gas ratio features are then incorporated into the first feature set to obtain the original feature set.

[0036] As a specific example, the concentration ratio of each pair of the above six gases is used to construct a one-dimensional feature, resulting in a total of C(6,2)=15-dimensional gas ratio features. After being incorporated into the first feature set, a 27-dimensional original feature set is obtained.

[0037] 1013. After standardizing each dimension of the original feature set, the comprehensive contribution of each dimension of the feature is determined based on the eigenvalue decomposition result of the covariance matrix. After sorting according to the comprehensive contribution, a preset number of dimensions of features are selected as the feature vector.

[0038] Specifically, the mathematical processing of principal component analysis is as follows: taking the sample size as... The original number of features is 27, and a feature matrix is ​​constructed. First, standardize each dimension of the feature matrix to obtain the standardized matrix. The standardization method is as follows: in , The first The mean and standard deviation of the dimensional features; Indicates the first The sample at the th The values ​​of each original feature This represents the corresponding value after standardization. The sample number is 1 to 1. , The original feature numbers are assigned values ​​from 1 to 27; then the covariance matrix is ​​calculated. And on Perform eigenvalue decomposition to obtain eigenvalues. , ...and their corresponding eigenvectors, sorted from largest to smallest eigenvalue. The variance contribution rate of each principal component: forward Cumulative contribution rate of each principal component: To preserve the physical meaning of the original features after dimensionality reduction, the comprehensive contribution of each of the 27 original features is further calculated based on the principal component loadings: in, For the first The overall contribution of each original feature For the first The original feature in the first Loadings on each principal component; The summation index for the original feature number, with values ​​ranging from 1 to 27; Principal component numbering, with values ​​ranging from 1 to... ; The index for summing eigenvalues ​​ranges from 1 to 27. Take cumulative contribution rate The number of principal components when the threshold is reached. For the first The original feature in the first The loads on each principal component, i.e., the corresponding load terms in the denominator when summing the squares of the 27 original characteristic loads, are calculated by dividing the 27 original characteristics according to... Sort the original features from high to low and select the top 10 as the model input.

[0039] This application acquires multi-source information from both the internal chemical state and external operating conditions of the transformer by simultaneously collecting gas data and non-gas auxiliary data. By introducing the gas ratio method to expand features and combining it with principal component analysis for dimensionality reduction, it can mine fault feature information from a more comprehensive perspective, while reducing the input complexity of the model, avoiding interference from low-contribution or noisy features on the fault diagnosis model, and retaining the physical meaning of the original features while reducing dimensionality.

[0040] In this embodiment, step 102 specifically includes: Figure 3 Step 102 is illustrated with six sub-steps: 1021 Multi-scale convolutional network extracts multi-scale local features; 1022 Channel attention and spatial attention weighted filtering; 1023 Gated recurrent unit short-term encoding, gated residual connections, and encoder long-term dependency capture; 1024 Cross-attention fusion and adaptive weighting; 1025 Classification prediction; and 1026 Backpropagation to update parameters. The data processing within each sub-step is described in detail below. Figure 4 The model structure shown.

[0041] Reference Figure 3 Step 102 specifically includes the following sub-steps: Figure 4 The internal structure of the preset temporal prediction model and the data flow between its modules are demonstrated. The feature vector is first input into a parallel multi-scale convolutional network 201 (containing three convolutional branches: first-scale, second-scale, and third-scale) to extract multi-scale local features. These features are then sequentially processed by a channel attention module 202 and a spatial attention module 203 for weighted filtering. The filtered features then enter two paths: a gated recurrent unit 204 for short-term temporal encoding, and a gated residual connection 205 as a residual branch. The output of the gated recurrent unit 204 and the residual branch are weighted and fused in the gated residual connection 205, and then fed into an encoder 206 to capture long-term dependencies, yielding temporal features. The attention-weighted multi-scale local features and the temporal features output by the encoder are then processed by global average pooling 207, followed by content-based interactive fusion in a cross-attention fusion module 208, and finally by a feature-level adaptive weighting layer 209 to obtain a comprehensive feature vector. Finally, a classification output layer 210 performs classification prediction based on the comprehensive feature vector, outputting prediction results for six fault types.

[0042] Reference Figure 4The temporal prediction model includes: a parallel multi-scale convolutional network 201 for extracting multi-scale local features from the input feature vector; a channel attention module 202 and a spatial attention module 203 for weighted filtering of the multi-scale local features; a gated recurrent unit 204 for short-term temporal encoding of the weighted multi-scale local features; a gated residual connection 205 for weighted fusion of the output sequence of the gated recurrent unit with the input sequence; an encoder 206 for capturing long-term dependencies in the fused sequence to obtain temporal features; global average pooling 207 for performing global average pooling on the multi-scale local features and the temporal features respectively; a cross-attention fusion module 208 for performing cross-attention fusion on the pooled multi-scale local features and temporal features; a feature-level adaptive weighting layer 209 for adaptively weighting the fused vector to obtain a comprehensive feature vector; and a classification output layer 210 for performing classification prediction based on the comprehensive feature vector and outputting the fault type prediction result. The specific data processing procedures for each module are as follows: 1021. Multi-scale local features are extracted from the input feature vector using a parallel multi-scale convolutional network 201. Specifically, feature extraction is performed on the feature vector at a first scale, a second scale, and a third scale, respectively. The first scale is used to extract short-term local features, the second scale is used to extract medium-term local features, and the third scale is used to extract long-term local features. The local features at the three scales are merged along the channel dimension to obtain multi-scale local features.

[0043] Specifically, the input is the feature vector obtained after processing in step 101. ,in Represents real numbers, The sequence length is represented by 10, and the feature dimension is 10. The multi-scale convolutional network 201 sets up three parallel convolutional branches: the first branch uses a size of... The first branch uses a small convolutional kernel designed to capture microscopic local patterns or rapid changes in the input feature sequence; the second branch uses a kernel of size 1. The third branch uses a medium-sized convolutional kernel designed to capture local feature changes at medium scales; Large convolutional kernels are designed to capture macroscopic local trends or patterns over a wider range, where Each branch independently performs a convolution operation on the input feature vector, and its calculation follows the standard convolution formula: in To output feature maps at time steps The value, The weights of the convolution kernel for this branch. Given the input sequence, For bias terms, for Activation function The kernel size for the corresponding branch , or Subsequently, the output feature maps of the three branches are merged along the channel dimension to form a feature sequence that incorporates multi-scale local information. ( and These represent the length of the pooled sequence and the number of channels after merging, respectively.

[0044] This application uses three different scales of multi-scale convolutional networks for feature extraction, which can capture short-term local mutation features, medium-term local change features and long-term local trend features in the input data respectively, so that the model can obtain local information at multiple time scales simultaneously during the feature extraction stage.

[0045] 1022. Apply channel attention module 202 and spatial attention module 203 sequentially to perform weighted filtering on multi-scale local features.

[0046] Specifically, firstly, a channel attention mechanism is applied to the multi-scale local feature sequence. Perform global average pooling and global max pooling respectively to obtain two channel descriptors. and ;Will and The data is fed into a shared multilayer perceptron (MLP), the two output vectors of the MLP are added element-wise, and then applied... Activation function to generate channel attention weight map : Will With the multi-scale local feature sequence Multiplying each channel sequentially yields the intermediate features. .

[0047] Next, spatial attention is applied: for intermediate features... Perform average pooling and max pooling along its channel dimension to obtain two spatial descriptors. and ;Will and The feature maps are concatenated along the channel dimension; a one-dimensional convolutional layer is applied to the concatenated feature maps, and then... Activation function generates spatial attention weight map : in for One-dimensional convolutional layer of size for Activation function, With intermediate features Multiplying by space, we obtain the final weighted and filtered multi-scale local features. .

[0048] This application enables the model to adaptively evaluate and enhance features more relevant to fault detection, filter irrelevant information, and improve the input quality of subsequent temporal coding by performing attention screening on multi-scale local features in both channel and spatial dimensions before entering temporal coding.

[0049] 1023. The weighted filtered multi-scale local features After passing through the gated loop unit 204 and the gated residual connection 205 in sequence, the data is input to the encoder 206 to obtain the timing characteristics.

[0050] Specifically, the gated loop unit 204 processes the multi-scale local features. Short-term timing coding is performed to obtain the output sequence of the gated recurrent unit 204. A gated residual connection 205 is used between the gated loop unit 204 and the encoder 206. The gated residual connection 205 receives two inputs: one input is the output sequence of the gated loop unit 204. The other input is the multi-scale local features. (In the projection path of gated residual connection 205, it is denoted as) The two inputs are weighted and fused by the gated residual connection 205, and the fused result is sent to the encoder 206. Its multi-head self-attention mechanism captures long-term dependencies in the sequence, dynamically controlling the transmission and updating of information from the gated recurrent unit 204. To ensure dimensionality consistency, a one-dimensional convolutional layer is first used to... Projected onto Same dimension: in To and Aligned dimensions Next, the gating signal is calculated. (A weight between 0 and 1), this gating determines how much new information from the gating loop unit 204 and how much from the gating loop unit 204 is retained. Original information: in, , , These are learnable parameters. yes Activation function. Finally, the gating signal is used. Dynamic fusion and The sequence entering the encoder is obtained. : in This represents element-wise multiplication. The sequence after gating and fusing. The data is fed into encoder 206, where a multi-head self-attention mechanism is used to capture long-term dependencies and obtain temporal features. .

[0051] As a specific example, the encoder described is a Transformer encoder. It is understood that other encoder structures with long sequence modeling capabilities may also be used in other embodiments.

[0052] This application first performs short-term temporal encoding on multi-scale local features through a gated recurrent unit, and then dynamically controls the information fusion ratio between the output of the gated recurrent unit and the original input through a learnable gating signal. This enables the model to adaptively decide how much short-term encoded information and how much long-term information of the original input to retain based on the actual characteristics of the current temporal data. This solves the problem that traditional gated recurrent units are difficult to handle long sequence dependencies, while retaining their efficiency advantage in short-term modeling.

[0053] This application first processes multi-scale local features through a gated loop unit 204. By performing short-term temporal encoding and then dynamically controlling the fusion ratio of the output of the gated recurrent unit 204 with the original input information through learnable gating signals, the model can adaptively determine how much short-term encoded information and how much long-term information of the original input to retain based on the actual characteristics of the current temporal data. This solves the problem that traditional gated recurrent units are difficult to handle long sequence dependencies while retaining their efficiency advantage in short-term modeling.

[0054] 1024. Multi-scale local features are fused using the cross-attention module 208. and time series characteristics Cross-attention fusion is performed, and then a comprehensive feature vector is obtained through feature-level adaptive weighting layer 209.

[0055] Specifically, the cross-attention fusion module 208 is positioned after the gated recurrent unit 204 and the encoder 206, and before the classification output layer 210. Multi-scale local features are processed through parallel sub-branches of global average pooling 207. and the time-series features Perform global average pooling to obtain local feature vectors. and time series feature vectors The calculation method is as follows: in yes The length of the sequence, It is the first in the sequence Feature vectors at each time step yes The length of the sequence, It is the first in the sequence The feature vectors at each time step, using time-series feature vectors As context, generate a gating signal. Dynamically filter local feature vectors The generation of this gating signal introduces a cross-attention mechanism, in which... As a query As keys and values: The computation process first uses three learnable weight matrices. , , Projection as query matrix Key matrix Sum matrix The calculation method is as follows: , , Given a learnable weight matrix, gated local features are obtained through attention calculation. The calculation method is as follows: in Let be the dimension of the key vector. For activation function, Key matrix The transpose of the matrix. Then the gated local features. and time series feature vectors The fusion vector is obtained by concatenating the vectors. = The feature vector is then weighted and fused using a feature-level adaptive weighting layer 209, allowing the layer to learn both shallow local features and deep temporal features simultaneously, generating a comprehensive feature vector for classification. .

[0056] The aforementioned feature-level adaptive weighting layer 209 is a feature-level attention fusion layer placed after cross-attention gating fusion and before the classification output layer 210. Its function is to adaptively weight each feature dimension or feature component in the fusion vector, based on the already completed concatenation of local and temporal features. This suppresses redundant and noisy information that may result from simple concatenation, highlighting key local anomaly patterns and long-term temporal evolution patterns that are more relevant to fault type discrimination.

[0057] Specifically, let the fusion vector = Finally, the attention layer first calculates the attention score using learnable weights, and then... Normalization yields the weight vector The fusion vector is then element-wise weighted, and the calculation method can be expressed as follows: in, , , , For learnable parameters, The hyperbolic tangent activation function is used. This is the intermediate attention score vector of the feature-level adaptive weighted layer 209. This is the attention weight vector of the feature-level adaptive weighted layer 209. This indicates element-wise multiplication. Through feature-level adaptive weighting layer 209, the model can further learn the relative importance between shallow local anomaly patterns and deep temporal evolution patterns before the output layer, improving the comprehensive feature vector used for classification. Discriminability and interpretability.

[0058] This application achieves content-based intelligent feature interaction fusion by replacing the traditional simple splicing or fixed gating method. It also performs adaptive weighting on the fusion vector before output classification, which can suppress the redundancy and noise that may be caused by simple splicing, highlight key feature patterns that are more relevant to fault type discrimination, and improve the discrimination ability of the comprehensive feature vector and the accuracy of the classification results.

[0059] 1025. Classification output layer 210 performs classification prediction based on comprehensive feature vectors and outputs the fault type prediction results.

[0060] Specifically, the classification output layer 210 receives the comprehensive feature vector. and through having =Calculate the prediction vector for a fully connected layer with 6 output units The calculation method is as follows: in This is the weight matrix of the final fully connected layer. This is the bias vector for the final fully connected layer. Scores for each fault category are then applied. The function converts it into a probability distribution and calculates each fault category. The probability of: in for The Middle Scores for each category From 1 to The summation index, output That is, for each fault category The probability is determined by the probability of the fault, with the highest probability being the most likely fault. The fault types include: normal, medium and low temperature overheating, high temperature overheating, partial discharge, low energy discharge, and high energy discharge.

[0061] 1026. Based on the difference between the fault type prediction result and the actual fault label, calculate the loss function, and update the parameters of the multi-scale convolutional network 201, the gated recurrent unit 204 and the encoder 206 through backpropagation.

[0062] Specifically, the feature vectors obtained in step 101 are divided into training sets. and verification set , training set The input is fed into the configured model, and the model weights are iteratively updated and trained through backpropagation until the model is validated on the validation set. Once the performance metrics meet the convergence condition or reach the preset maximum number of iterations, the trained model weights are saved to obtain the trained time-series prediction model.

[0063] This application achieves end-to-end closed-loop optimization of the model training process by updating model parameters in reverse based on the difference between classification prediction and true label during iterative training.

[0064] As a preferred embodiment of the present invention, before inputting the feature vector into the preset time series prediction model for iterative training, a hyperparameter global optimization step is further included: using a preset heuristic optimization algorithm to globally optimize the hyperparameters of the preset time series prediction model to determine the optimal hyperparameter combination; and using the optimal hyperparameter combination to configure the preset time series prediction model.

[0065] Specifically, the Kepler optimization algorithm is used for hyperparameter optimization. A hyperparameter search space for the model is defined, and the hyperparameters include the learning rate. Number of training iterations The kernel size of the multi-scale convolutional network , , The number of hidden layer nodes in the gated loop unit The number of hidden layer nodes of the encoder Define the optimization objective function ,in A set of hyperparameter combinations includes the above hyperparameters. Use parameters for the improved model The cross-entropy loss is evaluated on the independent validation set, and the Kepler optimization algorithm is used to iteratively optimize within the search space to minimize the objective function. With the objective function as the target, through the iterative search process of the Kepler algorithm, a set of results is ultimately determined and output to satisfy the objective function. Achieving the optimal combination of hyperparameters .use After configuring the preset time series prediction model, the above iterative training is then performed.

[0066] This application overcomes the limitations of traditional manual experience and grid search methods, which are prone to getting stuck in local optima, by globally optimizing the key hyperparameters of the preset time series prediction model, thus ensuring that the performance of the preset time series prediction model is fully utilized.

[0067] In summary, the embodiments of this application use multi-source operational data (gas data + non-gas auxiliary data) as input to provide a more comprehensive basis for fault diagnosis; through the cooperation of multi-scale convolutional networks with gated recurrent units and encoders, the model can simultaneously capture the short-term abrupt change features and long-term trend features of fault data; through a cross-attention fusion mechanism, content-based interactive fusion of local features and temporal features is achieved; and through iterative training, end-to-end closed-loop optimization of model parameters is achieved. This solves the problem that existing deep learning models cannot simultaneously accommodate temporal dependencies at two scales, thus improving the accuracy of transformer fault detection.

[0068] Accordingly, this application also provides a method for detecting transformer faults.

[0069] Figure 5 The process of applying a trained time-series prediction model for transformer fault detection is demonstrated. Step 301: Obtain multi-source operating data of the transformer under test (concentration of 6 gases and 6 non-gas auxiliary data), and obtain feature vectors according to the same feature engineering process as the training method (gas ratio method expansion and principal component analysis dimensionality reduction); Step 302: Input the feature vectors into the trained time-series prediction model, and the model outputs the fault type: normal, medium-low temperature overheating, high temperature overheating, partial discharge, low-energy discharge, or high-energy discharge.

[0070] Reference Figure 5 The detection method includes steps 301 to 302, and the specific steps are as follows: 301. Obtain multi-source operating data of the transformer under test.

[0071] The multi-source operating data includes dissolved gas concentration data in the six types of oils and six types of non-gas auxiliary data. Following the same feature engineering process as in step 101 (including gas ratio method expansion and principal component analysis dimensionality reduction), the feature vector corresponding to the transformer under test is obtained.

[0072] 302. Input the feature vector into the trained time-series prediction model, and determine the transformer fault type based on the model output. The time-series prediction model is obtained according to the training method described above in this application, and the fault types include normal, medium-low temperature overheating, high temperature overheating, partial discharge, low-energy discharge, and high-energy discharge.

[0073] Accordingly, this application also provides a training device for a time series prediction model.

[0074] Figure 6 The module composition and data transmission relationships of the training device are illustrated. The training device includes: a 401 data acquisition module, used to acquire multi-source runtime data, construct an original feature set, perform dimensionality reduction processing, and output feature vectors; and a 402 model training module, which receives the feature vectors and inputs them into a preset time-series prediction model for iterative training (internally containing...). Figure 2 The network consists of a multi-scale convolutional network 201, a channel attention module 202, a spatial attention module 203, a gated recurrent unit 204, a gated residual connection 205, an encoder 206, a global average pooling 207, a cross-attention fusion module 208, a feature-level adaptive weighted layer 209, and a classification output layer 210, until convergence is achieved and the trained temporal prediction model is output.

[0075] Reference Figure 6 The training device includes: The data acquisition module 401 is used to acquire multi-source operating data of the transformer, including gas data and non-gas auxiliary data, construct an original feature set, and perform dimensionality reduction processing on the original feature set to obtain a feature vector. The model training module 402 is used to input the feature vector into a preset temporal prediction model for iterative training until a preset convergence condition is met, thereby obtaining the trained temporal prediction model. In each iteration, multi-scale local features are extracted from the input feature vector through a parallel multi-scale convolutional network 201. The multi-scale local features are then passed through a gated recurrent unit 204 and a gated residual connection 205 and input into an encoder 206 to obtain temporal features. Global average pooling 207 is performed on the multi-scale local features and temporal features, and then the results are fed into a cross-attention fusion module 208 to obtain a comprehensive feature vector. Classification prediction is performed based on the comprehensive feature vector, and the fault type prediction result is output. The parameters of the multi-scale convolutional network 201, the gated recurrent unit 204, and the encoder 206 are updated according to the difference between the fault type prediction result and the actual fault label.

[0076] It is understood that the specific functions and implementation methods of the above modules can be referred to the detailed description of steps 101 and 102 in the above training method embodiment, and will not be repeated here.

[0077] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the training method of the aforementioned timing prediction model or the aforementioned transformer fault detection method.

[0078] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a time series prediction model, characterized in that, include: Multi-source operating data of the transformer, including gas data and non-gas auxiliary data, is acquired to construct an original feature set, and the original feature set is then subjected to dimensionality reduction processing to obtain a feature vector. The feature vector is input into a preset temporal prediction model for iterative training until a preset convergence condition is met, thereby obtaining the trained temporal prediction model. In each iteration, a parallel multi-scale convolutional network extracts multi-scale local features from the input feature vector. These multi-scale local features are then sequentially passed through a gated recurrent unit and a gated residual connection before being input into an encoder to obtain temporal features. Cross-attention fusion is performed on the multi-scale local features and the temporal features to obtain a comprehensive feature vector. Classification prediction is then performed based on this comprehensive feature vector, and a fault type prediction result is output. The parameters of the multi-scale convolutional network, the gated recurrent unit, and the encoder are updated according to the difference between the fault type prediction result and the actual fault label.

2. The training method for the time series prediction model according to claim 1, characterized in that, The step of sequentially passing the multi-scale local features through a gated loop unit and a gated residual connection before inputting them into the encoder includes: The gated recurrent unit performs short-term temporal encoding on the multi-scale local features to obtain the output sequence of the gated recurrent unit; After dimensionally aligning the output sequence of the gated loop unit with the input sequence of the gated loop unit, a gate signal is calculated. The output sequence of the gated loop unit and the input sequence of the gated loop unit are then weighted and fused using the gate signal, and the fused sequence is input into the encoder.

3. The training method for the time series prediction model according to claim 1, characterized in that, The process of performing cross-attention fusion on the multi-scale local features and the temporal features to obtain a comprehensive feature vector includes: Global average pooling is performed on the multi-scale local features and the temporal features respectively to obtain local feature vectors and temporal feature vectors; Using the local feature vector as the query and the temporal feature vector as the key and value, attention is calculated after projection through a learnable weight matrix to obtain gated local features; The gated local features are concatenated with the temporal feature vector to obtain a fusion vector; the fusion vector is then adaptively weighted to obtain the comprehensive feature vector.

4. The training method for the time series prediction model according to claim 1, characterized in that, The extraction of multi-scale local features from the input feature vector through parallel multi-scale convolution includes: Feature extraction is performed on the feature vector at a first scale, a second scale, and a third scale, respectively. The first scale is used to extract short-term local features, the second scale is used to extract medium-term local features, and the third scale is used to extract long-term local features. The local features at the three scales are then merged along the channel dimension.

5. The training method for the time series prediction model according to claim 1, characterized in that, Before inputting the multi-scale local features into the gated recurrent unit, the method further includes: The multi-scale local features are sequentially weighted and filtered using channel attention and spatial attention mechanisms, and the weighted and filtered multi-scale local features are then input into the gated loop unit.

6. The training method for the time series prediction model according to claim 1, characterized in that, The process involves acquiring multi-source operating data of the transformer, including gas data and non-gas auxiliary data, constructing an original feature set, and performing dimensionality reduction processing based on the original feature set to obtain a feature vector, including: Multi-source operating data of the transformer are collected. The gas data includes the concentration data of various gases dissolved in the oil. The non-gas auxiliary data includes load current, voltage fluctuation rate, top oil temperature, winding temperature, ambient temperature and ambient humidity. The concentration of each gas and each non-gas auxiliary data are each used as a one-dimensional feature to form the first feature set. According to the gas ratio method, the concentration ratio of each two gases in the multiple gases is used to form a one-dimensional feature. The resulting gas ratio features are then incorporated into the first feature set to obtain the original feature set. After standardizing each dimension of the original feature set, the comprehensive contribution of each dimension of the feature is determined based on the eigenvalue decomposition result of the covariance matrix. After sorting according to the comprehensive contribution, a preset number of dimensions of features are selected as the feature vector.

7. The training method for the time series prediction model according to claim 1, characterized in that, Before inputting the feature vector into the preset time-series prediction model for iterative training, the method further includes: A preset heuristic optimization algorithm is used to globally optimize the hyperparameters of the preset time series prediction model to determine the optimal combination of hyperparameters; The preset time series prediction model is configured using the optimal hyperparameter combination.

8. A method for detecting transformer faults, characterized in that, include: Acquire multi-source operating data of the transformer under test; The multi-source running data is input into the time series prediction model obtained by the training method according to any one of claims 1 to 7; The transformer fault type is determined based on the output of the time-series prediction model. The fault types include normal, medium-low temperature overheating, high temperature overheating, partial discharge, low-energy discharge, and high-energy discharge.

9. A training device for a time series prediction model, characterized in that, include: The data acquisition module is used to acquire multi-source operating data of the transformer, including gas data and non-gas auxiliary data, construct an original feature set, and perform dimensionality reduction processing on the original feature set to obtain a feature vector. The model training module is used to input the feature vector into a preset temporal prediction model for iterative training until a preset convergence condition is met, thereby obtaining the trained temporal prediction model. In each iteration, a parallel multi-scale convolutional network extracts multi-scale local features from the input feature vector. These multi-scale local features are then sequentially passed through a gated recurrent unit and a gated residual connection before being input into an encoder to obtain temporal features. Cross-attention fusion is performed on the multi-scale local features and the temporal features to obtain a comprehensive feature vector. Classification prediction is then performed based on this comprehensive feature vector, and a fault type prediction result is output. The parameters of the multi-scale convolutional network, the gated recurrent unit, and the encoder are updated according to the difference between the fault type prediction result and the actual fault label.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the training method of the time-series prediction model as described in any one of claims 1 to 7, or the transformer fault detection method as described in claim 8.

Images