A transformer oil dissolved gas content prediction method and system

Abstract

The application discloses a transformer oil dissolved gas content prediction method and system, belongs to the technical field of transformer fault prediction, solves the problems of limited transformer oil dissolved gas content abnormal data identification and insufficient model performance optimization in the prior art, realizes accurate identification of gas abnormal data by comprehensively considering the characteristics of local measurement and global measurement of the improved iNNE, and effectively improves data quality; the key hyperparameters of the Bert model are optimized by using the Optuna hyperparameter optimization framework, so that the limitations of manual parameter adjustment are effectively avoided; the Bert model is used to capture the long-distance dependence relationship and multi-scale features in the gas content time sequence, and when facing complex transformer oil dissolved gas information, accurate identification of transformer oil dissolved gas content abnormal information and rapid prediction of future gas content can be realized.

Description

Technical Field

[0001] This invention belongs to the field of transformer fault prediction technology, and relates to a method and system for predicting the dissolved gas content in transformer oil. Background Technology

[0002] Transformers are crucial equipment in the power grid transmission and transformation process, and their operating status directly affects the safe and stable operation of the power grid. Analyzing changes in the composition and content of dissolved gases in transformer oil is one of the most effective means of monitoring the safe operation of oil-immersed transformers. Predicting the dissolved gas content in transformer oil can help detect potential problems in transformers earlier, providing transformer operation and maintenance personnel with the ability to proactively eliminate safety hazards and avoid serious safety accidents, thus ensuring the stable and safe operation of the power system.

[0003] Currently, Recurrent Neural Networks (RNNs) are widely used in time series forecasting, with Long Short-Term Memory (LSTM) networks being one of the most common models. In the task of predicting dissolved gas content in transformer oil, LSTM models can make predictions in the following way: LSTM, through its unique structure including forget gates, input gates, and output gates, effectively captures and utilizes long-term dependencies in time series. When predicting dissolved gas content in transformer oil, the input to the LSTM model is typically a sequence of dissolved gas content data from historical moments. This data, after preprocessing and normalization, serves as the input sequence for the LSTM model. During the model training phase, the LSTM learns patterns and trends in the input sequence, progressively adjusting its internal state and weight parameters to optimize its ability to predict future changes in dissolved gas content. The model training process typically involves multiple epochs. In each epoch, the LSTM optimizes the loss function through backpropagation to reduce the error between predicted and observed values. Once training is complete, the LSTM model can be applied to future data predictions. When new transformer oil sample data is received, the LSTM generates a prediction of future dissolved gas content based on historical data and the current input state. This predictive capability makes LSTM excel at handling long-term dependencies and nonlinear dynamic patterns in time series data, making it suitable for applications requiring accurate prediction of future trends. For example, the invention patent with publication number CN112734028B discloses a modeling method for predicting dissolved gas concentration in transformer oil. This method selects important factors that may have a positive impact on the prediction of dissolved gases in transformer oil and establishes a variable environment. It sets that the prediction of each gas is positively influenced by all other variables, uses a Long Short-Term Memory (LSTM) network prediction model to predict and compare errors, verifies hypotheses, and extracts important factors that have a positive impact on the gas variables to be predicted.

[0004] The aforementioned and existing related technologies often suffer from the following drawbacks: Although LSTM is designed to capture long-term dependencies in time series, it may still encounter gradient vanishing or exploding problems when processing very long sequences. In such cases, the model may not be able to effectively remember the dissolved gas content information in transformer oil over long time intervals, affecting the accuracy and stability of long-term predictions. LSTM relies only on the input of the previous time step and its own state at each time step. This local information processing method limits its ability to understand the global context, especially when dealing with complex dissolved gas information in oil. Therefore, there is an urgent need to propose a prediction method that addresses the limitations in identifying anomalous dissolved gas content data in transformer oil and the inadequacy of model performance optimization. Summary of the Invention

[0005] The technical solution of the present invention is used to solve the problems of limited identification of abnormal data on dissolved gas content in transformer oil and insufficient optimization of model performance in the prior art.

[0006] The present invention solves the above-mentioned technical problems through the following technical solutions:

[0007] A method for predicting the dissolved gas content in transformer oil includes the following steps: S1. Obtain power transformer monitoring data as the original dataset, and divide the original dataset into training set, validation set and test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2. S2. Use the improved iNNE algorithm to identify abnormal data in the original dataset; S3. Use interpolation to correct the data; S4. Establish a prediction model for dissolved gas content in transformer oil and fine-tune it using the Optuna hyperparameter optimization framework. S41. Establish a prediction model for dissolved gas content in transformer oil based on BERT. S42. Convert the time series data of dissolved gases in transformer oil into a format suitable for BERT processing and perform data preprocessing; S43. Optuna uses EI as the optimization criterion for the objective function and selects the optimal parameter set based on the EI values ​​in different parameter sets. S5. Use the prediction model with the optimal parameter set to predict the test set, output the final result of the prediction model, and verify the model performance.

[0008] Further, the step S2, which involves using the improved iNNE algorithm to identify anomalous data in the original dataset, specifically involves: constructing a hypersphere centered at each data point in the original dataset with a radius equal to the Euclidean distance from that point to its nearest neighbor, thus isolating that point from other points; calculating the corrected isolation score for each point and converting it into anomaly probability; setting an anomaly probability threshold; and identifying various types of anomalous data. The expression for the hypersphere is as follows:

[0009]

[0010] in, It is a hypersphere. For the center of the ball, Let be the radius of the sphere. For the center of the ball The nearest neighbor, For the center of the ball The Euclidean distance between nearest neighbors, where x is a point in the original dataset D.

[0011] Furthermore, the input layer of the prediction model in step S41 includes: Word Embeddings, Sentence Embeddings, and Position Embeddings. The Word Embeddings are used to split the data processed in step S3 into word units, using words as the basic unit, and vectorizing the data at each time point. The Word Embeddings are used directly as the input of dissolved gas content in oil. The Sentence Embeddings are used to consider the relationship between two sentences, extract information from the entire sentence, and represent it as the relationship between dissolved gases in different types of oil. The Position Embeddings are used to consider the meaning expressed by each word in different positions, and the Position Embeddings are equivalent to timestamp embedding to provide time information of dissolved gases in oil.

[0012] Furthermore, the step S42, which involves converting the time series data of dissolved gases in transformer oil into a format suitable for BERT processing, specifically involves: dividing the time series data into multiple time windows, with each time point data within a window corresponding to a token in the BERT input sequence; for each time window, creating a sequence containing gas concentration data within that time period, which will be passed as input to BERT; and forming a multidimensional token by concatenating multiple gas concentration values ​​(such as H2, CH4, C2H6, etc.) for each day into a vector.

[0013] Further, the expression for the EI value in step S43 is as follows:

[0014] in, These are the parameter points to be optimized. At the parameter point The expected value of the model. It is the expected value of the currently known optimal solution. At the parameter point The standard deviation of the model at that location, It is the cumulative distribution function of the standard normal distribution. It is the probability density function of the standard normal distribution. It is the quantile of the standard normal distribution.

[0015] The present invention also provides a system for predicting the dissolved gas content in transformer oil, including a data partitioning module, a data identification module, a data correction module, a prediction model module, and a model verification module; The data partitioning module is used to acquire power transformer monitoring data as the raw dataset, and divide the raw dataset into training set, validation set and test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2. The data identification module is used to identify abnormal data in the original dataset using the improved iNNE algorithm; The data correction module is used to correct data using interpolation. The prediction model module is used to establish a prediction model for dissolved gas content in transformer oil and to fine-tune it using the Optuna hyperparameter optimization framework. Specifically, it establishes a prediction model for dissolved gas content in transformer oil based on BERT, converts the time series data of dissolved gases in transformer oil into a format suitable for BERT processing, and performs data preprocessing. Optuna uses EI as the optimization criterion for the objective function and selects the optimal parameter group based on the EI values ​​in different parameter groups. The model validation module is used to use the prediction model with the optimal parameter set to predict the test set, output the final result of the prediction model, and verify the model performance.

[0016] Furthermore, the data identification module utilizes the improved iNNE algorithm to identify anomalous data in the original dataset. Specifically, the improved iNNE algorithm is used to construct a hypersphere for each data point in the original dataset, centered at that point and with the Euclidean distance from that point to its nearest neighbor as its radius, thus isolating that point from other points. A corrected isolation score for each point is calculated and converted into anomaly probability. An anomaly probability threshold is set to identify various types of anomalous data. The hypersphere expression is as follows:

[0017]

[0018] in, It is a hypersphere. For the center of the ball, Let be the radius of the sphere. For the center of the ball The nearest neighbor, For the center of the ball The Euclidean distance between nearest neighbors, where x is a point in the original dataset D.

[0019] Furthermore, the input layer of the prediction model module includes: Word Embeddings, Sentence Embeddings, and Position Embeddings. Word Embeddings are used to split the data processed in step S3 into word units, using words as the basic unit, vectorizing the data at each time point, and directly using Word Embeddings as the input for dissolved gas content in the oil. Sentence Embeddings are used to consider the relationship between two sentences, extracting information from the entire sentence and representing it as the relationship between dissolved gases in different types of oil. Position Embeddings are used to consider the meaning expressed by each word in different positions, effectively embedding a timestamp to provide temporal information about dissolved gases in the oil.

[0020] Furthermore, the prediction model module converts the time series data of dissolved gases in transformer oil into a format suitable for BERT processing as follows: the time series data is divided into multiple time windows, and the time point data in each window corresponds to a token in the input sequence of BERT; for each time window, a sequence containing the gas concentration data within that time period is created, and this sequence is passed to BERT as input; a multidimensional token is formed by concatenating multiple gas concentration values ​​(such as H2, CH4, C2H6, etc.) for each day into a vector.

[0021] Furthermore, the expression for the EI value in the prediction model module is as follows:

[0022] in, These are the parameter points to be optimized. At the parameter point The expected value of the model. It is the expected value of the currently known optimal solution. At the parameter point The standard deviation of the model at that location, It is the cumulative distribution function of the standard normal distribution. It is the probability density function of the standard normal distribution. It is the quantile of the standard normal distribution.

[0023] The advantages of this invention are as follows: It achieves accurate identification of gas anomaly data by comprehensively considering the characteristics of local and global metrics through an improved iNNE, effectively improving data quality; it optimizes the key hyperparameters of the BERT model using the Optuna hyperparameter optimization framework, effectively avoiding the limitations of manual parameter tuning; and it captures long-distance dependencies and multi-scale features in the gas content time series by fine-tuning the BERT model structure and converting the time series data format, thus better understanding the complex relationships between gases. When faced with complex dissolved gas information in transformer oil, it can accurately identify anomalies in dissolved gas content and quickly predict gas content at future times, overcoming the shortcomings of limited anomaly data identification and insufficient model performance optimization in existing technologies. This comprehensive application method can effectively improve the efficiency of industrial equipment monitoring and maintenance, help to promptly detect anomalies in transformer oil, and take necessary preventive and remedial measures, thereby ensuring the safe operation and reliability of equipment. Attached Figure Description

[0024] Figure 1 This is a flowchart of a method for predicting dissolved gas content in transformer oil according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the Bert model input layer structure according to an embodiment of the present invention; Figure 3 This is a graph showing the error results of each model under different algorithms in this embodiment of the invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments: Example 1 like Figure 1 Specifically, a method for predicting the dissolved gas content in transformer oil is disclosed, including the following steps: S1. Obtain power transformer monitoring data as the original dataset, and divide the original dataset into training set, validation set and test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2. S2. Use the improved iNNE algorithm to identify abnormal data in the original dataset; Specifically, the improved iNNE algorithm is used to construct a hypersphere for each data point in the original dataset, with the point as the center and the Euclidean distance from the point to its nearest neighbor as the radius, thus isolating the point from other points; the corrected isolation score of each point is calculated and converted into anomaly probability, anomaly probability threshold is set, and various types of anomalous data are identified; the anomalous data includes isolated anomalies, continuous anomalies, zero points, etc.

[0027] Specifically, from the given original dataset D, a subset S of size m is randomly selected. Using each data point in subset S as the center and the Euclidean distance between that point and its nearest neighbor as the radius, a set of hyperspheres is constructed. The expression is as follows:

[0028]

[0029] in, For the center of the ball, Let be the radius of the sphere. For the center of the ball The nearest neighbor, For the center of the ball The Euclidean distance between nearest neighbors, where x is a point in the original dataset D.

[0030] This invention uses a hypersphere to isolate data from other data, wherein the radius... Characterized the center of the ball The degree of isolation, radius The larger the value, the more likely it is to be the center of the ball. The more isolated the area, the more isolated it is, and vice versa. However, this invention does not rely directly on the radius. Instead of using methods to identify abnormal data, the system calculates the isolation score for each data point to determine the probability of anomalies and thus identify abnormal data.

[0031] Compared to the traditional iNNE algorithm, which only uses local relative radii to calculate the isolation score (i.e., anomaly identification solely based on the ratio of the nearest neighbor hypersphere radius to the hypersphere radius of the sample point), this invention addresses the aforementioned influencing factors. By correcting the local relative radius of each data point, the isolation score calculation considers not only the local relative radius but also the relative size of the sample point's hypersphere radius to the largest hypersphere radius in the hypersphere group. This overcomes the inaccuracy of the traditional iNNE algorithm and effectively improves its recognition accuracy.

[0032] This invention utilizes an improved nearest-neighbor distance-based isolated anomaly detection method to identify corresponding anomaly data based on the characteristics of dissolved gas datasets in transformer oil. On one hand, dissolved gas data in transformer oil may have different feature scales; single-scale anomaly detection may fail to capture all types of anomalies and is prone to getting trapped in local optima. The improved iNNE algorithm can satisfy the need for local anomaly detection while also identifying anomaly data globally, achieving anomaly detection at multiple scales and reducing the risk of false positives and false negatives. On the other hand, dissolved gas data in transformer oil is typically continuously monitored and involves a large amount of data. The improved iNNE algorithm introduces a complexity optimization strategy in multi-scale anomaly detection, enabling efficient processing of large-scale data and improving the processing speed and detection efficiency of anomaly detection.

[0033] S3. Use interpolation to correct the data; This invention uses interpolation to infer the possible true values ​​of abnormal data based on the characteristics of normal data points around the abnormal data points obtained in step S2, accurately restores the original situation of the abnormal data, and thus corrects the impact of the existence of abnormal data on the model prediction results.

[0034] S4. Establish a prediction model for dissolved gas content in transformer oil and fine-tune it using the Optuna hyperparameter optimization framework. S41. Establish a prediction model for dissolved gas content in transformer oil based on BERT. BERT is a pre-trained language model developed by Google. As a deep learning model based on a self-attention mechanism, it was initially designed for processing text sequences. However, considering that the core mechanism of the BERT model is self-attention, it essentially adopts the Transformer architecture and excels at processing sequential data. Therefore, this invention applies it to processing time series data, especially when the time series has complex patterns and interdependencies, where the BERT model demonstrates excellent performance in predicting time series data. In the scenario of predicting dissolved gas content in transformer oil, there are complex dependencies between the gases. Therefore, the BERT model can be used to capture these relationships, thus achieving better gas content prediction. like Figure 2 As shown, since BERT typically processes discrete text tokens, the architecture of the prediction model needs to be adjusted to adapt to continuous time series data. The input layer of the prediction model includes: Word Embeddings, Sentence Embeddings, and Position Embeddings. Word Embeddings are used to split the data processed in step S3 into word units, using words as the basic unit, and vectorizing the data at each time point. Word Embeddings are used directly as the input for dissolved gas content in oil. Sentence Embeddings are used to consider the relationship between two sentences, extracting information from the entire sentence and representing it as the relationship between dissolved gases in different types of oil. Position Embeddings are used to consider the meaning expressed by each word in different positions, and Position Embeddings are equivalent to timestamp embedding to provide time information on dissolved gases in oil.

[0035] S42. Convert the time series data of dissolved gases in transformer oil into a format suitable for BERT processing and perform data preprocessing; Specifically, the time series data is divided into multiple time windows, with each time point within a window corresponding to a token in BERT's input sequence. For each time window, a sequence containing gas concentration data for that time period is created, and this sequence is passed as input to BERT. A multidimensional token is formed by concatenating multiple gas concentration values ​​(such as H2, CH4, C2H6, etc.) for each day into a vector. To ensure scale consistency across different time points, the gas concentration data is normalized or standardized, which avoids model training instability caused by excessively large differences in the absolute values ​​of gas concentrations.

[0036] This invention, through a finely tuned prediction model and converted time-series data, enables BERT to better understand the context when predicting dissolved gases in transformer oil by utilizing contextual information, including the relationships between preceding and following sentences and within sentences. BERT also utilizes its core self-attention mechanism to capture long-distance dependencies and multi-scale features in the gas content time series, thereby better understanding the complex relationships between gases and improving prediction accuracy.

[0037] S43. Optuna uses EI as the optimization criterion for the objective function and selects the optimal parameter set based on the EI values ​​in different parameter sets. Specifically, Optuna is a hyperparameter optimization framework based on Bayesian optimization. In this invention, Optuna uses Expected Improvement (EI) as the optimization criterion for the objective function. This criterion optimizes by measuring the expected improvement of the proposed parameter points relative to the currently known optimal solution. Compared to traditional optimization criteria such as Upper Confidence Bound (UCB) in Bayesian optimization, EI has the advantage of better balancing known and unknown solutions, thereby improving global search performance. Therefore, using EI demonstrates superior practical performance in the complex problem of predicting dissolved gas content in transformer oil. The expression for the EI value is as follows:

[0038] in, These are the parameter points to be optimized. At the parameter point The expected value of the model. It is the expected value of the currently known optimal solution. At the parameter point The standard deviation of the model at that location, It is the cumulative distribution function of the standard normal distribution. It is the probability density function of the standard normal distribution. It is the quantile of the standard normal distribution, usually 0.

[0039] The Optuna hyperparameter optimization framework uses the efficient optimization algorithm EI to search for the optimal set of hyperparameters based on a specified objective function, which helps the BERT model to perform parameter tuning more effectively. This reduces the time and effort required for manual experimentation and hyperparameter tuning, thereby accelerating the training and optimization process of the BERT model. At the same time, compared with traditional manual parameter tuning methods, this invention also has advantages in parameter tuning performance.

[0040] S5. Use the prediction model with the optimal parameter set to predict the test set, output the final result of the prediction model, and verify the model performance. In this embodiment, monitoring data of a 220kV power transformer from abroad is collected as the original dataset. After anomaly detection, the abnormal data is replaced by interpolation and directly input into the BERT-based transformer oil dissolved gas content prediction model according to the data conversion method in step S4 above.

[0041] To evaluate the predictive performance of the proposed model, this invention uses mean squared error (MSE) as the evaluation metric. MSE is a commonly used metric to measure the difference between the model's predictions and the true values. It is used to calculate the average of the squared errors between the model's predicted and true values, effectively reflecting the model's prediction error and fitting effect. The calculation formula is as follows:

[0042] Where n is the number of samples, This is the actual value of the dissolved gas content in the oil during the i-th training session. This is the predicted value of dissolved gas content in the oil during the i-th training iteration. The smaller the MSE, the smaller the difference between the model's prediction and the actual value, and the better the model fits the data.

[0043] like Figure 3 As shown, the prediction model is validated and analyzed. This embodiment uses LSTM, GRU, BERT, improved iNNE-BERT (MiNNE-BERT), Optuna-BERT, and improved iNNE-Optuna-BERT (MiNNE-Optuna-BERT) for comparative experiments. As can be seen from the training results in the figure below, BERT's prediction performance has a certain advantage over LSTM and GRU during model training. However, the optimized BERT's prediction performance far exceeds that of the three unoptimized models. The improved iNNE-BERT and Optuna-BERT show significant improvements in prediction performance compared to BERT. Furthermore, the improved iNNE-Optuna-BERT method proposed in this invention achieves the lowest error while reaching the optimum quickly.

[0044] This invention achieves accurate identification of gas anomaly data by comprehensively considering the characteristics of local and global metrics through an improved iNNE, effectively improving data quality. It optimizes key hyperparameters of the BERT model using the Optuna hyperparameter optimization framework, effectively avoiding the limitations of manual parameter tuning. By fine-tuning the BERT model structure and converting the time-series data format, it utilizes the BERT model to capture long-distance dependencies and multi-scale features in gas content time series, better understanding the complex relationships between gases. When faced with complex dissolved gas information in transformer oil, it can accurately identify anomalies in dissolved gas content and quickly predict gas content at future times, overcoming the shortcomings of limited anomaly data identification and insufficient model performance optimization in existing technologies. This comprehensive application method can effectively improve the efficiency of industrial equipment monitoring and maintenance, helping to promptly detect anomalies in transformer oil and take necessary preventative and remedial measures, thereby ensuring the safe operation and reliability of equipment.

[0045] The present invention also provides a system for predicting the dissolved gas content in transformer oil, including a data partitioning module, a data identification module, a data correction module, a prediction model module, and a model verification module; The data partitioning module is used to acquire power transformer monitoring data as the raw dataset, and divide the raw dataset into training set, validation set and test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2. The data identification module is used to identify abnormal data in the original dataset using the improved iNNE algorithm; Specifically, the improved iNNE algorithm is used to construct a hypersphere for each data point in the original dataset, with the point as the center and the Euclidean distance from the point to its nearest neighbor as the radius, thus isolating the point from other points; the corrected isolation score of each point is calculated and converted into anomaly probability, anomaly probability threshold is set, and various types of anomalous data are identified; the anomalous data includes isolated anomalies, continuous anomalies, zero points, etc.

[0046] Specifically, from the given original dataset D, a subset S of size m is randomly selected. Using each data point in subset S as the center and the Euclidean distance between that point and its nearest neighbor as the radius, a set of hyperspheres is constructed. The expression is as follows:

[0047]

[0048] in, For the center of the ball, Let be the radius of the sphere. For the center of the ball The nearest neighbor, For the center of the ball The Euclidean distance between nearest neighbors, where x is a point in the original dataset D.

[0049] This invention uses a hypersphere to isolate data from other data, wherein the radius... Characterized the center of the ball The degree of isolation, radius The larger the value, the more likely it is to be the center of the ball. The more isolated the area, the more isolated it is, and vice versa. However, this invention does not rely directly on the radius. Instead of using methods to identify abnormal data, the system calculates the isolation score for each data point to determine the probability of anomalies and thus identify abnormal data.

[0050] Compared to the traditional iNNE algorithm, which only uses local relative radii to calculate the isolation score (i.e., anomaly identification solely based on the ratio of the nearest neighbor hypersphere radius to the hypersphere radius of the sample point), this invention addresses the aforementioned influencing factors. By correcting the local relative radius of each data point, the isolation score calculation considers not only the local relative radius but also the relative size of the sample point's hypersphere radius to the largest hypersphere radius in the hypersphere group. This overcomes the inaccuracy of the traditional iNNE algorithm and effectively improves its recognition accuracy.

[0051] This invention utilizes an improved nearest-neighbor distance-based isolated anomaly detection method to identify corresponding anomaly data based on the characteristics of dissolved gas datasets in transformer oil. On one hand, dissolved gas data in transformer oil may have different feature scales; single-scale anomaly detection may fail to capture all types of anomalies, easily falling into local optima. The improved iNNE algorithm can satisfy the need for local anomaly detection while also identifying anomaly data globally, achieving anomaly detection at multiple scales and reducing the risk of false positives and false negatives. On the other hand, dissolved gas data in transformer oil is typically continuously monitored and involves a large amount of data. The improved iNNE algorithm introduces a complexity optimization strategy in multi-scale detection, enabling efficient processing of large-scale data and improving the processing speed and detection efficiency of anomaly detection.

[0052] The data correction module is used to correct data using interpolation. This invention utilizes interpolation to infer the possible true values ​​of abnormal data based on the characteristics of normal data points surrounding abnormal data points, accurately restoring the original state of the abnormal data and thus correcting the impact of the existence of abnormal data on the model's prediction results.

[0053] The prediction model module is used to establish a prediction model for the dissolved gas content in transformer oil, and fine-tunes it using the Optuna hyperparameter optimization framework. Establish a BERT-based prediction model for dissolved gas content in transformer oil; BERT is a pre-trained language model developed by Google. As a deep learning model based on a self-attention mechanism, it was initially designed for processing text sequences. However, considering that the core mechanism of the BERT model is self-attention, it essentially adopts the Transformer architecture and excels at processing sequential data. Therefore, this invention applies it to processing time series data, especially when the time series has complex patterns and interdependencies, where the BERT model demonstrates excellent performance in predicting time series data. In the scenario of predicting dissolved gas content in transformer oil, there are complex dependencies between the gases. Therefore, the BERT model can be used to capture these relationships, thus achieving better gas content prediction. like Figure 2 As shown, since BERT typically processes discrete text tokens, the architecture of the prediction model needs to be adjusted to adapt to continuous time series data. The input layer of the prediction model includes: Word Embeddings, Sentence Embeddings, and Position Embeddings. Word Embeddings are used to split the processed data into word units, using words as the basic unit, and vectorizing the data at each time point. Word Embeddings are used directly as the input for dissolved gas content in oil. Sentence Embeddings are used to consider the relationship between two sentences, extracting information from the entire sentence and representing it as the relationship between dissolved gases in different types of oil. Position Embeddings are used to consider the meaning expressed by each word in different positions, and Position Embeddings are equivalent to timestamp embedding to provide time information on dissolved gases in oil.

[0054] The time series data of dissolved gases in transformer oil were converted into a format suitable for BERT processing and preprocessed. Specifically, the time series data is divided into multiple time windows, with each time point within a window corresponding to a token in BERT's input sequence. For each time window, a sequence containing gas concentration data for that time period is created, and this sequence is passed as input to BERT. A multidimensional token is formed by concatenating multiple gas concentration values ​​(such as H2, CH4, C2H6, etc.) for each day into a vector. To ensure scale consistency across different time points, the gas concentration data is normalized or standardized, which avoids model training instability caused by excessively large differences in the absolute values ​​of gas concentrations.

[0055] This invention, through a finely tuned prediction model and converted time-series data, enables BERT to better understand the context when predicting dissolved gases in transformer oil by utilizing contextual information, including the relationships between preceding and following sentences and within sentences. BERT also utilizes its core self-attention mechanism to capture long-distance dependencies and multi-scale features in the gas content time series, thereby better understanding the complex relationships between gases and improving prediction accuracy.

[0056] Optuna uses EI as the optimization criterion for the objective function, and selects the optimal parameter set based on the EI values ​​in different parameter sets. Specifically, Optuna is a hyperparameter optimization framework based on Bayesian optimization. In this invention, Optuna uses Expected Improvement (EI) as the optimization criterion for the objective function. This criterion optimizes by measuring the expected improvement of the proposed parameter points relative to the currently known optimal solution. Compared to traditional optimization criteria such as Upper Confidence Bound (UCB) in Bayesian optimization, EI has the advantage of better balancing known and unknown solutions, thereby improving global search performance. Therefore, using EI demonstrates superior practical performance in the complex problem of predicting dissolved gas content in transformer oil. The expression for the EI value is as follows:

[0057] in, These are the parameter points to be optimized. At the parameter point The expected value of the model. It is the expected value of the currently known optimal solution. At the parameter point The standard deviation of the model at that location, It is the cumulative distribution function of the standard normal distribution. It is the probability density function of the standard normal distribution. It is the quantile of the standard normal distribution, usually 0.

[0058] The Optuna hyperparameter optimization framework uses the efficient optimization algorithm EI to search for the optimal set of hyperparameters based on a specified objective function, which helps the BERT model to perform parameter tuning more effectively. This reduces the time and effort required for manual experimentation and hyperparameter tuning, thereby accelerating the training and optimization process of the BERT model. At the same time, compared with traditional manual parameter tuning methods, this invention also has advantages in parameter tuning performance.

[0059] The model validation module is used to use the prediction model with the optimal parameter set to predict the test set, output the final result of the prediction model, and verify the model performance.

[0060] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the dissolved gas content in transformer oil, characterized in that, Includes the following steps: S1. Obtain power transformer monitoring data as the original dataset, and divide the original dataset into training set, validation set and test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2. S2. Use the improved iNNE algorithm to identify abnormal data in the original dataset; S3. Use interpolation to correct the data; S4. Establish a prediction model for dissolved gas content in transformer oil and fine-tune it using the Optuna hyperparameter optimization framework. S41. Establish a BERT-based prediction model for dissolved gas content in transformer oil; the input layer of the prediction model includes: Word Embeddings is used to split the data processed in step S3 into word units. Using words as the basic unit, the data at each time point is vectorized and Word Embeddings is used directly as the input of dissolved gas content in oil. Sentence Embeddings are used to consider the relationship between two sentences, extract information from the whole sentence, and represent the relationship between dissolved gases in different types of oil. Position Embeddings are used to consider the meaning of each word in different positions. Position Embeddings are equivalent to timestamp embeddings to provide time information of dissolved gases in oil. S42. Convert the time series data of dissolved gases in transformer oil into a format suitable for BERT processing. Specifically, divide the time series data into multiple time windows, and the time point data in each window corresponds to a token in the input sequence of BERT. For each time window, create a sequence containing the gas concentration data within that time period. This sequence will be passed as input to BERT. By concatenating multiple gas concentration values ​​for each day into a vector, a multidimensional token is formed, and data preprocessing is performed. S43. Optuna uses EI as the optimization criterion for the objective function and selects the optimal parameter set based on the EI values ​​in different parameter sets. S5. Use the prediction model with the optimal parameter set to predict the test set, output the final result of the prediction model, and verify the model performance.

2. The method for predicting dissolved gas content in transformer oil according to claim 1, characterized in that, Step S2, which involves using the improved iNNE algorithm to identify outliers in the original dataset, specifically involves: constructing a hypersphere centered at each data point in the original dataset with a radius equal to the Euclidean distance from that point to its nearest neighbor, thus isolating that point from other points; calculating the corrected isolation score for each point and converting it into an anomaly probability; setting an anomaly probability threshold; and identifying various types of outliers. The hypersphere expression is as follows: in, It is a hypersphere. For the center of the ball, Let be the radius of the sphere. For the center of the ball The nearest neighbor, For the center of the ball The Euclidean distance between nearest neighbors, where x is a point in the original dataset D.

3. The method for predicting dissolved gas content in transformer oil according to claim 1, characterized in that, The expression for the EI value in step S43 is as follows: in, These are the parameter points to be optimized. At the parameter point The expected value of the model. It is the expected value of the currently known optimal solution. At the parameter point The standard deviation of the model at that location, It is the cumulative distribution function of the standard normal distribution. It is the probability density function of the standard normal distribution. It is the quantile of the standard normal distribution.

4. A system for predicting the dissolved gas content in transformer oil, characterized in that, It includes a data partitioning module, a data identification module, a data correction module, a prediction model module, and a model validation module; The data partitioning module is used to acquire power transformer monitoring data as the raw dataset, and divide the raw dataset into training set, validation set and test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2. The data identification module is used to identify abnormal data in the original dataset using the improved iNNE algorithm; The data correction module is used to correct data using interpolation. The prediction model module is used to establish a prediction model for the dissolved gas content in transformer oil, and is fine-tuned using the Optuna hyperparameter optimization framework, specifically as follows: A BERT-based prediction model for dissolved gas content in transformer oil is established. The input layer of the prediction model includes: Word Embeddings is used to split the data processed in step S3 into word units. Using words as the basic unit, the data at each time point is vectorized and Word Embeddings is used directly as the input of dissolved gas content in oil. Sentence Embeddings are used to consider the relationship between two sentences, extract information from the whole sentence, and represent the relationship between dissolved gases in different types of oil. Position Embeddings are used to consider the meaning of each word in different positions. Position Embeddings are equivalent to timestamp embeddings to provide time information of dissolved gases in oil. The time series data of dissolved gases in transformer oil is converted into a format suitable for BERT processing. Specifically, the time series data is divided into multiple time windows, and the time point data in each window corresponds to a token in the BERT input sequence. For each time window, a sequence containing the gas concentration data within that time period is created, and this sequence is passed as input to BERT. By concatenating multiple gas concentration values ​​for each day into a vector, a multidimensional token is formed, and data preprocessing is performed. Optuna uses EI as the optimization criterion for the objective function and selects the optimal parameter set based on the EI values ​​in different parameter sets. The model validation module is used to use the prediction model with the optimal parameter set to make predictions on the test set, output the final result of the prediction model, and verify the model performance.

5. The system for predicting dissolved gas content in transformer oil according to claim 4, characterized in that, The data identification module utilizes the improved iNNE algorithm to identify anomalous data in the original dataset. Specifically, it constructs a hypersphere centered at each data point in the original dataset, with the Euclidean distance from that point to its nearest neighbor as its radius, thus isolating that point from other points. The corrected isolation score for each point is calculated and converted into anomaly probability. An anomaly probability threshold is set to identify various types of anomalous data. The hypersphere expression is as follows: in, It is a hypersphere. For the center of the ball, Let be the radius of the sphere. For the center of the ball The nearest neighbor, For the center of the ball The Euclidean distance between nearest neighbors, where x is a point in the original dataset D.

6. The system for predicting dissolved gas content in transformer oil according to claim 4, characterized in that, The expression for the EI value in the prediction model module is as follows: in, These are the parameter points to be optimized. At the parameter point The expected value of the model. It is the expected value of the currently known optimal solution. At the parameter point The standard deviation of the model at that location, It is the cumulative distribution function of the standard normal distribution. It is the probability density function of the standard normal distribution. It is the quantile of the standard normal distribution.

Images