Method and system for predicting content of dissolved gas in transformer oil

Abstract

The invention discloses a method and system for predicting the content of dissolved gas in transformer oil, belongs to the technical field of transformer fault prediction, and solves the problems of limited identification of abnormal data of the content of the dissolved gas in the transformer oil and insufficient model performance optimization in the prior art. According to the method, the characteristics of local measurement and global measurement are comprehensively considered through the improved iNNE, so that accurate recognition of the gas abnormal data is realized, and the data quality is effectively improved; key hyper-parameters of the Bert model are optimized by using an Optuna hyper-parameter optimization framework, so that the limitation of manual parameter adjustment is effectively avoided; a long-distance dependency relationship and multi-scale characteristics in a gas content time sequence are captured by using a Bert model, and accurate identification of abnormal information of the content of the dissolved gas in the transformer oil and rapid prediction of the gas content at the future moment can be realized in the face of complex information of the dissolved gas in the transformer oil.

Description

Technical Field

[0001] The invention belongs to the technical field of transformer fault prediction, and relates to a method and system for predicting dissolved gas content in transformer oil. Background Art

[0002] Transformers are important equipment in the power transmission and transformation process of power grids, and their operating status directly affects the safety and stability of power grid operation. In my country's current standard DL / T722-2000, analyzing the composition and content changes of dissolved gases in transformer oil is one of the most effective means to monitor the safe operation of oil-immersed transformers. Predicting the content of dissolved gas in transformer oil can detect possible problems in transformers earlier, providing transformer operation and maintenance personnel with the ability to actively eliminate safety hazards and avoid serious safety accidents, and providing guarantees for the stable and safe operation of the power system.

[0003] At present, recurrent neural networks (RNNs) are widely used in time series prediction, and one of the most common models is the long short-term memory (LSTM) network. In the task of predicting the dissolved gas content in transformer oil, the LSTM model can make predictions in the following ways: LSTM effectively captures and utilizes long-term dependencies in time series through its unique structure, including forget gates, input gates, and output gates. When predicting the dissolved gas content in transformer oil, the input of the LSTM model is usually a sequence of dissolved gas content data at historical moments. After preprocessing and normalization, these data are used as the input sequence of the LSTM model. During the model training phase, LSTM gradually adjusts its internal state and weight parameters by learning the patterns and trends in the input sequence to optimize the ability to predict future changes in dissolved gas content. The model training process usually includes multiple epochs, in each of which LSTM optimizes the loss function through the backpropagation algorithm to reduce the error between the predicted value and the actual observed value. Once the training is completed, the LSTM model can be applied to future data prediction. When new transformer oil sample data is received, LSTM generates a prediction of the future dissolved gas content based on historical data and the current input state. This predictive capability enables LSTM to perform well in processing long-term dependencies and nonlinear dynamic patterns in time series data, and is suitable for application scenarios that require accurate prediction of future trends. For example, the invention patent with the authorization publication number CN112734028B discloses a modeling method for predicting the concentration of dissolved gas in transformer oil. By selecting important factors that may have a positive impact on the prediction of dissolved gas in transformer oil and establishing a variable environment, the prediction of each gas is set to be positively affected by all other variables. The long short-term memory network (LSTM) prediction model is used to predict and compare errors separately, verify assumptions, and extract important factors that have a positive impact on the gas variables to be predicted.

[0004] The above-mentioned and existing related technologies often have the following defects: Although LSTM is designed to capture long-term dependencies in time series, it may still have the problem of gradient vanishing or exploding when processing very long sequences. In this case, the model may not be able to effectively remember the information of dissolved gas content in transformer oil at long time intervals, affecting the accuracy and stability of long-term predictions. LSTM only depends on the input of the previous time step and its own state at each time step. This local information processing method limits its ability to understand the global context, especially when processing complex dissolved gas information in oil. Therefore, it is urgent to propose a prediction method that solves the problems of limited recognition of abnormal data on dissolved gas content in transformer oil and insufficient optimization of model performance. Summary of the invention

[0005] The technical solution of the present invention is used to solve the problems in the prior art of limited recognition of abnormal data on dissolved gas content in transformer oil and insufficient optimization of model performance.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] A method for predicting dissolved gas content in transformer oil comprises the following steps:

[0008] S1. Obtain power transformer monitoring data as an original data set, and divide the original data set into a training set, a validation set, and a test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2;

[0009] S2, using the improved iNNE algorithm to identify abnormal data in the original data set;

[0010] S3, using interpolation method to correct data;

[0011] S4. Establish a prediction model for dissolved gas content in transformer oil and fine-tune it using the Optuna hyperparameter optimization framework.

[0012] S41. Establish a prediction model for dissolved gas content in transformer oil based on Bert;

[0013] S42, converting the time series data of dissolved gas in transformer oil into a format suitable for Bert processing, and performing data preprocessing;

[0014] S43, Optuna uses EI as the optimization criterion of the objective function and selects the optimal parameter group according to the EI values ​​in different parameter groups;

[0015] S5. Use the prediction model with the optimal parameter group to predict the test set, output the final result of the prediction model, and verify the model performance.

[0016] Furthermore, the step S2 of using the improved iNNE algorithm to identify abnormal data in the original data set is specifically as follows: using the improved iNNE algorithm to construct a hypersphere with the point as the center and the Euclidean distance from the point to its nearest neighbor as the radius for each data point in the original data set, isolating the point from other points; calculating the corrected isolation score of each point and converting it into an abnormal probability, setting an abnormal probability threshold, and identifying various types of abnormal data; the hypersphere expression is as follows:

[0017] B(c)={x:‖xc‖<τ(c)}

[0018] τ(c)=||c-η c ||

[0019] Where B(c) is a hypersphere, c is the center of the sphere, τ(c) is the radius of the sphere, η c is the nearest neighbor point of the sphere center c, ‖c-η c ‖ is the Euclidean distance between the nearest neighbor points of the sphere center c, and x is a point in the original data set D.

[0020] Furthermore, the input layer of the prediction model in step S41 includes: Word Embeddings, Sentence Embeddings and Position Embeddings; the Word Embeddings is used to split the data processed in step S3 into word units, and vectorize the data at each time point with the word as the basic unit, and use Word Embeddings directly as the input of the dissolved gas content in the oil; the Sentence Embeddings is used to consider the relationship between two sentences, extract information from the entire sentence, and express it as the relationship between dissolved gases in different types of oil; the Position Embeddings is used to consider the meaning expressed by each word at different positions, and Position Embeddings is equivalent to timestamp embedding to provide time information of dissolved gas in oil.

[0021] Furthermore, the step S42 of converting the time series data of dissolved gas in transformer oil into a format suitable for Bert processing is specifically as follows: dividing the time series data into multiple time windows, and the time point data in each window corresponds to a token in the input sequence of Bert; for each time window, creating a sequence containing the gas concentration data in the time period, and this sequence will be passed to Bert as input; by connecting multiple gas concentration values ​​of each day (such as H2, CH4, C2H6, etc.) into a vector, a multidimensional token is formed.

[0022] Further, the expression of the EI value in step S43 is as follows:

[0023]

[0024] Among them, y is the parameter point to be optimized, μ(y) is the expected value of the model at the parameter point y, and μ best is the expected value of the currently known optimal solution, σ(y) is the standard deviation of the model at the parameter point y, Φ is the cumulative distribution function of the standard normal distribution, φ is the probability density function of the standard normal distribution, and Z is the quantile of the standard normal distribution.

[0025] The present invention also provides a system for predicting dissolved gas content in transformer oil, comprising a data partitioning module, a data identification module, a data correction module, a prediction model module and a model verification module;

[0026] The data partitioning module is used to obtain the power transformer monitoring data as the original data set, and divide the original data set into a training set, a validation set and a test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2;

[0027] The data identification module is used to identify abnormal data on the original data set using the improved iNNE algorithm;

[0028] The data correction module is used to perform data correction using an interpolation method;

[0029] The prediction model module is used to establish a prediction model for dissolved gas content in transformer oil, and fine-tune it using the Optuna hyperparameter optimization framework; specifically, a prediction model for dissolved gas content in transformer oil based on Bert is established, the time series data of dissolved gas in transformer oil is converted into a format suitable for Bert processing, and data preprocessing is performed. Optuna uses EI as the optimization criterion of the objective function and selects the optimal parameter group according to the EI values ​​in different parameter groups;

[0030] The model verification module is used to use the prediction model of the optimal parameter group to predict the test set, output the final result of the prediction model, and verify the model performance.

[0031] Furthermore, the data identification module uses the improved iNNE algorithm to identify abnormal data in the original data set. Specifically, the improved iNNE algorithm is used to construct a hypersphere with the point as the center and the Euclidean distance from the point to its nearest neighbor as the radius for each data point in the original data set, isolating the point from other points; calculating the corrected isolation score of each point and converting it into an abnormal probability, setting an abnormal probability threshold, and identifying various types of abnormal data; the hypersphere expression is as follows:

[0032] B(c)={x:‖xc‖<τ(c)}

[0033] τ(c)=||c-η c ||

[0034] Where B(c) is a hypersphere, c is the center of the sphere, τ(c) is the radius of the sphere, η c is the nearest neighbor point of the sphere center c, ‖c-η c ‖ is the Euclidean distance between the nearest neighbor points of the sphere center c, and x is a point in the original data set D.

[0035] Furthermore, the input layer of the prediction model in the prediction model module includes: Word Embeddings, Sentence Embeddings and Position Embeddings; the Word Embeddings is used to split the data processed in step S3 into word units, use words as basic units, vectorize the data at each time point, and use Word Embeddings directly as the input of the dissolved gas content in oil; the Sentence Embeddings is used to consider the relationship between two sentences, extract information from the entire sentence, and express it as the relationship between dissolved gases in different types of oil; the Position Embeddings is used to consider the meaning expressed by each word at different positions, and Position Embeddings is equivalent to timestamp embedding to provide time information of dissolved gas in oil.

[0036] Furthermore, the prediction model module converts the time series data of dissolved gas in transformer oil into a format suitable for Bert processing as follows: the time series data is divided into multiple time windows, and the time point data in each window corresponds to a token in the input sequence of Bert; for each time window, a sequence containing the gas concentration data in the time period is created, and this sequence is passed to Bert as input; and a multidimensional token is formed by connecting multiple gas concentration values ​​of each day (such as H2, CH4, C2H6, etc.) into a vector.

[0037] Furthermore, the expression of the EI value in the prediction model module is as follows:

[0038]

[0039] Among them, y is the parameter point to be optimized, μ(y) is the expected value of the model at the parameter point y, and μ best is the expected value of the currently known optimal solution, σ(y) is the standard deviation of the model at the parameter point y, Φ is the cumulative distribution function of the standard normal distribution, φ is the probability density function of the standard normal distribution, and Z is the quantile of the standard normal distribution.

[0040] The advantages of the present invention are as follows: the present invention realizes accurate identification of gas abnormal data by comprehensively considering the characteristics of local and global metrics through improved iNNE, and effectively improves data quality; the key hyperparameters of the Bert model are optimized by using the Optuna hyperparameter optimization framework, which effectively avoids the limitations of manual parameter adjustment; by fine-tuning the Bert model structure and converting the time series data format, the Bert model is used to capture long-distance dependencies and multi-scale features in the gas content time series, and the complex relationship between gases is better understood. When facing complex information on dissolved gas in transformer oil, accurate identification of abnormal information on dissolved gas content in transformer oil and rapid prediction of gas content at future moments can be achieved, solving the defects of limited abnormal data identification and insufficient model performance optimization in the prior art. This comprehensive application method can effectively improve the efficiency of industrial equipment monitoring and maintenance, help timely discover abnormal conditions in transformer oil, and take necessary preventive and repair measures, thereby ensuring the safe operation and reliability of the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 is a flow chart of a method for predicting dissolved gas content in transformer oil according to an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of the input layer structure of the Bert model according to an embodiment of the present invention;

[0043] Figure 3It is a curve diagram of the error results of each model under different algorithms of the embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0046] Embodiment 1

[0047] like Figure 1 Specifically, a method for predicting dissolved gas content in transformer oil is disclosed, comprising the following steps:

[0048] S1. Obtain power transformer monitoring data as an original data set, and divide the original data set into a training set, a validation set, and a test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2;

[0049] S2, using the improved iNNE algorithm to identify abnormal data in the original data set;

[0050] Specifically, the improved iNNE algorithm is used to construct a hypersphere with each data point in the original data set as the center and the Euclidean distance from the point to its nearest neighbor as the radius, so as to isolate the point from other points; the corrected isolation score of each point is calculated and converted into anomaly probability, and anomaly probability thresholds are set to identify various types of abnormal data; the abnormal data include isolated anomalies, continuous anomalies, zero points, etc.

[0051] Specifically, a subset S of size m is randomly selected from a given original data set D, and a set of hyperspheres is constructed with each data point in the subset S as the sphere center and the Euclidean distance between the point and its nearest neighbor as the radius. The expression of the hypersphere B(c) is as follows:

[0052] B(c)={x:‖xc‖<τ(c)}

[0053] τ(c)=||c-η c ||

[0054] Where c is the center of the sphere, τ(c) is the radius of the sphere, and ηc is the nearest neighbor point of the sphere center c, ‖c-η c ‖ is the Euclidean distance between the nearest neighbor points of the sphere center c, and x is a point in the original data set D.

[0055] The present invention isolates data from other data through a hypersphere, wherein the radius τ(c) represents the degree of isolation of the center c. The larger the radius τ(c), the more isolated the center c is, and vice versa. However, the present invention does not directly identify based on the radius τ(c), but calculates the isolation score of each data to determine the abnormal probability of the data and complete the identification of abnormal data.

[0056] Compared with the traditional iNNE algorithm that only uses the local relative radius to calculate the isolation score, that is, only the ratio of the radius of the nearest neighbor hypersphere of the sample point to the radius of the hypersphere where it is located is used to identify anomalies, the traditional method can achieve a better recognition effect for local anomalies, but it is easy to misjudge the points where the radius of the nearest neighbor hypersphere is equal to the radius of the hypersphere where it is located, thereby causing recognition errors. The present invention takes into account the above-mentioned influencing factors, corrects the local relative radius of each data, and not only considers the size of the local relative radius when calculating the isolation score, but also considers the relative size of the hypersphere radius where the sample point is located and the maximum hypersphere radius in the hypersphere group, thereby overcoming the defect of inaccurate recognition of the traditional iNNE algorithm and effectively improving the recognition accuracy of the iNNE algorithm.

[0057] The present invention uses an improved isolated anomaly detection method based on the nearest neighbor distance to find the corresponding abnormal data for the data set characteristics of dissolved gas in transformer oil. On the one hand, the dissolved gas data in transformer oil may have different characteristic scales, and the anomaly detection of a single scale may not be able to capture all types of anomalies, and it is easy to fall into the defect of local optimality. The improved iNNE algorithm can meet the needs of local anomaly detection and can identify abnormal data globally, realize abnormal data detection at multiple scales, and reduce the risk of missed reports and false alarms. On the other hand, the dissolved gas data in transformer oil is usually continuous monitoring data with a large amount of data. The improved iNNE algorithm introduces a complexity optimization strategy in multi-scale anomaly detection, which can efficiently process large-scale data and improve the processing speed and detection efficiency of anomaly detection.

[0058] S3, using interpolation method to correct data;

[0059] The present invention uses the interpolation method to infer the possible real value of the abnormal data based on the characteristics of the normal data points around the abnormal data points obtained in step S2, accurately restore the original situation of the abnormal data, and then correct the influence of the existence of the abnormal data on the model prediction results.

[0060] S4. Establish a prediction model for dissolved gas content in transformer oil and fine-tune it using the Optuna hyperparameter optimization framework.

[0061] S41. Establish a prediction model for dissolved gas content in transformer oil based on Bert;

[0062] Bert is a pre-trained language model developed by Google. As a deep learning model based on the self-attention mechanism, it was originally designed to process text sequences. However, considering that the core mechanism of the Bert model is the self-attention mechanism, it essentially adopts the Transformer architecture and is good at processing sequence data. Therefore, the present invention applies it to process time series data, especially when the time series has complex patterns and interdependencies. The Bert model has excellent performance in processing time series data for prediction. In the scenario of predicting the dissolved gas content in transformer oil, there are complex dependencies between the gases, so the Bert model can be used to capture these relationships and better realize gas content prediction.

[0063] like Figure 2 As shown, since BERT usually processes discrete text tokens, in order to make the prediction model adapt to continuous time series data, the architecture of the prediction model needs to be adjusted. The input layer of the prediction model includes: Word Embeddings, Sentence Embeddings and Position Embeddings; the Word Embeddings is used to split the data processed in step S3 into word units, and vectorize the data at each time point with the word as the basic unit, and use Word Embeddings directly as the input of the dissolved gas content in oil; the Sentence Embeddings is used to consider the relationship between two sentences, extract information from the entire sentence, and express it as the relationship between dissolved gases in different types of oil; the Position Embeddings is used to consider the meaning expressed by each word at different positions, and Position Embeddings is equivalent to timestamp embedding to provide time information of dissolved gas in oil.

[0064] S42, converting the time series data of dissolved gas in transformer oil into a format suitable for Bert processing, and performing data preprocessing;

[0065] Specifically, the time series data is divided into multiple time windows, and the time point data in each window corresponds to a token in Bert's input sequence; for each time window, a sequence containing the gas concentration data in the time period is created, and this sequence will be passed to Bert as input; by connecting multiple gas concentration values ​​of each day (such as H2, CH4, C2H6, etc.) into a vector, a multidimensional token is formed. To ensure the scale consistency of the data between different time points, the gas concentration data is normalized or standardized to avoid model training instability caused by excessive differences in the absolute values ​​of gas concentrations.

[0066] Through fine-tuning the prediction model and converting the time series data in the format, the present invention enables BERT to use contextual information, including the association between the context and the sentence, to better understand the context when predicting dissolved gas in transformer oil; BERT uses its core self-attention mechanism to capture the long-range dependencies and multi-scale features in the gas content time series, better understand the complex relationship between gases, and thus improve the accuracy of prediction.

[0067] S43, Optuna uses EI as the optimization criterion of the objective function and selects the optimal parameter group according to the EI values ​​in different parameter groups;

[0068] Specifically, Optuna is a hyperparameter optimization framework based on Bayesian optimization. In the present invention, Optuna uses Expected Improvement (EI) as the optimization criterion of the objective function, which optimizes by measuring the expected improvement of the newly proposed parameter point relative to the currently known optimal solution. Compared with traditional optimization criteria such as Upper Confidence Bound (UCB) in Bayesian optimization, the advantage of EI is that it can better balance known solutions and unknown solutions, thereby improving global search performance. Therefore, the use of EI plays a more superior practical performance in the complex problem of predicting dissolved gas content in transformer oil. The expression of the EI value is as follows:

[0069]

[0070] Among them, y is the parameter point to be optimized, μ(y) is the expected value of the model at the parameter point y, and μ best is the expected value of the currently known optimal solution, σ(y) is the standard deviation of the model at the parameter point y, Φ is the cumulative distribution function of the standard normal distribution, φ is the probability density function of the standard normal distribution, and Z is the quantile of the standard normal distribution, which is usually 0.

[0071] The Optuna hyperparameter optimization framework uses the efficient optimization algorithm EI to search for the best hyperparameter set based on the specified objective function to help the Bert model perform parameter tuning more effectively, reducing the time and effort required for manual experiments and adjustment of hyperparameters, thereby accelerating the training and optimization process of the Bert model; at the same time, compared with the traditional manual parameter tuning method, the present invention also has advantages in parameter tuning performance.

[0072] S5. Use the prediction model of the optimal parameter group to predict the test set, output the final result of the prediction model, and verify the model performance;

[0073] In this embodiment, monitoring data of a 220 kV power transformer in a foreign country is collected as the original data set. After abnormality detection, the abnormal data is replaced by interpolation and directly input into the prediction model of dissolved gas content in transformer oil based on Bert according to the data conversion method of step S4 above.

[0074] In order to evaluate the prediction performance of the proposed model, the present invention uses mean square error (MSE) as the evaluation index of the model. Mean square error is a commonly used index to measure the difference between the model prediction result and the true value. It is used to calculate the average value of the square error between the model prediction value and the true value, which can effectively reflect the model prediction error and fitting effect. The calculation formula is as follows:

[0075]

[0076] Where n is the number of samples, y i is the actual value of the dissolved gas content in the oil in the i-th training, is the predicted value of the dissolved gas content in the oil for the i-th training. The smaller the MSE, the smaller the gap between the predicted result of the model and the true value, and the more closely the model fits.

[0077] like Figure 3 As shown, the prediction model is verified and analyzed. This embodiment uses LSTM, GRU, Bert, improved iNNE-Bert (MiNNE-Bert), Optuna-Bert and improved iNNE-Optuna-Bert (MiNNE-Optuna-Bert) for comparative experimental analysis. From the training results in the figure below, it can be seen that during the model training process, the prediction performance of Bert has certain advantages over LSTM and GRU. However, the prediction performance of the optimized Bert far exceeds that of the three unoptimized models. The improved iNNE-Bert and Optuna-Bert have significantly improved prediction performance compared to Bert. At the same time, the improved iNNE-Optuna-Bert method proposed in the present invention has the lowest error and can also reach the optimality at the fastest speed.

[0078] The present invention realizes accurate identification of gas abnormal data by comprehensively considering the characteristics of local and global metrics through improved iNNE, effectively improving data quality; uses Optuna hyperparameter optimization framework to optimize the key hyperparameters of the Bert model, effectively avoiding the limitations of manual parameter adjustment; through fine-tuning the Bert model structure and converting the time series data format, the Bert model is used to capture the long-distance dependencies and multi-scale characteristics in the gas content time series, better understand the complex relationship between gases, and when facing complex information on dissolved gas in transformer oil, it can realize accurate identification of abnormal information on dissolved gas content in transformer oil and rapid prediction of gas content at future moments, solving the defects of limited abnormal data identification and insufficient model performance optimization in the prior art. This comprehensive application method can effectively improve the efficiency of industrial equipment monitoring and maintenance, help timely discover abnormal conditions in transformer oil, and take necessary preventive and repair measures, thereby ensuring the safe operation and reliability of equipment.

[0079] The present invention also provides a system for predicting dissolved gas content in transformer oil, comprising a data partitioning module, a data identification module, a data correction module, a prediction model module and a model verification module;

[0080] The data partitioning module is used to obtain the power transformer monitoring data as the original data set, and divide the original data set into a training set, a validation set and a test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2;

[0081] The data identification module is used to identify abnormal data on the original data set using the improved iNNE algorithm;

[0082] Specifically, the improved iNNE algorithm is used to construct a hypersphere with each data point in the original data set as the center and the Euclidean distance from the point to its nearest neighbor as the radius, so as to isolate the point from other points; the corrected isolation score of each point is calculated and converted into anomaly probability, and anomaly probability thresholds are set to identify various types of abnormal data; the abnormal data include isolated anomalies, continuous anomalies, zero points, etc.

[0083] Specifically, a subset S of size m is randomly selected from a given original data set D, and a set of hyperspheres is constructed with each data point in the subset S as the sphere center and the Euclidean distance between the point and its nearest neighbor as the radius. The expression of the hypersphere B(c) is as follows:

[0084] B(c)={x:‖xc‖<τ(c)}

[0085] τ(c)=||c-η c ||

[0086] Where c is the center of the sphere, τ(c) is the radius of the sphere, and η c is the nearest neighbor point of the sphere center c, ‖c-η c ‖ is the Euclidean distance between the nearest neighbor points of the sphere center c, and x is a point in the original data set D.

[0087] The present invention isolates data from other data through a hypersphere, wherein the radius τ(c) represents the degree of isolation of the center c. The larger the radius τ(c), the more isolated the center c is, and vice versa. However, the present invention does not directly identify based on the radius τ(c), but calculates the isolation score of each data to determine the abnormal probability of the data and complete the identification of abnormal data.

[0088] Compared with the traditional iNNE algorithm that only uses the local relative radius to calculate the isolation score, that is, only the ratio of the radius of the nearest neighbor hypersphere of the sample point to the radius of the hypersphere where it is located is used to identify anomalies, the traditional method can achieve a better recognition effect for local anomalies, but it is easy to misjudge the points where the radius of the nearest neighbor hypersphere is equal to the radius of the hypersphere where it is located, thereby causing recognition errors. The present invention takes into account the above-mentioned influencing factors, corrects the local relative radius of each data, and not only considers the size of the local relative radius when calculating the isolation score, but also considers the relative size of the hypersphere radius where the sample point is located and the maximum hypersphere radius in the hypersphere group, thereby overcoming the defect of inaccurate recognition of the traditional iNNE algorithm and effectively improving the recognition accuracy of the iNNE algorithm.

[0089] The present invention uses an improved isolated anomaly detection method based on the nearest neighbor distance to find the corresponding abnormal data for the data set characteristics of dissolved gas in transformer oil. On the one hand, the data of dissolved gas in transformer oil may have different characteristic scales, and the anomaly detection of a single scale may not be able to capture all types of anomalies, and it is easy to fall into the defect of local optimality. The improved iNNE algorithm can not only meet the needs of local anomaly detection, but also identify abnormal data globally, realize abnormal data detection at multiple scales, and reduce the risk of missed reports and false alarms. On the other hand, the data of dissolved gas in transformer oil is usually continuous monitoring data with a large amount of data. The improved iNNE algorithm introduces a complexity optimization strategy in multi-scale detection, which can efficiently process large-scale data and improve the processing speed and detection efficiency of anomaly detection.

[0090] The data correction module is used to perform data correction using an interpolation method;

[0091] The present invention uses the interpolation method to infer the possible real value of the abnormal data based on the characteristics of the normal data points around the abnormal data points, accurately restore the original situation of the abnormal data, and then correct the influence of the existence of the abnormal data on the model prediction results.

[0092] The prediction model module is used to establish a prediction model for dissolved gas content in transformer oil, and fine-tune it using the Optuna hyperparameter optimization framework;

[0093] Establish a prediction model for dissolved gas content in transformer oil based on Bert;

[0094] Bert is a pre-trained language model developed by Google. As a deep learning model based on the self-attention mechanism, it was originally designed to process text sequences. However, considering that the core mechanism of the Bert model is the self-attention mechanism, it essentially adopts the Transformer architecture and is good at processing sequence data. Therefore, the present invention applies it to process time series data, especially when the time series has complex patterns and interdependencies. The Bert model has excellent performance in processing time series data for prediction. In the scenario of predicting the dissolved gas content in transformer oil, there are complex dependencies between the gases, so the Bert model can be used to capture these relationships and better realize gas content prediction.

[0095] like Figure 2 As shown in the figure, since BERT usually processes discrete text tokens, in order to make the prediction model adapt to continuous time series data, the architecture of the prediction model needs to be adjusted. The input layer of the prediction model includes: Word Embeddings, Sentence Embeddings and Position Embeddings; the Word Embeddings is used to split the processed data into word units, and vectorize the data at each time point with the word as the basic unit, and use Word Embeddings directly as the input of the dissolved gas content in the oil; the Sentence Embeddings is used to consider the relationship between two sentences, extract information from the entire sentence, and express it as the relationship between dissolved gases in different types of oil; the Position Embeddings is used to consider the meaning expressed by each word at different positions, and Position Embeddings is equivalent to timestamp embedding to provide time information of dissolved gas in oil.

[0096] Convert the time series data of dissolved gas in transformer oil into a format suitable for Bert processing and perform data preprocessing;

[0097] Specifically, the time series data is divided into multiple time windows, and the time point data in each window corresponds to a token in Bert's input sequence; for each time window, a sequence containing the gas concentration data in the time period is created, and this sequence will be passed to Bert as input; by connecting multiple gas concentration values ​​of each day (such as H2, CH4, C2H6, etc.) into a vector, a multidimensional token is formed. To ensure the scale consistency of the data between different time points, the gas concentration data is normalized or standardized to avoid model training instability caused by excessive differences in the absolute values ​​of gas concentrations.

[0098] Through fine-tuning the prediction model and converting the time series data in the format, the present invention enables BERT to use contextual information, including the association between the context and the sentence, to better understand the context when predicting dissolved gas in transformer oil; BERT uses its core self-attention mechanism to capture the long-range dependencies and multi-scale features in the gas content time series, better understand the complex relationship between gases, and thus improve the accuracy of prediction.

[0099] Optuna uses EI as the optimization criterion of the objective function and selects the optimal parameter group based on the EI values ​​in different parameter groups;

[0100] Specifically, Optuna is a hyperparameter optimization framework based on Bayesian optimization. In the present invention, Optuna uses Expected Improvement (EI) as the optimization criterion of the objective function, which optimizes by measuring the expected improvement of the newly proposed parameter point relative to the currently known optimal solution. Compared with traditional optimization criteria such as Upper Confidence Bound (UCB) in Bayesian optimization, the advantage of EI is that it can better balance known solutions and unknown solutions, thereby improving global search performance. Therefore, the use of EI plays a more superior practical performance in the complex problem of predicting dissolved gas content in transformer oil. The expression of the EI value is as follows:

[0101]

[0102] Among them, y is the parameter point to be optimized, μ(y) is the expected value of the model at the parameter point y, and μ best is the expected value of the currently known optimal solution, σ(y) is the standard deviation of the model at the parameter point y, Φ is the cumulative distribution function of the standard normal distribution, φ is the probability density function of the standard normal distribution, and Z is the quantile of the standard normal distribution, which is usually 0.

[0103] The Optuna hyperparameter optimization framework uses the efficient optimization algorithm EI to search for the best hyperparameter set based on the specified objective function to help the Bert model perform parameter tuning more effectively, reducing the time and effort required for manual experiments and adjustment of hyperparameters, thereby accelerating the training and optimization process of the Bert model; at the same time, compared with the traditional manual parameter tuning method, the present invention also has advantages in parameter tuning performance.

[0104] The model verification module is used to use the prediction model of the optimal parameter group to predict the test set, output the final result of the prediction model, and verify the model performance.

[0105] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting dissolved gas content in transformer oil, characterized in that: The following steps are involved: S1. Obtain power transformer monitoring data as an original data set, and divide the original data set into a training set, a validation set, and a test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2; S2, using the improved iNNE algorithm to identify abnormal data in the original data set; S3, using interpolation method to correct data; S4. Establish a prediction model for dissolved gas content in transformer oil and fine-tune it using the Optuna hyperparameter optimization framework. S41. Establish a prediction model for dissolved gas content in transformer oil based on Bert; S42, converting the time series data of dissolved gas in transformer oil into a format suitable for Bert processing, and performing data preprocessing; S43, Optuna uses EI as the optimization criterion of the objective function and selects the optimal parameter group according to the EI values ​​in different parameter groups; S5. Use the prediction model with the optimal parameter group to predict the test set, output the final result of the prediction model, and verify the model performance.

2. The method for predicting dissolved gas content in transformer oil according to claim 1, characterized in that: The step S2 of using the improved iNNE algorithm to identify abnormal data in the original data set is specifically as follows: using the improved iNNE algorithm to construct a hypersphere with the point as the center and the Euclidean distance from the point to its nearest neighbor as the radius for each data point in the original data set, isolating the point from other points; calculating the corrected isolation score of each point and converting it into an abnormal probability, setting an abnormal probability threshold, and identifying various types of abnormal data; the hypersphere expression is as follows: B(c)={x:‖xc‖<τ(c)} τ(c)=||c-η c || Where B(c) is a hypersphere, c is the center of the sphere, τ(c) is the radius of the sphere, η c is the nearest neighbor point of the sphere center c, ‖c-η c ‖ is the Euclidean distance between the nearest neighbor points of the sphere center c, and x is a point in the original data set D.

3. The method for predicting dissolved gas content in transformer oil according to claim 2, characterized in that: The input layer of the prediction model in step S41 includes: Word Embeddings, Sentence Embeddings and Position Embeddings; the Word Embeddings is used to split the data processed in step S3 into word units, use words as basic units, vectorize the data at each time point, and use Word Embeddings directly as the input of the dissolved gas content in oil; the Sentence Embeddings is used to consider the relationship between two sentences, extract information from the entire sentence, and express it as the relationship between dissolved gases in different types of oil; the Position Embeddings is used to consider the meaning expressed by each word at different positions, and Position Embeddings is equivalent to timestamp embedding to provide time information of dissolved gas in oil.

4. The method for predicting dissolved gas content in transformer oil according to claim 3, characterized in that: The step S42 of converting the time series data of dissolved gas in transformer oil into a format suitable for Bert processing is specifically as follows: dividing the time series data into multiple time windows, and the time point data in each window corresponds to a token in the input sequence of Bert; for each time window, creating a sequence containing the gas concentration data in the time period, and this sequence will be passed to Bert as input; by connecting multiple gas concentration values ​​of each day (such as H2, CH4, C2H6, etc.) into a vector, a multidimensional token is formed.

5. The method for predicting dissolved gas content in transformer oil according to claim 4, characterized in that: The expression of the EI value in step S43 is as follows: Among them, y is the parameter point to be optimized, μ(y) is the expected value of the model at the parameter point y, and μ best is the expected value of the currently known optimal solution, σ(y) is the standard deviation of the model at the parameter point y, Φ is the cumulative distribution function of the standard normal distribution, φ is the probability density function of the standard normal distribution, and Z is the quantile of the standard normal distribution.

6. A system for predicting dissolved gas content in transformer oil, characterized in that: It includes a data partitioning module, a data identification module, a data correction module, a prediction model module and a model verification module; The data partitioning module is used to obtain the power transformer monitoring data as the original data set, and divide the original data set into a training set, a validation set and a test set in a ratio of 7:2:1; the monitoring data includes time series data and gas concentration data of methane CH4, ethylene C2H4, ethane C2H6, acetylene C2H2, hydrogen H2, carbon monoxide CO, and carbon dioxide CO2; The data identification module is used to identify abnormal data on the original data set using the improved iNNE algorithm; The data correction module is used to perform data correction using an interpolation method; The prediction model module is used to establish a prediction model for dissolved gas content in transformer oil, and fine-tune it using the Optuna hyperparameter optimization framework; specifically, a prediction model for dissolved gas content in transformer oil based on Bert is established, the time series data of dissolved gas in transformer oil is converted into a format suitable for Bert processing, and data preprocessing is performed. Optuna uses EI as the optimization criterion of the objective function and selects the optimal parameter group according to the EI values ​​in different parameter groups; The model verification module is used to use the prediction model of the optimal parameter group to predict the test set, output the final result of the prediction model, and verify the model performance.

7. The system for predicting dissolved gas content in transformer oil according to claim 6, characterized in that: The data identification module uses the improved iNNE algorithm to identify abnormal data in the original data set. Specifically, the improved iNNE algorithm is used to construct a hypersphere with the point as the center and the Euclidean distance from the point to its nearest neighbor as the radius for each data point in the original data set, and the point is isolated from other points; the corrected isolation score of each point is calculated and converted into an abnormal probability, and an abnormal probability threshold is set to identify various types of abnormal data; the hypersphere expression is as follows: B(c)={x:‖xc‖<τ(c)} τ(c)=||c-η c || Where B(c) is a hypersphere, c is the center of the sphere, τ(c) is the radius of the sphere, η c is the nearest neighbor point of the sphere center c, ‖c-η c ‖ is the Euclidean distance between the nearest neighbor points of the sphere center c, and x is a point in the original data set D.

8. The system for predicting dissolved gas content in transformer oil according to claim 7, characterized in that: The input layer of the prediction model in the prediction model module includes: Word Embeddings, Sentence Embeddings and Position Embeddings; the Word Embeddings is used to split the data processed in step S3 into word units, use words as basic units, vectorize the data at each time point, and use Word Embeddings directly as the input of the dissolved gas content in oil; the Sentence Embeddings is used to consider the relationship between two sentences, extract information from the entire sentence, and express it as the relationship between dissolved gases in different types of oil; the Position Embeddings is used to consider the meaning expressed by each word at different positions, and Position Embeddings is equivalent to timestamp embedding to provide time information of dissolved gas in oil.

9. The system for predicting dissolved gas content in transformer oil according to claim 8, characterized in that: The prediction model module converts the time series data of dissolved gas in transformer oil into a format suitable for Bert processing as follows: the time series data is divided into multiple time windows, and the time point data in each window corresponds to a token in the input sequence of Bert; for each time window, a sequence containing the gas concentration data in the time period is created, and this sequence is passed to Bert as input; and a multidimensional token is formed by connecting multiple gas concentration values ​​(such as H2, CH4, C2H6, etc.) of each day into a vector.

10. The system for predicting dissolved gas content in transformer oil according to claim 9, characterized in that: The expression of the EI value in the prediction model module is as follows: Among them, y is the parameter point to be optimized, μ(y) is the expected value of the model at the parameter point y, and μ best is the expected value of the currently known optimal solution, σ(y) is the standard deviation of the model at the parameter point y, Φ is the cumulative distribution function of the standard normal distribution, φ is the probability density function of the standard normal distribution, and Z is the quantile of the standard normal distribution.

Images