Monitoring system function integrity evaluation method based on Stacking model

Abstract

The invention discloses a monitoring system function integrity evaluation method based on a Stacking model. According to the method, firstly, construction of a function integrity evaluation index system is studied, based on a transformer substation monitoring system function mechanism model, a measurement and control device is taken as an example, core functions of the measurement and control device are sorted and analyzed, and a basic index system of function integrity of the measurement and control device is constructed. Secondly, how to construct a Stacking integrated model is researched, three machine learning algorithms such as random forest are applied to form a base learning layer, a classifier is applied to serve as a meta-learner layer to form a Stacking model, and the functional integrity of the monitoring system is evaluated. Furthermore, initial function integrity score and grade giving are researched. And obtaining an initial function integrity score and grade by applying a fuzzy synthesis method. Moreover, the training and analysis of the model are researched, the model is subjected to iterative training, the optimal number of iterations is analyzed, the training result of the model is compared and analyzed, and the evaluation of the functional integrity of the monitoring system is completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power system automation, and particularly relates to a monitoring system function integrity evaluation method based on a stacking model. BACKGROUND

[0002] With the rapid development of new power systems, the instability of the supply and demand sides makes the power grid operation mode adjustment and centralized monitoring operation more complex and frequent, and the reliable execution of the monitoring system function is increasingly important for the power grid. However, the failure of the monitoring system function often occurs at present, which also causes serious power grid accidents, such as the misoperation of equipment and the mispulling of switches and knife switches due to the abnormality of the memory in the measurement and control device, resulting in power grid failure. And the existing researches mostly focus on the monitoring of a single device or loop, and lack of in-depth analysis of the overall function of the secondary system, which fails to combine each device and loop with the overall performance and operating state of the system, limiting the comprehensiveness of the evaluation. The function of the substation monitoring system is completed by the cooperation of monitoring devices such as monitoring host computers, gateway computers and measurement and control devices, and the state evaluation of the function can lay a foundation for the staff to scientifically master the function execution state and guide targeted patrol maintenance. At present, when a fault or alarm occurs in a substation, the abnormal information generated in the monitoring system is particularly large, and it is very difficult to quickly and accurately locate the fault or alarm position and find the fault cause. Moreover, the monitoring information caused by the abnormality of the monitoring system function is mostly scattered, the information utilization efficiency is low, and the value of data resources cannot be fully utilized. The existence of these problems makes the monitoring and evaluation of the substation monitoring function face certain challenges, and further research is still needed.

[0003] Integrity theory is mainly applied to scenarios with high requirements for system safety and data accuracy. For example, in the field of satellite navigation, it is used to measure the ability of the positioning system to timely alarm users when it cannot provide navigation or positioning services. The function of the substation monitoring system plays an important role in the stability of the power grid and the guarantee of power supply, and the reliable operation of the monitoring function is an important guarantee for the safety and stability of the power system. The integrity evaluation of the substation monitoring system function is a comprehensive evaluation of the running state of the core function of each device in the monitoring system based on data analysis. The core idea is to judge whether the key function of the monitoring system is in normal operation state through real-time collection and analysis of system operation data, and to timely trigger an alarm when the function is abnormal. The function integrity evaluation provides an intelligent operation guarantee means for the substation monitoring system. Through accurate evaluation of the function state, timely identification of potential abnormalities and alarm, it can better help the operation and maintenance personnel, and thus improve the reliability and safety of the overall operation of the substation.

[0004] Therefore, the patent proposes a monitoring system function integrity evaluation method based on a Stacking model, realizes a comprehensive evaluation of the function operation state of the substation monitoring system, and provides an intelligent operation guarantee means. SUMMARY

[0005] The purpose of the present application is to provide a monitoring system function integrity evaluation method based on a Stacking model, which can realize function integrity evaluation and improve the reliability and safety of the overall operation of the substation after application.

[0006] The purpose of the present application is realized by the following technical solutions:

[0007] A monitoring system function integrity evaluation method based on a Stacking model, the method comprising:

[0008] Step 1, research on the construction of the function integrity evaluation index system;

[0009] Step 2, research on the construction of the Stacking integrated model;

[0010] Step 3, research on the initial integrity score and grade;

[0011] Step 4, research on the training and analysis of the model;

[0012] From the above technical solutions provided by the present application, it can be seen that the above method is to establish a function integrity evaluation index system, and based on a Stacking model, the function integrity of the substation monitoring system is evaluated and analyzed, and an intelligent operation guarantee means is provided for the substation monitoring system. Accurate evaluation of function state, timely identification of potential abnormalities and alarm, improve the reliability and safety of the function operation of the substation. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0014] Figure 1 The substation monitoring system function integrity evaluation process described in the embodiment of the present application;

[0015] Figure 2 The function integrity basic index framework of the measurement and control device described in the embodiment of the present application. DETAILED DESCRIPTION

[0016] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the protection scope of the present application.

[0017] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings. The method comprises:

[0018] Step 1, research on construction of a function integrity evaluation index system;

[0019] In the step 1, the specific process is as follows:

[0020] Research on technical specifications and other related standards of devices such as a monitoring host, a measurement and control device and a gateway machine of a substation monitoring system, based on a function mechanism model of the substation monitoring system, the core functions of the measurement and control device are sorted out and analyzed, the input and output requirements of the functions are clarified, and the dependency relationship between the functions is understood. According to the flow direction and processing process of the function data, the division principle of the whole process module is determined. For the control operation, the electrical quantity measurement, the state quantity acquisition and the cross-interval interlocking function, the processing chain from the input to the output is analyzed in detail, including the process and requirements of the data acquisition, transmission, encoding and decoding, processing and feedback. On the basis of the processing chain analysis, the key node information in the function processing chain of each type is screened out, which usually includes data acquisition, processing and transmission. A basic index framework of the function integrity of the measurement and control device is constructed, which contains four functions of the measurement and control device, 30 nodes, and a total of 92 detection indexes under the nodes. Therefore, an initial index pool containing 92 indexes is established, and Table 1 gives the initial index number and typical index examples of the four functions of the measurement and control device.

[0021] Table 1: Initial index pool of function integrity of measurement and control device

[0022]

[0023] Step 2, research on construction of a stacking integrated model;

[0024] In the step 2, the specific process is as follows:

[0025] Stacking is an integrated learning technique. Its core idea is to train a “meta-learner” model to combine the prediction results of multiple “base-learner” models, so as to obtain better generalization performance than any single base-learner. It belongs to a high-level form of model fusion strategy. Stacking usually involves at least two levels of models:

[0026] The first layer: base learner layer, contains multiple, different types (heterogeneous) of machine learning models. These models are called base learners. Common choices: decision tree, random forest, support vector machine, K-nearest neighbor, neural network, linear / logistic regression, gradient boosting tree, etc. The diversity of models is key. These base learners are trained on the original training data.

[0027] The second layer: meta-learner layer, only one model, called meta-learner or combiner. Its input is not the original features, but the prediction results of all base learners on the training samples (these prediction results are treated as new "meta-features"), and the output is the final prediction of the entire Stacking model. Common choices are relatively simple, possibly slightly better in terms of interpretability, and not prone to overfitting models, such as: linear regression (for regression problems), logistic regression (for classification problems), ridge regression / Lasso, simple decision trees, etc. Sometimes powerful models like GBDT or neural networks are also used, but be wary of the risk of overfitting.

[0028] The Stacking model is applied to evaluate the function integrity of the monitoring system by combining the prediction abilities of different models and the characteristics of system data, effectively analyzing and evaluating the function integrity of the monitoring system, and providing strong support for the prediction and decision of the device business function state. Gradient boosting regressor and classifier are used as meta-models to build a Stacking integrated model and complete the evaluation of the function integrity of the monitoring system.

[0029] Step 3, study the initial function integrity score and grade given;

[0030] In step 3, the specific process is:

[0031] (1) Construction of function integrity evaluation index system. According to the function integrity evaluation index system established in step 1, a tree structure evaluation factor set U = {u1, u2,..., un} is formed.

[0032] (2) Division of evaluation grades. Establish a four-element evaluation grade set V = {good, qualified, attention, serious} = {V1, V2, V3, V4}, which corresponds to the gradual change process of function integrity state from normal to serious degradation.

[0033] "Good" - all function indicators are within the ideal range and can run well;

[0034] "Qualified" - all function indicators are within a certain range around the ideal value, with good function performance and normal operation;

[0035] "Attention" - the function of multiple indicators have deviated and exceeded the system specified range of trends, indicators fluctuation, close to the threshold, alarm;

[0036] "Serious" - the function of some indicators exceed the threshold, in abnormal state, has been unable to normal operation, serious alarm.

[0037] (3) Construction of fuzzy relation matrix. Through the calculation of its degradation, get each index on each evaluation grade membership, construct fuzzy relation matrix.

[0038] (4) The determination of weight vector. Inviting experts to determine the importance of each index, using G1 method to determine the weight.

[0039] (5) Multi-level fuzzy comprehensive evaluation. The fuzzy relation matrix and weight vector operation, get fuzzy vector of comprehensive evaluation.

[0040] (6) The results of the analysis. The fuzzy comprehensive evaluation results are defuzzified to get the specific evaluation results as shown in Table 2, namely the function of integrity level. Analysis of the function of integrity level, give the possible conditions under this level.

[0041] Table 2 Fuzzy comprehensive evaluation method comment set

[0042]

[0043] Step 4, training and analysis of the research model;

[0044] In the step 4, the specific process is:

[0045] Using the built intelligent substation monitoring system fault injection test platform for experiment, obtain the required index data, for the evaluation of function integrity and the training of machine learning model. A total of 2000 groups of measurement and control device index data set, according to the proportion of 6:2:2 is divided into training set, validation set and test set.

[0046] Before training the data using machine learning algorithm, each group of data needs to be "labeled" so that the supervised learning algorithm can learn the integrity result corresponding to each group of data. Therefore, according to the content described in step 3, the functional integrity of the core monitoring device is evaluated. The membership matrix of each level index is multiplied by (35, 75, 85, 95)T to obtain the integrity of each index, and finally a "integrity score" and "integrity level" (serious, attention, qualified, good) of a complete set of data are obtained, both of which can be used as the integrity result of the monitoring device. The score is used as the regression label, and the level is used as the classification label. Finally, the integrity of the 2000 groups of data obtained is "good" about 500 groups (25%), "qualified" about 1200 groups (60%), "attention" about 280 groups (14%), and "serious" about 20 groups (1%).

[0047] Using the smart substation monitoring system fault injection test platform, a diversified data covering four states of "serious", "attention", "qualified" and "good" is generated by simulating the real substation operating environment and artificially injecting multiple types of abnormalities (such as communication packet loss, logic lockout failure, sampling precision deviation, etc.), which can effectively enhance the sensitivity of the machine learning model to functional abnormal conditions and extreme failures.

[0048] Through training iteration, the relationship between the minimum target value of the Stacking model and the number of iterations is analyzed. The minimum target value refers to the minimum value of the loss function. As a result, the difference between the observed and estimated minimum target values is more obvious in the initial stage; after 25 iterations, both gradually converge to a lower level, indicating that the model has entered a convergent state or reached a global optimal solution, and subsequent iterations are only fine-tuned or remain unchanged. Therefore, when training the model, 30 is selected as the final number of iterations of the model.

[0049] In terms of evaluation results, the predictions of the four models are concentrated between 83 and 91 points, showing the same concentration trend. However, in terms of data dispersion, compared with the other three models, the concentration trend and dispersion degree of the Stacking model are more consistent with the actual scores, and there is a clear and slender tail effect in the segment below 70. The prediction score of the Stacking model is more consistent with the actual score, and the other models have a certain deviation.

[0050] It is worth noting that the contents not described in detail in the embodiments of the present application belong to the prior art known to those skilled in the art.

[0051] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any changes or replacements within the technical scope disclosed by the present application, which can be easily thought by those skilled in the art, should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for evaluating the functional integrity of a monitoring system based on a Stacking model, characterized in that: The method comprises: Step 1: Research the construction of functional integrity evaluation index system; Step 2: Study the construction of Stacking integration model; Step 3: Study the initial integrity score and grade; Step 4: Research model training and analysis.

2. The method for evaluating the functional integrity of a monitoring system based on a Stacking model according to claim 1 is characterized in that: In step 1, the construction of the functional integrity assessment index system is studied, and the specific process is as follows: Study the technical specifications of the substation monitoring system monitoring host, measurement and control devices, gateway equipment and other equipment and other relevant standards. Based on the functional mechanism model of the substation monitoring system, taking the measurement and control device as an example, sort out and analyze the core functions of the measurement and control device, clarify the input and output requirements of the function, and understand the dependency relationship between the functions. Based on the flow direction and processing process of functional data, the division principle of the whole process module is determined. For the control operation, electrical quantity measurement, status quantity acquisition, and cross-interval interlocking functions, the processing chain from input to output is analyzed in detail, including the processes and requirements of data acquisition, transmission, encoding and decoding, processing and feedback. Based on the processing chain analysis, the key node information in various functional processing chains is screened out, which usually includes data acquisition, processing and transmission. The basic indicator framework for the functional integrity of the measurement and control device is constructed, which includes the four major functions of the measurement and control device, 30 nodes, and a total of 92 detection indicators under the nodes. Therefore, an initial indicator pool containing 92 indicators is established. Table 1 below gives the initial indicator quantity and typical indicator examples of the four major functions of the measurement and control device. Table 1 Initial indicator pool for functional integrity of measurement and control devices 3. The method for evaluating the functional integrity of a monitoring system based on a stacking model according to claim 1, characterized in that: In step 2, the construction of the Stacking integration model is studied. The specific process is as follows: Stacking is an ensemble learning technique whose core idea is to train a "meta-learner" model to combine the predictions of multiple "base learner" models, thereby achieving better generalization performance than any single base learner. It is an advanced form of model fusion strategy. Stacking typically involves at least two layers of models: The first layer, the base learner layer, contains multiple, heterogeneous machine learning models. These models are called base learners. Common choices include decision trees, random forests, support vector machines, K-nearest neighbors, neural networks, linear regression / logistic regression, and gradient boosting trees. Model diversity is key. These base learners are trained on the original training data. The second layer, the meta-learner layer, consists of a single model, called the meta-learner or combiner. Its input is not the original features, but rather the predictions of all base learners in the first layer for the training examples (these predictions are treated as new "meta-features"). The output is the final prediction of the entire stacking model. Relatively simple models with potentially better interpretability and a low risk of overfitting are often chosen, such as linear regression (for regression problems), logistic regression (for classification problems), ridge regression / lasso, and simple decision trees. Powerful models such as GBDT or neural networks are sometimes used, but the risk of overfitting should be carefully considered. Random forests, support vector machines, XGBoost, and a stacking model that integrates these three machine learning models are used to assess the functional integrity of the monitoring system. By integrating the predictive capabilities of different models and combining them with system data characteristics, the functional integrity of the monitoring system is effectively analyzed and evaluated, providing strong support for predicting and making decisions about the functional status of equipment. A gradient boosting regressor and classifier are used as meta-models to construct a stacking integrated model to complete the functional integrity assessment of the monitoring system.

4. The method for evaluating the functional integrity of a monitoring system based on a stacking model according to claim 1, characterized in that: In step 3, the initial integrity score and grade are given, and the specific process is as follows: (1) Construction of functional integrity evaluation index system: Based on the functional integrity evaluation index system established in step 1, a tree-structured evaluation factor set U = {u1, u2, ..., un} is formed. (2) Classification of evaluation levels. A four-element evaluation level set V = {good, qualified, caution, severe} = {V1, V2, V3, V4} is established, corresponding to the gradual change of functional integrity status from normal to severe deterioration. "Good" - all functional indicators are within the ideal range and can operate well; "Qualified" means that all functional indicators fluctuate within a certain range near the ideal value, the functional performance is good, and normal operation is normal; "Attention" - Multiple indicators of the function have a tendency to deviate from and exceed the range specified by the system, the indicators fluctuate greatly, and approach the threshold, and an alarm is issued; "Critical" - Certain indicators of the function exceed the threshold and are in an abnormal state. Normal operation can no longer be performed and a serious alarm is issued. (3) Construction of fuzzy relationship matrix. By calculating its degradation degree, the membership degree of each indicator to each evaluation level is obtained and the fuzzy relationship matrix is ​​constructed. (4) Determination of weight vectors: Invite experts to determine the importance of each indicator and use the G1 method to determine the weight. (5) Multi-level fuzzy comprehensive evaluation: The fuzzy relationship matrix is ​​operated with the weight vector to obtain the fuzzy vector of comprehensive evaluation. (6) Results and Analysis. The fuzzy comprehensive evaluation results are defuzzified to obtain the specific evaluation results shown in Table 2, namely the functional integrity level. The functional integrity level is analyzed to provide the possible conditions that may occur at this level. Table 2. Comments on fuzzy comprehensive evaluation method 5. The method for evaluating the functional integrity of a monitoring system based on a Stacking model according to claim 1, characterized in that: In step 4, the training and analysis of the research model are as follows: Experiments were conducted using a fault injection test platform for smart substation monitoring systems to obtain the required indicator data for functional integrity assessment and machine learning model training. A total of 2,000 measurement and control device indicator datasets were obtained and divided into training, validation, and test sets in a 6:2:2 ratio. Before using machine learning algorithms to train data, each data set must be labeled so that the supervised learning algorithm can learn the integrity results corresponding to each data set. Therefore, according to the procedures described in Step 3, the functional integrity of the core monitoring equipment is assessed. The membership matrix of each indicator is multiplied by (35, 75, 85, 95)T to obtain the integrity of each indicator. Ultimately, an "integrity score" and "integrity level" (critical, caution, acceptable, good) for the entire data set are obtained. Both can be used as integrity results for the measurement and control device. Using the score as the regression label and the level as the classification label, the integrity of 2,000 data sets is finally obtained: approximately 500 groups (25%) are rated "good", approximately 1,200 groups (60%) are rated "acceptable", approximately 280 groups (14%) are rated "caution", and approximately 20 groups (1%) are rated "critical". By using the fault injection test platform for the intelligent substation monitoring system, by simulating the real substation operating environment and artificially injecting multiple types of anomalies (such as communication packet loss, logical lock failure, sampling accuracy deviation, etc.), we actively generate diverse data covering four status categories: "serious", "caution", "qualified", and "good", which can effectively enhance the sensitivity of machine learning models to functional abnormalities and extreme faults. Through training iterations, we analyzed the relationship between the minimum target value of the Stacking model and the number of iterations. The minimum target value refers to the lowest value of the loss function. The results show that the difference between the observed and estimated minimum target values ​​is obvious in the initial stage. After 25 iterations, the two gradually converged and remained at a low level, indicating that the model had entered a state of convergence or reached the global optimal solution. Subsequent iterations only required fine-tuning or remained essentially unchanged. Therefore, when training the model, 30 iterations were selected as the final number of model iterations. The evaluation results show that the predictions of the four models all converged between 83 and 91 points, demonstrating a common central tendency. However, in terms of data dispersion, the Stacking model's central tendency and dispersion were more consistent with the actual scores than the other three models, with a noticeable and elongated tail effect in the range below 70. The Stacking model's predicted scores corresponded more closely to the actual scores, while the other models exhibited some deviation.

Images