Power grid dispatching AI test data credibility evaluation method and device

Abstract

The invention provides a power grid dispatching AI test data credibility evaluation method and device. The method comprises the following steps: preprocessing power grid raw data; arranging the preprocessed data into a plurality of data pairs according to a power grid AI application scene; according to the plurality of data pairs, constructing a fuzzy set of distribution robust optimization based on a Wasserstein distance; and according to the fuzzy set, measuring a data distribution offset degree, and according to the data distribution offset degree, evaluating the credibility of the power grid dispatching AI test data. According to the method, credible factors except for data missing and abnormal problems in the data generation and transmission process of the power grid AI test data are explained through data offset, and the data offset is measured through a credibility index based on the Wasserstein distance, so that credibility evaluation of the power grid dispatching AI test data is achieved. According to the method, the credibility index is designed through the distributed robust optimization framework, so that robustness and interpretability can be provided for a power grid AI test data set and subsequent power grid dispatching AI.

Description

Technical Field

[0001] The present invention relates to the field of smart grid technology, and in particular to a method and device for evaluating the credibility of power grid dispatching AI test data. Background Art

[0002] Under the dual carbon goals, building a new power system with new energy as the main body faces highly complex and highly random challenges. For power grid dispatching problems, traditional solution methods usually require a large amount of calculation and a long running time. In order to cope with the limitations and shortcomings of traditional methods, the digitalization and intelligence of power systems based on artificial intelligence algorithms are receiving more and more attention, aiming to build a new power system with new energy as the main body.

[0003] While AI solutions to power grid dispatching problems provide convenience, they also face many challenges. First, most AI methods (such as neural networks) are considered black boxes and lack sufficient interpretability and performance guarantees. In addition, the training and maintenance of AI models require a large amount of data and computing resources. The model's ability to infer trends in new instances from the initial training data depends largely on the quality of the input data. Finally, if there are biases and discriminations in the training data, machine learning models may inherit these biases, which may lead to unfair or discriminatory predictions. Therefore, credibility, interpretability, and performance guarantees are essential for the practical implementation of AI models for power grid dispatching scenarios.

[0004] Artificial intelligence solutions to power grid dispatching problems involve techniques such as machine learning, neural networks, and deep learning, which can all be mathematically summarized as learning models. The effectiveness of learning models is directly affected by the quality of the data used for training. Reliable, accurate, and representative data is essential for developing artificial intelligence models that can be effectively applied to new data. From a data-driven perspective, data distribution deviation is an important factor that makes learning algorithms unreliable.

[0005] For power system data, although the credibility assessment research on using it to train power grid dispatching artificial intelligence is still insufficient, there are already mature methods for identifying bad power system data. Generally, it is based on the weighted least squares state estimation method, first assuming that the measurement error follows a normal distribution, and then using the hypothesis test method to identify bad measurement data based on the residual. For data duplication, missing data, anomalies, data imbalance, malicious attacks and other problems in the power grid field, traditional data processing methods are relatively more reliable after data cleaning, verification, fusion, and feature extraction.

[0006] As for the data itself, it is important to emphasize that even after traditional data processing, the processed data still has the problem of data distribution deviation. This refers to the deviation between the distribution of data samples used for model training and the real data distribution in actual application scenarios. Existing power grid data evaluation technology ignores the credibility analysis from the important perspective of data distribution deviation. If the analysis is not performed, it is difficult to guarantee the credibility of the final power grid dispatching artificial intelligence.

[0007] In the field of power grids, the existing technologies for evaluating the credibility of power grid data basically only involve specific equipment and specific steps in the process of data generation, collection, and transmission, and lack technologies for evaluating whether the overall power grid dispatching data is credible; the few methods that start from an overall perspective are relatively subjective, and do not start from the data itself. In addition, most data credibility assessment methods in the power grid field do not specifically consider the scenarios of power grid dispatching artificial intelligence, and the degree of adaptation is insufficient. If the data evaluation method cannot be combined with the ultimate purpose of power grid dispatching AI to provide a certain guarantee for the algorithm performance that can be generated under the data, it will be difficult to provide strong persuasiveness.

[0008] In summary, adapting to the scenarios of power grid dispatching artificial intelligence, focusing on the impact of data distribution deviation on the reliability of artificial intelligence algorithms, developing evaluation methods for evaluating the credibility of power grid dispatching artificial intelligence training data, and providing credible and reliable training data are still urgent issues to be solved in the field of smart grid technology. Summary of the invention

[0009] The present invention proposes a method and device for evaluating the credibility of power grid dispatching AI test data, which can measure the distribution deviation degree of training data to achieve credibility evaluation of power grid dispatching AI test data.

[0010] In a first aspect, the present application provides a method for evaluating the credibility of power grid dispatching AI test data, including: preprocessing power grid data; organizing the preprocessed data into multiple data pairs according to the power grid AI application scenario; constructing a distributed robust optimized fuzzy set based on the Wasserstein distance based on the multiple data pairs; based on the fuzzy set, measuring the degree of data distribution deviation, and evaluating the credibility of the power grid dispatching AI test data according to the degree of data distribution deviation.

[0011] In some embodiments, according to the power grid AI application scenario, the preprocessed data is organized into multiple data pairs, including: when the application scenario is new energy prediction or load prediction, the input data includes historical data, and the output data includes new data to be predicted; when the application scenario is unit combination optimization, the input data includes the actual operating parameters of the power grid, and the output data includes the start and stop status of the generator set and the unit output.

[0012] In some embodiments, according to the plurality of data pairs, constructing a distributed robust optimized fuzzy set based on Wasserstein distance, comprising:

[0013] Based on Wasserstein distance, a distributed robust optimization fuzzy set is constructed. The fuzzy set is centered on the nominal distribution and has ε as the radius to form a Wasserstein ball. The data set {ξ 1 ,ξ 2 ,…,ξ N}, each data point ξ i The dimension is d, in the value space The empirical distribution is Let P(U) be The set of all possible probability distributions on , the Wasserstein fuzzy set is expressed as:

[0014]

[0015] in Is the p-order Wasserstein distance between two distributions. For two distributions P and Q on P(U), the p-order Wasserstein distance between them is defined as:

[0016]

[0017] Where p=1.

[0018] In some embodiments, according to the plurality of data pairs, a distributed robust optimization fuzzy set is constructed based on the Wasserstein distance, including: in the case of 1-Wasserstein distance, given a confidence level of 1-β, the fuzzy set radius is ε N (β), calculated as:

[0019]

[0020] Where D is a constant, which can be understood as the diameter of the fuzzy set, and is calculated as follows:

[0021]

[0022] where ξ k is a data sample, is the sample mean;

[0023] At this point, the fuzzy set can be guaranteed contains the true distribution with a confidence level of 1-β, i.e.

[0024]

[0025] The performance of the theoretical optimal algorithm under the real data distribution in the actual application scenario is denoted as E P [l(f * ,ξ)], the performance of the data-driven distributed robust algorithm under the real distribution is E P [l(f * dro ,ξ)], then the difference between the two has the following upper bound with a confidence level of 1-β:

[0026]

[0027] where ||l(f * ,·)|| Lip is the Lipschitz norm of the objective loss function, defined as:

[0028]

[0029] Where the supremum norm ||f|| ∞ is the supremum of a function over its domain.

[0030] In some embodiments, the degree of data distribution deviation is measured according to the fuzzy set, and the credibility of the power grid dispatching AI test data is evaluated according to the degree of data distribution deviation, including: integrating the fuzzy set radius in the confidence value interval to obtain the expected distance between the nominal distribution and the true distribution, and using the expected distance to reflect the degree of data distribution deviation. The smaller the expected distance, the less the degree of data distribution deviation.

[0031] In some embodiments, according to the fuzzy set, the degree of data distribution deviation is measured, and according to the degree of data distribution deviation, the credibility of the power grid dispatching AI test data is evaluated, including:

[0032] Suppose the confidence level is changed from 1-β to 1-β+Δβ, then the radius of the fuzzy set will increase from ε N (β) increases to ε N (β-Δβ);

[0033] The true distribution lies within a radius from ε N (β) takes the value to ε N (β-Δβ), thickness is ε N (β-Δβ)-ε N (β) is in the Wasserstein shell, with the nominal distribution located at the center of the sphere;

[0034] Let Δβ approach zero, for the fuzzy set radius ε N (β) Integrate over the confidence interval β∈[0,1] to obtain the empirical distribution The expected distance from the true distribution P reflects the degree of data deviation with the expected distance;

[0035] The expected radius of the fuzzy set is obtained by integration as a data credibility evaluation index to measure the degree of data distribution deviation. Where D is a constant; when the number of samples N in the data set is given, the smaller the indicator means the less data deviation.

[0036] In a second aspect, the present application provides a power grid dispatching AI test data credibility assessment device, comprising:

[0037] A preprocessing unit, used for preprocessing the data generated by the power grid;

[0038] A sorting unit, used to sort the pre-processed data into multiple data pairs according to the power grid AI application scenario;

[0039] A construction unit, used for constructing a distributed robust optimized fuzzy set based on Wasserstein distance according to a plurality of the data pairs;

[0040] An evaluation unit is used to measure the degree of data distribution deviation according to the fuzzy set, and evaluate the credibility of the power grid scheduling AI test data according to the degree of data distribution deviation.

[0041] In some embodiments, the sorting unit is used for: when the application scenario is new energy prediction or load prediction, the input data includes historical data, and the output data includes new data to be predicted; when the application scenario is unit combination optimization, the input data includes the actual operating parameters of the power grid, and the output data includes the start and stop status of the generator set and the unit output.

[0042] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any of the above methods when executed by a processor.

[0043] In a fourth aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above methods when executing the program.

[0044] The power grid dispatching AI test data credibility assessment method and device of the present application embodiment have the following beneficial effects:

[0045] The power grid dispatching AI test data credibility assessment method and device of the present application have the following beneficial effects:

[0046] In this application, the trust factors other than data missing and abnormal problems in the data generation and transmission process of the power grid AI test data are explained by data offset, and the data offset is measured by the credibility index based on the Wasserstein distance to achieve the credibility assessment of the power grid dispatching AI test data. This application designs the credibility index through the framework of distributed robust optimization, so it can provide robustness and explainability for the power grid AI test data set and subsequent power grid dispatching AI. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flow chart of a method for evaluating the credibility of AI test data for power grid dispatching according to an embodiment of the present application;

[0048] Figure 2 This is a schematic diagram of the structure of the power grid dispatching AI test data credibility assessment device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] In the following introduction, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The following introduction provides multiple embodiments of the present invention, and different embodiments can be replaced or combined, so this application can also be considered to include all possible combinations of the same and / or different embodiments recorded. Therefore, if one embodiment includes features A, B, and C, and another embodiment includes features B and D, then this application should also be considered to include embodiments containing one or more of all other possible combinations of features A, B, C, and D, even though the embodiment may not be clearly recorded in the following text.

[0050] The terms involved in this application are explained as follows:

[0051] 1) Smart Grid: It is the intelligence of the power grid, also known as "Grid 2.0". It is based on an integrated, high-speed two-way communication network, through the application of advanced sensing and measurement technology, advanced equipment technology, advanced control methods and advanced decision support system technology.

[0052] 2) Grid dispatching: Based on the data information fed back by various information collection devices or the information provided by monitoring personnel, combined with the actual operating parameters of the grid, such as voltage, current, frequency, load, etc., comprehensively consider the progress of various production work, judge the safe and economical operation status of the grid, issue operating instructions through telephone or automatic system, and command on-site operators or automatic control systems to make adjustments, such as adjusting generator output, load distribution, switching capacitors and reactors, etc., so as to ensure the continuous safe and stable operation of the grid.

[0053] 3) Grid dispatching AI: that is, artificial intelligence for grid dispatching, which is generally based on technologies such as machine learning, neural networks, and deep learning, and is used in typical grid dispatching scenarios including but not limited to new energy forecasting, load forecasting, unit combination optimization, and optimal power flow calculation.

[0054] 4) Power grid dispatching AI test data: data used for power grid dispatching artificial intelligence training, used to build test data sets, and highly correlated with the final performance and credibility of artificial intelligence.

[0055] 5) Wasserstein distance: also known as the bulldozer distance, it is used to indicate the similarity between two distributions. In mathematics, it refers to the distance function between probability distributions on a given metric space M. Intuitively, each probability distribution can be regarded as a pile of a unit number of soil piled on M. Then the Wasserstein metric refers to the minimum cost of turning one pile of soil into another pile of soil, which can be defined as the amount of soil that needs to be moved multiplied by the average distance that needs to be moved.

[0056] 6) Distributionally robust optimization: Distributionally robust optimization is a method for dealing with uncertainty in optimization problems. Unlike traditional robust optimization, distributionally robust optimization assumes partial knowledge of the probability distribution of uncertain parameters, rather than complete ignorance. Specifically, distributionally robust optimization models uncertainty by considering a set of possible probability distributions (called fuzzy sets) and optimizing the worst-case expected cost within this set of distributions. This approach is particularly suitable for scenarios that require careful decision making under uncertainty, such as financial portfolio design, supply chain management, and machine learning model training.

[0057] 7) Generalization ability of the learning model: that is, under the setting of the learning problem, the ability of the trained artificial intelligence model to make accurate predictions when faced with new data that it has never seen. The training model minimizes the expected loss on the probability distribution of the training data, but the probability distribution of real-world data is different from that of the training data. The mismatch between the two causes the performance of the model to deteriorate in real applications. The stronger the generalization ability of the learning model, the closer the performance it can provide in real-world applications to that on the training set, and the more credible the model is.

[0058] 8) Data distribution shift: refers to the deviation between the distribution of data samples used for model training and the actual data distribution in the actual application scenario. Data distribution shift can be divided into three categories: covariate shift, prior probability shift, and conditional probability shift. Common causes of data distribution shift are sample selection bias and non-stationary environment, that is, the samples collected from the total population are not uniform enough, and the data will change over time and space in actual applications. From a data-driven perspective, data distribution shift is an important factor that leads to unreliable learning algorithms.

[0059] Embodiment 1

[0060] like Figure 1 As shown, the present application provides a method for evaluating the credibility of power grid dispatching AI test data, including: S11, preprocessing the power grid data; S13, organizing the preprocessed data into multiple data pairs according to the power grid AI application scenario; S15, constructing a distributed robust optimized fuzzy set based on the Wasserstein distance according to the multiple data pairs; S17, measuring the degree of data distribution deviation according to the fuzzy set, and evaluating the credibility of the power grid dispatching AI test data according to the degree of data distribution deviation.

[0061] The power grid dispatching AI test data credibility assessment method provided in this application is different from the heuristic and subjective power grid data credibility assessment method. This method is based on the framework of distributional robust optimization and uses learning theory and statistical analysis to design data-driven credibility indicators. It can provide robustness and interpretability for power grid AI test data sets and subsequent power grid dispatching AI, and solves the problems of existing power grid data assessment technology lacking consideration of data distribution offset of power grid dispatching data, lacking adaptation in power grid dispatching artificial intelligence scenarios, and lacking analysis of the impact of test data on algorithm performance.

[0062] Embodiment 2

[0063] The power grid dispatching AI test data credibility assessment method provided in this application includes:

[0064] Step 101: perform data preprocessing. The raw data of the power grid is cleaned and verified to solve the problems of duplication, missing, anomaly, etc. on the data. On this basis, data fusion and feature extraction can be further performed according to the specific application scenario and data structure to improve data quality and facilitate subsequent processing.

[0065] Step 102: According to the specific power grid AI application scenario, the data is organized into a unified format, that is, a binary relationship pair such as (data input, data output), with a total of N data pairs. In the examples of new energy forecasting and load forecasting, the data input is historical data, and the data output is the new data to be predicted. In the unit combination optimization example, the input is the actual operating parameters of the power grid such as voltage, current, frequency, load, line topology, etc., and the output is the start and stop status of the generator set and the unit output.

[0066] Step 103: Based on the data-driven Wasserstein distribution robust optimization framework, a Wasserstein fuzzy set is constructed. The Wasserstein fuzzy set is a Wasserstein ball centered on the nominal distribution. Under the confidence level 1-β that contains the true distribution, the fuzzy set radius is ε N (β). One example of a calculation method is as follows:

[0067]

[0068] Where D is a constant, which can be understood as the diameter of the fuzzy set and can be obtained in the following data-driven way:

[0069]

[0070] where ξ k is a data sample, is the sample mean.

[0071] Step 104: fuzzy set radius ε N (β) Integrate over the confidence interval β∈[0,1] to obtain the expected distance between the nominal distribution and the true distribution This reflects the degree of data deviation.

[0072] Starting from the perspective of data distribution shift, this application can explain the problem that even after traditional data processing, the trained artificial intelligence model may still have poor generalization. After modeling classic problems of power grid dispatching such as new energy forecasting, load forecasting, and unit combination optimization as learning problems, the framework of distributed robust optimization can provide guarantees for the generalization performance of data-driven learning algorithms based on the characteristics of the data and the modeling of the problem. The data credibility assessment method based on Wasserstein distributed robust optimization design can measure the degree of distribution shift of training data to achieve credibility assessment of power grid dispatching AI test data.

[0073] Embodiment 3

[0074] The power grid dispatching AI test data credibility assessment method provided in this application includes the following steps:

[0075] Step (1) uses traditional data processing methods to pre-process the grid data.

[0076] Considering that the amount of power grid data may be very large, it is recommended to use computer programs to assist in processing, use code to read raw data and implement data cleaning operations such as statistical methods. However, it is strongly recommended not to use artificial intelligence-based methods such as neural networks in this step, otherwise the black box will affect the credibility of the data.

[0077] First, after obtaining the power grid data, the duplicate items in the data are processed. By comparing the unique identifiers of the data records, such as timestamps, device IDs, etc., duplicate records are identified and deleted to ensure that each data record exists uniquely in the data set. Next, the missing values ​​in the data are processed. For cases with fewer missing values, interpolation or mean filling can be used to complete them; for cases with more missing values, it is necessary to combine business logic and data characteristics to select a suitable filling method or directly delete records with too many missing values. In addition, outliers in the data need to be processed. By setting a reasonable threshold range or using statistical analysis methods, such as least squares regression, box plots, Z scores, etc., obviously abnormal data points can be identified and eliminated to ensure the rationality of the data.

[0078] After data cleaning is completed, data verification is performed. The purpose of data verification is to ensure that no errors occur during data transmission and storage. First, format verification is performed to ensure that the format of the data meets expectations, such as time format, numerical range, etc. Secondly, consistency verification is performed to check whether the logical relationship between data is reasonable, such as whether the physical relationship of parameters such as voltage and current conforms to the basic laws of the power system. Finally, integrity verification is performed to ensure the integrity of data records, such as whether each record contains all necessary fields and whether the field value is empty.

[0079] On this basis, you can choose to further perform data fusion and feature extraction according to the specific application scenario and data structure. You can choose but you don’t have to. Step (1) is to ensure that the data does not have missing, duplicated, or obvious anomalies. Specifying data fusion and feature extraction according to the application scenario and data structure may make subsequent processing more convenient.

[0080] Possible implementation examples of load forecasting scenarios are given below, which are only used for illustrative purposes and are not intended to limit the specific content of data fusion and feature extraction operations.

[0081] Load forecasting scenario: In load forecasting, data fusion can choose to integrate historical load data, weather data, holidays and other information. First, match the historical load data with the weather data of the corresponding time period. Align the two types of data through timestamps to ensure that the load data at each time point has corresponding weather data. Then, add holiday information to the data set. Holidays may have a significant impact on power load, so holidays and working days need to be marked in the data set. In terms of feature extraction, time features such as year, month, day, and hour can be extracted from timestamps. These features can capture the periodic changes of load. Or extract features such as temperature, humidity, and wind speed from weather data, and further extract statistics of these features to better reflect the impact of weather on load. In addition, the load values ​​of the previous day, week, and month are extracted from historical load data as features. These features can capture short-term and long-term trends of load.

[0082] Step (2) normalizes the data format and unifies it into input-output pairs according to specific power grid AI application scenarios.

[0083] For multi-source heterogeneous data, if data fusion operations or other methods are not performed in the previous step to unify the data structure, their structures need to be unified at this time. Data from different data sources may be stored in different formats such as relational databases (such as MySQL, PostgreSQL), NoSQL databases (such as MongoDB), CSV files, Excel tables, etc., and these data need to be standardized and unified. First, connect and extract data. For relational databases, SQL query statements can be used to extract the required data; for NoSQL databases, corresponding query languages ​​or APIs can be used to extract data; for CSV files and Excel tables, data processing tools (such as Pandas) can be used to read data. The extracted data may include historical load data, weather data, equipment status data, etc. Next, the extracted data is formatted and standardized. The data formats of different data sources may be different, such as the format of timestamps, the units of values, etc. These data need to be converted into a unified format. For example, convert all timestamps into a unified time format (such as ISO 8601), and convert all values ​​into a unified unit (such as temperature is unified into degrees Celsius and voltage is unified into volts). In addition, the data needs to be standardized, such as normalizing or standardizing numerical data to eliminate the impact between different dimensions.

[0084] In the setting of learning problems, we need to consider a problem of mapping from input x∈X to output y∈Y. We do not know the real underlying mapping relationship f:X→Y. What we can do is to use the existing training data set (x, y)~D trainThe model of the training learning algorithm is used to simulate the mapping f. Therefore, it is necessary to organize the data into a unified format, that is, a binary relationship data pair such as (data input, data output), according to the power grid AI application scenario.

[0085] In the examples of new energy forecasting and load forecasting, the data input is mainly historical data, and the data output is the new data to be predicted, which can be multiple time points for a single time point or multiple time points for multiple time points. In the example of unit combination optimization, the data input is the actual operating parameters of the power grid, such as voltage, current, frequency, load, line topology, etc., and the data output is the start and stop status of the generator set and the unit output. In this way, the data can form a unified and standardized training data set for training artificial intelligence models.

[0086] However, even after processing through steps (1) and (2), the distribution of the obtained data still deviates from the actual distribution in real-world application scenarios. The reasons may include: the diversity and heterogeneity of data generation sources, changes in generation environments and conditions, differences between simulated data and actual data, the impact of data sampling and resolution; signal loss and distortion, delay and packet loss in data transmission, and security and integrity issues in data transmission. The data distribution deviation caused by these reasons cannot be completely eliminated, and we must estimate the credible impact of data distribution deviation on power grid dispatching AI.

[0087] Step (3) constructs data-driven Wasserstein fuzzy sets.

[0088] The present invention constructs fuzzy sets under the framework of distributed robust optimization. In power grid dispatching, whether it is optimization problems such as unit combination, economic dispatch and optimal power flow, or renewable energy or load forecasting, in the paradigm of learning problems, they are all capturing mapping relationships such as f:x→y, and this mapping is described by us in a parameterized form. Therefore, we design a credibility indicator and construct a data set consisting of data points ξ=(x,y), which ultimately serves the following distributed robust learning problem, namely min f∈F max P∈P E ξ～P [l(f,ξ)], in the hope that the learned parameterized model has strong generalization ability.

[0089] For the fuzzy set P of distributed robust optimization, it is constructed based on the Wasserstein distance, with the nominal distribution as the center and ε as the radius to form a Wasserstein ball. The nominal distribution generally takes the empirical distribution. Consider a data set consisting of N data samples {ξ 1 ,ξ 2 ,…,ξ N}, each data point ξ i The dimension is d, in the value space The empirical distribution is remember yes The set of all possible probability distributions on . In this case, the Wasserstein fuzzy set is represented as:

[0090]

[0091] in is the p-order Wasserstein distance between two distributions. The p-order Wasserstein distance between two distributions P and Q on is defined as follows:

[0092]

[0093] Among them, p=1, that is, 1-Wasserstein distance is the most commonly used case.

[0094] In the case of 1-Wasserstein distance, a desirable fuzzy set radius embodiment is as follows: Given a confidence level of 1-β, the fuzzy set radius ε is selected N (β), the formula is:

[0095]

[0096] Where D is a constant, which can be understood as the diameter of the fuzzy set and can be obtained in the following data-driven way

[0097]

[0098] where ξ k is a data sample, is the sample mean

[0099] At this point, the fuzzy set can be guaranteed contains the true distribution with a confidence level of 1-β, i.e.

[0100]

[0101] The performance of the theoretical optimal algorithm under the real data distribution in the actual application scenario is E P [l(f * ,ξ)], the performance of the data-driven distributed robust algorithm under the real distribution is E P [l(f * dro ,ξ)]. Then the difference between the two with a confidence level of 1-β has the following upper bound:

[0102]

[0103] where ||l(f * ,·)|| Lip is the Lipschitz norm of the objective loss function, defined as:

[0104]

[0105] Where the supremum norm ||f|| ∞ is the supremum of a function over its domain.

[0106] Step (4) Calculate the credibility index of the power grid dispatching AI test data.

[0107] Given the size of the data sample and the number of sample points in it, the radius of the fuzzy set depends only on the confidence level. Under the same 1-β confidence level, the radius of the Wasserstein ball ε N The larger the (β) is, the more empirical the distribution is. The farther away from the true distribution P, the higher the degree of data deviation of the sample data, and the lower the corresponding data reliability. However, this only gives a probability result under a certain confidence level, and there is a trade-off between the confidence level and the size of the radius. To reduce the uncertainty of the judgment, the smaller the significance level β is, the smaller the radius of the fuzzy set ε is. N (β) inevitably becomes larger.

[0108] The goal of indicator design is to evaluate and compare the deviation of different data sets from the true distribution. The larger the deviation, the larger the indicator. This is used to distinguish the degree of deviation of different data sets and select the data set with smaller deviation, that is, more credible.

[0109] In order to obtain an evaluation indicator that does not depend on β, we consider the following process: Imagine making a slight change in the confidence level, from 1-β to 1-β+Δβ, then the radius of the fuzzy set will increase from ε N (β) increases to ε N (β-Δβ). Therefore, with probability Δβ we believe that the true distribution lies within a radius from ε N (β) takes the value to ε N (β-Δβ), thickness is ε N (β-Δβ)-ε N (β) is in the Wasserstein shell, and the nominal distribution is at the center of the sphere. Let Δβ approach zero, and we consider the fuzzy set radius ε N (β) By integrating over the confidence interval β∈[0,1], we can obtain the empirical distribution The expected distance from the true distribution P is used to reflect the degree of data deviation. The upper and lower bounds of the integral can also be adjusted according to specific needs.

[0110] Omitting the integration process, the expected radius of the fuzzy set is obtained as a data credibility evaluation index to measure the degree of data distribution deviation. The constant D is given by the complete data drive. When the number of samples N in the data set is given, the smaller the indicator is, the less data deviation it has. In addition, as the sample size increases, we can estimate the deviation between the true distribution and the empirical distribution we use more and more accurately.

[0111] The credibility assessment method of power grid dispatching AI test data provided by the present invention can measure the distribution deviation of training data to achieve credibility assessment of power grid dispatching AI test data. This method explains the credibility factors other than data missing and abnormal problems in the data generation and transmission process of power grid AI test data through data deviation, and the data deviation is measured by a credibility indicator based on Wasserstein distance to achieve credibility assessment of power grid dispatching AI test data. This method designs credibility indicators through a distributed robust optimization framework, which can provide robustness and interpretability for power grid AI test data sets and subsequent power grid dispatching AI.

[0112] Embodiment 4

[0113] like Figure 2 As shown, an embodiment of the present application provides a credibility assessment device for power grid dispatching AI test data, including: a preprocessing unit 31, used to preprocess power grid generated data; a sorting unit 32, used to sort the preprocessed data into multiple data pairs according to the power grid AI application scenario; a construction unit 33, used to construct a distributed robust optimized fuzzy set based on the Wasserstein distance according to the multiple data pairs; an evaluation unit 34, used to measure the degree of data distribution deviation according to the fuzzy set, and evaluate the credibility of the power grid dispatching AI test data according to the degree of data distribution deviation.

[0114] The sorting unit 32 is used for: when the application scenario is new energy prediction or load prediction, the input data includes historical data, and the output data includes new data to be predicted; when the application scenario is unit combination optimization, the input data includes the actual operating parameters of the power grid, and the output data includes the start and stop status of the generator set and the unit output.

[0115] The construction unit 33 is used to construct a distributed robust optimization fuzzy set based on the Wasserstein distance. The fuzzy set takes the nominal distribution as the center and ε as the radius to form a Wasserstein ball. The data set data {ξ 1 ,ξ 2 ,…,ξ N}, each data point ξ iThe dimension is d, in the value space The empirical distribution is remember yes The set of all possible probability distributions on , the Wasserstein fuzzy set is expressed as:

[0116]

[0117] in is the p-order Wasserstein distance between two distributions, for The p-order Wasserstein distance between two distributions P and Q on is defined as:

[0118]

[0119] Where p=1.

[0120] The construction unit 33 is also used to: in the case of 1-Wasserstein distance, given a confidence level of 1-β, the fuzzy set radius is ε N (β), calculated as:

[0121]

[0122] Where D is a constant, which can be understood as the diameter of the fuzzy set, and is calculated as follows:

[0123]

[0124] where ξ k is a data sample, is the sample mean;

[0125] At this point, the fuzzy set can be guaranteed contains the true distribution with a confidence level of 1-β, i.e.

[0126]

[0127] The performance of the theoretical optimal algorithm under the real data distribution in the actual application scenario is E P [l(f * ,ξ)], the performance of the data-driven distributed robust algorithm under the real distribution is E P [l(f * dro ,ξ)], then the difference between the two has the following upper bound with a confidence level of 1-β:

[0128]

[0129] where ||l(f * ,·)||Lip is the Lipschitz norm of the objective loss function, defined as:

[0130]

[0131] Where the supremum norm ||f|| ∞ is the supremum of a function over its domain.

[0132] The evaluation unit 34 is used to integrate the fuzzy set radius in the confidence interval to obtain the expected distance between the nominal distribution and the true distribution, and use the expected distance to reflect the degree of deviation of the data distribution. The smaller the expected distance, the less the degree of deviation of the data distribution.

[0133] The device provided by the present application explains the trust factors other than data missing and abnormal problems in the data generation and transmission process of the power grid AI test data through data offset, and the data offset is measured by the trust index based on the Wasserstein distance to achieve the trustworthiness assessment of the power grid dispatching AI test data. The present application designs the trustworthiness index through the framework of distributed robust optimization, so as to provide robustness and explainability for the power grid AI test data set and the subsequent power grid dispatching AI.

[0134] In the present application, the embodiment of the power grid AI test data credibility assessment device is basically similar to the embodiment of the power grid AI test data credibility assessment method. For relevant matters, please refer to the introduction of the embodiment of the power grid AI test data credibility assessment method.

[0135] The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the above-mentioned power grid AI test data credibility assessment method steps when executing the program. The computer device of the present application is, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The computer device of the present application includes a processor, a memory, an input device, and an output device. The processor, the memory, the input device, and the output device may be connected via a bus or other means. A computer program is stored on the memory, which can be executed on the processor, and the above-mentioned power grid AI test data credibility assessment method steps are implemented when the processor executes the program.

[0136] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned power grid AI test data credibility assessment method steps are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0137] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0138] In the various embodiments of the present invention, all functional units may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0139] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for evaluating the credibility of power grid dispatching AI test data, characterized in that: include: Preprocessing of power grid data; According to the power grid AI application scenario, the pre-processed data is sorted into multiple data pairs; According to the plurality of data pairs, constructing a distributed robust optimized fuzzy set based on Wasserstein distance; According to the fuzzy set, the degree of data distribution deviation is measured, and the credibility of the power grid dispatching AI test data is evaluated according to the degree of data distribution deviation.

2. According to the power grid dispatching AI test data credibility assessment method of claim 1, it is characterized in that: According to the AI ​​application scenarios of power grids, the preprocessed data are organized into multiple data pairs, including: when the application scenario is new energy forecasting or load forecasting, the input data includes historical data, and the output data includes new data to be predicted; when the application scenario is unit combination optimization, the input data includes the actual operating parameters of the power grid, and the output data includes the start and stop status of the generator set and the unit output.

3. The method for evaluating the credibility of power grid dispatching AI test data according to claim 1 or 2, characterized in that: According to the plurality of data pairs, a distributed robust optimized fuzzy set is constructed based on Wasserstein distance, including: Based on Wasserstein distance, a distributed robust optimization fuzzy set is constructed. The fuzzy set is centered on the nominal distribution and has ε as the radius to form a Wasserstein ball. The data set {ξ1,ξ2,…,ξ N }, each data point ξ i The dimension is d, in the value space The empirical distribution is remember yes The set of all possible probability distributions on , the Wasserstein fuzzy set is expressed as: in is the p-order Wasserstein distance between two distributions, for The p-order Wasserstein distance between two distributions P and Q on is defined as: Where p=1.

4. According to the power grid dispatching AI test data credibility assessment method of claim 3, it is characterized in that: According to the plurality of data pairs, a distributed robust optimization fuzzy set is constructed based on the Wasserstein distance, including: in the case of 1-Wasserstein distance, given a confidence level of 1-β, the fuzzy set radius is ε N (β), calculated as: Where D is a constant, which can be understood as the diameter of the fuzzy set, and is calculated as follows: where ξ k is a data sample, is the sample mean; At this point, the fuzzy set can be guaranteed contains the true distribution with a confidence level of 1-β, i.e. The performance of the theoretical optimal algorithm under the real data distribution in the actual application scenario is E P [l(f * ,ξ)], the performance of the data-driven distributed robust algorithm under the real distribution is E P [l(f * dro ,ξ)], then the difference between the two has the following upper bound with a confidence level of 1-β: where ||l(f * ,·)|| Lip is the Lipschitz norm of the objective loss function, defined as: Where the supremum norm ||f|| ∞ is the supremum of a function over its domain.

5. The method for evaluating the credibility of power grid dispatching AI test data according to claim 1 or 2, characterized in that: According to the fuzzy set, the degree of data distribution deviation is measured, and the credibility of the power grid dispatching AI test data is evaluated according to the degree of data distribution deviation, including: integrating the fuzzy set radius in the confidence value interval to obtain the expected distance between the nominal distribution and the true distribution, and using the expected distance to reflect the degree of data distribution deviation. The smaller the expected distance, the less the degree of data distribution deviation.

6. According to the power grid dispatching AI test data credibility assessment method of claim 4, it is characterized in that: According to the fuzzy set, the degree of data distribution deviation is measured, and the credibility of the power grid dispatching AI test data is evaluated according to the degree of data distribution deviation, including: Suppose the confidence level is changed from 1-β to 1-β+Δβ, then the radius of the fuzzy set will increase from ε N (β) increases to ε N (β-Δβ); The true distribution lies within a radius from ε N (β) takes the value to ε N (β-Δβ), thickness is ε N (β-Δβ)-ε N (β) is in the Wasserstein shell, with the nominal distribution located at the center of the sphere; Let Δβ approach zero, for the fuzzy set radius ε N (β) Integrate over the confidence interval β∈[0,1] to obtain the empirical distribution The expected distance from the true distribution P reflects the degree of data deviation with the expected distance; The expected radius of the fuzzy set is obtained by integration as a data credibility evaluation index to measure the degree of data distribution deviation. Where D is a constant; when the number of samples N in the data set is given, the smaller the indicator means the less data deviation.

7. A power grid dispatching AI test data credibility assessment device, characterized in that: include: A preprocessing unit, used for preprocessing the data generated by the power grid; A sorting unit, used to sort the pre-processed data into multiple data pairs according to the power grid AI application scenario; A construction unit, used for constructing a distributed robust optimized fuzzy set based on Wasserstein distance according to a plurality of the data pairs; An evaluation unit is used to measure the degree of data distribution deviation according to the fuzzy set, and evaluate the credibility of the power grid scheduling AI test data according to the degree of data distribution deviation.

8. The power grid dispatching AI test data credibility assessment device according to claim 7, characterized in that: The sorting unit is used for: when the application scenario is new energy prediction or load prediction, the input data includes historical data, and the output data includes new data to be predicted; when the application scenario is unit combination optimization, the input data includes actual operating parameters of the power grid, and the output data includes the start and stop status of the generator set and the unit output.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 6 are implemented.

Images