Power grid data purification method, system, equipment and medium

Through deep learning and reinforcement learning, the power grid data purification method in the existing technology is solved, and the data purification method cannot adapt to the problem of emergency scheduling flexibility of urban power grids and the sparsity of multi-field characteristics data, realizing the selection and accurate prediction of high-quality data sets. Reduced calculation error.

CN118378761BActive Publication Date: 2024-11-26PEKING UNIV
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410817532.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-24
Publication Date
2024-11-26
Estimated Expiration
2044-06-24

AI Technical Summary

Technical Problem

The existing grid data purification methods based on information entropy cannot adjust the evaluation indicators based on the flexibility requirements of urban power grid emergency scheduling, and when processing multi-field characteristic data, calculation errors are caused by sparsity, which affects the acquisition of high-quality data sets.

Method used

Adopt the power grid data purification method based on deep learning and reinforcement learning, update the data value evaluation model and reinforcement learning algorithm, select suitable data subsets to train the prediction model, and filter high-quality data sets based on the prediction results to adapt to the power grid prediction targets and reduce Calculation error.

Benefits of technology

It realizes the flexibility of selecting high-quality data sets based on the power grid prediction targets, matching the flexibility of urban power grid emergency scheduling, reducing the impact of multi-field characteristic data sparseness on data purification quality, and reducing calculation errors.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118378761B_ABST
    Figure CN118378761B_ABST
Patent Text Reader

Abstract

The embodiments of the present invention provide a method, system, device and medium for purifying power grid data, which belongs to the field of power and big data processing. The method includes: inputting the initial data set into a data value assessment model constructed based on a deep learning algorithm to output the data value corresponding to each data point, and the data value shows the probability that the corresponding data point is selected for training the prediction model; and selecting a subset from the initial data set according to the sorting of data value to train the prediction model. If the prediction result fails to meet the power grid prediction target, the data value assessment model is updated based on the reinforcement learning algorithm, otherwise the data points are screened according to the data value sorting to form a high-quality data set. The embodiments of the present invention implement a power grid data purification scheme based on deep learning and reinforcement learning. On the one hand, it can specifically match the flexibility requirements of power grid decision-making, and on the other hand, it avoids the impact of the sparsity of multi-field feature data on the quality of data purification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of electric power technology and big data processing, and in particular to a power grid data purification method, system, equipment and medium. Background Art

[0002] At present, my country's power industry generates PB-level grid data every year, and it will continue to grow exponentially in the future. These grid big data contain multi-faceted status information of the power system, which is crucial for data-driven power system decision-making.

[0003] However, maximizing the potential of data-driven methods in accurate decision-making in power systems depends largely on the quality of input data. Taking the urban power grid with inherent uncertainty as an example, in the process of training decision-making models, not all data contribute information equally: some high-value data provide key information to help grasp the core laws of model training; while some low-value data provide interference information, misleading the model to make correct judgments. Therefore, how to purify data, that is, to selectively screen high-quality data sets, is critical to improving the effectiveness of data-driven methods in urban power grid decision-making.

[0004] In this regard, the prior art has proposed a data purification method using information entropy as a statistical feature. Information entropy is an indicator for measuring the amount of information in a data set. It comes from information theory and was proposed by Claude Shannon. The specific expression is as follows:

[0005]

[0006] in Representation dataset The information entropy of is the number of data points, For data points The probability of occurrence, represents the i-th data point. The core idea of ​​information entropy is to evaluate the uncertainty or randomness in a data set. The larger the information entropy, the richer the information contained in the data set and the higher the uncertainty. Figure 1 As shown in the figure, taking the urban power grid as an example, the data purification process based on information entropy is as follows: after inputting the initial data set, the information entropy is calculated, and high-quality data sets are screened out according to the size of the information entropy index, and finally applied to decision-making such as emergency dispatch of urban power grids.

[0007] However, the inventors of the present application found in the process of implementing the present application that the data purification method based on information entropy has at least the following two defects:

[0008] First, the calculation method of information entropy is fixed, but the emergency dispatch target of urban power grid is not fixed. For example, there are different trade-offs between safety and economy in different scenarios. Therefore, the data quality calculation based on information entropy cannot well match the flexibility requirements of urban power grid emergency dispatch, and cannot adjust the evaluation indicators according to the actual decision-making goals, which is biased.

[0009] Second, in the actual application of urban power grids, it is often necessary to obtain a large amount of characteristic data in multiple fields such as meteorology and energy, and then use data-driven models to accurately characterize renewable uncertainty. However, when describing the amount of information in a multi-feature data set, information entropy relies on approximate algorithms due to the sparsity of specific data, which not only increases the calculation error, but also affects the acquisition of high-quality data sets due to the calculation error.

[0010] Therefore, a more flexible and adaptable grid data purification solution is needed. Summary of the invention

[0011] The purpose of the embodiments of the present invention is to provide a power grid data purification method, system, device and medium, which are used to at least partially solve the above technical problems.

[0012] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides a method for purifying power grid data, comprising: inputting an initial data set into a data value assessment model constructed based on a deep learning algorithm to output a data value corresponding to each data point in the initial data set, wherein the initial data set includes several data points of multi-domain characteristic data about the power grid, and the data value is configured to show the probability of the corresponding data point being selected for training a prediction model, wherein the prediction model is constructed based on a power grid prediction target and is used to predict future power grid data through historical power grid data; selecting a subset from the initial data set according to the sorting of the data value to train the prediction model and obtain corresponding prediction results, and if the prediction result fails to meet the power grid prediction target, updating the data value assessment model based on a reinforcement learning algorithm until the selected subset enables the prediction result of the prediction model to meet the power grid prediction target; and for the data points that meet the power grid prediction target, screening the data points according to the sorting of the data value to form a high-quality data set.

[0013] Optionally, the power grid is a city power grid, and the multi-field characteristic data of the city power grid includes meteorological data and energy data, wherein the energy data includes wind power data and / or photovoltaic data.

[0014] Optionally, the power grid prediction objective is to minimize the prediction loss of the prediction model, and / or the reinforcement learning algorithm is a policy gradient algorithm.

[0015] Optionally, the prediction model is trained based on the following formula:

[0016] ,

[0017] ,

[0018] In the formula, Generally refers to the prediction model class that meets the requirements. represents the trained prediction model, represents the loss function of the prediction model, represents the power grid prediction target, D represents the raw data from the initial dataset, represents the validation set, , is the input feature vector of the data points in the corresponding dataset, and , yes , The corresponding feature vectors to be predicted are: It is a data value assessment model The output of is discretized to obtain the filter function, where is capable of being optimized to update the data value assessment model Parameters.

[0019] Optionally, the data value assessment model is updated based on the following formula:

[0020]

[0021] In the formula, is the original data from the initial dataset D Medium Parameters The probability distribution of the selection of a specified vector under It means that it can be optimized by The updated data value assessment model, represents the optimization goal of executing the policy gradient algorithm, For the original data D Medium Parameters The prediction results of the prediction model under the specified vector The probability distribution of choice, For parameters gradient.

[0022] Optional, is represented as follows:

[0023]

[0024] In the formula, the data point , For data points The input feature vector is For data points The feature vector to be predicted, Represents data points In the specified vector The corresponding amount.

[0025] Optionally, the power grid data purification method also includes: using a pre-built power grid decision model to evaluate the corresponding power grid revenue when the high-quality data set is used as input, and if the expected revenue is not met, adjusting the data set size for the data point screening.

[0026] Optionally, evaluating the grid revenue includes: constructing the grid decision model with the grid revenue as the decision, and integrating the prediction model into the grid decision model; inputting the high-quality data set into the prediction model to obtain corresponding prediction results; and using the constructed grid decision model to continue processing the obtained prediction results to evaluate the grid revenue.

[0027] Optionally, adjusting the data set size includes: initializing an upper bound and a lower bound of the data set size, wherein the upper bound and the lower bound are respectively a preset maximum percentage and a preset minimum percentage of the data set size; and when a difference between the upper bound and the lower bound is greater than a set threshold, adapting to the power grid revenue corresponding to each of the upper bound and the lower bound, reducing the upper bound and increasing the lower bound accordingly.

[0028] On the other hand, an embodiment of the present invention further provides a power grid data purification system, comprising: a preprocessing module, used to input an initial data set into a data value assessment model constructed based on a deep learning algorithm to output a data value corresponding to each data point in the initial data set, wherein the initial data set includes several data points of multi-domain characteristic data about the power grid, and the data value is configured to show the probability of the corresponding data point being selected for training a prediction model, wherein the prediction model is constructed based on a power grid prediction target and is used to predict future power grid data through historical power grid data; an updating module, used to select a subset from the initial data set according to the ranking of the data value to train the prediction model and obtain corresponding prediction results, and if the prediction result fails to meet the power grid prediction target, update the data value assessment model based on a reinforcement learning algorithm until the selected subset enables the prediction result of the prediction model to meet the power grid prediction target; and a screening module, used to screen data points that meet the power grid prediction target according to the ranking of the data value to form a high-quality data set.

[0029] Optionally, the power grid data purification system also includes: an adjustment module, which is used to use a pre-built power grid decision model to evaluate the corresponding power grid benefit when the high-quality data set is used as input. If the expected benefit is not met, the data set size of the screening module for screening the data points is adjusted.

[0030] On the other hand, an embodiment of the present invention further provides an electronic device, comprising: a memory storing a program that can be run on a processor; and the processor, which is configured to implement any of the above-mentioned power grid data purification methods when executing the program.

[0031] On the other hand, an embodiment of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute any of the above-mentioned power grid data purification methods.

[0032] Through the above technical scheme, the embodiment of the present invention realizes a power grid data purification scheme based on deep learning and reinforcement learning, which can adapt to the power grid prediction objectives to select high-quality data sets, so as to match the flexibility requirements of power grid decision-making in a targeted manner, and through iterative learning to process multi-domain feature data, it avoids the impact of the sparsity of multi-domain feature data on the quality of data purification and reduces calculation errors.

[0033] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0035] Figure 1 It is a schematic diagram of the data purification process based on information entropy in the prior art;

[0036] Figure 2 is a flow chart of a method for purifying power grid data according to a first embodiment of the present invention;

[0037] Figure 3 is a schematic diagram of a process for implementing data purification in an example of Embodiment 1 of the present invention;

[0038] Figure 4 is a structural schematic diagram of a power grid data purification system according to a second embodiment of the present invention;

[0039] Figure 5 is a schematic structural diagram of an electronic device according to a third embodiment of the present invention;

[0040] Figure 6is a schematic diagram of an IEEE-30 node test system of application example 1 of the present invention;

[0041] Figure 7 It is a schematic diagram of experimental results of data purification applied to the IEEE-30 node test system according to the first application example of the present invention;

[0042] Figure 8 is a schematic diagram of an IEEE-118 node test system of application example 2 of the present invention; and

[0043] Fig. 9 It is a schematic diagram of experimental results of data purification applied to the IEEE-118 node test system according to the second application example of the present invention.

[0044] Description of Reference Numerals

[0045] 100, pre-processing module; 200, updating module; 300, screening module; 400, adjusting module. DETAILED DESCRIPTION

[0046] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0047] Before specifically introducing the embodiments of the present invention, some terms that will be involved are introduced here to facilitate a better understanding of the embodiments of the present invention.

[0048] 1. Data purification: refers to the process of selectively screening and using high-quality data through a series of technical means and methods in data processing and analysis, while eliminating low-quality data, data containing noise or deviations. Compared with conventional data screening, the data purification of the embodiment of the present invention is a further selection of high-quality data, which is equivalent to "purifying" on the basis of conventional data screening to obtain a more accurate and reliable data set. The data purification process is often used in machine learning, data science and other fields that require high-precision data support, and is a key step in achieving efficient data analysis and optimizing model performance.

[0049] 2. Data-driven: It means organizing massive amounts of data into information, then integrating and refining relevant information, and further training and fitting to form an automated decision-making model.

[0050] 3. Multi-field characteristic data: refers to data with characteristics of different fields. Taking the urban power grid as an example, different fields cover various aspects such as meteorology and energy, and its multi-field characteristic data includes multiple data with several characteristics based on different fields, such as meteorological data and energy data, where energy data includes wind power data and / or photovoltaic data. For example, there are 200 meteorological data, each of which has multiple characteristics such as wind speed, temperature, speed, etc., and there are 200 wind power data and photovoltaic data, each of which has multiple characteristics such as the number of units, unit start-up and shutdown costs, and power generation costs.

[0051] 4. Data value and data value assessment model: In the embodiment of the present invention, taking a data point (which can be understood as a piece of data) as an example, data value refers to the probability that the data point is selected for training the prediction model for the power grid. The higher the probability, the higher its value for power grid prediction. Accordingly, the data value assessment model is a model that can be pre-built based on a given training set and a deep learning algorithm, which aims to determine the data set to be further screened from the massive data of the initial data set through data value assessment.

[0052] 5. Prediction model: refers to a model that can be pre-built based on the power grid prediction target and is used to predict future power grid data through historical power grid data. In an embodiment of the present invention, the data points with higher data value determined by the aforementioned data value assessment model will be used to train the prediction model, and the prediction model will feed back the prediction effect to the data value assessment model to determine whether the data value assessment model needs to be updated.

[0053] 6. Grid decision model: It is a simulation model that reflects the connection and interaction relationship between the components in the grid by building data models of the components, devices and systems in the grid, such as the grid emergency dispatch model. The grid decision model can integrate the above prediction models and optimize the grid emergency dispatch decision based on its prediction results, including estimating the grid benefits of executing the grid emergency dispatch decision.

[0054] 7. Deep learning and reinforcement learning: In the embodiments of the present invention, deep learning is a machine learning method based on artificial neural networks, which learns and extracts high-order features of data through a multi-level neural network structure (i.e., a deep model), thereby achieving effective representation and classification of complex data; reinforcement learning is a goal-oriented way of learning and decision-making. Both deep learning and reinforcement learning are autonomous learning. The difference is that the former learns from a data set and then applies the learning to a new data set, while the latter learns dynamically by adjusting actions based on continuous feedback to get infinitely close to the goal. However, those skilled in the art should understand that deep learning and reinforcement learning are not mutually exclusive, and the combination of the two forms deep reinforcement learning (DRL). In the embodiments of the present invention, the DRL can be understood as both deep learning and reinforcement learning.

[0055] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0056] Embodiment 1

[0057] Figure 2 FIG. 1 is a flow chart of a method for purifying power grid data according to Embodiment 1 of the present invention. Figure 2 As shown, the power grid data purification method may include the following steps S100-S300.

[0058] Step S100, inputting the initial data set into a data value assessment model constructed based on a deep learning algorithm to output the data value corresponding to each data point in the initial data set.

[0059] The initial data set includes several data points of multi-domain characteristic data about the power grid, the data value is configured to show the probability of the corresponding data point being selected for training the prediction model, and the deep learning algorithm is, for example, a neural network algorithm.

[0060] For this step S100, for example, an initial data set containing multi-domain feature data is input into a data value assessment model constructed by a neural network to fully explore the data value characteristics of such multi-dimensional data. In addition, the data value output by the data value assessment model is continuous, for example, it can be any value between 0 and 0.99999, which makes it easy to sort by the specific value of the data value in subsequent data screening. However, considering that the output of the data value assessment model close to 1 indicates an increased probability of being selected as high-quality data, it will be more conducive to identifying high-quality data if its output is converted into discrete data in the form of 0, 1 (0 represents low quality, 1 represents high quality) based on a probability screening mechanism. Therefore, the output of the data value assessment model can also be discretized to obtain the data value in a discretized form, and the subsequent prediction model training and data value assessment model update are performed accordingly. This will also be described in detail in conjunction with examples below, so it will not be repeated here.

[0061] Step S200, according to the ranking of the data value, a subset is selected from the initial data set to train the prediction model and obtain the corresponding prediction result. When the prediction result fails to meet the power grid prediction target, the data value assessment model is updated based on the reinforcement learning algorithm until the selected subset can make the prediction result of the prediction model meet the power grid prediction target.

[0062] Among them, the power grid prediction target can also be understood as the prediction requirements pre-set for the prediction model. For example, the power grid prediction target is to minimize the prediction loss of the prediction model (or to achieve the set prediction accuracy). According to step S200, it is evaluated whether the prediction result of the prediction model can meet the corresponding power grid prediction target. If it cannot be met, the data value evaluation model is iteratively updated to obtain different data subsets to continue training the prediction model, that is, to continuously improve the accuracy of the data value output by the data value evaluation model. Among them, the reinforcement learning algorithm used for iteratively updating the data value evaluation model can be a policy gradient algorithm, which can update the data value evaluation model by optimizing the specified parameters of the data value evaluation model, which will be described in detail below through examples, and will not be repeated here. Here, by updating the data value evaluation model, it can output data values ​​that are more suitable for the current power grid prediction target and more accurate, thereby improving the prediction effect of the prediction model.

[0063] Step S300, for data points that meet the power grid prediction target, data points are screened according to the ranking of the data values ​​to form a high-quality data set.

[0064] For example, once the prediction target is met, the correspondence between the data points and their data values ​​will be retained, and the data values ​​will be sorted from high to low. A data screening mechanism will be implemented to determine the scale of the high-quality data set, such as screening out the top 10% of the data to form a high-quality data set for subsequent power grid emergency dispatch decisions.

[0065] Here, through steps S100-S300, taking the urban power grid as an example, an embodiment of the present invention implements a data purification scheme for the urban power grid based on deep learning and reinforcement learning, which can adapt to the power grid prediction target to select high-quality data sets, so as to match the flexibility requirements of the emergency dispatch of the urban power grid in a targeted manner, and through iterative learning to process multi-domain feature data, it avoids the impact of the sparsity of multi-domain feature data on the quality of data purification and reduces calculation errors.

[0066] However, the high-quality data set obtained through the above steps S100-S300 has not been evaluated for power grid benefits, and it is not possible to determine whether it is indeed a high-quality data set that optimizes power grid benefits. In this regard, in a preferred embodiment, the power grid data purification method further includes the following step S400.

[0067] Step S400, using a pre-built power grid decision model to evaluate the power grid revenue corresponding to the high-quality data set as input, if the expected revenue is not met, adjusting the size of the data set for data point screening.

[0068] For example, evaluating the power grid revenue includes: constructing the power grid decision model with the power grid revenue as the decision, and integrating the prediction model into the power grid decision model; inputting the high-quality data set into the prediction model to obtain the corresponding prediction result; and using the constructed power grid decision model to continue processing the obtained prediction result to evaluate the power grid revenue. Taking the power grid decision model as an urban power grid emergency dispatch model as an example, it is determined whether the urban power grid emergency dispatch revenue estimated based on the current high-quality data set meets the preset requirements. If not, the data set size for data point screening in step S300 is adjusted, and step S400 is returned to re-estimate the revenue. If it is satisfied, the iteration is completed, thereby achieving the acquisition of high-quality data sets and the improvement of the benefits of urban power grid emergency dispatch.

[0069] For further example, adjusting the data set size includes: initializing the upper and lower bounds of the data set size, wherein the upper and lower bounds are respectively the preset maximum percentage and the preset minimum percentage of the data set size; and when the difference between the upper and lower bounds is greater than a set threshold, adapting to the power grid benefits corresponding to the upper and lower bounds, correspondingly reducing the upper bound and increasing the lower bound. This adjustment scheme is performed, for example, by the golden section method, and will be described in conjunction with examples below, so it will not be repeated here.

[0070] The following further describes the implementation of the power grid data purification method according to the embodiment of the present invention with reference to examples. Figure 3 FIG. 1 is a flow chart of implementing data purification in an example of an embodiment of the present invention. Figure 3 , corresponding to the above steps S100-S400, this example is aimed at urban power grid emergency dispatch, and its process covers the complete path from data value assessment to decision optimization, which not only improves the accuracy of power grid prediction and decision-making, but also provides an efficient data-driven solution for urban power grid management.

[0071] Specifically, Figure 3 As shown, the process of this example may include the following steps S1-S7.

[0072] Step S1, obtaining an initial data set of several data points covering multi-domain characteristic data about a power grid.

[0073] In the example, this initial dataset can be written as of the form, where , is the input feature vector of the data point, is the feature vector to be predicted. Get the initial data set D , which can be used to extract high-quality subsets that are beneficial to the subsequent predictive modeling process.

[0074] Step S2, input the initial data set into the pre-built data value assessment model.

[0075] In the example, the data value assessment model receives the initial data set and outputs the data value of each data point in the initial data set selected for subsequent predictive modeling. That is, the data value assessment model implements the data value assessment of the initial data set, and in order to simplify the discrete data filtering process, the ability of neural networks to capture the laws of multi-domain feature data can be used to construct a data value assessment model. , whose output continuously indicates the selection probability of the data point. is capable of being optimized to update the data value assessment model The parameters will be introduced in detail below and will not be described here.

[0076] Furthermore, in the example, a probability-based Bernoulli probability mechanism is introduced Discretized data value assessment model The output is expressed as: . The operator " " represents mapping multiplication, It is a data value assessment model The output of is discretized to obtain the filter function, which is recorded as Through this filter function, the selection probability of the data point is interpreted as the data value in discretized form for the next step of predictive model training.

[0077] Step S3, construct a prediction model and determine whether its prediction result meets the prediction requirements. If not, execute step S4, otherwise continue to execute step S5.

[0078] In the example, the model is evaluated based on the data value The output data value selects a subset of the initial dataset for training the prediction model. This subset is the original data from the initial dataset and can still be used D It should be noted that the following is also expressed as raw data D And, further set the validation set of the prediction model , where the meaning of related parameters can be found in the above data set D In addition, the grid prediction target of the prediction model is, for example, to minimize the prediction loss of the prediction model, which is also the current prediction requirement. D , validation set In the case of power grid forecasting targets, the prediction model can be constructed by using the time series prediction method based on LightGBM. f θ , and determine whether it uses the data value assessment model Whether the filtered data set meets the prediction requirements, if so, continue with the next step of data purification process, otherwise optimize the data value assessment model Perform prediction iterations to ensure that the prediction requirements are finally met. Based on this, it can be summarized as training the prediction model through the following formula:

[0079] ,

[0080] ,

[0081] In the formula, Generally refers to the prediction model class that meets the requirements. represents the trained prediction model, represents the loss function of the prediction model, represents the power grid prediction target, D represents the raw data from the initial dataset, represents the validation set, , is the input feature vector of the data points in the corresponding dataset, and , yes , The corresponding feature vectors to be predicted are: It is a data value assessment model The output of is discretized to obtain the filter function, where is capable of being optimized to update the data value assessment model The parameter “st” represents the constraint condition.

[0082] Step S4, update the data value assessment model based on the reinforcement learning algorithm, and return to step S2.

[0083] In this example, the main goal of this step is to optimize the parameters , thereby minimizing the prediction loss in the process of predicting the prediction model, thereby improving the prediction performance, that is, each update selects the filter function Parameters Finally, the newly determined "high-quality" subset is used to train the prediction model. , this iterative training aims to minimize the prediction loss The ultimate goal is to reduce The prediction loss on . Wherein, the reinforcement learning algorithm can adopt a policy gradient algorithm, thereby updating the data value assessment model based on the following formula:

[0084]

[0085] In the formula, is the original data from the initial dataset D Medium Parameters The probability distribution of the selection of a specified vector under It means that by optimizing its parameters The updated data value assessment model, represents the optimization goal of executing the policy gradient algorithm, For the original data D Medium Parameters The specified vector in the prediction results of the prediction model under The probability distribution of choice, For parameters The gradient of . Furthermore, the vector Can be described as , each component of which represents the selection (1) or non-selection (0) of the corresponding data point, then for a specific vector The probability of selection is:

[0086] .

[0087] In the formula, the data point , For data points The input feature vector is For data points The feature vector to be predicted, Represents data points In the specified vector The corresponding amount.

[0088] Based on this, the original gradient calculation is transferred to the prediction model utility and selection vector The product of probability gradients can effectively realize the data value assessment model according to the chain rule of gradients. Updates.

[0089] Step S5, obtaining a high-quality data set based on a screening mechanism.

[0090] In this example, if the data value assessment model in the previous step converges, the data screening mechanism here sorts the data in descending order according to the data value (denoted as m ) to screen data points to form a high-quality data set. It should be noted that the sorting at this time is based on the continuous output of the data value assessment model, which can be any value between 0 and 0.99999, rather than the output after discretization, so that it is easy to achieve sorting based on the specific value of the data value.

[0091] Step S6, using the urban power grid emergency dispatch model to calculate the benefits.

[0092] In the example, the urban power grid emergency dispatch model is a part of the power grid decision model, which takes the power grid emergency dispatch profit as the decision, aims to optimize the power grid operation strategy, and is constructed through a variety of optimization algorithms such as stochastic optimization technology.

[0093] Step S7, determine whether the income meets the preset requirements, if so, complete the iteration, otherwise return to step S5 to adjust the data set size.

[0094] In this example, the segmentation search method is used to adjust the size of the data set. The corresponding data screening mechanism is described as follows:

[0095] 1) Initialize the upper bound of the dataset size and the lower bound ,in , for example, the upper bound is full selection, while the lower bound is partial selection.

[0096] 2) Calculate the benefits of urban power grid emergency dispatch decision-making under two data set sizes, namely, data set and and its corresponding benefits , ,inm Indicates the data value sorting.

[0097] 3) If and If the difference is less than the pre-set threshold, exit.

[0098] 4) Adjust the size of the dataset, and ,in Convergence coefficient, easy to know upper bound Reduce and lower bound Increase.

[0099] 5) Calculate the benefits of urban power grid emergency dispatch decision-making under two new data set sizes, namely, data set and and its corresponding benefits , .

[0100] 6) If , ;on the contrary, , and return to the previous step for judgment. In this way, the optimal scale convergence is achieved in an iterative manner by continuously reducing the upper and lower bounds, where and All located in and between, through and Corresponding to each and The mutual replacement between them can reduce the interval size and eventually achieve convergence.

[0101] Through the above examples, it can be seen that the first embodiment of the present invention implements a power grid data purification method based on deep reinforcement learning, which can implement flexible and differentiated data purification strategies according to specific decision-making goals such as the emergency dispatch benefits of urban power grids. Specifically, the execution of this data purification strategy based on deep reinforcement learning can efficiently and accurately mine the value patterns in multi-field feature big data, thereby obtaining targeted high-quality data sets, and significantly improving the application efficiency and practical utility of data-driven methods in urban power grids. That is, not only the data quality assessment process is optimized, but also the adaptability and accuracy of data processing in complex application environments are enhanced.

[0102] Embodiment 2

[0103] Based on the same inventive concept as the power grid data purification method of the above-mentioned embodiment 1, the second embodiment of the present invention provides a power grid data purification system. Figure 4As shown, the power grid data purification system includes: a preprocessing module 100, which is used to input the initial data set into a data value assessment model constructed based on a deep learning algorithm to output the data value corresponding to each data point in the initial data set, wherein the initial data set includes several data points of multi-domain characteristic data about the power grid, and the data value is configured to show the probability of the corresponding data point being selected for training a prediction model, wherein the prediction model is constructed based on a power grid prediction target and is used to predict future power grid data through historical power grid data; an updating module 200, which is used to select a subset from the initial data set according to the ranking of the data value to train the prediction model and obtain the corresponding prediction result, and in the case where the prediction result fails to meet the power grid prediction target, update the data value assessment model based on a reinforcement learning algorithm until the selected subset can make the prediction result of the prediction model meet the power grid prediction target; and a screening module 300, which is used to screen data points according to the ranking of the data value for data points that meet the power grid prediction target to form a high-quality data set.

[0104] In a preferred embodiment, corresponding to step S400 of embodiment one, the power grid data purification system may further include: an adjustment module 400, for using a pre-built power grid decision model to evaluate the corresponding power grid benefit when the high-quality data set is used as input, and if the expected benefit is not met, adjusting the data set size of the screening module 300 for screening the data points.

[0105] For more implementation details and effects of the power grid data purification system, reference may be made to the aforementioned embodiment 1 of the power grid data purification method, which will not be described in detail here.

[0106] Embodiment 3

[0107] Embodiment 3 of the present invention provides an electronic device, such as Figure 5 As shown, it includes: a memory storing a program that can be run on a processor; and the processor, which is configured to implement any of the above-mentioned power grid data purification methods when executing the program.

[0108] The electronic device is, for example, a computer or a server.

[0109] More implementation details and effects of the electronic device can be referred to the aforementioned embodiment 1, which will not be described in detail here.

[0110] It should be noted that the processor includes a kernel, and the kernel retrieves the corresponding program unit from the memory. One or more kernels can be set, and the corresponding power grid data purification is achieved by adjusting the kernel parameters. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0111] The rationality of the high-quality data set obtained by data purification in the above embodiment of the present invention will be specifically explained below through two application examples.

[0112] Application Example 1

[0113] In this application example 1, Figure 6 Taking the IEEE-30 node test system as an example, the problem of power grid data purification of thermal power units combined with wind power generation is discussed.

[0114] As an efficient and clean energy source, wind power can significantly improve the efficiency of urban power systems and reduce costs. However, the volatility and randomness of wind power generation also bring considerable challenges, increasing the uncertainty and complexity of power grid dispatch in emergency situations. In order to solve this problem, this example 1 adopts the data purification technology based on deep reinforcement learning in the above embodiment to achieve more accurate wind power forecasting, thereby improving the dispatch efficiency of urban power grids in response to emergencies.

[0115] Specifically, in terms of the prediction model, a time series prediction method based on LightGBM is used. The model uses data from the past 48 hours to predict wind power output within the next 24 hours, while combining key meteorological data (such as wind speed, temperature, and humidity) to improve the accuracy and reliability of the prediction. , expressed as the mean percentage error.

[0116] In terms of the power grid decision model, a Gaussian distribution model is constructed based on the wind power data output by the prediction model and combined with the historical prediction error. The Gaussian distribution model is integrated into the power grid decision model through random optimization technology to optimize the operation strategy of the power grid and deal with potential risks, ensuring the stability and security of the power grid. The specific optimization model is as follows:

[0117]

[0118] in G represents the set of all thermal power units, and Represents thermal power units exist A binary variable that is turned on or off at the moment, and Separate units i The startup and shutdown costs of Expressing hope, represents the randomness of wind power, It's the crew i exist t The amount of electricity generated at any time, is the power generation cost function, where and All parameters W It means that the overall first item is the economic cost of generating electricity. W represents all wind power generation nodes, is the error penalty term, and Represent the positive and negative errors of actual wind power and predicted wind power, respectively. This term mainly represents the safety cost of urban power grid dispatching. In addition, the model also needs to meet operation constraints, including start-stop constraints, minimum on / off time constraints, lifting and lowering power constraints, and power balance constraints.

[0119] In the urban power grid scheduling of Example 1, the data purification method based on deep reinforcement learning in the example of Example 1 of the present invention is used, wherein the specific parameters of the data purification method and the urban power grid decision model are shown in the following table.

[0120] Table 1 Data purification parameter settings for Example 1

[0121]

[0122] Table 2 Node parameter settings for Example 1

[0123]

[0124] Specific test results such as Figure 7As shown in the figure, it is shown that the data purification algorithm based on deep reinforcement learning has a significant regulatory effect on the emergency dispatch of the urban power grid in Example 1. As the amount of screened data decreases from 100%, that is, the data purification effect is enhanced, it can be observed that the power grid cost shows a significant downward trend. At the same time, the prediction error is also reduced. This finding verifies how the data-driven method can effectively reduce power grid costs and increase revenue in accurate prediction, and also highlights the key role of data purification in this process. The measurement results also show that the optimal high-quality data set size is about 60% of the original data set size, and a data set size that is too small will not lead to higher dispatch benefits. This is because when the data is over-screened, key feature data may be lost, resulting in insufficient data, which in turn affects the stability and effectiveness of the data-driven model. Therefore, it is crucial to reasonably control the degree of data purification to maintain model performance and accuracy.

[0125] Application Example 2

[0126] Continuing to focus on the emergency dispatch problem of urban power grid, the IEEE-118 node test system is selected as the research object of Example 2. Its structure is as follows Figure 8 As shown. Considering the need to cope with a wider range of application scenarios, multiple wind power output curves are introduced to enhance the system's ability to cope with different wind power generation conditions. At the same time, in order to meet more complex load requirements, more thermal power nodes are added in this example 2.

[0127] The specific experimental results are as follows Fig. 9 As shown, "+100%" means that no data purification technology is applied, while "+60%" means that after the data purification technology is applied, 60% of the original data set is retained for high-quality data. In contrast, "-60%" means that after the data purification technology is used, 40% of the high-quality data is removed, forming an original data set containing only 60% low-quality data. By comparing the grid costs under these different data application strategies, it can be clearly seen that: on the one hand, the high-quality data set obtained by using data purification technology can significantly reduce the cost of urban power grids in emergency dispatch and effectively improve the decision-making efficiency; on the other hand, the lack of high-quality data will lead to out-of-control of grid decisions, especially when facing problems such as uncertainty in wind power, showing poor decision-making performance, thereby increasing the operating cost of the grid. This finding emphasizes the importance of high-quality data in ensuring grid stability and optimizing emergency response.

[0128] Based on this, the calculation results of Example 1 and Example 2 not only confirm the practical value of the data purification method of the embodiment of the present invention, but also provide important technical support for the intelligent management of urban power grid data, which has far-reaching significance for improving the efficiency and accuracy of power grid decision-making and ensuring the stability of power grid operation. For example, it is significant in the following two aspects.

[0129] First, the proportion of renewable energy incorporated into urban power grids is increasing, and its inherent uncertainty poses a huge challenge to the safe and stable operation of the entire system. In this context, the data purification scheme of the embodiment of the present invention uses digital technologies such as reinforcement learning and deep learning to accurately predict the output of renewable energy, thereby effectively managing the uncertainty in the urban power grid, which plays a vital role in improving the efficiency of system decision-making.

[0130] In the second aspect, embodiments of the present invention relate to the value assessment of data sets, identifying and isolating those bad data elements that may affect the accuracy of analysis results and model performance. Through data purification, the accuracy and reliability of data sets can be guaranteed, thereby improving the effectiveness and efficiency of data-driven decision-making.

[0131] An embodiment of the present invention further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to enable a machine to execute the power grid data purification method described in any of the above embodiments. The machine is, for example, a computer or a server.

[0132] An embodiment of the present invention further provides a processor, which is used to run a program, wherein the power grid data purification method described in any of the above embodiments is executed when the program is run.

[0133] An embodiment of the present invention further provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing the steps of the power grid data purification method described in any of the above embodiments.

[0134] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0135] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0136] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0137] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0138] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0139] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0140] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0141] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0142] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for purifying power grid data, characterized in that: include: Input the initial data set into a pre-built data value assessment model based on a given training set and deep learning algorithm To output the data value corresponding to each data point in the initial data set, is capable of being optimized to update the data value assessment model The initial data set includes a plurality of data points of multi-domain characteristic data about a power grid, and the data values ​​are configured to show the probability of the corresponding data points being selected for training a prediction model, wherein the prediction model is constructed based on a power grid prediction target and is used to predict future power grid data through historical power grid data, wherein the power grid prediction target is to minimize the prediction loss of the prediction model; A subset is selected from the initial data set according to the ranking of the data value to train the prediction model and obtain the corresponding prediction results. If the prediction results fail to meet the power grid prediction target, the parameters are optimized based on the reinforcement learning algorithm. to update the data value assessment model until the selected subset enables the prediction result of the prediction model to meet the power grid prediction target, wherein the reinforcement learning algorithm is a policy gradient algorithm; and For data points that meet the power grid prediction target, screening the data points according to the ranking of the data values ​​to form a high-quality data set; The prediction model is trained based on the following formula: , , In the formula, Generally refers to the prediction model class that meets the requirements. represents the trained prediction model, represents the loss function of the prediction model, represents the power grid prediction target, D represents the raw data from the initial dataset, represents the validation set, , is the input feature vector of the data points in the corresponding dataset, and , yes , The corresponding feature vectors to be predicted are: It is a data value assessment model The output of is discretized to obtain the filter function, where is capable of being optimized to update the data value assessment model Parameters of Update the data value assessment model based on the following formula : In the formula, is the original data from the initial dataset D Medium Parameters The probability distribution of the selection of a specified vector under It means that by optimizing its parameters The updated data value assessment model, represents the optimization goal of executing the policy gradient algorithm, For the original data D Medium Parameters The prediction results of the prediction model under the specified vector The probability distribution of choice, For parameters The gradient of in, is represented as follows: In the formula, the data point , For data points The input feature vector is For data points The feature vector to be predicted, Represents data points In the specified vector The corresponding amount.

2. The power grid data purification method according to claim 1, characterized in that: The power grid is a city power grid, and the multi-field characteristic data of the city power grid includes meteorological data and energy data, wherein the energy data includes wind power data and / or photovoltaic data.

3. The power grid data purification method according to claim 1 or 2, characterized in that: The power grid data purification method further includes: A pre-built power grid decision model is used to evaluate the power grid benefit corresponding to the high-quality data set as input. If the expected benefit is not met, the size of the data set for data point screening is adjusted.

4. The method for purifying power grid data according to claim 3, characterized in that: Evaluating grid benefits includes: Constructing the power grid decision model by taking power grid revenue as decision, and integrating the prediction model into the power grid decision model; Inputting the high-quality data set into the prediction model to obtain corresponding prediction results; and The constructed power grid decision model is used to further process the obtained prediction results to evaluate the power grid benefits.

5. The method for purifying power grid data according to claim 3, characterized in that: Sizing the dataset includes: Initializing an upper bound and a lower bound of the size of the data set, wherein the upper bound and the lower bound are respectively a preset maximum percentage and a preset minimum percentage of the size of the data set; and When the difference between the upper bound and the lower bound is greater than a set threshold, the upper bound is reduced and the lower bound is increased accordingly according to the power grid revenues corresponding to the upper bound and the lower bound.

6. A power grid data purification system, characterized in that: include: The preprocessing module is used to input the initial data set into the data value assessment model pre-built based on the given training set and deep learning algorithm To output the data value corresponding to each data point in the initial data set, is capable of being optimized to update the data value assessment model The initial data set includes a plurality of data points of multi-domain characteristic data about a power grid, and the data values ​​are configured to show the probability of the corresponding data points being selected for training a prediction model, wherein the prediction model is constructed based on a power grid prediction target and is used to predict future power grid data through historical power grid data, wherein the power grid prediction target is to minimize the prediction loss of the prediction model; An updating module is used to select a subset from the initial data set according to the ranking of the data value to train the prediction model and obtain the corresponding prediction results, and when the prediction results fail to meet the power grid prediction target, optimize the parameters based on the reinforcement learning algorithm to update the data value assessment model until the selected subset enables the prediction result of the prediction model to meet the power grid prediction target, wherein the reinforcement learning algorithm is a policy gradient algorithm; as well as A screening module, for screening data points that meet the power grid prediction target according to the ranking of the data values ​​to form a high-quality data set; The prediction model is trained based on the following formula: , , In the formula, Generally refers to the prediction model class that meets the requirements. represents the trained prediction model, represents the loss function of the prediction model, represents the power grid prediction target, D represents the raw data from the initial dataset, represents the validation set, , is the input feature vector of the data points in the corresponding dataset, and , yes , The corresponding feature vectors to be predicted are: It is a data value assessment model The output of is discretized to obtain the filter function, where is capable of being optimized to update the data value assessment model Parameters of Update the data value assessment model based on the following formula : In the formula, is the original data from the initial dataset D Medium Parameters The probability distribution of the selection of a specified vector under It means that by optimizing its parameters The updated data value assessment model, represents the optimization goal of executing the policy gradient algorithm, For the original data D Medium Parameters The prediction results of the prediction model under the specified vector The probability distribution of choice, For parameters The gradient of in, is represented as follows: In the formula, the data point , For data points The input feature vector is For data points The feature vector to be predicted, Represents data points In the specified vector The corresponding amount.

7. The power grid data purification system according to claim 6, characterized in that: The power grid data purification system also includes: The adjustment module is used to use a pre-built power grid decision model to evaluate the power grid benefit corresponding to the high-quality data set as input, and if the expected benefit fails to meet the expected benefit, adjust the data set size for data point screening by the screening module.

8. An electronic device, characterized in that: include: A memory storing a program that can be executed on the processor; as well as The processor is configured to implement the power grid data purification method according to any one of claims 1 to 5 when executing the program.

9. A machine-readable storage medium having instructions stored thereon, the instructions being used to enable a machine to execute the power grid data purification method as described in any one of claims 1 to 5.

Citation Information

Patent Citations

  • A power network reliability prediction method

    CN109242150A

  • Comprehensive optimization method for power grid in high-permeability region of distributed power supply

    CN112507614A

  • Unit load and dynamic regulation capability online prediction method in automatic power generation control

    CN116258055A

  • Electric quantity prediction method based on neural network

    CN117439069A

  • Short-term wind power probability prediction method and system and storage medium

    CN117689082A