Identification and correction method of bad data in new energy stations based on deep learning

Abstract

The present invention proposes a method for identifying and correcting bad data of a new energy station based on deep learning, including: obtaining historical operation data of an identification object in a new energy station, marking historical normal data and historical bad data in the historical operation data; establishing an identification model, and performing deep learning training on the identification model according to historical normal data; establishing a correction model, inputting historical bad data into the correction model and the trained identification model, and performing deep learning training on the correction model in combination with the output of the identification model; obtaining real-time operation data of the identification object, and distinguishing real-time normal data and real-time bad data in the real-time operation data by inputting the real-time operation data into the trained identification model; and inputting the real-time bad data into the trained correction model to obtain the correction value of the real-time bad data. The present invention can significantly improve the efficiency of bad data identification and correction, and ensure the real-time safe and stable operation of new energy power stations.

Description

Technical Field

[0001] The present invention belongs to the field of energy data management, and in particular relates to a method for identifying and correcting bad data of new energy stations based on deep learning. Background Art

[0002] With the continuous deepening and advancement of the construction of new energy stations, the data collection of new energy stations shows a trend of mass and high-dimensionality. At the same time, the problem of bad data is becoming increasingly prominent. The real-time data collected by new energy stations often have bad data such as missing, invalid, repeated and wrong. Bad data is usually caused by two reasons: one is the failure of the power system of new energy, such as the temporary interruption of a data channel in the data collection system, which leads to untrue data; the other is due to special events such as sudden and accidental fluctuations of some large industrial loads and sudden adverse environments, which cause irregular oscillations in the data. The existence of bad data distorts the state estimation results of new energy stations, affects the operation and dispatching and stable operation of the power system, and may even cause unknown safety consequences.

[0003] The amount of various types of data in new energy stations is huge, and there are many relationships between the data. The data of stations, units, environments, etc. are coupled with each other, and there is also a coupling relationship between their internal data. With the continuous development of modern information technology, artificial intelligence has been applied to various fields, among which deep learning has been widely used in new energy pattern recognition, classification and load forecasting scenarios. Deep learning has a strong ability to adapt to the time-varying regularity of time series, has memory and association functions for historical information, and can continuously learn massive and coupled data. Therefore, deep learning can be used to identify and correct bad data in new energy stations to ensure the safe and stable operation of new energy stations. Summary of the invention

[0004] In order to solve the shortcomings and deficiencies in the prior art, the present invention proposes a method for identifying and correcting bad data of new energy stations based on deep learning, including:

[0005] S100: Acquire historical operation data of the identified object in the new energy station, and mark historical normal data and historical bad data in the historical operation data;

[0006] S200: Establishing an identification model, and performing deep learning training on the identification model based on historical normal data;

[0007] S300: Establish a correction model, input historical bad data into the correction model and the trained identification model, and perform deep learning training on the correction model in combination with the output of the identification model;

[0008] S400: acquiring real-time operation data of the identification object, and distinguishing real-time normal data and real-time bad data in the real-time operation data by inputting the real-time operation data into a trained identification model;

[0009] S500: Input the real-time defect data into the trained correction model to obtain a correction value of the real-time defect data.

[0010] Optionally, the identification objects include wind power generation parameters, photovoltaic power generation parameters, and installed capacity and active power of units in new energy stations;

[0011] Wherein, the wind power generation parameters include wind speed, temperature, wind direction cosine value, humidity and pressure;

[0012] The photovoltaic power generation parameters include irradiation intensity, irradiation duration and component area.

[0013] Optionally, the S200 includes:

[0014] S210: dividing the historical normal data into training data and test data in time series order, and initializing the hyperparameters of the recognition model;

[0015] S220: inputting the training data into the recognition model for training, and calculating the predicted value of the recognition object in the time series corresponding to the test data through the recognition model;

[0016] S230: Calculate whether the convergence accuracy of the identification model meets a preset threshold. The calculation formula of the convergence accuracy is:

[0017]

[0018] Among them, a is the convergence accuracy, n is the total number of predicted values, and x fi is the i-th test data, x i is the i-th predicted value;

[0019] S240: If the convergence accuracy meets the preset conditions, the training ends; otherwise, the hyperparameters of the identification model are adjusted, and S220-S230 are repeated until the convergence accuracy does not exceed the preset conditions.

[0020] Optionally, the method further includes: before S200, preprocessing the historical operation data, including:

[0021] Identify missing values ​​in historical operation data, obtain historical operation data belonging to the same category as the missing value, obtain an interpolation value for the missing value by calculating the same category mean of the missing value, and replace the missing value with the interpolation value. The calculation formula for the interpolation value is:

[0022]

[0023] Among them, a i is the mean coefficient. When the historical running data s of the i-th input i If missing, it is 0, otherwise it is 1. m is the total amount of historical operation data of the same type. is the interpolation value.

[0024] Optionally, the S300 includes:

[0025] S310: Initialize the hyper parameters of the identification model;

[0026] S320: Inputting the historical bad data into the trained recognition model, and taking the predicted value of the historical bad data output by the recognition model as the accurate value;

[0027] S330: inputting the historical bad data into the correction model for training, analyzing the characteristics of the historical bad data through the correction model, and outputting the correction value of the historical bad data according to the analysis result;

[0028] S340: Analyze the error degree of the corrected value relative to the accurate value and the error dispersion degree. When the analysis result meets the preset conditions, end the training. Otherwise, adjust the hyperparameters of the correction model and repeat S320-S330 until the preset conditions are met.

[0029] Optionally, the error degree and error dispersion degree of the analyzed correction value relative to the accurate value include:

[0030] Analyze the degree of error by calculating the mean absolute error of the corrected value relative to the accurate value;

[0031] The error dispersion is analyzed by calculating the root mean square deviation of the corrected value relative to the accurate value.

[0032] Optionally, the method further includes:

[0033] While executing S500, a preset proportion of normal data is selected and input into the correction model, and the mean absolute error and root mean square error between the output value of the correction model and the normal data are calculated;

[0034] When any one of the mean absolute error and the root mean square error does not meet the preset condition, the real-time operation data input when executing S400 is used as training data to retrain the identification model and the correction model.

[0035] Optionally, the identification model is provided with a solver in the output layer, and the solver is a softmax function.

[0036] Optionally, the solver outputs the probability of real-time bad data by comparing the calculated prediction value with the error of the actual measured real-time operation data, and outputs the identified real-time bad data and the time series position of the real-time bad data according to the probability.

[0037] The beneficial effects brought by the technical solution provided by the present invention are:

[0038] The present invention combines deep learning to establish and jointly train identification models and correction models, and uses the obtained models to perform real-time and rapid identification and correction of bad data collected in new energy sites, which can significantly improve the efficiency of bad data identification and correction, support real-time analysis applications of new energy power stations, and ensure real-time, safe and stable operation of new energy power stations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0040] Figure 1 A flowchart of a method for identifying and correcting bad data of a new energy station based on deep learning proposed in an embodiment of the present invention;

[0041] Figure 2 A line graph showing the correlation between the number of neurons in a neural network and the convergence accuracy. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein.

[0044] It should be understood that in various embodiments of the present invention, the size of the sequence number of each process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0045] It should be understood that in the present invention, "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products or apparatuses.

[0046] It should be understood that in the present invention, "plurality" refers to two or more than two. "And / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "Contains A, B and C", "Contains A, B, C" means that A, B, and C are all included, "Contains A, B or C" means that one of A, B, and C is included, and "Contains A, B and / or C" means that any one, any two, or any three of A, B, and C are included.

[0047] It should be understood that in the present invention, "B corresponding to A", "B corresponding to A", "A corresponds to B" or "B corresponds to A" means that B is associated with A and B can be determined based on A. Determining B based on A does not mean determining B based only on A, but B can also be determined based on A and / or other information. A and B match when the similarity between A and B is greater than or equal to a preset threshold.

[0048] Depending on the context, "if" as used herein may be interpreted as "when" or "when" or "in response to determining" or "in response to detecting."

[0049] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0050] like Figure 1 As shown, this embodiment proposes a method for identifying and correcting bad data of new energy stations based on deep learning, including:

[0051] S100: Acquire historical operation data and real-time operation data of the identified object in the new energy station, and mark historical normal data and historical bad data in the historical operation data;

[0052] S200: Establishing an identification model, and performing deep learning training on the identification model based on historical normal data;

[0053] S300: Establish a correction model, input historical bad data into the correction model and the trained identification model, and perform deep learning training on the correction model in combination with the output of the identification model;

[0054] S400: Differentiate between normal real-time data and bad real-time data in the real-time operation data by inputting the real-time operation data into a trained recognition model;

[0055] S500: Input the real-time defect data into the trained correction model to obtain a correction value of the real-time defect data.

[0056] This embodiment uses a deep neural network algorithm to learn and train the historical data collected by the new energy power station to obtain a deep neural network identification model that meets the accuracy requirements; the next step is to input the real-time collected data into the identification model to obtain the predicted value of the deep neural network, take the predicted value as the accurate value, and set the deviation threshold. By comparing the real collected data with the accurate value, if it exceeds the threshold range, it is regarded as bad data, and finally the collected bad data is identified in real time. Through the above process, deep learning has a strong adaptability to the time-varying regularity characteristics of time series, has memory and association functions for historical information, and can continuously learn massive and coupled data, thereby realizing real-time and rapid identification and correction of bad data, which can significantly improve the efficiency of bad data identification and correction, support the real-time analysis application of new energy power stations, and ensure the real-time safe and stable operation of new energy power stations.

[0057] In this embodiment, the identification objects include wind power generation parameters, photovoltaic power generation parameters, and installed capacity and active power of units in new energy stations; wherein the wind power generation parameters include wind speed, temperature, wind direction cosine value, humidity and pressure; the photovoltaic power generation parameters include irradiation intensity, irradiation duration and component area.

[0058] In this embodiment, both historical operation data and real-time operation data are collected and acquired through the SCADA system. Specifically, the SCADA system collects data once every 15 minutes. Because the collected data may have some obvious data errors due to uncontrollable random failures in the SCADA system, which will affect the accuracy of the subsequent training model, this embodiment pre-processes the historical operation data and the real-time operation data before S200, including:

[0059] Identify missing values ​​in historical operation data, obtain historical operation data belonging to the same category as the missing value, obtain an interpolation value for the missing value by calculating the same category mean of the missing value, and replace the missing value with the interpolation value. The calculation formula for the interpolation value is:

[0060]

[0061] in, is the interpolation value, a i is the mean coefficient. When the i-th input historical operation data or real-time operation data s i It is 0 when missing, otherwise it is 1. m is the total amount of historical operation data or real-time operation data of the same type.

[0062] After data preprocessing, this embodiment obtains 35040 sample data sets with a time span of 1 year. According to the time coherence characteristics, the last 1000 sample data sets are intercepted as real-time recognition data sets, and the remaining sample data sets are used as historical operation data to build and train recognition models and correction models, and historical normal data and historical bad data are marked in the historical operation data in advance based on experience.

[0063] First, the recognition model is trained, and the step S200 includes:

[0064] S210: dividing the historical normal data into training data and test data in time series order, and initializing the hyperparameters of the recognition model;

[0065] S220: inputting the training data into the recognition model for training, and calculating the predicted value of the recognition object in the time series corresponding to the test data through the recognition model;

[0066] S230: Calculate whether the convergence accuracy of the identification model meets a preset threshold. The calculation formula of the convergence accuracy is:

[0067]

[0068] Among them, a is the convergence accuracy, n is the total number of predicted values, and x fi is the i-th test data, x i is the i-th predicted value;

[0069] S240: If the convergence accuracy meets the preset conditions, the training ends; otherwise, the hyperparameters of the identification model are adjusted, and S220-S230 are repeated until the convergence accuracy does not exceed the preset conditions.

[0070] In this embodiment, the hyperparameters of the identification model are first set, including parameters such as the initial network structure, network threshold, and weight. The identification problem of bad data is essentially a classification problem, so the output layer of the identification model is provided with a solver and uses a softmax function, and the activation function uses a sigmoid function. The solver outputs the probability of real-time bad data by comparing the error between the calculated prediction value and the actual measured real-time operation data, and outputs the identified real-time bad data and the time series position of the real-time bad data according to the probability.

[0071] Specifically, during the training process, the identification model predicts the running data of the identification object in the subsequent time series through the training data. Here, the subsequent time series is consistent with the time series corresponding to the test data, and then the identification accuracy of the identification model is judged according to the error between the predicted value and the test data. When S400 is subsequently executed, the identification model predicts the next time series based on the real-time running data of the previous time series, and compares the predicted result with the real-time running data of the next time series, that is, the measured value of the identification object in the next time series. The identification model sets a floating threshold through the solver of the output layer, and judges the accurate value of the output and the corresponding measured value according to the threshold Δe. If it exceeds the threshold range, it is regarded as bad data. The threshold Δe is:

[0072]

[0073] x max is the maximum measured value, x min is the minimum measured value.

[0074] Since the number of samples used in this embodiment is relatively large, considering the accuracy and processing speed, the use of double hidden layers is sufficient to meet the requirements. Figure 2 The line graph of the correlation of the convergence accuracy of bad data identification corresponding to the number of neurons of 5, 10, 15, 20, 25, and 30 is shown. The horizontal axis is the number of neurons and the vertical axis is the corresponding convergence accuracy. For the convenience of comparison, the number of neurons in each layer is the same. During the training process, it was found that with the increase of the number of neurons, the simulation time almost doubled, and the accuracy of the identification model gradually increased. After the number of hidden layer nodes of the identification model was 20 and 20 respectively, the identification results were not significantly improved. This is because the deep model has been increasing and tending to be stable, which shows that the performance of the identification model is gradually optimized with the increase of hidden layers. According to the comprehensive consideration of the relationship between model time and performance, this paper selects the number of hidden layer nodes of the identification model as 20, and the final bad data results are shown in Table 1. It can be seen from Table 1 that the model accuracy of each type of data reaches more than 97%, indicating that the accuracy of using the identification model to identify different types of bad data is high and the convergence is good.

[0075] Table 1

[0076]

[0077]

[0078] After the training of the identification model is completed, the correction model is trained. In this embodiment, the correction model is a BP neural network, and the trainbr algorithm is used. This algorithm has better functional ability and faster convergence speed than the basic gradient algorithm, and is more suitable for mutually coupled data sets.

[0079] Specifically, the S300 includes:

[0080] S310: Initialize the hyperparameters of the recognition model, that is, determine the neural network structure, network threshold and weight;

[0081] S320: After normalization, the historical bad data is input into the trained recognition model, and the predicted value of the historical bad data output by the recognition model is used as the accurate value. Specifically, the historical bad data is normalized using the mapminmax function in Matlab;

[0082] S330: Inputting historical bad data into the correction model, analyzing the characteristics of the historical bad data through the correction model, and outputting correction values ​​for the historical bad data according to the analysis results;

[0083] S340: Analyze the error degree of the corrected value relative to the accurate value and the error dispersion degree. When the analysis result meets the preset conditions, end the training. Otherwise, adjust the hyperparameters of the correction model, that is, adjust the neural network structure, network threshold and weights, and repeat S320-S330 until the preset conditions are met.

[0084] In this embodiment, the hyperparameters of the recognition model are first set, including parameters such as the initial network structure, network threshold, and weights. For the selection of the number of hidden layer nodes and the number of hidden layers of the neural network, if the number of hidden layer nodes is too small, the network cannot have the necessary learning ability and information processing ability. On the contrary, if the number of hidden layer nodes is too large, not only will the network become more complex and the processing speed slower, but the network will also be more likely to fall into local minima during the learning process. Therefore, this embodiment takes into account that when the output of the recognition model is multiple, the use of more than two layers of hidden layers is better for the recognition and modification of bad data, and the number of nodes in each layer and other parameters are obtained during training.

[0085] The error degree and error dispersion degree of the analysis correction value relative to the accurate value include:

[0086] The error degree is analyzed by calculating the mean absolute error (MAE) of the corrected value relative to the accurate value. The specific calculation formula is:

[0087]

[0088] y i is the i-th correction value, y ti is the i-th accurate value, and l is the total number of corrected values.

[0089] The error dispersion degree is analyzed by calculating the root mean square error (RMSE) of the corrected value relative to the accurate value. The specific calculation formula is:

[0090]

[0091] Specifically, during the training process of the correction model, the error degree of the correction value relative to the accurate value and the error dispersion degree are shown in Table 2. The values ​​of the correction evaluation indicators RMSE and MAE are very small, which are relatively close to the actual values, indicating that this method has a good correction effect on various types of bad data of new energy stations and meets the preset conditions. Therefore, the training of the correction model is completed.

[0092] Table 2

[0093] Data Types RMSE MAE Active Power 0.47706 0.07211 Wind speed 0.51107 0.05084 Wind direction cosine 0.17631 0.13896 temperature 0.08850 0.00889 pressure 0.35447 0.05220 humidity 0.41174 0.03381

[0094] In order to cope with the random changes in the operation of new energy stations, this embodiment also includes: while executing S500, selecting a preset proportion of normal data to input into the correction model, calculating the mean absolute error and root mean square error between the output value of the correction model and the normal data; when any of the mean absolute error and the root mean square error does not meet the preset conditions, the real-time operation data input when executing S400 is used as training data to retrain the identification model and the correction model. Through the above process, the hyperparameters of the identification model and the correction model are adjusted in time to meet the accuracy optimization of the identification model and the correction model.

[0095] In this embodiment, while the real-time normal data remains unchanged and the real-time bad data is corrected, a part of the real-time normal data is also input into the trained correction model for correction. The corrected related error indicators are statistically compared to obtain the corrected related error indicators. If the correction evaluation indicators RMSE and MAE both meet the preset conditions, it means that the current model accuracy meets the requirements, and the hyperparameters of the identification model and the correction model do not need to be adjusted for the time being.

[0096] The serial numbers in the above embodiments are for description only and do not represent the sequence of the components during assembly or use.

[0097] The above descriptions are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying and correcting bad data in new energy stations based on deep learning, characterized in that: include: S100: Acquire historical operation data and real-time operation data of the identified object in the new energy station, and mark historical normal data and historical bad data in the historical operation data; S200: Establishing an identification model, and performing deep learning training on the identification model based on historical normal data; S300: Establish a correction model, input historical bad data into the correction model and the trained identification model, and perform deep learning training on the correction model in combination with the output of the identification model; S400: Differentiate between normal real-time data and bad real-time data in the real-time operation data by inputting the real-time operation data into a trained recognition model; S500: inputting the real-time bad data into the trained correction model to obtain the correction value of the real-time bad data; The S300 includes: S310: Initialize the hyperparameters of the identification model; S320: After normalization, the historical bad data is input into the trained recognition model, and the predicted value of the historical bad data output by the recognition model is used as the accurate value; S330: inputting the historical bad data into the correction model for training, analyzing the characteristics of the historical bad data through the correction model, and outputting the correction value of the historical bad data according to the analysis result; S340: Analyze the error degree and error dispersion degree of the corrected value relative to the accurate value. When the analysis result meets the preset conditions, the training is terminated. Otherwise, the hyperparameters of the correction model are adjusted, and S320-S330 are repeated until the preset conditions are met. The error degree and error dispersion degree of the analysis correction value relative to the accurate value include: Analyze the degree of error by calculating the mean absolute error of the corrected value relative to the accurate value; The error dispersion is analyzed by calculating the root mean square deviation of the corrected value relative to the accurate value.

2. According to the method for identifying and correcting bad data of new energy stations based on deep learning in claim 1, it is characterized in that: The identification objects include wind power generation parameters, photovoltaic power generation parameters, and installed capacity and active power of units in new energy stations; Wherein, the wind power generation parameters include wind speed, temperature, wind direction cosine value, humidity and pressure; The photovoltaic power generation parameters include irradiation intensity, irradiation duration and component area.

3. According to the method for identifying and correcting bad data of new energy stations based on deep learning in claim 1, it is characterized in that: The method further includes: before S200, preprocessing the historical operation data and the real-time operation data, including: Identify missing values ​​in historical operation data, obtain historical operation data belonging to the same category as the missing value, obtain an interpolation value for the missing value by calculating the same category mean of the missing value, and replace the missing value with the interpolation value. The calculation formula for the interpolation value is: ； in, is the interpolation value, is the mean coefficient. When the i-th input historical operation data or real-time operation data It is 0 when missing, otherwise it is 1. m is the total amount of historical operation data or real-time operation data of the same type.

4. According to claim 1, a method for identifying and correcting bad data of new energy stations based on deep learning is characterized in that: The S200 includes: S210: dividing the historical normal data into training data and test data in time series order, and initializing the hyperparameters of the recognition model; S220: inputting the training data into the recognition model for training, and calculating the predicted value of the recognition object in the time series corresponding to the test data through the recognition model; S230: Calculate whether the convergence accuracy of the identification model meets a preset threshold. The calculation formula of the convergence accuracy is: ； in, is the convergence accuracy, n is the total number of predicted values, is the i-th test data, is the i-th predicted value; S240: If the convergence accuracy meets the preset conditions, the training ends; otherwise, the hyperparameters of the identification model are adjusted, and S220-S230 are repeated until the convergence accuracy does not exceed the preset conditions.

5. According to claim 1, a method for identifying and correcting bad data of new energy stations based on deep learning is characterized in that: The method further comprises: While executing S500, a preset proportion of normal data is selected and input into the correction model, and the mean absolute error and root mean square error between the output value of the correction model and the normal data are calculated; When any one of the mean absolute error and the root mean square error does not meet the preset condition, the real-time operation data input when executing S400 is used as training data to retrain the identification model and the correction model.

6. According to the method of claim 1, which is characterized by: The identification model is provided with a solver at the output layer, and the solver is a softmax function.

7. A method for identifying and correcting bad data of new energy stations based on deep learning according to claim 6, characterized in that: The solver outputs the probability of real-time bad data by comparing the calculated prediction value with the error of the actually measured real-time operation data, and outputs the identified real-time bad data and the time series position of the real-time bad data according to the probability.

Images