Energy storage power station data cleaning system method, system, equipment and medium
By establishing an evaluation model and a variety of cleaning methods, the data imbalance of energy storage power stations is solved, and the data balance effect and analysis accuracy are improved.
Patent Information
- Application Number
- CN202510625531.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-15
- Publication Date
- 2025-08-26
AI Technical Summary
In the prior art, the data on the source grid and load side of the energy storage power station are missing or imbalanced due to mechanical or human reasons, resulting in incomplete and inaccurate refinement identification, which affects the potential assessment of distributed energy storage power stations.
Establish an evaluation model, determine the data set through unbalanced evaluation value, set different cleaning thresholds and cleaning methods, including deletion of duplicate data, filling missing values, data normalization and neural network model processing, and adapting to different data sets for cleaning.
It realizes effective balanced processing of energy storage power station data, and improves the accuracy and refined identification capabilities of subsequent analysis.
Smart Images
Figure CN120541373A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of energy storage power station data processing technology, and in particular to a data cleaning system method, system, equipment, and medium for an energy storage power station. Background Art
[0002] Energy storage power stations are systems that store, convert, and release cyclical electrical energy through electrochemical cells or electromagnetic energy storage media. Due to various factors, including mechanical and human factors, data on the source, grid, and load sides can become missing or imbalanced. This directly leads to incomplete and inaccurate refined identification, which in turn affects the subsequent potential assessment of distributed energy storage power stations. Therefore, it is necessary to appropriately balance and complete the data on the source, grid, and load sides.
[0003] In existing technologies, data class imbalance is typically addressed by building a library of methods for tackling multi-source heterogeneous data class imbalance on the source, grid, and load sides. Undersampling and oversampling are used to balance data classifications. However, this data cleaning approach is overly simplistic and ineffective in balancing and completing data, resulting in inaccurate analysis results. Summary of the Invention
[0004] The purpose of the present invention is to provide a data cleaning system method, system, equipment and medium for energy storage power station to solve the above problems in the prior art.
[0005] The present invention is achieved through the following technical solutions:
[0006] In a first aspect, the present invention provides a method for cleaning data of an energy storage power station, comprising:
[0007] Obtain several types of data sets of the current energy storage power station, establish an evaluation model, perform imbalance evaluation on each type of data set, and obtain an imbalance evaluation value for each data set;
[0008] Obtaining a comprehensive imbalance evaluation value of the current energy storage power station based on the plurality of imbalance evaluation values, setting a first evaluation threshold and a second evaluation threshold respectively, and determining whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold;
[0009] If not, the data set smaller than the second evaluation threshold is cleaned using the first data cleaning method, and the data set greater than or equal to the second evaluation threshold is cleaned using the second data cleaning method;
[0010] If so, the data set with a value less than the second evaluation threshold is cleaned using the second data cleaning method, and the data set with a value greater than or equal to the second evaluation threshold is cleaned using the third data cleaning method;
[0011] The first data cleaning method, the second data cleaning method and the third data cleaning method are three different data cleaning methods, and the number of cleaning steps gradually increases.
[0012] Preferably, the establishing of the evaluation model includes:
[0013] The data set includes a plurality of parameter values collected at different times, and the number of missing values in the data set is obtained;
[0014] Establish an evaluation function:
[0015]
[0016] Where, E k is the imbalance evaluation value of the kth data set, N c is the total number of parameters in the data set, N l is the number of missing values, X i is the value of the i-th parameter, X a is the average value of the parameters of the current data set, E o is the comprehensive imbalance evaluation value, and T is the total number of data set types of the current energy storage power station.
[0017] Preferably, the setting of the first evaluation threshold and the second evaluation threshold includes:
[0018] The first evaluation threshold includes:
[0019]
[0020] The first evaluation threshold includes:
[0021] E k,α =0.95E o,α
[0022] Where, E o,α is the first evaluation threshold, E k,α is the second evaluation threshold.
[0023] Preferably, the first data cleaning method includes:
[0024] Determine whether there is complete duplicate data in the data set, where the complete duplicate data includes the acquisition time being completely consistent with the acquisition data value. If so, delete it; if not, do not process it.
[0025] Preferably, the second data cleaning method includes:
[0026] Get the missing values in the current data set, delete the missing values, and set the filling start threshold;
[0027] If the number of data in the current data set is greater than the filling start threshold, the average value of the two adjacent data of the current missing value is obtained and the average value is used to fill the gap of the missing value;
[0028] If the number of data in the current data set is not greater than the filling start threshold, no processing is performed.
[0029] Preferably, the third data cleaning method includes:
[0030] Normalize the data in the data set, build a neural network model, obtain historical data sets, train the neural network model with the historical data sets, and output the trained neural network model;
[0031] Input the data in the data set into the neural network model, obtain the predicted value at the current moment, calculate the difference between the data in the data set and the predicted value, and set the difference threshold. If the difference is less than or equal to the difference threshold, it is judged as normal data. If the difference is greater than the difference threshold, it is judged as an abnormal parameter.
[0032] After replacing the abnormal parameters, data cleaning is completed.
[0033] Preferably, the established neural network model includes a TCN model and an LSTM model, wherein the TCN model is used to extract hidden features in the measurement data, and the LSTM model is used to predict data:
[0034] The TCN model includes dilated convolution, which includes:
[0035]
[0036] The LSTM model includes:
[0037]
[0038] Where F(s) is the output value after convolution calculation, f(a) is the weight of the convolution kernel at the ath position, and x s-d*a is the sd*ath element of the corresponding convolution kernel, d is the expansion factor, K is the convolution kernel size, h t is the forward parameter, h ′ t is the reverse parameter, h() and g() are activation functions, ω1, ω2, ω3, ω4 and ω5 are weight parameters, x t is the input value at time t, y t is the output value at time t, h t-1 is the positive parameter of the previous moment, h t+1 is the reverse parameter at the next moment.
[0039] In a second aspect, the present invention further provides a data cleaning system for an energy storage power station, comprising:
[0040] a data acquisition module configured to acquire several types of data sets of the current energy storage power station, establish an evaluation model, perform an imbalance evaluation on each type of data set, and obtain an imbalance evaluation value for each data set;
[0041] The data cleaning module is configured to obtain a comprehensive imbalance evaluation value of the current energy storage power station based on a plurality of imbalance evaluation values, set a first evaluation threshold and a second evaluation threshold respectively, and determine whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold; if not, clean the data set less than the second evaluation threshold using a first data cleaning method, and clean the data set greater than or equal to the second evaluation threshold using a second data cleaning method; if so, clean the data set less than the second evaluation threshold using a second data cleaning method, and clean the data set greater than or equal to the second evaluation threshold using a third data cleaning method; the first data cleaning method, the second data cleaning method, and the third data cleaning method are three different data cleaning methods, and the number of cleaning steps gradually increases;
[0042] The main control device is connected to the data acquisition module and the data cleaning module, and is used to execute the above-mentioned energy storage power station data cleaning method.
[0043] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned energy storage power station data cleaning system method when executing the computer program.
[0044] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned energy storage power station data cleaning system method when executed by a processor.
[0045] The technical solution of the present invention has at least the following advantages and beneficial effects:
[0046] The method provided by the present invention mainly includes establishing an evaluation model, performing imbalance evaluation on each type of data set, and obtaining an imbalance evaluation value for each data set; obtaining a comprehensive imbalance evaluation value of the current energy storage power station based on several imbalance evaluation values, setting a first evaluation threshold and a second evaluation threshold respectively, judging whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold, and whether the imbalance evaluation value is approximately the second evaluation value, and providing different data cleaning methods respectively. In this way, more complex data cleaning can be performed on data in relatively poor conditions to obtain better results, and data in good conditions only requires simple data cleaning. In this way, different data are adapted respectively by multiple data cleaning methods, and finally a data group with better balancing effect is obtained, making subsequent analysis results more accurate. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.
[0048] Figure 1 It is a control flow diagram of the present invention;
[0049] Figure 2 Schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION
[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.
[0051] The terms "first," "second," and so on, in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. The naming or numbering of steps in this application does not necessarily imply that the steps in the method flow must be executed in the chronological or logical order indicated by the naming or numbering. Named or numbered process steps may be executed in a different order based on the desired technical objectives, as long as the same or similar technical effects are achieved.
[0052] Independently described modules or submodules may or may not be physically separate; they may be implemented in software or hardware. Some modules or submodules may be implemented in software, with the processor invoking the software to implement the functionality of these modules or submodules, while other modules or submodules may be implemented in hardware, such as hardware circuits. Furthermore, some or all of the modules may be selected based on actual needs to achieve the objectives of the present application.
[0053] Please refer to Figure 1-Figure 2 The present invention provides a method for cleaning data of an energy storage power station, comprising:
[0054] S101: Acquire several types of data sets of the current energy storage power station, establish an evaluation model, perform imbalance evaluation on each type of data set, and obtain an imbalance evaluation value for each data set;
[0055] Energy storage power stations collect a variety of data, including voltage parameters, operating temperature parameters, output power parameters, state of charge parameters, and so on. When collecting these parameters, some invalid data is inevitable. These invalid data include obvious abnormal data, missing values, repeated values at the same time, and so on. These invalid data cause an imbalance in the data set, that is, cause errors in subsequent analysis.
[0056] S102: Obtaining a comprehensive imbalance evaluation value of the current energy storage power station based on the plurality of imbalance evaluation values, setting a first evaluation threshold and a second evaluation threshold respectively, and determining whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold;
[0057] By judging the comprehensive imbalance evaluation value and the first evaluation threshold, the overall imbalance of all data sets of the energy storage power station is reflected. Based on the current overall situation, the situation of each type of data set is judged, and different data cleaning methods are given for data cleaning.
[0058] S103: If not, the data set smaller than the second evaluation threshold is cleaned using the first data cleaning method, and the data set greater than or equal to the second evaluation threshold is cleaned using the second data cleaning method;
[0059] S104: If yes, the data set smaller than the second evaluation threshold is cleaned using the second data cleaning method, and the data set greater than or equal to the second evaluation threshold is cleaned using the third data cleaning method;
[0060] S105: The first data cleaning method, the second data cleaning method and the third data cleaning method are three different data cleaning methods, and the number of cleaning steps gradually increases.
[0061] The method provided by the present invention mainly includes establishing an evaluation model, performing imbalance evaluation on each type of data set, and obtaining an imbalance evaluation value for each data set; obtaining a comprehensive imbalance evaluation value of the current energy storage power station based on several imbalance evaluation values, setting a first evaluation threshold and a second evaluation threshold respectively, judging whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold, and whether the imbalance evaluation value is approximately the second evaluation value, and providing different data cleaning methods respectively. In this way, more complex data cleaning can be performed on data in relatively poor conditions to obtain better results, and data in good conditions only requires simple data cleaning. In this way, different data are adapted respectively by multiple data cleaning methods, and finally a data group with better balancing effect is obtained, making subsequent analysis results more accurate.
[0062] In an exemplary embodiment of the present invention, establishing the evaluation model includes:
[0063] The data set includes a plurality of parameter values collected at different times, and the number of missing values in the data set is obtained;
[0064] Establish an evaluation function:
[0065]
[0066]
[0067] Where, E k is the imbalance evaluation value of the kth data set, N c is the total number of parameters in the data set, N l is the number of missing values, X i is the value of the i-th parameter, X a is the average value of the parameters of the current data set, E o is the comprehensive imbalance evaluation value, and T is the total number of data set types of the current energy storage power station.
[0068] In this scheme, since missing values not only account for the number of a numerical value, but the specific value is also 0, which has a greater impact on the entire data set, the main consideration is the impact of missing values on the imbalance of the entire data set. The more missing values there are, the greater the overall imbalance evaluation value.
[0069] Secondly, setting the first evaluation threshold and the second evaluation threshold includes:
[0070] The first evaluation threshold includes:
[0071]
[0072] The first evaluation threshold includes:
[0073] Ek,α =0.95E o,α
[0074] Where, E o,α is the first evaluation threshold, E k,α is the second evaluation threshold.
[0075] Of course, the first evaluation threshold and the second evaluation threshold can also be set to specific fixed values according to the specific situation. In this solution, the advantage of setting them as floating thresholds is that the more missing values there are, the smaller the threshold is. This is equivalent to increasing the impact of the number of missing values on the judgment and improving the accuracy as much as possible.
[0076] In an exemplary embodiment of the present invention, the first data cleaning method includes:
[0077] Determine whether there is complete duplicate data in the data set, where the complete duplicate data includes the acquisition time being completely consistent with the acquisition data value. If so, delete it; if not, do not process it.
[0078] In terms of missing data mechanism, missing data can be divided into three categories: missing completely at random, missing at random, and missing not completely at random. Different methods are used to deal with different categories. Direct deletion is the most direct way to deal with missing data.
[0079] Secondly, the second data cleaning method includes:
[0080] Get the missing values in the current data set, delete the missing values, and set the filling start threshold;
[0081] If the number of data in the current data set is greater than the filling start threshold, the average value of the two adjacent data of the current missing value is obtained and the average value is used to fill the gap of the missing value;
[0082] If the number of data in the current data set is not greater than the filling start threshold, no processing is performed.
[0083] In an exemplary embodiment of the present invention, the third data cleaning method includes the first data cleaning method and the second data cleaning method, and further includes:
[0084] Normalize the data in the data set, build a neural network model, obtain historical data sets, train the neural network model with the historical data sets, and output the trained neural network model;
[0085] Input the data in the data set into the neural network model, obtain the predicted value at the current moment, calculate the difference between the data in the data set and the predicted value, and set the difference threshold. If the difference is less than or equal to the difference threshold, it is judged as normal data. If the difference is greater than the difference threshold, it is judged as an abnormal parameter.
[0086] Among them, regarding the setting of the threshold, the measurement accuracy parameter and the average relative error can be selected for comprehensive judgment.
[0087] After replacing the abnormal parameters, data cleaning is completed. The replacement method can be achieved using conventional technical means.
[0088] Specifically, the neural network model includes a TCN model and an LSTM model. The TCN model is used to extract hidden features in the measurement data, and the LSTM model is used to predict the data:
[0089] The TCN model includes dilated convolution, which includes:
[0090]
[0091] The LSTM model includes:
[0092]
[0093] Where F(s) is the output value after convolution calculation, f(a) is the weight of the convolution kernel at the ath position, and x s-d*a is the sd*ath element of the corresponding convolution kernel, d is the expansion factor, K is the convolution kernel size, h t is the forward parameter, h ′ t is the reverse parameter, h() and g() are activation functions, ω1, ω2, ω3, ω4 and ω5 are weight parameters, x t is the input value at time t, y t is the output value at time t, h t-1 is the positive parameter of the previous moment, h t+1 is the reverse parameter at the next moment.
[0094] Specifically, the TCN model consists of causal convolution, dilated convolution, and residual connection modules. Causal convolution ensures the causal relationship between the input and output sequences, preventing interference from future data, so that the model's output at a given moment depends only on current and past inputs. Dilated convolution solves the linear superposition problem caused by information capture and uses interval reading to access data, expanding the receptive field and capturing more historical information. As a result, it can capture higher-dimensional feature information from data at different time intervals.
[0095] The neural network model constructed using the above scheme includes normalization of historical measurement data of different dimensions in the database to ensure effective neural network training. Then, the historical measurement sequence data from time k+1-T to time k is input into the neural network in a sliding window format, and the measurement data at time k+1 is output. The TCN parameters are set as follows: number of layers, number of convolution kernels, 64, kernel size, and dilation factors of 1, 2, 4, 8, and 16, respectively. The BILSTM network parameters are set as follows: number of layers, number of neurons, 100, activation function, Adam optimizer, learning rate, 0.001, loss function, 150 iterations, and batch size, 64.
[0096] A data cleaning system for an energy storage power station, comprising:
[0097] a data acquisition module configured to acquire several types of data sets of the current energy storage power station, establish an evaluation model, perform an imbalance evaluation on each type of data set, and obtain an imbalance evaluation value for each data set;
[0098] The data cleaning module is configured to obtain a comprehensive imbalance evaluation value of the current energy storage power station based on a plurality of imbalance evaluation values, set a first evaluation threshold and a second evaluation threshold respectively, and determine whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold; if not, clean the data set less than the second evaluation threshold using a first data cleaning method, and clean the data set greater than or equal to the second evaluation threshold using a second data cleaning method; if so, clean the data set less than the second evaluation threshold using a second data cleaning method, and clean the data set greater than or equal to the second evaluation threshold using a third data cleaning method; the first data cleaning method, the second data cleaning method, and the third data cleaning method are three different data cleaning methods, and the number of cleaning steps gradually increases;
[0099] The main control device is connected to the data acquisition module and the data cleaning module, and is used to execute the above-mentioned energy storage power station data cleaning method.
[0100] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0101] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, optical disks, and other media that can store program code.
[0102] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.
Claims
1. A method for cleaning data of an energy storage power station, characterized in that: include: Obtain several types of data sets of the current energy storage power station, establish an evaluation model, perform imbalance evaluation on each type of data set, and obtain an imbalance evaluation value for each data set; Obtaining a comprehensive imbalance evaluation value of the current energy storage power station based on the plurality of imbalance evaluation values, setting a first evaluation threshold and a second evaluation threshold respectively, and determining whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold; If not, the data set smaller than the second evaluation threshold is cleaned using the first data cleaning method, and the data set greater than or equal to the second evaluation threshold is cleaned using the second data cleaning method; If so, the data set with a value less than the second evaluation threshold is cleaned using the second data cleaning method, and the data set with a value greater than or equal to the second evaluation threshold is cleaned using the third data cleaning method; The first data cleaning method, the second data cleaning method and the third data cleaning method are three different data cleaning methods, and the number of cleaning steps gradually increases.
2. The energy storage power station data cleaning method according to claim 1, characterized in that: The establishment of the evaluation model comprises: The data set includes a plurality of parameter values collected at different times, and the number of missing values in the data set is obtained; Establish an evaluation function: Where, E k is the imbalance evaluation value of the kth data set, N c is the total number of parameters in the data set, N l is the number of missing values, X i is the value of the i-th parameter, X a is the average value of the parameters of the current data set, E o is the comprehensive imbalance evaluation value, and T is the total number of data set types of the current energy storage power station.
3. The energy storage power station data cleaning method according to claim 2, characterized in that: The setting of the first evaluation threshold and the second evaluation threshold includes: The first evaluation threshold includes: The first evaluation threshold includes: E k,α =0.95E o,α Where, E o,α is the first evaluation threshold, E k,α is the second evaluation threshold.
4. The energy storage power station data cleaning method according to claim 3, characterized in that: The first data cleaning method includes: Determine whether there is complete duplicate data in the data set, where the complete duplicate data includes the acquisition time being completely consistent with the acquisition data value. If so, delete it; if not, do not process it.
5. The energy storage power station data cleaning method according to claim 4, characterized in that: The second data cleaning method includes: Get the missing values in the current data set, delete the missing values, and set the filling start threshold; If the number of data in the current data set is greater than the filling start threshold, the average value of the two adjacent data of the current missing value is obtained and the average value is used to fill the gap of the missing value; If the number of data in the current data set is not greater than the filling start threshold, no processing is performed.
6. The energy storage power station data cleaning method according to claim 5, characterized in that: The third data cleaning method includes: Normalize the data in the data set, build a neural network model, obtain historical data sets, train the neural network model with the historical data sets, and output the trained neural network model; Input the data in the data set into the neural network model, obtain the predicted value at the current moment, calculate the difference between the data in the data set and the predicted value, and set the difference threshold. If the difference is less than or equal to the difference threshold, it is judged as normal data. If the difference is greater than the difference threshold, it is judged as an abnormal parameter. After replacing the abnormal parameters, data cleaning is completed.
7. The energy storage power station data cleaning method according to claim 6, characterized in that: The neural network model includes a TCN model and an LSTM model. The TCN model is used to extract hidden features in the measurement data, and the LSTM model is used to predict data: The TCN model includes dilated convolution, which includes: The LSTM model includes: Where F(s) is the output value after convolution calculation, f(a) is the weight of the convolution kernel at the ath position, and x s-d*a is the sd*ath element of the corresponding convolution kernel, d is the expansion factor, K is the convolution kernel size, h t is the forward parameter, h′ t is the reverse parameter, h() and g() are activation functions, ω1, ω2, ω3, ω4 and ω5 are weight parameters, x t is the input value at time t, y t is the output value at time t, h t-1 is the positive parameter of the previous moment, h t+1 is the reverse parameter at the next moment.
8. A data cleaning system for an energy storage power station, characterized in that: include: a data acquisition module configured to acquire several types of data sets of the current energy storage power station, establish an evaluation model, perform an imbalance evaluation on each type of data set, and obtain an imbalance evaluation value for each data set; The data cleaning module is configured to obtain a comprehensive imbalance evaluation value of the current energy storage power station based on a plurality of imbalance evaluation values, set a first evaluation threshold and a second evaluation threshold respectively, and determine whether the comprehensive imbalance evaluation value is greater than or equal to the first evaluation threshold; if not, clean the data set less than the second evaluation threshold using a first data cleaning method, and clean the data set greater than or equal to the second evaluation threshold using a second data cleaning method; if so, clean the data set less than the second evaluation threshold using a second data cleaning method, and clean the data set greater than or equal to the second evaluation threshold using a third data cleaning method; the first data cleaning method, the second data cleaning method, and the third data cleaning method are three different data cleaning methods, and the number of cleaning steps gradually increases; A main control device is connected to the data acquisition module and the data cleaning module, and is used to execute the energy storage power station data cleaning method according to any one of claims 1 to 7.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for cleaning energy storage power station data according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for cleaning a system for cleaning energy storage power stations according to any one of claims 1 to 7 is implemented.