System and method for sensing and predicting real-time data traffic of power grid

Abstract

The invention discloses a power grid real-time data traffic sensing and predicting system and method, and belongs to the field of power grid monitoring. The invention discloses a power grid real-time data traffic sensing and predicting system and method. The system comprises a data sensing unit, a feature extraction unit, a data traffic predicting unit and a visualization unit. According to the method, the problem of prediction errors in the prior art is solved, real-time data traffic in a power grid system is sensed and preprocessed, then feature extraction is performed on the data traffic, the data traffic is divided into a training set and a test set, a machine learning algorithm is adopted to construct a real-time data traffic prediction model, and training and testing are performed. And the trained real-time data traffic prediction model is applied to practice, so that future data traffic information is provided for a power grid system, and meanwhile, the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction result of the future data traffic can be visually displayed through a visualization tool.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid monitoring, and in particular to a power grid real-time data flow perception and prediction system and method. Background Art

[0002] In future power grid systems, real-time data flow monitoring becomes an indispensable part, aiming to effectively cope with the ever-changing task load and volatility of real-time data flow. In power grid systems, monitoring real-time data flow can help predict the peak period of energy demand, so as to prepare for the adjustment of computing resources in advance. In smart grids, by monitoring the real-time data flow of users' electricity consumption behavior, the system can more flexibly respond to the spatiotemporal changes of users' electricity consumption and improve electricity efficiency. In terms of power grid security management, the real-time monitoring system can quickly detect abnormal events and improve the anti-attack and anti-fault capabilities of the power grid system. However, the existing power grid real-time data flow perception and prediction system has large prediction errors, such as the influence of external factors, and cannot provide reliable prediction results; therefore, it does not meet the existing needs. In this regard, we propose a power grid real-time data flow perception and prediction system and method. Summary of the invention

[0003] The purpose of the present invention is to provide a power grid real-time data flow perception and prediction system and method, which senses the real-time data flow in the power grid system and performs preprocessing, then extracts features of the data flow and divides it into a training set and a test set, and then uses a machine learning algorithm to build a real-time data flow prediction model and performs training and testing. The trained real-time data flow prediction model is applied in practice, thereby providing future data flow information for the power grid system. At the same time, the real-time data flow prediction model can be used to visualize the fitting effect of the monitored data and the prediction results of the future data flow through visualization tools, thereby solving the problems raised in the above-mentioned background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a real-time data flow perception and prediction system for a power grid, comprising:

[0005] Data sensing unit, used to:

[0006] Sense the real-time data traffic in the power grid system and pre-process the sensed data traffic;

[0007] Feature extraction unit, used to:

[0008] Extract features from the monitored data traffic and divide the extracted features;

[0009] The data traffic prediction unit is used to:

[0010] A machine learning algorithm is used to build a real-time data flow prediction model, which provides future data flow information for the power grid system.

[0011] Visualization unit for:

[0012] Visualize the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction results of future data traffic.

[0013] Furthermore, the data sensing unit includes:

[0014] Data monitoring module for:

[0015] Real-time monitoring of data traffic generated in the power grid system;

[0016] Data processing module for:

[0017] The monitored data traffic is preprocessed, including normalization and standardization processing as well as missing value and outlier processing.

[0018] Furthermore, the preprocessing is specifically as follows:

[0019] Normalization processing: used to scale the values ​​of data flow characteristics to the range of [0,1];

[0020] Standardization processing: used to convert data traffic characteristics into a standard normal distribution with zero mean and unit variance;

[0021] Missing value processing: delete the data flow where the missing value is located, or use interpolation method to fill the missing value;

[0022] Outlier processing: Delete outliers or use alternative values ​​to repair them.

[0023] Furthermore, the feature extraction unit comprises:

[0024] Feature extraction module for:

[0025] Perform feature extraction on the preprocessed data flow, and use the data flow after feature extraction to build a data flow database;

[0026] Data partitioning module, used to:

[0027] The data traffic in the data traffic database is divided into a training set and a test set, wherein the training set is used to train the real-time data traffic prediction model, and the test set is used to test the real-time data traffic prediction model.

[0028] Furthermore, the data flow database includes the following functions:

[0029] Periodic update function: used to set the time interval for periodic collection and collect new data flow information at the set time interval;

[0030] Backup and recovery function: used to regularly back up the data in the data traffic database and restore from the backup data when the original data is damaged or lost.

[0031] Furthermore, the data partitioning module is specifically:

[0032] Set the data division ratio, where the division ratio standard is 7:3 or 8:2;

[0033] All required data are extracted from the data flow database, and the data are allocated to the training set and the test set according to the set division ratio, wherein the data are allocated randomly.

[0034] Furthermore, the data flow prediction unit includes:

[0035] Model building modules for:

[0036] Use machine learning algorithms to build real-time data traffic prediction models;

[0037] Model training module, used to:

[0038] The real-time data traffic prediction model is trained using the training set. At the same time, cross-validation technology and hyperparameter tuning are used to optimize the performance of the real-time data traffic prediction model.

[0039] Model evaluation module for:

[0040] Use the test set to test the real-time data traffic prediction model, understand the overall performance of the intelligent detection model through testing, and further optimize the performance of the real-time data traffic prediction model based on the model test results;

[0041] Model application module for:

[0042] The trained real-time data flow prediction model is applied to practice, and the newly perceived data flow is predicted and analyzed through the real-time data flow prediction model, thereby providing future data flow information for the power grid system.

[0043] Furthermore, the cross-validation technique and hyperparameter tuning are specifically:

[0044] Cross-validation technology: using time series cross-validation and sliding window cross-validation;

[0045] Time series cross validation, which is used to divide the training set and test set into multiple sets according to the time sequence of the data;

[0046] Sliding window cross validation, in time series, uses a sliding window approach, each time sliding the window a certain step length, using previous data for training, while ensuring that the data in the test set is future observations;

[0047] Hyperparameter tuning: Optimize the performance of the real-time data traffic prediction model by adjusting hyperparameters. Hyperparameter tuning methods include grid search, random search, and Bayesian optimization.

[0048] Furthermore, the visualization unit provides visualization tools through Prophet, and uses line charts, bar charts, time series charts, heat maps and maps to visualize the model fitting effects and prediction results.

[0049] A method for implementing a real-time data flow perception and prediction system for a power grid comprises the following steps:

[0050] The real-time data flow in the power grid system is sensed through the data sensing unit, and the sensed data flow is normalized and standardized and missing values ​​and abnormal values ​​are processed;

[0051] The feature extraction unit extracts features from the monitored data traffic, and divides the data traffic after feature extraction into a training set and a test set;

[0052] The data traffic prediction unit uses machine learning algorithms to build a real-time data traffic prediction model;

[0053] The real-time data traffic prediction model is trained using the training set, and the real-time data traffic prediction model is tested using the test set;

[0054] Apply the trained real-time data flow prediction model to practice, and provide future data flow information for the power grid system through the real-time data flow prediction model;

[0055] The visualization unit uses the visualization tools provided by Prophet to visualize the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction results of future data traffic.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention senses the real-time data flow in the power grid system through a data sensing unit and performs preprocessing. The preprocessing can improve the quality of the data flow for subsequent feature extraction. The feature extraction unit extracts features from the data flow and divides it into a training set and a test set. The data flow prediction unit uses a machine learning algorithm to construct a real-time data flow prediction model and performs training and testing. The trained real-time data flow prediction model is applied in practice to provide future data flow information for the power grid system, thereby supporting intelligent scheduling and resource optimization of the power grid system. At the same time, the fitting effect of the real-time data flow prediction model on the monitored data and the prediction results of the future data flow can be visualized through visualization tools. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a structural schematic diagram of the power grid real-time data flow perception and prediction system of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] In order to solve the technical problem that the existing power grid real-time data flow perception and prediction system has large prediction errors, such as the influence of external factors, and cannot provide reliable prediction results, please refer to Figure 1 , this embodiment provides the following technical solutions:

[0061] A power grid real-time data flow perception and prediction system, comprising:

[0062] Data sensing unit, used to:

[0063] Sense the real-time data traffic in the power grid system and pre-process the sensed data traffic;

[0064] Feature extraction unit, used to:

[0065] Extract features from the monitored data traffic and divide the extracted features;

[0066] The data traffic prediction unit is used to:

[0067] A machine learning algorithm is used to build a real-time data flow prediction model, which provides future data flow information for the power grid system.

[0068] Visualization unit for:

[0069] Visualize the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction results of future data traffic.

[0070] The technical effect of the above content is: the real-time data flow in the power grid system is sensed and preprocessed through the data sensing unit. The preprocessing can improve the quality of the data flow, thereby improving the efficiency of subsequent feature extraction, so as to facilitate subsequent feature extraction. The feature extraction unit extracts features from the data flow and divides it into a training set and a test set. By extracting useful features, it is beneficial to the subsequent training and testing of the model. The data flow prediction unit uses a machine learning algorithm to construct a real-time data flow prediction model and performs training and testing. The trained real-time data flow prediction model is applied in practice to provide future data flow information for the power grid system, which helps power grid managers to better plan and manage their resources to avoid problems caused by sudden increases or decreases in data flow, thereby supporting intelligent scheduling and resource optimization of the power grid system. At the same time, the real-time data flow prediction model can be used to visualize the fitting effect of the monitored data and the prediction results of future data flow through visualization tools. The intuitive visualization is easy for staff to understand.

[0071] Data sensing unit, including:

[0072] Data monitoring module for:

[0073] Real-time monitoring of data traffic generated in the power grid system;

[0074] Data processing module for:

[0075] The monitored data traffic is preprocessed, including normalization and standardization processing as well as missing value and outlier processing.

[0076] Preprocessing, specifically:

[0077] Normalization processing: used to scale the values ​​of data flow characteristics to the range of [0,1];

[0078] Standardization processing: used to convert data traffic characteristics into a standard normal distribution with zero mean and unit variance;

[0079] Missing value processing: delete the data flow where the missing value is located, or use interpolation method to fill the missing value;

[0080] Outlier processing: Delete outliers or use alternative values ​​to repair them.

[0081] The technical effect of the above content is: the perception of real-time data flow is crucial in the power grid system. Through in-depth analysis of real-time data flow, the power grid system can better adapt to different workloads and make reasonable scheduling decisions to meet the power grid system's requirements for high real-time and reliability. The monitored data flow is preprocessed. Preprocessing is a key step in the development of real-time data flow prediction models. Its purpose is to improve the accuracy, stability and robustness of the model by effectively processing the raw data. Through normalization and standardization, the numerical range of different characteristics of the data flow can be ensured to be consistent, thereby improving the stability and convergence speed of model training. Missing value and outlier processing ensure the quality and consistency of the input data. Through preprocessing, the accuracy of the data can be improved, thereby improving the accuracy of the model prediction.

[0082] Feature extraction unit, including:

[0083] Feature extraction module for:

[0084] Perform feature extraction on the preprocessed data flow, and use the data flow after feature extraction to build a data flow database;

[0085] Data partitioning module, used to:

[0086] Dividing the data traffic in the data traffic database into a training set and a test set, wherein the training set is used to train the real-time data traffic prediction model, and the test set is used to test the real-time data traffic prediction model;

[0087] Among them, the data flow database includes the following functions:

[0088] Periodic update function: used to set the time interval for periodic collection and collect new data flow information at the set time interval;

[0089] Backup and recovery function: used to regularly back up the data in the data flow database and restore from the backup data when the original data is damaged or lost;

[0090] Data partitioning module, specifically:

[0091] Set the data division ratio, where the division ratio standard is 7:3 or 8:2;

[0092] All required data are extracted from the data flow database, and the data are allocated to the training set and the test set according to the set division ratio, wherein the data are allocated randomly.

[0093] The technical effect of the above content is: the feature extraction module extracts features from the preprocessed data flow to capture the key information therein, and the extracted features will be used to build a data flow database to facilitate subsequent model training and prediction. The data partitioning module will be responsible for partitioning the data flow in the data flow database for training and testing. This module will set a data partition ratio, such as 7:3 or 8:2, which means that part of the data will be used to train the model and the other part of the data will be used to test the model. Then, the module will extract all the data needed for training and testing from the database, and distribute the data to the training set and the test set according to the set ratio. This process is random to ensure the balance of the two sets of data sets. In addition, the functions of the data flow database include regular updates, which can ensure the timeliness of the data by setting regular time intervals. The functions of the data flow database also include backup and recovery functions, which can be restored from backup data in the event of data damage or loss, thereby ensuring the efficient, safe and reliable use of data.

[0094] The data traffic prediction unit includes:

[0095] Model building modules for:

[0096] Use machine learning algorithms to build real-time data traffic prediction models;

[0097] Model training module, used to:

[0098] The real-time data traffic prediction model is trained using the training set. At the same time, cross-validation technology and hyperparameter tuning are used to optimize the performance of the real-time data traffic prediction model.

[0099] Among them, cross-validation techniques: time series cross-validation and sliding window cross-validation are used;

[0100] Time series cross validation, which is used to divide the training set and test set into multiple sets according to the time sequence of the data;

[0101] Sliding window cross validation, in time series, uses a sliding window approach, each time sliding the window a certain step length, using previous data for training, while ensuring that the data in the test set is future observations;

[0102] Hyperparameter tuning: Optimize the performance of the real-time data traffic prediction model by adjusting hyperparameters. Hyperparameter tuning methods include grid search, random search, and Bayesian optimization.

[0103] Grid search, which is a commonly used hyperparameter tuning method by searching for the best combination in a predefined hyperparameter space;

[0104] Random search, compared to grid search, random search randomly selects combinations from the hyperparameter space, which can more efficiently find hyperparameter combinations with better performance;

[0105] Bayesian optimization is a hyperparameter tuning method based on a probability model. It gradually converges to the global optimal solution by continuously selecting the next set of possible hyperparameters for evaluation.

[0106] Model evaluation module for:

[0107] Use the test set to test the real-time data traffic prediction model, understand the overall performance of the intelligent detection model through testing, and further optimize the performance of the real-time data traffic prediction model based on the model test results;

[0108] Model application module for:

[0109] The trained real-time data flow prediction model is applied to practice, and the newly perceived data flow is predicted and analyzed through the real-time data flow prediction model, thereby providing future data flow information for the power grid system.

[0110] The technical effect of the above content is: the model construction module uses a machine learning algorithm to construct a real-time data flow prediction model. In practical applications, it is usually necessary to select a suitable algorithm according to the specific situation. Regression models, support vector machines and deep learning models each have their own advantages and disadvantages. Therefore, the choice of model should be based on the nature of the data, the complexity of the problem, and the availability of computing resources. Factors such as weighing, or using a model integration method to combine the prediction results of multiple models to obtain a more robust and accurate prediction. After the model is built, the real-time data flow prediction model can be trained and tested. Through training and testing, the real-time data flow prediction model is continuously adjusted and optimized, so that the real-time data flow prediction model has stronger generalization ability and better performance in practical applications. Finally, the model application module applies the trained real-time data flow prediction model to practice, so as to provide future data flow information for the power grid system, so as to facilitate intelligent scheduling and resource optimization of the power grid system.

[0111] The visualization unit provides visualization tools through Prophet, and uses line charts, bar charts, time series charts, heat maps and maps to visualize the model fitting effect and prediction results.

[0112] The technical effect of the above content is: through the various visualization tools and methods provided by Prophet, the staff can intuitively understand the fitting effect of the model's real-time data traffic prediction model on the monitored data and the prediction results of future data traffic. Through various visualization methods such as line charts, bar charts, time series charts, heat maps and maps, multiple dimensions and indicators of the model can be displayed at the same time, making the data richer and more three-dimensional.

[0113] Specifically, this embodiment also proposes an implementation method of a power grid real-time data flow perception and prediction system, comprising the following steps:

[0114] The real-time data flow in the power grid system is sensed through the data sensing unit, and the sensed data flow is normalized and standardized and missing values ​​and abnormal values ​​are processed;

[0115] The feature extraction unit extracts features from the monitored data traffic, and divides the data traffic after feature extraction into a training set and a test set;

[0116] The data traffic prediction unit uses machine learning algorithms to build a real-time data traffic prediction model;

[0117] The real-time data traffic prediction model is trained using the training set, and the real-time data traffic prediction model is tested using the test set;

[0118] Apply the trained real-time data flow prediction model to practice, and provide future data flow information for the power grid system through the real-time data flow prediction model;

[0119] The visualization unit uses the visualization tools provided by Prophet to visualize the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction results of future data traffic.

[0120] Working principle: The real-time data flow in the power grid system is sensed and preprocessed through the data sensing unit. The preprocessing can improve the quality of the data flow for subsequent feature extraction. The feature extraction unit extracts features from the data flow and divides it into training sets and test sets, which is conducive to the subsequent training and testing of the model. The data flow prediction unit uses machine learning algorithms to build a real-time data flow prediction model and conducts training and testing. The trained real-time data flow prediction model is applied in practice to provide future data flow information for the power grid system, thereby supporting the intelligent scheduling and resource optimization of the power grid system, and helping power grid managers to better plan and manage their resources. At the same time, the real-time data flow prediction model can be used through visualization tools to visualize the fitting effect of the monitored data and the prediction results of future data flow. The intuitive visualization is easy for staff to understand.

[0121] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0122] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. A power grid real-time data flow perception and prediction system, characterized in that: include: Data sensing unit, used to: Sense the real-time data traffic in the power grid system and pre-process the sensed data traffic; Feature extraction unit, used to: Extract features from the monitored data traffic and divide the extracted features; The data traffic prediction unit is used to: A real-time data flow prediction model is constructed using machine learning algorithms to provide future data flow information to the power grid system. Visualization unit for: Visualize the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction results of future data traffic.

2. A power grid real-time data flow perception and prediction system according to claim 1, characterized in that: The data sensing unit comprises: Data monitoring module for: Real-time monitoring of data traffic generated in the power grid system; Data processing module for: The monitored data traffic is preprocessed, including normalization and standardization processing as well as missing value and outlier processing.

3. A power grid real-time data flow perception and prediction system according to claim 2, characterized in that: The preprocessing is specifically as follows: Normalization: used to scale the value of data flow characteristics to the range of [0, 1]; Standardization processing: used to convert data traffic characteristics into a standard normal distribution with zero mean and unit variance; Missing value processing: delete the data flow where the missing value is located, or use interpolation method to fill the missing value; Outlier processing: Delete outliers or use alternative values ​​to repair them.

4. A power grid real-time data flow perception and prediction system according to claim 1, characterized in that: The feature extraction unit comprises: Feature extraction module for: Perform feature extraction on the preprocessed data flow, and use the data flow after feature extraction to build a data flow database; Data partitioning module, used to: The data traffic in the data traffic database is divided into a training set and a test set, wherein the training set is used to train the real-time data traffic prediction model, and the test set is used to test the real-time data traffic prediction model.

5. A power grid real-time data flow perception and prediction system according to claim 4, characterized in that: The data flow database includes: Used to set the time interval for regular collection and collect new data flow information at the set time interval; It is used to regularly back up the data in the data traffic database and restore it from the backup data when the original data is damaged or lost.

6. A power grid real-time data flow perception and prediction system according to claim 5, characterized in that: The data partitioning module is specifically: Set the data division ratio, where the division ratio standard is 7:3 or 8:2; All required data are extracted from the data flow database, and the data are allocated to the training set and the test set according to the set division ratio, where the data are allocated randomly.

7. A power grid real-time data flow perception and prediction system according to claim 1, characterized in that: The data flow prediction unit comprises: Model building modules for: Use machine learning algorithms to build real-time data traffic prediction models; Model training module, used to: The real-time data traffic prediction model is trained using the training set. At the same time, cross-validation technology and hyperparameter tuning are used to optimize the performance of the real-time data traffic prediction model. Model evaluation module for: Use the test set to test the real-time data traffic prediction model, understand the overall performance of the intelligent detection model through testing, and further optimize the performance of the real-time data traffic prediction model based on the model test results; Model application module for: The trained real-time data flow prediction model is applied to practice, and the newly perceived data flow is predicted and analyzed through the real-time data flow prediction model, thereby providing future data flow information for the power grid system.

8. A power grid real-time data flow perception and prediction system according to claim 7, characterized in that: The cross-validation technique and hyperparameter tuning are specifically: Cross-validation technology: using time series cross-validation and sliding window cross-validation; Time series cross validation, which is used to divide the training set and test set into multiple sets according to the time sequence of the data; Sliding window cross validation, in time series, uses a sliding window approach, each time sliding the window a certain step length, using previous data for training, while ensuring that the data in the test set is future observations; Hyperparameter tuning: Optimize the performance of the real-time data traffic prediction model by adjusting hyperparameters. Hyperparameter tuning methods include grid search, random search, and Bayesian optimization.

9. A power grid real-time data flow perception and prediction system according to claim 1, characterized in that: The visualization unit provides visualization tools through Prophet, and uses line charts, bar charts, time series charts, heat maps and maps to visualize the model fitting effect and prediction results.

10. A method for sensing and predicting real-time data flow in a power grid, implemented based on the real-time data flow sensing and prediction system for a power grid according to any one of claims 1 to 9, characterized in that: The following steps are involved: The real-time data flow in the power grid system is sensed through the data sensing unit, and the sensed data flow is normalized and standardized and missing values ​​and abnormal values ​​are processed; The feature extraction unit extracts features from the monitored data traffic, and divides the data traffic after feature extraction into a training set and a test set; The data traffic prediction unit uses machine learning algorithms to build a real-time data traffic prediction model; The real-time data traffic prediction model is trained using the training set, and the real-time data traffic prediction model is tested using the test set; Apply the trained real-time data flow prediction model to practice, and provide future data flow information for the power grid system through the real-time data flow prediction model; The visualization unit uses the visualization tools provided by Prophet to visualize the fitting effect of the real-time data traffic prediction model on the monitored data and the prediction results of future data traffic.

Images