Short-term power load rapid prediction method based on spark framework

Abstract

The invention discloses a short-term power load rapid prediction method based on a spark framework. According to the method, two models are trained; a BIRCH parallelization algorithm is used to cluster historical load data and weather data to obtain a model for anomaly detection;, historical load data and weather data are trained through a light GBM algorithm based on the spark technology, a loadprediction model is obtained, and then the two models are sent to a Spark Streaming cluster to be used for clustering and predicting real-time data streams; in clustering and prediction of real-time data streams, a kafka cluster is used to receive power load data streams sent from various terminals; the data flow is transmitted to a Spark Steaming cluster to be processed; real-time feature extraction and normalization processing is completed on a Spark Steaming cluster, real-time clustering is performed by using an anomaly detection model to find out whether abnormal data exists or not, and then a load value of a next time period is predicted by using a load prediction model by using non-abnormal load data.

Description

technical field [0001] The invention relates to the technical field of electric load forecasting, in particular to a short-term fast electric load forecasting method based on a spark framework. Background technique [0002] With the development of the State Grid, the number of intelligent power consumption terminals and collection terminals is increasing day by day, and data acquisition is becoming more and more convenient, which makes various types of power automation data grow geometrically, showing "large volume" and "multiple types". ", "low density" and "fast growth" are typical characteristics of big data. [0003] Power system load forecasting is an important part of various safety technical measures of the power system. Like relay protection, stability calculation, and short-circuit calculation, it plays a very important role in the safe, economical and stable operation of the power system. Load forecasting is the basis of power grid planning and operation. Accurate...

AI Technical Summary

Problems solved by technology

For short-term load forecasting, most of the existing methods focus on the improvement or innovation of the forecasting model, such as using innovative machine learning models for power load forecasting, using improved machine learning models for power load forecasting etc. Although the accuracy of data-driven intelligent models is getting higher and higher, as the amount of data and data dimensions increase, the complexity and training time of the intelligent model will increase exponentially while the accuracy of the intelligent model increases, making it even difficult to converge.

Moreover, the traditional data management mode is inefficient when faced with the storage and processing of massive data. The traditional power load data analysis mode is still based on static offline data analysis. For example, the collected data is stored, and then periodically Analyzing data, this method is not closely integrated with the production and operation system, which is not conducive to quickly predicting the load status of the system and finding abnormal phenomena in time

Images