Energy demand prediction system based on big data analysis

Abstract

The invention discloses an energy demand prediction system based on big data analysis. The system comprises a data acquisition module used for acquiring historical energy data of a selected area in a preset time period; the feature engineering module is used for performing feature selection on the historical energy data to obtain core features; the data expansion module is used for obtaining an initial data set according to the core features and expanding the initial data set to obtain a training data set; the model construction module is used for constructing an energy demand prediction model and performing training based on the training data set; the demand prediction module is used for acquiring real-time energy data and predicting energy demand based on the energy demand prediction model and the real-time energy consumption data. According to the method, the accuracy and efficiency of energy demand prediction are improved by comprehensively utilizing the big data technology and the machine learning algorithm, and the method is of great significance to resource optimization configuration and sustainable development of the energy industry.

Description

Technical Field

[0001] The present invention belongs to the technical field of energy supply, and in particular relates to an energy demand prediction system based on big data analysis. Background Art

[0002] Energy demand forecasting is a core part of many fields such as power systems and urban energy planning. By accurately forecasting energy demand, it can help relevant decision makers to make more reasonable resource allocation and management, reduce operating costs, and ensure system stability and security.

[0003] In today's era of rapid development of computing power and big data, the application of machine learning and artificial intelligence in many fields, including energy demand forecasting, has become more and more extensive. In particular, the emergence of methods such as deep learning and neural networks has enabled high-dimensional and nonlinear problems to be effectively handled. At the same time, fast and robust classifier methods such as naive Bayes and extreme learning machines have also greatly promoted progress in this field.

[0004] Energy demand may be affected by factors such as energy markets, industrial production, seasons, weather, holidays, etc., showing obvious temporal volatility. In addition, the amount of data generated by the energy industry is huge. Therefore, it is urgent to comprehensively consider the temporal volatility of energy demand and the huge amount of data, and design an energy demand forecasting system based on big data analysis. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes an energy demand forecasting system based on big data analysis to solve the problems existing in the above-mentioned prior art.

[0006] To achieve the above objectives, the present invention provides an energy demand forecasting system based on big data analysis, comprising:

[0007] Data acquisition module, feature engineering module, model building module, demand forecasting module;

[0008] The data acquisition module is used to acquire historical energy data within a preset time period of a selected area;

[0009] The feature engineering module is used to perform feature selection on historical energy data to obtain core features;

[0010] The data expansion module is used to obtain an initial data set according to the core features, and expand the initial data set to obtain a training data set;

[0011] The model building module is used to build an energy demand prediction model and perform training based on the training data set;

[0012] The demand forecasting module is used to obtain real-time energy data and forecast energy demand based on the energy demand forecasting model and the real-time energy consumption data;

[0013] The data acquisition module, the feature engineering module, the data expansion module, the model building module and the demand forecasting module are connected in sequence.

[0014] Optionally, the historical energy data acquired by the data acquisition module includes meteorological data, energy market data, economic indicators, industrial production data, regional equipment utilization rate, regional equipment aging rate, social activity information and energy consumption data.

[0015] Optionally, the feature engineering module includes a preprocessing unit, a feature screening unit, and a feature selection unit;

[0016] The preprocessing unit is used to delete abnormal data and erroneous data in the historical energy data, and merge the historical energy data into a unified data framework;

[0017] The feature screening unit is used to analyze the pre-processed historical energy data using descriptive statistics and EDA, and determine the initial key feature range according to the analysis results; calculate the correlation coefficients of different variables in the historical energy data, identify the correlation between the variables, and select features that meet the correlation requirements based on the correlation to obtain multiple key features;

[0018] The feature selection unit is used to further screen multiple key features through a decision tree to obtain core features.

[0019] Optionally, the preprocessing unit adjusts the time resolution of the historical energy data by resampling, fills in missing values ​​by linear interpolation, aligns the time indexes of the historical energy data, and merges the historical energy data into a unified data framework.

[0020] Optionally, the data expansion module includes an expansion unit and a verification unit;

[0021] The expansion unit is used to perform two sample expansions on the initial data set;

[0022] The verification unit is used to verify whether the two sample expansions meet the preset conditions. If not, the initial data set is selected as the training data set. If so, the results of the two sample expansions are merged to obtain the training data set.

[0023] Optionally, the expansion unit uses a Borderline-SMOTE algorithm and a nonlinear interpolation method to expand the initial data set to obtain a first expanded data set and a second expanded data set.

[0024] Optionally, the model building module includes a data conversion unit, a model building unit, and a training unit;

[0025] The data conversion unit is used to perform one-dimensional vector conversion on the data in the training data set;

[0026] The model building unit is used to build an energy demand prediction model;

[0027] The training unit is used to divide the training data set into equal intervals according to set time intervals, input the training data set segments into the energy demand prediction model, determine the deviation value between the model output value and the energy consumption value of the next segment, and adjust the model parameters of the energy demand prediction model based on the deviation value until the preset requirements are met.

[0028] Optionally, the model building unit uses a convolutional neural network as a basic model, introduces a causal convolutional network structure and a residual network structure into the basic model for optimization, and obtains an energy demand prediction model.

[0029] Optionally, after the training unit divides the training data set into equal intervals according to set time intervals, each group of data sets obtained after segmentation is split into channels based on core features, and inputs the channels into the energy demand prediction model.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] The data acquisition module of the present invention is used to obtain historical energy data within a preset time period of a selected area; the feature engineering module is used to perform feature selection on the historical energy data to obtain core features; the data expansion module is used to obtain an initial data set based on the core features, expand the initial data set, and obtain a training data set; the model construction module is used to construct an energy demand prediction model and perform training based on the training data set; the demand prediction module is used to obtain real-time energy data and predict energy demand based on the energy demand prediction model and real-time energy consumption data.

[0032] The present invention obtains a variety of historical energy data including meteorological data, energy market data, economic indicators, industrial production data, etc., which is helpful for more comprehensive analysis and prediction of energy demand; by effectively preprocessing, feature screening and selection of historical energy data, the core features that have an important impact on the prediction of energy demand are obtained; the initial data set is expanded, and the BorderlineSMOTE algorithm and nonlinear interpolation method are used to increase data diversity and improve the generalization ability of the model; a convolutional neural network is used as the basic model, and a causal convolutional network structure and a residual network structure are introduced for optimization, which is helpful for dealing with high-dimensional and nonlinear problems and improving the accuracy of prediction; the present invention improves the accuracy and efficiency of energy demand prediction by comprehensively utilizing big data technology and machine learning algorithms, which is of great significance to the optimal resource allocation and sustainable development of the energy industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0034] Figure 1 Schematic diagram of the system structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Embodiment 1

[0038] like Figure 1 As shown, this embodiment provides an energy demand prediction system based on big data analysis, including:

[0039] Data acquisition module, feature engineering module, model building module, demand forecasting module;

[0040] The data acquisition module is used to acquire historical energy data within a preset time period of a selected area;

[0041] In some specific embodiments, the historical energy data acquired by the data acquisition module includes meteorological data, energy market data, economic indicators, industrial production data, regional equipment utilization rate, regional equipment aging rate, social activity information and energy consumption data.

[0042] Specifically, meteorological data includes temperature, humidity, wind speed, etc.; economic indicators include GDP growth rate, population growth rate; social activity information includes holidays, special events or emergencies.

[0043] The feature engineering module is used to perform feature selection on historical energy data to obtain core features;

[0044] In some specific embodiments, the feature engineering module includes a preprocessing unit, a feature screening unit, and a feature selection unit;

[0045] The preprocessing unit is used to delete abnormal data and erroneous data in the historical energy data, and merge the historical energy data into a unified data framework;

[0046] The feature screening unit is used to analyze the pre-processed historical energy data using descriptive statistics and EDA, and determine the initial key feature range according to the analysis results; calculate the correlation coefficients of different variables in the historical energy data, identify the correlation between the variables, and select features that meet the correlation requirements based on the correlation to obtain multiple key features;

[0047] Specifically, descriptive statistical analysis is performed on the data set to calculate statistics such as the mean, median, standard deviation, minimum value, and maximum value in order to understand the basic distribution characteristics of the data.

[0048] Specifically, correlation coefficients between different variables, such as Pearson correlation coefficient, Spearman rank correlation, etc., are calculated to identify linear or nonlinear relationships between variables.

[0049] The feature selection unit is used to further screen multiple key features through a decision tree to obtain core features.

[0050] In some specific embodiments, the preprocessing unit adjusts the time resolution of the historical energy data by resampling, fills in missing values ​​by linear interpolation, aligns the time indexes of the historical energy data, and merges the historical energy data into a unified data framework.

[0051] The data expansion module is used to obtain an initial data set according to the core features, and expand the initial data set to obtain a training data set;

[0052] In some specific embodiments, the data expansion module includes an expansion unit and a verification unit;

[0053] The expansion unit is used to perform two sample expansions on the initial data set;

[0054] The verification unit is used to verify whether the two sample expansions meet the preset conditions. If not, the initial data set is selected as the training data set. If so, the results of the two sample expansions are merged to obtain the training data set.

[0055] In some specific embodiments, the expansion unit uses a Borderline-SMOTE algorithm and a nonlinear interpolation method to expand the initial data set to obtain a first expanded data set and a second expanded data set.

[0056] The model building module is used to build an energy demand prediction model and perform training based on the training data set;

[0057] In some specific embodiments, the model building module includes a data conversion unit, a model building unit, and a training unit;

[0058] The data conversion unit is used to perform one-dimensional vector conversion on the data in the training data set;

[0059] The model building unit is used to build an energy demand prediction model;

[0060] The training unit is used to divide the training data set into equal intervals according to set time intervals, input the training data set segments into the energy demand prediction model, determine the deviation value between the model output value and the energy consumption value of the next segment, and adjust the model parameters of the energy demand prediction model based on the deviation value until the preset requirements are met.

[0061] In some specific embodiments, the model building unit uses a convolutional neural network as a basic model, introduces a causal convolutional network structure and a residual network structure into the basic model for optimization, and obtains an energy demand prediction model.

[0062] Specifically, the energy demand forecasting model includes:

[0063] Input layer: accepts multi-dimensional input data, such as time series data, weather conditions, holiday information, etc.

[0064] Causal convolution layer: Use causal convolution to ensure the causal relationship of time series data, that is, the current output depends only on past inputs and not on future inputs. This is usually achieved by using padding in the convolution operation to ensure that the convolution kernel only extracts information from past time steps.

[0065] Residual Blocks: Residual blocks are introduced in CNN to solve the gradient vanishing problem in deep network training, allowing the network to learn the identity mapping and thus train a deeper network structure. Each residual block contains multiple convolutional layers, activation functions (usually ReLU) and batch normalization layers.

[0066] Batch Normalization Layer: Batch Normalization is used in convolutional layers and residual blocks to speed up the training process and improve model stability.

[0067] Activation function: Use nonlinear activation functions (such as ReLU or its variants) to introduce nonlinearity and enhance the expressiveness of the model.

[0068] Fully connected layers: After the convolutional layers, fully connected layers are used to map features to the final prediction output.

[0069] Output layer: The final output layer typically uses a linear activation function to output continuous energy demand forecasts.

[0070] In some specific embodiments, after the training unit divides the training data set into equal intervals according to set time intervals, each group of data sets obtained after segmentation is split into channels based on core features and input into the energy demand prediction model.

[0071] The demand forecasting module is used to obtain real-time energy data and forecast energy demand based on the energy demand forecasting model and the real-time energy consumption data;

[0072] The data acquisition module, the feature engineering module, the data expansion module, the model building module and the demand forecasting module are connected in sequence.

[0073] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An energy demand forecasting system based on big data analysis, characterized in that: The following steps are involved: Data acquisition module, feature engineering module, model building module, demand forecasting module; The data acquisition module is used to acquire historical energy data within a preset time period of a selected area; The feature engineering module is used to perform feature selection on historical energy data to obtain core features; The data expansion module is used to obtain an initial data set according to the core features, and expand the initial data set to obtain a training data set; The model building module is used to build an energy demand prediction model and perform training based on the training data set; The demand forecasting module is used to obtain real-time energy data and forecast energy demand based on the energy demand forecasting model and the real-time energy consumption data; The data acquisition module, the feature engineering module, the data expansion module, the model building module and the demand forecasting module are connected in sequence.

2. The energy demand forecasting system based on big data analysis according to claim 1 is characterized in that: The historical energy data acquired by the data acquisition module includes meteorological data, energy market data, economic indicators, industrial production data, regional equipment utilization rate, regional equipment aging rate, social activity information and energy consumption data.

3. The energy demand forecasting system based on big data analysis according to claim 1 is characterized in that: The feature engineering module includes a preprocessing unit, a feature screening unit, and a feature selection unit; The preprocessing unit is used to delete abnormal data and erroneous data in the historical energy data, and merge the historical energy data into a unified data framework; The feature screening unit is used to analyze the pre-processed historical energy data using descriptive statistics and EDA, and determine the initial key feature range according to the analysis results; calculate the correlation coefficients of different variables in the historical energy data, identify the correlation between the variables, and select features that meet the correlation requirements based on the correlation to obtain multiple key features; The feature selection unit is used to further screen multiple key features through a decision tree to obtain core features.

4. The energy demand forecasting system based on big data analysis according to claim 3 is characterized in that: The preprocessing unit adjusts the time resolution of the historical energy data by resampling, fills in missing values ​​by linear interpolation, aligns the time indexes of the historical energy data, and merges the historical energy data into a unified data framework.

5. The energy demand forecasting system based on big data analysis according to claim 1 is characterized in that: The data expansion module includes an expansion unit and a verification unit; The expansion unit is used to perform two sample expansions on the initial data set; The verification unit is used to verify whether the two sample expansions meet the preset conditions. If not, the initial data set is selected as the training data set. If so, the results of the two sample expansions are merged to obtain the training data set.

6. The energy demand forecasting system based on big data analysis according to claim 5 is characterized in that: The expansion unit expands the initial data set by using a Borderline-SMOTE algorithm and a nonlinear interpolation method to obtain a first expanded data set and a second expanded data set.

7. The energy demand forecasting system based on big data analysis according to claim 1 is characterized in that: The model building module includes a data conversion unit, a model building unit, and a training unit; The data conversion unit is used to perform one-dimensional vector conversion on the data in the training data set; The model building unit is used to build an energy demand prediction model; The training unit is used to divide the training data set into equal intervals according to set time intervals, input the training data set segments into the energy demand prediction model, determine the deviation value between the model output value and the energy consumption value of the next segment, and adjust the model parameters of the energy demand prediction model based on the deviation value until the preset requirements are met.

8. The energy demand forecasting system based on big data analysis according to claim 7 is characterized in that: The model building unit uses a convolutional neural network as a basic model, introduces a causal convolutional network structure and a residual network structure into the basic model for optimization, and obtains an energy demand prediction model.

9. The energy demand forecasting system based on big data analysis according to claim 7 is characterized in that: After the training unit divides the training data set into equal intervals according to the set time intervals, each group of data sets obtained after the segmentation is split into channels based on core features, and inputs the channels into the energy demand prediction model.

Images