Load characteristic prediction method based on Informer hyper-parameter optimization model

Abstract

The invention relates to the technical field of power system load prediction, and discloses a load characteristic prediction method based on an Informer hyper-parameter optimization model, which comprises the following steps of: firstly, acquiring long-time weather and power grid data of a power supply company region and preprocessing to obtain a characteristic data set corresponding to load data; and carrying out normalization processing on the feature data set corresponding to the load data. Then, inputting the processed data into an Informer model for training, fully exploring the time correlation among the data, and utilizing a multi-head sparse self-attention mechanism to mine the correlation between the states before and after the feature information; and then determining hyper-parameters needing to be optimized and a value range, and performing hyper-parameter optimization on the Informer model by adopting a TPE algorithm to obtain an optimal combination of the hyper-parameters. And finally, obtaining a prediction result through the optimized model, then establishing a model comparison scheme, and carrying out prediction result comparative analysis on the Informer parameter optimization model, the LSTM and the GRU to embody the feasibility of the method.

Description

Technical Field

[0001] The present invention relates to the technical field of power system load forecasting, and in particular to a load characteristic forecasting method based on an Informer hyperparameter optimization model. Background Art

[0002] Electricity is an important basic industry and public utility related to the national economy and people's livelihood, and is an indispensable means of production and living material for the national economy and social development. The safe, stable and reliable supply of electricity is related to the comprehensive, coordinated and sustainable development of the national economy, the harmony and stability of society, and the happiness and well-being of the people.

[0003] Power system load forecasting is one of the important tasks of the power sector. It is not only a prerequisite for formulating a reasonable power macro-planning, but also an important guarantee for the safe and economic operation of the power system. Relying on accurate load forecasting, the start and stop of the internal generator sets of the power grid can be arranged economically and reasonably, the safe and stable operation of the power grid can be maintained, unnecessary spinning reserve capacity can be reduced, and the maintenance of the units can be arranged reasonably to ensure the normal production of the society and the normal life of the people, effectively reduce the cost of power generation, and improve economic and social benefits. The results of load forecasting can also be used to determine the capacity and site selection of new generator sets, determine the capacity increase and reconstruction of the power grid, etc., provide information reference and decision-making assistance for the approval of power projects, and are an important basis for the construction and development of the power system. Therefore, the level of power load forecasting has become one of the signs of measuring whether the management of a country's energy department, a power grid enterprise or a power generation enterprise is moving towards modernization. However, with the rapid development of my country's power industry, the separation of power plants and grids, the steady progress of power market transactions, and the vigorous development of smart grids, the operation of the power system has become more complex, the changes in the load of the power system have become more diverse, and the power load is more affected by various sudden political and economic events. These have put forward new and higher requirements for the load forecasting work of the power system. More accurate power load forecasting has become an important and urgent task facing power workers.

[0004] In load forecasting, the forecast time range is usually from a few hours to a few days. This forecast is crucial for the daily operation and dispatch of the power grid, and can help power companies reasonably dispatch power generation resources to ensure the stability of the power grid and the reliability of power supply. The time range of medium-term and long-term load forecasts is longer, and may cover several months or even years. This forecast is mainly used for the planning and development strategy of the power system. Through accurate medium-term and long-term load forecasts, power companies can reasonably plan the construction and expansion of the power grid, power supply configuration and their investment decisions.

[0005] With the development of big data, artificial intelligence and Internet of Things technologies, the methods and accuracy of load forecasting are also constantly improving. Due to the influence of various factors such as weather changes, social activities and festival types, the load is manifested as a non-stationary random process in the time series, but most of the factors affecting the system load have regularity, which lays the foundation for effective prediction. In recent years, load forecasting technology has experienced significant development and innovation, especially in the application of data analysis methods, computing technology and artificial intelligence.

[0006] Based on the actual demand of the above distribution network, it is necessary to develop a load forecasting model. The difficulties are: (1) The accuracy of the power load forecast results requires data enhancement of the available power load historical data, and it is difficult to process such data; (2) Load forecasting involves multiple variables and time scales, and the decisions between multiple stages affect each other, which makes the model difficult to solve. (3) In load forecasting, climate factors, economic factors, and holiday factors are usually considered to be the main external factors affecting the load characteristic indicators of the power grid. How to conduct sensitivity analysis on these factors is also a difficult point that needs to be studied. Summary of the invention

[0007] In view of the above problems, the purpose of the present invention is to provide a load characteristic prediction method based on the Informer hyperparameter optimization model, which ensures the load characteristic analysis and accurate power prediction taking into account uncertain factors, and the prediction accuracy is greatly improved compared with the traditional optimization method. The technical solution is as follows:

[0008] A load characteristic prediction method based on an Informer hyperparameter optimization model comprises the following steps:

[0009] Step 1: Obtain weather and power grid data for a specific period of time in the area to be predicted, use the historical load data as the label sequence, and the remaining historical data as the feature sequence; preprocess the acquired data to obtain the feature data set corresponding to the load data, and normalize the feature data set corresponding to the load data;

[0010] Step 2: Input the normalized feature data set into the Informer model training to fully explore the temporal correlation between the data and use the multi-head sparse self-attention mechanism to mine the correlation between the previous and next states of the feature information;

[0011] Step 3: Determine the hyperparameters and value ranges that need to be optimized, and use a non-standard Bayesian optimization algorithm to optimize the hyperparameters of the Informer model to obtain the optimal combination of hyperparameters;

[0012] Step 4: Perform load forecasting using the optimized Informer model to obtain load forecasting results.

[0013] Furthermore, the step 1 is specifically as follows:

[0014] Step 1.1: Perform preliminary screening and classification on the acquired weather and power grid data to obtain a preliminary screening data set; the preliminary screening data set is represented as: X = {T t ,V t ,R t ,W t ,U t ,P t}, where T t represents the average daily temperature, V t represents the average daily wind speed, R t represents the average daily rainfall, W t represents the daily average humidity, U t Indicates the power consumption in this area, P t Indicates the load in this area;

[0015] Step 1.2: Preprocess the initial screening data set to obtain a preprocessed data set; the preprocessing includes analyzing the data, removing obviously erroneous data and supplementing missing data, and standardizing the data elements in the initial screening data set; the preprocessed data set is represented by: D = {D T ,D V ,D R ,D W ,D U ,D P}, where D T represents the standardized daily average temperature, D V represents the standardized daily average wind speed, D R represents the standardized daily average rainfall, D W represents the standardized daily average humidity, D U represents the standardized electricity consumption in the region, D P Indicates the standardized load in the region;

[0016] Step 1.3: Divide the processed feature dataset into training set, validation set and test set in proportion.

[0017] Furthermore, the method of supplementing the missing data in the initial screening data set in step 1.2 is the hot card filling method, that is, finding an object most similar to it in the complete data, and then filling it with the value of this most similar object.

[0018] Furthermore, the method for standardizing the data elements in the preliminary screening data set in step 1.2 is: using the MinMaxScaler function to scale the original data according to the difference between the minimum and maximum values, so that the scaled data range is between [0,1]. The calculation formula for the data after standardization adjustment is as follows:

[0019]

[0020] Among them, D X represents the data after standardization, X0 represents the original data input at the beginning, and X max Represents the maximum value in the input data set, X min Represents the minimum value in the input data set.

[0021] Furthermore, in step 2, the preprocessed feature data set is input into the Informer prediction model according to the set time step, and the load at several future time points is predicted to obtain a preliminary load prediction result, which is specifically:

[0022] Step 2.1: Perform word vector encoding and position encoding on the input time series data, and weight these two encodings to obtain the input data of the model:

[0023] Time series data input at time t in, Represents the i-th input data at time t, i=1,2…L x , L x Indicates the data length, Represents the characteristic dimension of the input data; encodes this time series data;

[0024] For the time series data X input at time t t and p types of global timestamps, whose feature dimension is d model ,The embedding consists of three separate parts, namely the scalar projection, local timestamp, and global timestamp;

[0025] For local timestamps, context information is saved by embedding at fixed positions. The expression is as follows:

[0026]

[0027]

[0028] Among them, PE represents position encoding, pos represents position, and d model represents the feature dimension, j represents the feature dimension d model The jth dimension in , j∈{1,...,[d model / 2]}, indicating the range 1-Lx;

[0029] Using learnable stamp embedding SE (pos) , that is, the location timestamp represents the global timestamp;

[0030] Scalar projection is to use a one-dimensional convolution filter to transform the input data Map to d model Dimensional vector The input representation vector fed into the model is:

[0031]

[0032] in, represents the model input data sequence, t represents time, i represents the i-th data, α represents the factor of balancing the size between scalar projection and local / global embedding, PE represents position encoding, L x Indicates the data length, p indicates the type of position encoding, and SE indicates the embedded position stamp;

[0033] Step 2.2: Input the encoded time series data and use the probabilistic sparse self-attention mechanism instead of the traditional self-attention mechanism:

[0034] The attention of the mth Query in the tuple input in the traditional self-attention mechanism is defined as a kernel smoother in the form of probability. The specific formula is as follows:

[0035]

[0036] Among them, A represents the calculation method of the probabilistic sparse self-attention mechanism, q m The query vector of the mth row of the Query matrix Q is represented, K represents the matrix of Key, and V represents the matrix of Value; k represents the correlation vector between the queried information and other information, n represents the nth data, and l represents the vector length. Expresses expectation; k n is the key vector of the nth data in the query information, k l is the key vector of the lth data in the query information, v n is the value vector of the nth data of the query information;

[0037] and k(q m ,k n ) Select the asymmetric exponential kernel Self-attention is achieved by calculating the probability p(k n ∣q m ) to combine these values ​​and obtain the output; d represents the dimension of the K matrix;

[0038] Using the probabilistic sparse self-attention mechanism, each Key is allowed to focus only on u main queries. The formula is as follows:

[0039]

[0040] Among them, Softmax() represents the normalized exponential function operation, K T represents the transpose of the matrix representing Key; Represents a sparse matrix containing only the first u queries;

[0041] Step 2.3: And use the self-attention distillation method to reduce the network dimension:

[0042] The distillation operation is used to give higher weights to the dominant features, and a focused self-attention feature map is generated in the next layer. The formula of the self-attention feature map is as follows:

[0043]

[0044] in,() att represents the attention module, Represents the h-th layer matrix, Convld() represents a one-dimensional convolution operation on the time series, and uses the exponential linear unit ELU as the activation function. Finally, the maximum pooling layer MaxPooling is passed to halve the amount of calculation of each layer, and the model retains the information of the long input time series;

[0045] Step 2.4: Input The time series enters the decoder, and the input time series is divided into two parts, namely the known sequence before the predicted point and forecast series that require masking of future load data Its form is as follows:

[0046]

[0047] Among them, L token and L y Respectively represent the length of the known sequence and the length of the predicted sequence, d model Represents the dimension of the model, Concat() represents the concatenation operation, represents the input time series, Represents the known sequence before the predicted point, Indicates the forecast series for which future load data need to be masked.

[0048] Furthermore, in step 3, the non-standard Bayesian optimization algorithm uses the Bayesian formula to optimize the expected improvement function

[0049]

[0050] Where x is the hyperparameter to be optimized, y is the loss, P(x|y) is the probability of the hyperparameter x when the loss is y; y * is the threshold of y; P(x) is the probability of x occurring, and P(y) is the probability of y occurring;

[0051] Let γ = P(y < y*), that is, the probability that y < y*, representing a certain quantile of the non-standard Bayesian optimization algorithm, used to divide g(x) and h(x), and construct:

[0052] P(x) = ∫P(x|y)P(y)dy = γg(x) + (1 - γ)h(x)

[0053] Where g(x) is the observed density when the loss of f(x) is less than the threshold y * ; h(x) is the observed density when the loss of f(x) is greater than or equal to the threshold y * ;

[0054] Finally, the expected improvement function is simplified to

[0055]

[0056] Where ∝ means proportional to;

[0057] After iterating a specific number of times, return the candidate hyperparameter x* with the expected improvement function .

[0058] Furthermore, the hyperparameter optimization process in step 3 is as follows:

[0059] Step 3.1: Initialization

[0060] Randomly generate a set of initial parameter configurations and evaluate the performance of the initial parameter configurations;

[0061] Step 3.2: Modeling and Sampling

[0062] Use two different probability density functions to model the conditional probability distribution of the parameters, including the probability density function Exploitation for modeling well-performing parameter configurations and the probability density function Exploration for modeling unknown parameter configurations;

[0063] Sampling: In each iteration, according to the observed initial parameter configurations and the performance of the parameter configurations, use the Exploitation model and the Exploration model to generate candidate parameter configurations respectively;

[0064] Step 3.3: Evaluation and Update

[0065] Evaluation: By running the Informer model and calculating the loss function, the generated candidate parameter configurations are evaluated to obtain the performance of the candidate parameter configurations;

[0066] Update: Update the Exploitation model and Exploration model based on the evaluation results;

[0067] Step 3.4: Repeat the iteration

[0068] Repeat the modeling, sampling, evaluation, and updating steps until a predetermined number of iterations is reached or a stopping condition is met;

[0069] Step 3.5: Output the optimal parameters

[0070] After the iteration, the best performing parameter configuration is output, which is the optimal parameter of the Informer model.

[0071] The beneficial effects of the present invention are as follows: the improved Informer algorithm proposed in the present invention can well grasp the nonlinear relationship between the load to be predicted and a large number of input features, and compared with LSTM and GRU, it has higher prediction accuracy and greatly improves the prediction accuracy; after the parameters of the non-standard Bayesian optimization algorithm are optimized, the prediction speed can also be accelerated. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is the load characteristic prediction method and feasibility comparison analysis process based on the Informer hyperparameter optimization model of the present invention.

[0073] Figure 2 This is the structure of the Informer prediction model of the present invention.

[0074] Figure 3 This is a comparison chart between the predicted value and the actual value of the load forecasting method based on Informer hyperparameter optimization.

[0075] Figure 4 This is a comparison chart between the predicted value and the actual value of the LSTM-based load forecasting method.

[0076] Figure 5 This is a comparison chart between the predicted value and the actual value of the GRU-based load forecasting method. DETAILED DESCRIPTION

[0077] The embodiment of the present invention provides a method for predicting load characteristics taking into account uncertain factors based on an Informer hyperparameter optimization model, the steps of which are as follows: Figure 1As shown: 1. Obtain long-term weather and power grid data in the power supply company area and preprocess them to obtain the feature data set corresponding to the load data, and normalize the feature data set corresponding to the load data. 2. Input the processed data into the Informer model training to fully explore the temporal correlation between the data, and use the multi-head sparse self-attention mechanism to mine the correlation between the state before and after the feature information. Then determine the hyperparameters and value ranges that need to be optimized, and use the TPE algorithm to optimize the hyperparameters of the Informer model to obtain the optimal combination of hyperparameters. 3. Obtain the prediction results through the optimized model, and then establish a model comparison plan to compare and analyze the prediction results of this Informer parameter optimization model with LSTM and GRU to reflect the feasibility of this method.

[0078] S1: Obtain long-term weather and power grid data in the power supply company area and preprocess them to obtain a feature data set corresponding to the load data, and normalize the feature data set corresponding to the load data.

[0079] Specifically, long-term weather and power grid data in the area where the power supply company is located are obtained, historical load data are used as label sequences, other relevant historical data are used as feature sequences, and the obtained data are preprocessed to obtain a feature data set corresponding to the load data.

[0080] The acquired weather and power grid data are preliminarily screened and classified to obtain a preliminarily screened data set, where the preliminarily screened data set is represented by X = {T t ,V t ,R t ,W t ,U t ,P t}, T t represents the average daily temperature, V t represents the average daily wind speed, R t represents the average daily rainfall, W t represents the average daily humidity, U t Indicates the power consumption in this area, P t Indicates the load in this area.

[0081] The initial screening data set is then preprocessed to obtain a preprocessed data set, wherein the preprocessing includes analyzing and processing the data, removing obviously erroneous data and supplementing missing data, and standardizing the data elements in the initial screening data set to obtain a processed data set D = {D T ,D V ,D R ,D W ,D U ,D P}, where D T represents the standardized daily average temperature, D Vrepresents the standardized daily average wind speed, D R represents the standardized daily average rainfall, D W represents the standardized daily average humidity, D U represents the standardized electricity consumption in the region, D P It represents the standardized load in the region. The processed feature data set is divided into training set, validation set and test set in the ratio of 7:1:2.

[0082] The method to supplement the missing data in the initial screening data set is the hot card filling method, that is, to find an object that is most similar to it in the complete data, and then fill it with the value of this most similar object.

[0083] The method for standardizing the data elements in the initial screening data set is to use the MinMaxScaler function to scale the original data according to the difference between the minimum and maximum values, so that the scaled data range is between [0,1]. The calculation formula for the data after standardization adjustment is as follows:

[0084]

[0085] Among them, D represents the data after standardization, X represents the original data input at the beginning, and X max Represents the maximum value in the input data set, X min Represents the minimum value in the input data set.

[0086] S2: Input the processed data into the Informer model training, fully explore the temporal correlation between the data, and use the multi-head sparse self-attention mechanism to mine the correlation between the previous and next states of the feature information. Then determine the hyperparameters and value ranges that need to be optimized, and use the TPE algorithm to optimize the hyperparameters of the Informer model to obtain the optimal combination of hyperparameters.

[0087] S21: Preprocess the feature data set, input the preprocessed feature data set into the informer prediction model according to a certain time step, predict the load at several future time points, and obtain a preliminary load prediction result. The informer prediction model is as follows. The informer prediction model includes a data input unit, an encoder, a decoder, and a data output unit, such as Figure 2 shown.

[0088] The input time series data is encoded with word vectors and positions, and these two encodings are weighted to obtain the input data of the model.

[0089] At time t input Time series data, where X t represents the input sequence at time t, represents the i-th input at time t, Represents the feature dimension of the input data and encodes this time series data sample. Compared with the encoding of the traditional Transformer model, the Informer prediction model requires not only local time series information, but also hierarchical timestamps (week, month, year) and unknown timestamps (holidays, events).

[0090] Preferably, for t sequences, input X t and p types of global timestamps, the feature dimension after input representation is d model ,The input embedding consists of three separate parts, namely the scalar projection, the local timestamp, and the global timestamp.

[0091] The local timestamp is the same as the traditional Transformer model position encoding, which saves context information through fixed position embedding. The expression is as follows:

[0092]

[0093]

[0094] Among them, PE represents position encoding, pos represents position, and d model represents the feature dimension, L x Indicates data length, j indicates d model The jth dimension in , j∈{1,...,[d model / 2]}, indicating a range of 1-L x .

[0095] For global timestamps, use learnable timestamp embedding SE (pos) (Location timestamp) indicates a global timestamp. Its vocab size is limited to 60 at most, i.e., the finest granularity is one minute.

[0096] Scalar projection is to use a one-dimensional convolution filter (default width is 3, stride is 1) to transform the scalar Map to d model Dimensional vector Get the input representation vector fed into the model:

[0097]

[0098] in, represents the model input data sequence, t represents time, i represents the i-th data, α represents the factor of balancing the size between scalar projection and local / global embedding, Indicates that the scalar Map to dmodel dimensional vector, PE represents position encoding, L x Indicates the data length, p indicates the type of position encoding, and SE indicates the embedded location stamp.

[0099] The encoded time series data is input, and the ProbSparse self-attention mechanism is used to replace the traditional self-attention mechanism. The self-attention distillation method is used to reduce the network dimension.

[0100] The traditional self-attention mechanism is defined based on the tuple input (Query, Key and Value), which expresses the proportional dot product as, Among them, A represents the calculation method of the probabilistic sparse self-attention mechanism, Q represents the Query matrix, K represents the Key matrix, and V represents the Value matrix. d represents the dimension of the K matrix, Softmax() represents the normalization function and q m , k m 、v m Represent the mth row in Q, K, and V respectively. The attention of the mth Query is defined as a kernel smoother in the form of probability. The specific formula is as follows:

[0101]

[0102] Among them, A represents the calculation method of the probabilistic sparse self-attention mechanism, q m The query vector of the mth row of the Query matrix Q is represented, K represents the matrix of Key, and V represents the matrix of Value; k represents the correlation vector between the queried information and other information, n represents the nth data, and l represents the vector length. Expresses expectation; k n is the key vector of the nth data in the query information, k l is the key vector of the lth data in the query information, v n is the value vector of the nth data in the query information.

[0103] and k(q m ,k n ) Select the asymmetric exponential kernel Self-attention is achieved by calculating the probability p(k n ∣q m ) to combine these values ​​and obtain the output; d represents the dimension of the K matrix.

[0104] Since there are two ∑s in the calculation, the time complexity and space complexity are both Indicates the complexity, L Q and L KRespectively represent the length of Q and K. According to the above A(q m ,K,V) formula, the attention of the mth query to all keys is defined as the probability p(k n ∣q m ), whose output is a combination of and V. A few major dot product pairs drive the attention probability distribution of the corresponding Query away from the uniform distribution.

[0105] To solve the above problem, the ProbSparse self-attention mechanism is used to allow each Key to focus only on u main queries. The formula is as follows:

[0106]

[0107] Among them, A represents the calculation method of the probabilistic sparse self-attention mechanism; Softmax() represents the normalized exponential function operation, V represents the matrix representing Value, and K T represents the transpose of the matrix representing Key; Represents a sparse matrix containing only the first u queries.

[0108] The distillation operation is used to give higher weights to the dominant features with main features, and a focused self-attention feature map is generated in the next layer. The formula of the self-attention feature map is as follows:

[0109]

[0110] in,() att represents the attention module, Represents the h-th layer matrix, Convld() represents a one-dimensional convolution operation on the time series, and uses exponential linear units (ELU) as the activation function. Finally, it passes through a maximum pooling (MaxPooling) layer with a step size of 2, which reduces the amount of calculation for each layer by half, and the model can retain the information of long input time series.

[0111] enter The time series enters the decoder, and the input time series is divided into two parts, namely the known sequence before the predicted point and forecast series that require masking of future load data Its form is as follows:

[0112]

[0113] Among them, L token and L y Respectively represent the length of the known sequence and the length of the predicted sequence, d modelrepresents the dimension of the model, and Concat() represents the concatenation operation. represents the input time series. represents the known sequence before the predicted point. represents the prediction sequence that needs to mask the future load data.

[0114] S22. The non-standard Bayesian optimization algorithm (Tree-structured parzen estimator, TPE) is used to optimize the hyperparameters of the Informer model. TPE is a non-standard Bayesian optimization algorithm based on tree-structured Parzen density estimation. Traditional Bayesian optimization algorithms may suffer from the curse of dimensionality in high-dimensional spaces. Compared with other Bayesian optimization algorithms, TPE has better performance in high-dimensional spaces. In the TPE algorithm, the Bayesian formula is used to optimize the expected improvement function.

[0115]

[0116] In the formula, P(x|y) is the probability of hyperparameter x when the loss is y; g(x) is the observed density when the loss of f(x) is less than the threshold y. * When; h(x) is the observed density when the loss of f(x) is greater than or equal to the threshold y. * When. Let γ = P(y < y*), which represents a certain quantile of the TPE algorithm, used to divide g(x) and h(x), and construct

[0117] P(x) = ∫P(x|y)P(y)dy = γg(x) + (1 - γ)h(x)

[0118] Finally, the expected improvement function is simplified to

[0119]

[0120] From the above formula, to maximize the expected improvement function, it is necessary to ensure that the probability of g(x) is high and the probability of h(x) is low at point x. TPE uses h(x) / g(x) to evaluate the selected candidate. After a certain number of iterations, the algorithm returns the candidate x* with the maximum EI. Through this method, the parameters of the informer model are optimized. After iterative convergence, the model with optimized parameters is used for load prediction to obtain the load prediction result Y. t .

[0121] The process of optimizing the parameters of the TPE-improved Informer model is as follows:

[0122] (1). Initialization

[0123] First, a set of initial parameter configurations are randomly generated and their performance is evaluated.

[0124] (2) Modeling and sampling

[0125] Modeling: The TPE algorithm uses two different probability density functions to model the conditional probability distribution of parameters. One is the probability density function for modeling good parameter configurations (Exploitation), and the other is the probability density function for modeling unknown parameter configurations (Exploration).

[0126] Sampling: In each iteration, the Exploitation model and the Exploration model are used to generate candidate parameter configurations based on the observed parameter configurations and their performance. In this way, the TPE algorithm can search in the parameter space in a targeted manner and is more likely to find the global optimal solution.

[0127] (3) Evaluation and update

[0128] Evaluation: Evaluate the generated candidate parameter configurations to get their performance. This is done by running the Informer model and calculating the loss function.

[0129] Update: Based on the evaluation results, the Exploitation and Exploration models are updated so that in the next iteration, the algorithm can generate more promising candidate parameter configurations based on more accurate models.

[0130] (4) Repeated iteration

[0131] Repeat the above modeling, sampling, evaluation and updating steps until the predetermined number of iterations is reached or the stopping condition is met. During the iteration process, the TPE algorithm continuously accumulates knowledge about the parameter space and gradually approaches the optimal parameter configuration.

[0132] (5) Output optimal parameters

[0133] After the iteration, the TPE algorithm will output the best performing parameter configuration, which is the optimal parameter of the Informer model.

[0134] S3: The obtained Y t The prediction results are denormalized, and then a model comparison scheme is established to compare and analyze the prediction results of this Informer parameter optimization model with those of LSTM and GRU.

[0135] Specifically, the comparative analysis uses (RMSE) and mean absolute error (MAE) to evaluate the predictive performance of machine learning models. Among them, RMSE is a measure of accuracy used to compare the prediction errors of different models on a specific data set, while MAE is the average of the absolute values ​​of the difference between the true value and the predicted value. When comparing the performance of different models, the smaller the RMSE and MAE, the better the model performance. The calculation formulas for the two indicators are as follows:

[0136]

[0137]

[0138] Among them, m represents the number of samples, y true represents the true value of the load, represents the load prediction value, and i represents the i-th data.

[0139] Forecast the load Perform denormalization output to obtain the final load prediction result.

[0140] The experiment of this project uses Tensorflow as the framework, the programming environment is Python3.8, and the simulation is implemented on a computer equipped with an Inter core i9-13900HX CPU and an NVIDIA RTX4070 graphics card. The input data for daily time scale data processing are: daily average temperature, daily rainfall, daily average humidity, daily wind speed, daily electricity consumption, and daily load. The data is normalized and used as the input value of the model.

[0141] The daily data from 2018 to 2022 are used as the data set. The first 1000 data of daily load are used as the training set, and 1000-1300 data are used as the validation set. 1300-1400 data are used as the test set. The data are normalized and used as the input value of the model. Figure 3-5 The prediction results are shown in the figure Figure 3 Load forecasting based on Informer hyperparameter optimization, Figure 4 LSTM-based load forecasting, and Figure 5 The load prediction performance comparison based on GRU is shown in Table 1.

[0142] Table 1 Comparison of prediction results between the Informer parameter optimization model and LSTM and GRU

[0143] Model RMSE MAPE GRU 7.75 0.064 LSTM 7.01 0.061 Informer hyperparameters improve the model 6.61 0.054

[0144] As can be seen from Table 1, since the improved Informer algorithm can well grasp the nonlinear relationship between the load to be predicted and a large number of input features, it has higher prediction accuracy than LSTM and GRU, and the prediction accuracy is greatly improved.

Claims

1. A load characteristic prediction method based on the Informer hyperparameter optimization model, characterized in that: It includes the following steps: Step 1: Obtain the weather and power grid data in a specific area for a specific period of time. Use the historical load data as the label sequence and the remaining historical data as the feature sequence. Preprocess the obtained data to obtain the feature dataset corresponding to the load data, and normalize the feature dataset corresponding to the load data. Step 2: Input the normalized feature dataset into the Informer model for training, fully explore the temporal correlation between data, and use the multi-head sparse self-attention mechanism to mine the correlation between the front and back states of feature information. Step 3: Determine the hyperparameters to be optimized and their value ranges, and use the non-standard Bayesian optimization algorithm to optimize the hyperparameters of the Informer model to obtain the optimal combination of hyperparameters. Step 4: Perform load forecasting through the optimized Informer model to obtain the load forecasting result.

2. The load characteristic prediction method based on the Informer hyperparameter optimization model according to claim 1 is characterized in that: The specific content of Step 1 is as follows: Step 1.1: Perform preliminary screening and classification on the acquired weather and power grid data to obtain a preliminary screening data set; the preliminary screening data set is represented as: X = {T t ,V t ,R t ,W t ,U t ,P t }, where T t represents the average daily temperature, V t represents the average daily wind speed, R t represents the average daily rainfall, W t represents the daily average humidity, U t Indicates the power consumption in this area, P t Indicates the load in this area; Step 1.2: Preprocess the preliminary screening dataset to obtain the preprocessed dataset. The preprocessing includes analyzing and processing the data, removing obviously erroneous data and supplementing missing data, and standardizing the data elements in the initial screening data set. The preprocessing data set is represented by: D = {D T ,D V ,D R ,D W ,D U ,D P }, where D T represents the standardized daily average temperature, D V represents the standardized daily average wind speed, D R represents the standardized daily average rainfall, D W represents the standardized daily average humidity, D U represents the standardized electricity consumption in the region, D P Indicates the standardized load in the region; Step 1.3: Divide the processed feature dataset into a training set, a validation set, and a test set according to a certain proportion.

3. The load characteristic prediction method based on the Informer hyperparameter optimization model according to claim 2 is characterized in that: In Step 1.2, the method for supplementing the missing data in the preliminary screening dataset is the hot deck imputation method, that is, find a most similar object in the complete data, and then use the value of this most similar object for filling.

4. The load characteristic prediction method based on the Informer hyperparameter optimization model according to claim 2 is characterized in that: In Step 1.2, the method for standardizing the data elements in the preliminary screening dataset is as follows: Use the MinMaxScaler function to scale the original data according to the difference between the minimum value and the maximum value, so that the range of the scaled data is between [0,1]. The calculation formula for the data after standardization adjustment is as follows: Among them, D X represents the data after standardization, X0 represents the original data input at the beginning, and X max Represents the maximum value in the input data set, X min Represents the minimum value in the input data set.

5. The load characteristic prediction method based on the Informer hyperparameter optimization model according to claim 1 is characterized in that: In Step 2, input the preprocessed feature dataset into the Informer prediction model according to the set time step to predict the load volume at several future time points to obtain the preliminary load forecasting result. Specifically: Step 2.1: Perform word vector encoding and position encoding on the input time series data, and weight these two encodings to obtain the input data of the model: Time series data input at time t in, Represents the i-th input data at time t, i=1,2…L x , L x Indicates the data length, Represents the characteristic dimension of the input data; encodes this time series data; For the time series data X input at time t t and p types of global timestamps, whose feature dimension is d model ,The embedding consists of three separate parts, namely the scalar projection, local timestamp, and global timestamp; For local timestamps, save the context information through fixed position embedding, and the expression is as follows: Among them, PE represents position encoding, pos represents position, and d model represents the feature dimension, j represents the feature dimension d model The jth dimension in , j∈{1,...,[d model / 2]}, indicating a range of 1-L x ; Using learnable stamp embedding SE (pos) , that is, the location timestamp represents the global timestamp; Scalar projection is to use a one-dimensional convolution filter to transform the input data Map to d model Dimensional vector The input representation vector fed into the model is: in, represents the model input data sequence, t represents time, i represents the i-th data, α represents the factor of balancing the size between scalar projection and local / global embedding, PE represents position encoding, L x Indicates the data length, p indicates the type of position encoding, and SE indicates the embedded position stamp; Step 2.2: Input the encoded time series data, and use the probabilistic sparse self-attention mechanism to replace the traditional self-attention mechanism: Define the attention of the m-th Query in the tuple input in the traditional self-attention mechanism as a kernel smoother in the form of probability. The specific formula is as follows: Among them, A represents the calculation method of the probabilistic sparse self-attention mechanism, q m The query vector of the mth row of the Query matrix Q is represented, K represents the matrix of Key, and V represents the matrix of Value; k represents the correlation vector between the queried information and other information, n represents the nth data, and l represents the vector length. Expresses expectation; k n is the key vector of the nth data in the query information, k l is the key vector of the lth data in the query information, v n is the value vector of the nth data of the query information; and k(q m ,k n ) Select the asymmetric exponential kernel Self-attention is achieved by calculating the probability p(k n ∣q m ) to combine these values ​​and obtain the output; d represents the dimension of the K matrix; Use the probabilistic sparse self-attention mechanism, allowing each Key to only focus on u main queries. The formula is as follows: Among them, Softmax() represents the normalized exponential function operation, K T represents the transpose of the matrix representing Key; Represents a sparse matrix containing only the first u queries; Step 2.3: And use the method of self-attention distillation to reduce the network dimension: Use the distillation operation to assign higher weights to the dominant features and generate a focused self-attention feature map in the next layer. The formula for the self-attention feature map is as follows: in,() att represents the attention module, Represents the h-th layer matrix, Convld() represents a one-dimensional convolution operation on the time series, and uses the exponential linear unit ELU as the activation function. Finally, the maximum pooling layer MaxPooling is passed to halve the amount of calculation of each layer, and the model retains the information of the long input time series; Step 2.4: Input The time series enters the decoder, and the input time series is divided into two parts, namely the known sequence before the predicted point and forecast series that require masking of future load data Its form is as follows: Among them, L token and L y Represent the length of the known sequence and the length of the predicted sequence, d model Represents the dimension of the model, Concat() represents the concatenation operation, represents the input time series, Represents the known sequence before the predicted point, Indicates the forecast series for which future load data need to be masked.

6. The load characteristic prediction method based on the Informer hyperparameter optimization model according to claim 1 is characterized in that: In step 3, the non-standard Bayesian optimization algorithm uses the Bayesian formula to optimize the expected improvement function Where x is the hyperparameter to be optimized, y is the loss, P(x|y) is the probability of hyperparameter x when the loss is y; y * is the threshold of y; P(x) is the probability of x occurring, and P(y) is the probability of y occurring; Let γ = P(y < y*), that is, the probability of y < y*, representing a certain quantile of the non-standard Bayesian optimization algorithm, used to divide g(x) and h(x), and construct: P(x) = ∫P(x|y)P(y)dy = γg(x) + (1 - γ)h(x) In the formula, g(x) is the loss of f(x) less than the threshold y * Observation density when: h(x) is the loss of f(x) greater than or equal to the threshold y * The observation density at ; Finally, we expect the improved function Simplified to: In the formula, ∝ represents proportional to; After a certain number of iterations, return a function with the expected improvement Candidate hyperparameters x* for .

7. The load characteristic prediction method based on the Informer hyperparameter optimization model according to claim 1 is characterized in that: The hyperparameter optimization process in Step 3 is as follows: Step 3.1: Initialization Randomly generate a set of initial parameter configurations and evaluate the performance of the initial parameter configurations; Step 3.2: Modeling and Sampling Use two different probability density functions to model the conditional probability distribution of parameters, including the probability density function Exploitation for modeling well-performing parameter configurations, and the probability density function Exploration for modeling unknown parameter configurations; Sampling: In each iteration, the Exploitation model and the Exploration model are used to generate candidate parameter configurations based on the performance of the observed initial parameter configurations and parameter configurations, respectively. Step 3.3: Evaluate and Update Evaluation: By running the Informer model and calculating the loss function, the generated candidate parameter configurations are evaluated to obtain the performance of the candidate parameter configurations; Update: Update the Exploitation model and Exploration model based on the evaluation results; Step 3.4: Repeat the iteration Repeat the modeling, sampling, evaluation, and updating steps until a predetermined number of iterations is reached or a stopping condition is met; Step 3.5: Output the optimal parameters After the iteration, the best performing parameter configuration is output, which is the optimal parameter of the Informer model.

Images