Short-term power load prediction method, system and device based on VAE-BiLSTM hybrid neural network and Bayesian optimization, and medium

Abstract

The invention discloses a short-term power load prediction method, system and device based on a VAE-BiLSTM hybrid neural network and Bayesian optimization, and a medium. The method comprises the following steps: preprocessing obtained load data related to a power system and feature data influencing a load; performing feature dimension reduction on the processed data by using kernel principal component analysis; performing hyper-parameter optimization on the network structures of the variational auto-encoder and the bidirectional long-short-term memory network by using a Bayesian optimization method; training the dimensionality-reduced data containing the data principal component by using a variational auto-encoder with an optimal hyper-parameter; inputting a coding layer containing data potential representation in the trained variational auto-encoder network structure into the bidirectional long-short-term memory network model with the optimal hyper-parameter for load prediction to obtain a short-term power load prediction result; performing accuracy evaluation on the prediction result; according to the method, reasonable selection of time sequence correlation and hyper-parameters can be fully considered in power load prediction, and the accuracy of power load prediction is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of power load forecasting, and in particular relates to a short-term power load forecasting method, system, equipment and medium based on VAE-BiLSTM hybrid neural network and Bayesian optimization. Background Art

[0002] With the rapid popularization of digitalization and information and communication technology in recent years, the development of power systems is becoming more and more modernized. Power systems need to maintain a balance between power generation and load demand to avoid interference with the operation of the power grid due to overload. To solve the above problems, power load forecasting has emerged. Accurate load forecasting can not only optimize power grid dispatching and ensure the safe and stable operation of the entire power system, but also improve economic and social benefits, and ensure the normal life of humans and the normal production of society.

[0003] At present, the methods for short-term load forecasting mainly include traditional statistical methods and prediction methods based on machine learning. Among them, statistical methods mainly include multiple linear regression (MLR), time series analysis and Kalman filtering. The principles and modeling of these methods are simple, but when the data sample size is large, the prediction effect is general. Another type of method is based on machine learning algorithms, such as gray system, artificial neural network, support vector machine (SVM) and Gaussian process (GP). Among them, back propagation (BP) neural network and SVM are the most widely used. However, the above methods lack consideration of time series correlation, and there is a problem of ineffective convergence when there are many training samples. In addition, since the selection of hyperparameters in neural networks is very difficult and underfitting or overfitting is prone to occur, the selection of hyperparameters also greatly affects the accuracy of load forecasting.

[0004] The patent application document with publication number CN118117577A discloses a load forecasting method, system, device and readable storage medium, which uses big data and deep learning technology to build and train load forecasting models through data collection, cleaning and preprocessing, and handles load forecasting uncertainty through probabilistic modeling and Monte Carlo simulation, thereby improving the accuracy and stability of load forecasting and providing reliable decision support for power system management. However, since probabilistic modeling and Monte Carlo simulation methods are often based on simplified statistical assumptions, they may not be able to fully capture the complex nonlinear structure and long-term dependence of data, and therefore have problems such as not being able to better capture the correlation and patterns in time series data. Summary of the invention

[0005] In order to overcome the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a short-term power load forecasting method, system, device and medium based on VAE-BiLSTM hybrid neural network and Bayesian optimization, which removes noise and redundancy in the original power data through a variational autoencoder (VAE), extracts potential features that are more critical to load forecasting, and provides a more refined and representative input for subsequent prediction models; uses a bidirectional long short-term memory network (BiLSTM) to simultaneously capture past and future information in the time series, more comprehensively understands the dynamic changes and long-term dependencies in the power data, and fully considers the impact of time series correlation on the prediction results, thereby achieving higher accuracy prediction in power load forecasting; in addition, a Bayesian optimization algorithm is used to ensure efficient and systematic selection of hyperparameters, avoid overfitting or underfitting, and provide a higher level of robustness and faster convergence speed.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization includes the following steps:

[0008] Step 1: preprocessing the acquired load data related to the power system and characteristic data affecting the load, wherein the preprocessing includes removing missing values, outliers and normalizing to obtain a preprocessed data set;

[0009] Step 2: Perform feature dimensionality reduction on the data set preprocessed in step 1 using kernel principal component analysis (KPCA) to obtain data containing principal components after dimensionality reduction;

[0010] Step 3: Use the Bayesian optimization method to optimize the hyperparameters of the network structure of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), and obtain the variational autoencoder (VAE) model and the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters;

[0011] Step 4, using the variational autoencoder (VAE) with the best hyperparameters obtained in step 3 to train the data containing the principal components of the data after dimensionality reduction obtained in step 2, to obtain the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure;

[0012] Step 5, input the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure obtained in step 4 into the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters obtained in step 3 to perform load forecasting, and obtain the result of short-term power load forecasting;

[0013] Step six, evaluating the accuracy of the short-term power load forecasting results obtained in step five.

[0014] The specific method of step one is:

[0015] Obtain the load data related to the power system and the characteristic data affecting the load, detect the missing values ​​and abnormal values ​​therein, replace the missing values ​​and abnormal values ​​with the average value of all normal values ​​in the obtained load data related to the power system, and perform minimum-maximum normalization processing on the replaced data, which is expressed as:

[0016]

[0017] Among them, x' represents the data after normalization processing, x is the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load, min(x) is the minimum value of the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load, max(x) is the maximum value of the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load, and the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load are scaled to between (0, 1) by normalization.

[0018] The specific method of step 2 is:

[0019] Select the kernel function of nonlinear processing, and for each sample point in the data set preprocessed in step 1, use the selected kernel function to calculate its kernel function value with all other sample points, and construct the kernel matrix K; the element K of the kernel matrix K ij It is expressed as:

[0020] K ij = k(x i ,x j )

[0021] Among them, k is the kernel function, x i , x j is the sample point in the data set after preprocessing in step 1; next, the kernel matrix K is centered, and principal component analysis PCA is applied in high-dimensional space to centralize the kernel matrix It is expressed as:

[0022]

[0023] Among them, k is the kernel matrix, e is the all-1 vector, and n is the number of samples. After obtaining the centralized kernel matrix, perform eigendecomposition on it and solve the largest eigenvalue λ of the dimension m to which it is reduced i and the corresponding eigenvector ui ; Use the selected eigenvector u i Construct a mapping matrix to map the data set preprocessed in step 1 to the new principal component space; the mapping matrix T is expressed as:

[0024]

[0025] Among them, u i is the kernel matrix K corresponding to the i-th eigenvalue λ i , m is the number of principal components selected; the data set preprocessed in step 1 is mapped to the new principal component space through the mapping matrix T to obtain the data containing the principal components after dimensionality reduction.

[0026] The specific method of step three is:

[0027] For the hyperparameters in the network structure of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), the Bayesian optimization algorithm is used to obtain the optimal hyperparameter combination of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM); the hyperparameter space Θ is defined, which is expressed as:

[0028] Θ={θ1, θ2, θ3,..., θ n}

[0029] Among them, θ i Represent the hyperparameters of the network structure including the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), establish a probability model for the hyperparameters, and establish an objective function f(θ) using the mean square error, which is expressed as:

[0030] f(θ)=MSE validation (Model(θ))

[0031] Using the Bayesian algorithm, a point θ is selected from the hyperparameter space for sampling. According to the performance of the sampling point, the posterior distribution of the hyperparameter is updated, and finally a variational autoencoder (VAE) model and a bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters are obtained.

[0032] The specific method of step 4 is:

[0033] A variational autoencoder (VAE) with optimal hyperparameters is used to extract features from the data containing the principal components of the data obtained in step 2 after dimensionality reduction and learn the potential distribution of the data. The variational autoencoder (VAE) includes an encoder, reparameterization, a latent space, and a decoder. During the training process of the variational autoencoder (VAE), the probability distribution q(z|x) represents the distribution of the latent variable z in the latent space when the input data x is given, which satisfies:

[0034] q(z|x)=N(z|μ(x),σ(x) 2 )

[0035] Among them, μ(x) and σ(x) represent the mean and standard deviation parameters of the latent variable z respectively. The reparameterization technique is introduced to optimize the variational autoencoder (VAE) through the back propagation algorithm:

[0036] z=μ(x)+σ(x)☉∈

[0037] Among them, ∈ is random noise sampled from the standard normal distribution N(0,i); the difference between the output data reconstructed by the decoder part of the variational autoencoder (VAE) and the data containing the principal components of the data after dimensionality reduction obtained in step 2 is measured by using the mean square error (MSE), and the KL divergence is used as a regularization term. The two are combined to form the total loss function, which is expressed as:

[0038]

[0039] Among them, x is the original data, is the output of the decoder network in the variational autoencoder (VAE) structure, that is, the reconstructed data. β is a hyperparameter used to balance the weight between the reconstruction loss and the KL divergence. The Adam optimizer is selected to minimize the loss function. The data containing the principal components of the data after dimensionality reduction obtained in step 2 is input into the variational autoencoder (VAE) for training, and the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure is obtained.

[0040] The specific method of step five is:

[0041] Define the network layer of the BiLSTM network: BiLSTM = LSTM (forward) + LSTM (backward), where LSTM (forward) processes the forward sequence and LSTM (backward) processes the reverse sequence. Connect the information in both directions to merge and define the output of the BiLSTM network. Select the Adam algorithm for optimization and define the loss function. Input the encoded features into the trained BiLSTM model for prediction. The predicted value can be expressed as:

[0042]

[0043] Among them, g is the activation function, W y and b y are the weights and biases of the output layer, BiLSTM(h t ) is the output of the bidirectional long short-term memory network (BiLSTM), expressed as:

[0044] BiLSTM(h t )=[LSTM f (h t ); LSTM b (h t )]

[0045] Among them, LSTM f and LSTM b are the hidden states of the forward and reverse LSTM networks (LSTM) at time step t, respectively.

[0046] The specific method of step six is:

[0047] The mean square error (MSE) and mean absolute percentage error (MAPE) are used to evaluate the accuracy of the prediction results, expressed as:

[0048]

[0049] Where N is the number of samples, y i is the actual value of the i-th sample, is the model's predicted value.

[0050] A short-term power load forecasting system based on VAE-BiLSTM hybrid neural network and Bayesian optimization, comprising:

[0051] A data acquisition and preprocessing module is used to preprocess the acquired load data related to the power system and the characteristic data affecting the load, wherein the preprocessing includes removing missing values, abnormal values ​​and normalizing to obtain a preprocessed data set;

[0052] The feature extraction module is used to perform feature dimension reduction on the preprocessed data set using kernel principal component analysis (KPCA) to obtain data containing principal components after dimension reduction;

[0053] The neural network optimization module is used to optimize the hyperparameters of the network structures of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM) using the Bayesian optimization method to obtain the variational autoencoder (VAE) model and the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters;

[0054] A neural network training module is used to train the data containing the principal components of the data after dimensionality reduction using a variational autoencoder (VAE) with optimal hyperparameters, and obtain an encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure;

[0055] The prediction result output module is used to input the encoding layer containing the potential representation of data in the trained variational autoencoder (VAE) network structure into a bidirectional long short-term memory network (BiLSTM) model with optimal hyperparameters for load forecasting, and obtain the result of short-term power load forecasting;

[0056] The forecast result evaluation module is used to evaluate the accuracy of the results of short-term power load forecasting.

[0057] The present invention also provides a short-term power load forecasting device based on VAE-BiLSTM hybrid neural network and Bayesian optimization, comprising:

[0058] Memory: a computer program for storing the above-mentioned short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization, which is a computer-readable device;

[0059] Processor: used to implement the short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization when executing the computer program.

[0060] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1. The present invention converts high-dimensional data into low-dimensional feature representation through kernel principal component analysis (KPCA). As an effective dimensionality reduction technique, KPCA can capture the nonlinear structure in the data, thereby providing a more compact and information-rich feature representation for the variational autoencoder (VAE). In addition, the reduced-dimensional data helps to avoid overfitting, improve the generalization ability of the model, and ensure that the variational autoencoder (VAE) can make accurate predictions on new or unseen data.

[0063] 2. The present invention provides a powerful prediction framework by combining a variational autoencoder (VAE) with a bidirectional long short-term memory network (BiLSTM), wherein the variational autoencoder (VAE) first extracts the deep features and potential representations of the data through its encoder, which capture the complex structure and pattern of the input data by learning the distribution of the data; subsequently, these features are used as the input of the bidirectional long short-term memory network (BiLSTM), which uses its bidirectional understanding ability of time series to further analyze and predict the future trend of the series. In this process, the variational autoencoder (VAE) handles the high dimensionality and nonlinearity of the data, while the bidirectional long short-term memory network (BiLSTM) focuses on the dynamic characteristics of the time series. The synergy of the two not only enhances the prediction accuracy of the model for time series data, but also improves the generalization ability to new situations, making the overall model more effective and robust in dealing with prediction tasks with complex time dependencies.

[0064] 3. The present invention provides a probability model between hyperparameters and model performance constructed by Bayesian optimization, which can intelligently explore the hyperparameter space within a small number of iterations and quickly converge to the optimal solution. In the VAE-BiLSTM framework, this means that key hyperparameters such as learning rate and number of hidden layer neurons can be automatically adjusted for the reparameterization technique of the variational autoencoder VAE and the time series prediction ability of the bidirectional long short-term memory network BiLSTM, thereby improving the prediction accuracy and generalization ability of the model. In addition, the sampling strategy of Bayesian optimization reduces the demand for computing resources, making the VAE-BiLSTM combined model training process more efficient, and can significantly shorten the time of model tuning while ensuring prediction performance.

[0065] In summary, the power load forecasting model of the present invention realizes an accurate, efficient and robust forecasting framework through dimensionality reduction processing of kernel principal component analysis (KPCA), deep feature extraction of variational autoencoder (VAE), time series prediction of bidirectional long short-term memory network (BiLSTM) and hyperparameter adjustment of Bayesian optimization, which provides strong technical support for load management and optimal scheduling of power systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 The figure is a flow chart of the implementation method of the present invention.

[0067] Figure 2 A schematic diagram of the flow of the kernel principal component analysis (KPCA) method provided by the present invention.

[0068] Figure 3 A schematic diagram of the variational autoencoder (VAE) neural network structure provided for an embodiment of the present invention.

[0069] Figure 4 A schematic diagram of the long short-term memory network (LSTM) structure provided for an embodiment of the present invention.

[0070] Figure 5 A schematic diagram of the structure of a bidirectional long short-term memory network (BiLSTM) provided in an embodiment of the present invention.

[0071] Figure 6 A schematic diagram of the Bayesian optimization method flow chart provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0072] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0073] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0074] Figure 1 The schematic diagram of the implementation flow of the short-term power load forecasting method based on the VAE-BiLSTM hybrid neural network and Bayesian optimization provided in the embodiment of the present application includes the following steps:

[0075] Step 101, preprocessing the acquired load data related to the power system and characteristic data affecting the load, wherein the preprocessing includes removing missing values, abnormal values ​​and normalizing to obtain a preprocessed data set;

[0076] By obtaining historical load data and other observation data that affects historical load from power companies, meteorological services, and power market operators, including but not limited to: power generation data, power demand data, meteorological data (such as temperature, humidity, wind speed, etc.) and power trading price related data information.

[0077] Optionally, after obtaining the relevant information, the visualization of the relevant data on the power grid side can be realized, so as to avoid the problem of obtaining electricity consumption-related data directly from various places for many times, which is cumbersome and consumes a lot of financial, material and human resources to process, and promote the effective acquisition of data.

[0078] In this embodiment, load data and corresponding observation data can be obtained by setting a specific sampling frequency, and common sampling frequencies include once every 15 minutes or 20 minutes. This sampling strategy allows researchers to capture short-term fluctuations in power demand and provide necessary time series information for load forecasting.

[0079] Subsequently, based on the framework of sliding time windows, historical load data and observed data are used to construct a prediction model. Specifically, the load data at continuous time points are selected as the feature set, and the data at the specific time point immediately following it is used as the label set. In this framework, the feature set is used to predict the load value at the immediately following time point, that is, the label. For example, if the selected time window length is m, the load data at the first m time points are selected as feature inputs, and the load data at the m+1th time point is used as the predicted target label. By training the model to learn the mapping relationship between features and labels, the load value at a certain time point in the future can be predicted. As time progresses, the time window slides forward to generate a new time window sequence with the same time window length. This process is repeated continuously, thereby accumulating multiple sets of time window sequence data. In this way, the model can continuously update its predictions to adapt to the time dependency and dynamic change characteristics of the load data.

[0080] Taking into account the situation that the acquired data may contain missing values ​​and outliers due to various external factors, such as sensor failure, data transmission interruption, measurement error, equipment failure or external emergencies, a series of data preprocessing steps must be performed after obtaining the original load data and its observation data to ensure the reliability and validity of the data set, for example, estimating missing values ​​through interpolation methods, using the sliding average of time series or using statistical information of historical data to fill in, or using statistical methods or machine learning techniques to identify and correct these outliers, such as the IQR method based on box plots or outlier detection based on clustering algorithms.

[0081] In order to make the model learn more efficiently and extract effective features from the data, the processed data must be normalized. Normalization usually involves scaling the data to a uniform range, such as the (0,1) interval, or converting it to a distribution with zero mean and unit variance. Through this step, the acquired load data related to the power system and the characteristic data affecting the load are scaled to between (0, 1) after replacing missing values ​​and outliers. This not only helps to speed up the convergence of model training, but also avoids weight initialization problems caused by differences in numerical scales.

[0082] like Figure 2 As shown, step 102, using kernel principal component analysis (KPCA) to perform feature dimension reduction on the data set preprocessed in step 101, to obtain data containing data principal components after dimension reduction;

[0083] In this embodiment, kernel principal component analysis, as a dimensionality reduction technique, has significant advantages in processing raw data, especially in processing data sets with nonlinear structures. The core of this method is that it can map raw data to a high-dimensional feature space through a kernel function, and apply linear principal component analysis in this space to capture the main change direction of the data. Due to the introduction of kernel techniques, kernel principal component analysis can reveal the intrinsic structure of data that cannot be directly observed in the original space, which is impossible in traditional linear principal component analysis. Another advantage of kernel principal component analysis is its flexibility and adaptability. By selecting kernel functions that can be processed nonlinearly according to different characteristics of the data, such as RBF kernel, polynomial kernel and Sigmoid kernel, the complexity and fit of the model are adjusted according to the specific characteristics of the data. In time series prediction tasks such as power load prediction, the dimensionality reduction effect of kernel principal component analysis not only helps to reduce the storage requirements and computational complexity of data, but also removes noise and redundant features, thereby improving the efficiency and accuracy of subsequent analysis or model training. In addition, kernel principal component analysis can effectively extract key features reflecting load changes from historical data, laying the foundation for building an accurate prediction model.

[0084] The implementation of kernel principal component analysis in this embodiment can be implemented by the scientific computing library scikit-learn in Python. For each sample point in the data set preprocessed in step 101, the selected kernel function is used to calculate the kernel function value of the sample point and all other sample points, and the kernel matrix K is constructed. The elements K of the kernel matrix ij It is expressed as:

[0085] K ij = k(x i ,x j )

[0086] Among them, k is the kernel function, x i , x j is the sample point in the data set preprocessed in step 101. Next, the kernel matrix K is centered by applying principal component analysis in high-dimensional space. It can be expressed as:

[0087]

[0088] Among them, K is the kernel matrix, e is the all-1 vector, and n is the number of samples; after obtaining the centralized kernel matrix, perform eigendecomposition on it and solve its m (the number of dimensions to be reduced) largest eigenvalues ​​λ i and the corresponding eigenvector u i ; Use the selected eigenvector u i Construct a mapping matrix to map the data set preprocessed in step 101 to the new principal component space; the mapping matrix T can be expressed as:

[0089]

[0090] Among them, u i is the kernel matrix K corresponding to the i-th eigenvalue λ i , m is the number of principal components selected; the data set preprocessed in step 101 is mapped to the new principal component space through the mapping matrix T to obtain the data containing the principal components after dimensionality reduction.

[0091] like Figure 6 As shown, in step 103, the network structure of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM) is optimized using the Bayesian optimization method to obtain a variational autoencoder (VAE) model and a bidirectional long short-term memory network (BiLSTM) model with optimal hyperparameters;

[0092] like Figure 3 The diagram shows the structure of the variational autoencoder (VAE). The variational autoencoder (VAE) includes an input layer for inputting power-related data; an encoding layer responsible for converting the input data into a latent space representation; wherein the latent space representation is usually the logarithm of the mean and variance, because the logarithmic variance is easier to handle; a decoding layer responsible for converting the latent space representation back to the data space; and an output layer for the final reconstructed data or new samples.

[0093] like Figure 5 The figure shows the structure of a bidirectional long short-term memory network (BiLSTM). In a bidirectional long short-term memory network (BiLSTM), each bidirectional long short-term memory network layer consists of two long short-term memory network units, corresponding to the forward and backward propagation of the data sequence, respectively. The forward LSTM unit LSTM (forward) processes the information of the input sequence from the current time step to the future time step, and the backward LSTM unit LSTM (backward) processes the information of the input sequence from the current time step to the past time step. Multiple layers of LSTM are stacked together to capture more complex time dependencies, and each layer further abstracts and learns features based on the previous layer.

[0094] Considering that hyperparameters directly affect the performance, generalization ability and final application effect of the model, Bayesian optimization is an efficient global optimization algorithm. The core advantage of this method lies in its ability to construct a probabilistic model of hyperparameters and use prior knowledge and historical evaluation results to guide the search process, thereby quickly locating the optimal or near-optimal hyperparameter combination with fewer evaluations.

[0095] In this embodiment, Bayesian optimization is used to perform hyperparameter optimization on the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), and the hyperparameter space Θ is defined, which is expressed as:

[0096] θ={θ1,θ2,θ3,...,θ n}

[0097] Among them, θ i Represents hyperparameters including learning rate, batch size, number of network layers, regularization parameters, etc., establishes a probability model for the hyperparameters, and establishes an objective function f(θ) using mean square error, which is expressed as:

[0098] f(θ)=MSE validation (Model(θ))

[0099] Using the Bayesian algorithm, a point θ is selected from the hyperparameter space for sampling. According to the performance of the sampling point, the posterior distribution of the hyperparameter is updated, and finally a variational autoencoder (VAE) model and a bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters are obtained.

[0100] Step 104, using the variational autoencoder (VAE) with the best hyperparameters obtained in step 103 to train the data containing the data principal components after dimensionality reduction obtained in step 102, to obtain a coding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure;

[0101] Considering that kernel principal component analysis may not fully capture the complex patterns and structures of the data, variational autoencoders (VAEs) can provide a nonlinear conversion mechanism that enables variational autoencoders to discover more complex data relationships in high-dimensional feature spaces. Variational autoencoders (VAEs) can also further extract deep features based on the reduced-dimensional data. Variational autoencoders can learn more abstract and advanced representations of data through their encoder networks. In addition, the reparameterization technique and KL divergence regularization term of variational autoencoders help prevent model overfitting and improve the generalization ability of the model.

[0102] Combination Figure 3 , design the architecture of the variational autoencoder model, including the encoder and decoder networks. The encoder is responsible for mapping the input data to the parameters (mean and variance) of the latent space, and the decoder is responsible for reconstructing the points in the latent space back to the data space, passing the data processed by the kernel principal component analysis as input to the variational autoencoder for training:

[0103] The conditional probability distribution q(z|x) is given to represent the distribution of the latent variable z given the input data x, which satisfies:

[0104] q(z|x)=N(z|μ(x),σ(x) 2 )

[0105] Among them, μ(x) and σ(x) represent the mean and standard deviation parameters of the latent variable z respectively. Next, we introduce the reparameterization technique to optimize the variational autoencoder (VAE) through the back propagation algorithm:

[0106] z=μ(x)+σ(x)☉∈

[0107] Among them, ∈ is random noise sampled from the standard normal distribution N(0,i); the difference between the output data reconstructed by the decoder part of the variational autoencoder (VAE) and the data containing the principal components of the data after dimensionality reduction obtained in step 2 is measured by using the mean square error (MSE), and the KL divergence is used as a regularization term. The two are combined to form the total loss function, which is expressed as:

[0108]

[0109] Among them, x is the original data, is the output of the decoder network in the variational autoencoder (VAE) structure, that is, the reconstructed data, and β is a hyperparameter used to balance the weight between the reconstruction loss and the KL divergence; the Adam optimizer is selected to minimize the loss function, and the data containing the principal components of the data after dimensionality reduction obtained in step 102 is input into the variational autoencoder (VAE) for training, so as to obtain the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure.

[0110] Step 105, inputting the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure obtained in step 104 into the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters obtained in step 103 to perform load forecasting, and obtaining the result of short-term power load forecasting;

[0111] like Figure 4 As shown in the figure, considering that in applications such as power load forecasting, time series data usually exhibits strong time dependence and dynamics, the BiLSTM network can learn the long-term dependencies hidden in the sequence data through its forward and reverse LSTM units, and can understand and predict the periodic and trend changes in the data. In addition, the gating mechanism of the BiLSTM network (including input gate, forget gate and output gate) provides the network with effective memory and forgetting capabilities, enabling it to ignore irrelevant information while maintaining important information.

[0112] There may be complex nonlinear relationships in power load data. Through weight updates during training, the bidirectional long short-term memory network can automatically learn the nonlinear features in the data without complex feature engineering, which provides additional advantages for prediction tasks.

[0113] In this embodiment, it is ensured that the data format of the output feature representation of the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure in step 104 can be used as the input of the bidirectional long short-term memory network (BiLSTM) for short-term load forecasting to obtain the forecast result. Define the network layer of the bidirectional long short-term memory network (BiLSTM): BiLSTM = LSTM (forward) + LSTM (backward), where LSTM (forward) processes the forward sequence and LSTM (backward) processes the reverse sequence. Connect the information in both directions to merge the output of the bidirectional long short-term memory network (BiLSTM), select the Adam algorithm for optimization and define the loss function, input the encoded features into the trained bidirectional long short-term memory network (BiLSTM) model for forecasting, and the forecast value can be expressed as:

[0114]

[0115] Among them, g is the activation function, W y and b y are the weights and biases of the output layer, BiLSTM(h t ) is the output of the bidirectional long short-term memory network (BiLSTM), expressed as:

[0116] BiLSTM(h t )=[LSTM f (h t ); LSTM b (h t )]

[0117] Among them, LSTM f and LSTM b are the hidden states of the forward and reverse LSTM networks (LSTM) at time step t, respectively.

[0118] Combination Figure 5, the encoding layer of the variational autoencoder (VAE) is responsible for mapping the input data into a low-dimensional representation of a latent space. When these latent variables are used as inputs to the bidirectional long short-term memory network (BiLSTM), the bidirectional long short-term memory network (BiLSTM) can focus on using these refined feature representations to learn the temporal dynamics of sequence data. Since the encoding layer of the variational autoencoder has reduced the dimension and extracted features of the original data, the bidirectional long short-term memory network no longer needs to process the noise and redundant information in the original data, thereby improving the efficiency of model training and the accuracy of prediction.

[0119] The output of the encoding layer of the variational autoencoder is deterministic and continuous, which makes it easier for the bidirectional long short-term memory network to process input data and perform effective time series forecasting on this basis. The bidirectional long short-term memory network can consider both historical and future information through its unique bidirectional architecture, further enhancing the model's ability to understand time series data.

[0120] Step 106, evaluating the accuracy of the short-term power load forecasting result obtained in step 5.

[0121] In this embodiment, the output results can be evaluated using mean square error (MSE) and mean absolute percentage error (MAPE), expressed as:

[0122]

[0123] Where N is the number of samples, y i is the actual value of the i-th sample, is the model's predicted value.

[0124] The present invention relates to a short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization. The method comprises: replacing null values ​​and outliers with mean values ​​on the acquired power data, and performing minimum-maximum normalization processing; performing feature dimension reduction by performing kernel principal component analysis on the processed data; performing further feature extraction on the reduced data using a variational autoencoder and learning the potential distribution of the data; ensuring that the feature representation output by the variational autoencoder is serialized, and then using a bidirectional long short-term memory network for load forecasting; using Bayesian optimization to optimize the hyperparameters of the variational autoencoder and the bidirectional long short-term memory network; using mean square error and mean absolute percentage error to evaluate performance and compare with other models. The present invention can fully consider the time series correlation and the reasonable selection of hyperparameters in power load forecasting, and improve the accuracy of power load forecasting.

[0125] In addition, the present invention also provides a short-term power load forecasting system based on VAE-BiLSTM hybrid neural network and Bayesian optimization, including:

[0126] A data acquisition and preprocessing module, used to implement preprocessing of the load data related to the power system and the characteristic data affecting the load acquired in step 1, wherein the preprocessing includes removing missing values, outliers and normalizing to obtain a preprocessed data set;

[0127] The feature extraction module is used to implement feature dimension reduction using kernel principal component analysis (KPCA) on the data set preprocessed in step 1 in step 2, and obtain data containing data principal components after dimension reduction;

[0128] A neural network optimization module is used to implement the hyperparameter optimization of the network structure of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM) using the Bayesian optimization method in step 3, and obtain a variational autoencoder (VAE) model and a bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters;

[0129] A neural network training module is used to implement step 4, using the variational autoencoder (VAE) with the best hyperparameters obtained in step 3 to train the data containing the principal components of the data obtained in step 2 after dimensionality reduction, to obtain a coding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure;

[0130] The prediction result output module is used to implement step 5, inputting the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure obtained in step 4 into the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters obtained in step 3 to perform load forecasting, and obtain the result of short-term power load forecasting;

[0131] The prediction result evaluation module is used to implement the accuracy evaluation of the short-term power load prediction result obtained in step five in step six.

[0132] The present invention also provides a short-term power load forecasting device based on VAE-BiLSTM hybrid neural network and Bayesian optimization, comprising:

[0133] Memory: a computer program for storing the above-mentioned short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization, which is a computer-readable device;

[0134] Processor: used to implement the short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization when executing the computer program.

[0135] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization.

[0136] The embodiment of the present invention obtains historical load data and other observation data that affects historical load, and performs null value, outlier and normalization processing on the obtained data; uses kernel principal component analysis to perform feature dimension reduction on the processed data; uses the data after feature dimension reduction to train the variational autoencoder and bidirectional long short-term memory network to achieve the optimal prediction model; and uses the above model to output the short-term power load prediction results. The implementation of the present invention, by integrating advanced data processing technology and machine learning models, makes the prediction method more complete and the prediction results more accurate, showing significant comprehensive performance advantages, and providing strong technical support for the intelligent management and optimized scheduling of power systems.

[0137] Optionally, in some embodiments of the present invention, the historical load data obtained and other observation data affecting the historical load may include power generation data, power demand data, meteorological data (such as temperature, humidity, wind speed, etc.) and power trading prices and other related information obtained from power companies, meteorological services, and power market operators.

[0138] Optionally, in some embodiments of the present invention, the sum function selected using kernel principal component analysis includes but is not limited to: polynomial kernel, radial basis function kernel, sigmoid kernel, Laplace kernel, histogram kernel, etc. In addition to the above-mentioned standard kernel functions, researchers can also design custom kernel functions according to specific applications.

[0139] The various embodiments of the present invention are described in a related manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments.

[0140] In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0141] Those skilled in the art will appreciate that the templates, units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0142] If the module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the embodiment of the short-term power load forecasting method proposed in the present invention, whether it is the entire process or part of the process, can be driven by a computer program to execute the relevant hardware. The computer program is intended to convert the coded instructions into a machine-understandable format to implement the various steps of power load forecasting. Further, the computer program can be encoded in a variety of forms, including but not limited to source code, object code, executable files or intermediate state codes. The physical carrier of these codes, that is, the computer-readable storage medium, can be diversified, covering from traditional physical recording media to modern digital storage devices, such as USB flash drives, mobile hard disks, magnetic disks, optical disks, and storage components inside computers, such as random access memory (RAM), read-only memory (ROM), etc. In addition, wireless transmission methods such as electric carrier signals and telecommunication signals, as well as software distribution through the Internet, have also become effective ways to spread the computer program code.

[0143] The technical application of the present invention is not limited to the technical means previously disclosed, but includes technical solutions composed of various combinations of these technical features. Although the preferred embodiments of the present invention are described in detail above, it is worth emphasizing that ordinary technicians engaged in the relevant technical field may still make various improvements and modifications without departing from the principles of the present invention. These improvements and modifications are also considered to be part of the protection scope of the present invention. Therefore, the protection scope of the present invention should cover all equivalent embodiments that meet the combination of technical features, and is not limited to the specifically disclosed embodiments.

Claims

1. A short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization, characterized in that: The following steps are involved: Step 1: preprocessing the acquired load data related to the power system and characteristic data affecting the load, wherein the preprocessing includes removing missing values, outliers and normalizing to obtain a preprocessed data set; Step 2: Perform feature dimensionality reduction on the data set preprocessed in step 1 using kernel principal component analysis (KPCA) to obtain data containing principal components after dimensionality reduction; Step 3: Use the Bayesian optimization method to optimize the hyperparameters of the network structure of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), and obtain the variational autoencoder (VAE) model and the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters; Step 4, using the variational autoencoder (VAE) with the best hyperparameters obtained in step 3 to train the data containing the principal components of the data after dimensionality reduction obtained in step 2, to obtain the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure; Step 5, input the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure obtained in step 4 into the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters obtained in step 3 to perform load forecasting, and obtain the result of short-term power load forecasting; Step six, evaluating the accuracy of the short-term power load forecasting results obtained in step five.

2. A short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization according to claim 1, characterized in that: The specific method of step one is: Obtain the load data related to the power system and the characteristic data affecting the load, detect the missing values ​​and abnormal values ​​therein, replace the missing values ​​and abnormal values ​​with the average value of all normal values ​​in the obtained load data related to the power system, and perform minimum-maximum normalization processing on the replaced data, which is expressed as: Among them, x' represents the data after normalization processing, x is the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load, min(x) is the minimum value of the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load, max(x) is the maximum value of the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load, and the data after replacing missing values ​​and abnormal values ​​of the acquired load data related to the power system and the characteristic data affecting the load are scaled to between (0, 1) by normalization.

3. A short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization according to claim 1, characterized in that: The specific method of step 2 is: Select the kernel function of nonlinear processing, and for each sample point in the data set preprocessed in step 1, use the selected kernel function to calculate its kernel function value with all other sample points, and construct the kernel matrix K; the element K of the kernel matrix K ij It is expressed as: K ij =k(x i ,x j ) Among them, k is the kernel function, x i , x j is the sample point in the data set after preprocessing in step 1; next, the kernel matrix K is centered, and principal component analysis PCA is applied in high-dimensional space to centralize the kernel matrix It is expressed as: Among them, K is the kernel matrix, e is the all-1 vector, and n is the number of samples; after obtaining the centralized kernel matrix, perform eigendecomposition on it and solve the largest eigenvalue λ of the dimension m to which it is reduced i and the corresponding eigenvector u i ; Use the selected eigenvector u i Construct a mapping matrix to map the data set preprocessed in step 1 to the new principal component space; the mapping matrix T is expressed as: Among them, u i is the kernel matrix K corresponding to the i-th eigenvalue λ i , m is the number of principal components selected; the data set preprocessed in step 1 is mapped to the new principal component space through the mapping matrix T to obtain the data containing the principal components after dimensionality reduction.

4. A short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization according to claim 1, characterized in that: The specific method of step three is: For the hyperparameters in the network structure of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), the Bayesian optimization algorithm is used to obtain the optimal hyperparameter combination of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM); the hyperparameter space Θ is defined, which is expressed as: Θ={θ1,θ2,θ3,…,θ n } Among them, θ i Represent the hyperparameters of the network structure including the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM), establish a probability model for the hyperparameters, and establish an objective function f(θ) using the mean square error, which is expressed as: f(θ)=MSE validation (Model(θ)) Using the Bayesian algorithm, a point θ is selected from the hyperparameter space for sampling. According to the performance of the sampling point, the posterior distribution of the hyperparameter is updated, and finally a variational autoencoder (VAE) model and a bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters are obtained.

5. The short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization according to claim 1 is characterized in that: The specific method of step 4 is: A variational autoencoder (VAE) with optimal hyperparameters is used to extract features from the data containing the principal components of the data obtained in step 2 after dimensionality reduction and learn the potential distribution of the data. The variational autoencoder (VAE) includes an encoder, reparameterization, a latent space, and a decoder. During the training process of the variational autoencoder (VAE), the probability distribution q(z|x) represents the distribution of the latent variable z in the latent space when the input data x is given, which satisfies: q(z|x)=N(z|μ(x),σ(x) 2 ) Among them, μ(x) and σ(x) represent the mean and standard deviation parameters of the latent variable z respectively. The reparameterization technique is introduced to optimize the variational autoencoder (VAE) through the back propagation algorithm: z=μ(x)+σ(x)⊙∈ Among them, ∈ is random noise sampled from the standard normal distribution N(0,i); the difference between the output data reconstructed by the decoder part of the variational autoencoder (VAE) and the data containing the principal components of the data after dimensionality reduction obtained in step 2 is measured by using the mean square error (MSE), and the KL divergence is used as a regularization term. The two are combined to form the total loss function, which is expressed as: Among them, x is the original data, is the output of the decoder network in the variational autoencoder (VAE) structure, that is, the reconstructed data. β is a hyperparameter used to balance the weight between the reconstruction loss and the KL divergence. The Adam optimizer is selected to minimize the loss function. The data containing the principal components of the data after dimensionality reduction obtained in step 2 is input into the variational autoencoder (VAE) for training, and the encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure is obtained.

6. A short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization according to claim 1, characterized in that: The specific method of step five is: Define the network layer of the BiLSTM network: BiLSTM = LSTM (forward) + LSTM (backward), where LSTM (forward) processes the forward sequence and LSTM (backward) processes the reverse sequence. Connect the information in both directions to merge and define the output of the BiLSTM network. Select the Adam algorithm for optimization and define the loss function. Input the encoded features into the trained BiLSTM model for prediction. The predicted value can be expressed as: Among them, g is the activation function, W y and b y are the weights and biases of the output layer, BiLSTM(h t ) is the output of the bidirectional long short-term memory network (BiLSTM), expressed as: BiLSTM(h t )=[LSTM f (h t );LSTM b (h t )] Among them, LSTM f and LSTM b are the hidden states of the forward and reverse LSTM networks (LSTM) at time step t, respectively.

7. The short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization according to claim 1 is characterized in that: The specific method of step six is: The mean square error (MSE) and mean absolute percentage error (MAPE) are used to evaluate the accuracy of the prediction results, expressed as: Where N is the number of samples, y i is the actual value of the i-th sample, is the model's predicted value.

8. A short-term power load forecasting system based on a VAE-BiLSTM hybrid neural network and Bayesian optimization based on the method according to any one of claims 1 to 7, characterized in that: include: A data acquisition and preprocessing module is used to preprocess the acquired load data related to the power system and the characteristic data affecting the load, wherein the preprocessing includes removing missing values, abnormal values ​​and normalizing to obtain a preprocessed data set; The feature extraction module is used to perform feature dimension reduction on the preprocessed data set using kernel principal component analysis (KPCA) to obtain data containing principal components after dimension reduction; The neural network optimization module is used to optimize the hyperparameters of the network structures of the variational autoencoder (VAE) and the bidirectional long short-term memory network (BiLSTM) using the Bayesian optimization method to obtain the variational autoencoder (VAE) model and the bidirectional long short-term memory network (BiLSTM) model with the best hyperparameters; A neural network training module is used to train the data containing the principal components of the data after dimensionality reduction using a variational autoencoder (VAE) with optimal hyperparameters, and obtain an encoding layer containing the potential representation of the data in the trained variational autoencoder (VAE) network structure; The prediction result output module is used to input the encoding layer containing the potential representation of data in the trained variational autoencoder (VAE) network structure into a bidirectional long short-term memory network (BiLSTM) model with optimal hyperparameters for load forecasting, and obtain the result of short-term power load forecasting; The forecast result evaluation module is used to evaluate the accuracy of the results of short-term power load forecasting.

9. A short-term power load forecasting device based on VAE-BiLSTM hybrid neural network and Bayesian optimization, characterized in that: include: Memory: a computer program storing a short-term power load forecasting method based on a VAE-BiLSTM hybrid neural network and Bayesian optimization as described in any one of claims 1 to 7, which is a computer-readable device; Processor: used to implement a short-term power load forecasting method based on VAE-BiLSTM hybrid neural network and Bayesian optimization as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it can implement a short-term power load forecasting method based on a VAE-BiLSTM hybrid neural network and Bayesian optimization as described in any one of claims 1 to 7.

Images