A power short-term load forecasting method based on deep reinforcement learning and transfer learning
By employing deep reinforcement learning and transfer learning methods, and using GRU networks and multi-core maximum mean difference algorithms, the problem of limited training samples in short-term load forecasting of power systems is solved, achieving higher accuracy in load forecasting.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2023-01-16
- Publication Date
- 2026-05-29
AI Technical Summary
Existing short-term load forecasting methods for power systems suffer from gradient explosion and gradient vanishing problems when training samples are limited, and traditional methods cannot effectively utilize historical data from other communities for load forecasting.
We employ a method based on deep reinforcement learning and transfer learning, using a GRU network to mitigate the gradient explosion and vanishing problems, and a multi-kernel maximum mean difference algorithm for dynamic distribution alignment, utilizing source domain data for target domain prediction.
It improves the accuracy of power load forecasting when training samples are limited, solves the gradient explosion and vanishing problems, and achieves more accurate short-term load forecasting.
Smart Images
Figure CN116306229B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of power data processing, specifically relating to a short-term power load forecasting method based on deep reinforcement learning and transfer learning. Background Technology
[0002] Short-term load forecasting is an important daily task for power system dispatching and operation departments. The accuracy of the forecast directly affects the safety, economy, and power supply quality of the power system. According to statistical theory, the accuracy of model forecasting is related to the amount of data by a power of -1 / 2. This means that the amount of data plays a crucial role in improving the accuracy of load forecasting models and is a significant driving force for improving load forecasting accuracy.
[0003] Throughout the development of power systems, researchers both domestically and internationally have proposed various schemes for short-term load forecasting. These can be broadly categorized into two types based on their development stages: one is the traditional, classical method, primarily based on mathematical theory. Traditional classical methods are represented by multiple linear regression, time series forecasting, Kalman filtering, and wavelet analysis. These methods rely heavily on pure mathematical derivation, starting from the data itself to find and explore patterns. The other type is newer methods based on artificial intelligence and machine learning theories. Newer methods are represented by support vector machines, random forests, GBDT (Gradient Boosting Decision Tree) ensemble learning models, BPNN (Back Propagation Neural Network), and model combination forecasting methods. These models can learn the relationships between data and possess strong fitting capabilities even under complex conditions. Therefore, research on short-term load forecasting methods for power systems is essential.
[0004] Short-term load forecasting plays a crucial role in the effective operation of modern power systems. Accurate load forecasting helps to control the operation of modern power grids more effectively and safely by avoiding unnecessary interruptions or shortages. In recent years, various deep learning-based time series forecasting algorithms have demonstrated superior performance. Typically, existing time series forecasting algorithms require a sufficient number of training samples to learn a reliable forecasting model. However, there are also situations where only a limited number of training samples are available, such as forecasting the power load of newly built communities. Under these stringent conditions, due to the limited training samples, classical deep learning-based time series forecasting algorithms suffer from high prediction errors and overfitting problems. On the other hand, in real-world scenarios, a large amount of historical data can be collected from other communities with similar backgrounds, which may be helpful in learning load forecasting models. Therefore, the problem of power load forecasting with limited training samples can be addressed from the perspective of transfer learning. The goal is to learn a reliable load forecasting model for a target domain with only a limited number of training samples using a large amount of historical data from other source domains.
[0005] Traditional short-term power load forecasting methods mainly include regression analysis, trend extrapolation, expert systems, and time series methods. Trend extrapolation and regression analysis are primarily suitable for load data exhibiting clear trends and relatively stable load changes. Expert systems rely on the subjective experience of experts, leading to significant individual differences. Time series methods only consider the autocorrelation of the load itself, neglecting in-depth analysis of load influencing factors. Furthermore, due to the significant volatility and randomness in short-term load data, traditional forecasting methods often fail to achieve accurate predictions. Meanwhile, currently popular methods based on artificial intelligence and machine learning theories are only applicable to specific situations. Each model has its own advantages and disadvantages; for example, linear models are more suitable for load data with strong linear correlations, BPNN is suitable for scenarios with complex data relationships, ARIMA autoregressive time series forecasting models can be used for data lacking features, and SVR or GBDT models can be considered for data with sufficient features.
[0006] Current power system load forecasting suffers from various drawbacks. For example, in source domain load forecasting, the popular recurrent neural network forecasting methods suffer from gradient explosion and gradient vanishing problems. Similarly, for target domain load forecasting with only limited or no training samples, current popular load forecasting methods cannot solve the problem of forecasting with too few or no samples. Based on this, this patent proposes a short-term power load forecasting method based on deep reinforcement learning and transfer learning. Summary of the Invention
[0007] To address the problems existing in current technologies, this application proposes a short-term power load forecasting method based on deep reinforcement learning and transfer learning. The method includes:
[0008] Multiple sets of different source domain power load data and target domain power load data are acquired, and the source domain power load data and the target domain power load data are preprocessed respectively.
[0009] Multiple power load source domain prediction models based on GRU networks were trained using preprocessed multiple sets of source domain power load data and corresponding power fusion features.
[0010] The preprocessed target domain power load data is used as input, and the source domain prediction model is selected based on a time series dual-depth Q network.
[0011] The multi-core maximum mean difference algorithm is used to dynamically align the power load data of the source domain and the power load data of the target domain, and the model parameters of the selected source domain prediction model are adjusted.
[0012] The model parameters of the adjusted source domain prediction model are transferred to the power load target domain prediction model based on the GRU network.
[0013] The preprocessed target domain power load data and corresponding power fusion features are input into the target domain prediction model, and the prediction results of the target domain power load data in the short term are output.
[0014] The beneficial effects of this invention are:
[0015] This invention is divided into two parts: source domain load prediction and target domain load prediction. In the source domain load prediction, to address the gradient explosion and vanishing problems inherent in currently popular recurrent neural network (RNN) prediction methods, a gated recurrent unit (GRU) structure is introduced to alleviate these problems, simplify the structure and computation of the RNN, and improve prediction capabilities. Simultaneously, for target domain load prediction with only a limited number of training samples, a transfer learning method is used to apply the source domain prediction model to the target domain, thereby predicting the power system load in the target domain with limited training samples. To address the problem of insufficient training samples in the target domain, this application trains multiple source domains to obtain a power load source domain prediction model for each source domain. Then, the TS-DDQN algorithm is used to select from these multiple power load source domain prediction models, and the MK-MMD algorithm is used to fine-tune the parameters during the transfer process. Finally, the fine-tuned source domain prediction model is used to predict the power load in the target domain, outputting the short-term prediction results. Attached Figure Description
[0016] Figure 1 This is a flowchart of the short-term power load forecasting method based on deep reinforcement learning and transfer learning of the present invention;
[0017] Figure 2 This is a structural diagram of the gated loop unit used in an embodiment of the present invention;
[0018] Figure 3 This is a diagram of the power system load forecasting model based on the GRU network used in this embodiment of the invention.
[0019] Figure 4 This is a flowchart of the source prediction model selected by TS-DDQN in an embodiment of the present invention;
[0020] Figure 5 This is a flowchart of the two-stage training method used in an embodiment of the present invention;
[0021] Figure 6 This is a flowchart of the power system prediction process based on transfer learning used in an embodiment of the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] This application can be divided into two parts: learning the source domain prediction model and learning the target domain prediction model using transfer learning methods. The latter, learning the target domain prediction model using transfer reinforcement learning methods, is further divided into selecting a proxy for the source prediction model using deep reinforcement learning, and using transfer learning for power system load forecasting in the target domain.
[0024] The first part of the learning process employs a Gated Recurrent Unit (GRU) structure to learn the source domain prediction model. GRU avoids gradient explosion and vanishing gradient phenomena by introducing internal states and a gate mechanism, and it has been proven to discover long-term dependencies between sequence data. GRU is an improved version based on LSTM, merging the input gate and forget gate into an update gate, resulting in a simpler structure, faster convergence during training, and higher accuracy. After training each source domain dataset, the second part uses the TS-DDQN algorithm to select which source model to use for prediction in the target domain, followed by model transfer. After transfer, the MK-MMD algorithm is used to appropriately adjust the network structure, maximizing the value of the data and preventing negative transfer, finally outputting the results. The flowchart of the entire method is as follows: Figure 1 As shown, to better illustrate the method of the present invention, please refer to the process of steps 101-106:
[0025] 101. Acquire multiple sets of different source domain power load data and target domain power load data, and preprocess the source domain power load data and the target domain power load data respectively;
[0026] In this embodiment of the invention, considering that power system load data is time series data generated in the power system operating environment, it not only has the characteristics of time series itself, but also the characteristics of the power system environment.
[0027] The load data has the following time series characteristics.
[0028] Long-term trend: The load value will change over time, showing a slow but long-term trend of continuous rise, fall and stagnation.
[0029] Periodicity: Load data shows obvious periodic changes, with load values similar at the same time of year.
[0030] Randomness: Load data may exhibit random, irregular, and unpredictable fluctuations during certain periods; Stability: Load data exhibits stability in its temporal distribution, with the load data at each point in time generally not differing significantly from the load data at its immediate preceding and following points.
[0031] Recentness: Load data has the characteristic of "larger values in the near future and smaller values in the distant future". The load value at the current moment is closely related to the load value at recent historical moments, but less related to the non-periodic synchronous load value at distant historical moments.
[0032] Load data often contains missing values and outliers; this patent addresses both types of issues separately.
[0033] First, there's the issue of missing values. Based on the actual load data, the imputation of missing values should be analyzed from two perspectives: long-term (years) and short-term (weeks).
[0034] Based on long-term annual analysis: If workdays (weekends) are missing, first collect load data for the same date in previous years. If it is found that the same date in a certain year is not a workday (weekend), then the nearest corresponding workday (weekend) is taken. If statutory holidays are missing, since the corresponding dates of statutory holidays vary from year to year, load data for the same period in previous years is collected.
[0035] Based on short-cycle analysis: If there is a missing workday (weekend), the load data of the three historical identical workdays for that day are used, and statutory holidays are removed.
[0036] Secondly, there is the issue of abnormal data. Abnormal power load data can be divided into two main categories: bad data and distorted data. Bad data is mainly caused by problems in the data acquisition system and includes four types: missing data, data to be supplemented, load spikes, maxima, and minima. Distorted data is caused by special external events, which cause fluctuations in normal data and distort load values. This is mainly caused by random and sudden events in political, economic, and social life. Based on this, this patent uses an improved zscore-3σ method to detect abnormal data in load data. This improved anomaly detection method includes the following:
[0037] Suppose the load dataset is X = [x0, x1, x2, ..., x t For daily load data with a time unit of days, a 7-day period is used. For the load x of a given day... t Select historical load data from the same period in history to obtain the historical load sequence [x]. t-7 ,x t-14 ,…,x t-7*k (k is a positive integer). Given sufficient data, the distribution of historical load sequences from the same period approximately follows a normal distribution, allowing for anomaly detection using the zscore-3σ anomaly detection method. Calculate [x] t-7 ,x t-14 ,…,x t-7*k The mean μ and standard deviation σ of x are then calculated. t zscore value The result is then compared with the threshold value th. If the result exceeds the range of the th value, it indicates that the data is abnormal. The th value is an adjustable floating-point number in the range of [1,3], and is generally set to 3.
[0038] The significant difference between load data on holidays and normal working days can impact anomaly detection for the current time period. Collecting historical data from the same period requires removing data from statutory holidays. For example, to detect anomalies on Monday of this week, load data from historical Monday load data should be removed from those related to statutory holidays. The advantage of the improved zscore-3σ anomaly detection method based on historical load data is that it considers the cyclical changes in load data and uses the distribution of historical data to assess the current data, resulting in more accurate assessments.
[0039] To eliminate the impact of order-of-magnitude differences, varying data ranges, and unclear data trends in the raw performance data samples collected from smart meters on model training, and to improve model accuracy and network training speed, all data need to be of the same order of magnitude for comparability. The data preprocessing stage requires normalization of the numerical features. This patent employs a linear min-max normalization method.
[0040] 102. Multiple power load source domain prediction models based on GRU networks are trained using preprocessed multiple sets of source domain power load data and corresponding power fusion features.
[0041] In this embodiment of the invention, considering that power load data is a time series, and time itself is a continuous variable, but observations are made at specific points in time, time series are generally displayed as discrete observation results. Time series data essentially reflects a certain trend in the variable's continuous change over time. In a time series, adjacent observations are usually correlated; the technique for analyzing correlation is called time series analysis. Recurrent Neural Networks (RNNs) are a type of neural network with short-term memory capabilities. Based on this loop structure, RNNs are well-suited for processing data with rich contextual information and strong correlations, making them well-suited for processing time series data. However, RNNs also have obvious drawbacks: they suffer from gradient explosion and vanishing gradient problems, and they are not suitable for processing sequences with long time intervals. In a power system, the load at the next moment is closely related to the load at the previous moment. If a sudden fault occurs in the current power system, the power system at the next moment will inevitably also be faulty, and vice versa. Therefore, this patent uses a GRU network to discover the correlation between power load data.
[0042] A gated loop unit contains two gate structures: an update gate and a reset gate. The GRU represents its internal state with a linear dependency. The internal structure of a repeating module in a GRU is shown in the diagram below. Figure 2 As shown, the network state h at time t-1 is input. t-1 Combined with the candidate states h′ of the network at time t t and the GRU reset gate r at time t t and update gate z t After processing through vector addition, vector multiplication, the tanh function, and the sigmoid function, the final output is the network state h at time t. t .
[0043] To reduce the error between the output and the real network on the basic GRU network, this patent uses the backpropagation algorithm to iteratively update the GRU network parameters. To minimize the target loss function, gradient descent is used to optimize the weights layer by layer. The Adam algorithm has advantages in computational efficiency and convergence speed compared to other parameter optimization algorithms; therefore, this patent employs the Adam algorithm with an adaptive learning rate to accelerate the convergence of the GRU network model. The Adam algorithm estimates the learning rate of each parameter in the model using the moments of the gradient, with the update formula as follows:
[0044]
[0045]
[0046] Where θ is the iteration parameter. s t and r t Let these be the first-order and second-order matrices of the gradient. and The bias corrections are for the first-order moment estimate and the second-order moment estimate of the gradient. α1 and α2 are both attenuation coefficients.
[0047] The power system load source domain prediction model based on GRU network in this embodiment has the following network structure: Figure 3 As shown, the network has two inputs. Input 1 is the source domain power load data, such as the hourly power consumption of different applications (i.e., gas, cooling, fans, and lights) in the source domain. Input 2 is the power fusion features, such as auxiliary information like the date, weather, and temperature of the prediction day. The date information includes Monday to Sunday and holidays; the weather information includes sunny, cloudy, and rainy days; and the temperature information is the temperature of the day in degrees Celsius. This is because residential users' power demand differs on weekdays, weekends, and public holidays. Similarly, users' power consumption varies under different weather conditions and temperatures. After fusing the features of the two inputs, normalization and dropout processing are performed. In neural networks, dropout refers to randomly deleting some nodes in the input and hidden layers with probability p. All forward and backward connections to the deleted nodes are temporarily removed, creating a new network architecture, which reduces overfitting.
[0048] In this embodiment of the invention, since different source domain power load data are used, each type of source domain power load data can be used to train a power load source domain prediction model based on a GRU network. Assuming there are N types of source domain power load data, there are N corresponding power load source domain prediction models. Here, the N types of source domain power load data do not refer to the categories of power load data, but rather to the number of datasets of source domain power load data.
[0049] 103. Using the preprocessed target domain power load data as input, select the source domain prediction model based on the time series dual-depth Q network;
[0050] In this embodiment of the invention, a prediction model that best matches the target domain power load data is selected from multiple source domain prediction models based on a Time-Series Double Deep Q Network (TS-DDQN), and the agent selection is completed using reinforcement learning (RL).
[0051] At time step t, the RL agent, based on the given state s t Take action t The environment outputs the next state s. t+1 and reward r t Standard RL problems can typically be formulated as Markov Decision Processes (MDPs), denoted as (MDP). Where S is the state set and A is the action set. (S) is the probability of the next possible state s′ given any state s and action a. Here, π is the reward function, and γ∈[0,1) is the discount factor. The goal of the RL agent is to find the optimal policy π. * (a|s): S→Δ(A), which is the distribution of the state-action mapping to the action, maximizing the expected cumulative discount reward; the performance of the policy can be defined by the Q-value function, which is based on the Bellman expectation equation, which relates the value of the current state-action pair (s,a) to the expectation of the next state-action pair (s′,a′).
[0052] To find the optimal strategy, value-based RL algorithms obtain the optimal Q-value function by solving the Bellman optimality equation. This process also yields the optimal action. In practical applications, deep reinforcement learning algorithms typically use neural networks. Approximation is achieved by minimizing the timing difference (TD) error.
[0053] When using reinforcement learning for model selection, the model selection problem can be defined as a sequential decision problem. Specifically, the corresponding MDP... The task of selecting the prediction model in this invention can be summarized as follows:
[0054] A set of states S. States This describes information about the time series with time step t, where T is the range of the input sequence, and d s This refers to the dimension of the input features. The state set refers to the time-series data.
[0055] Action set A. In The action at a given time step is a discrete action, corresponding to the index of the selected base model within time step t. In this invention, the action set refers to the set of source domain prediction models.
[0056] Let represent the probability of the next possible state s′ given any state s and action a. It is worth noting here that... In fact, it is unrelated to the selected action, which means that the choice of the base model will not affect the next state s. t+1 The future time series.
[0057] Let represent any state s and action a, the possible next state s′, and the expected value of the next step. Reward It can be any user-defined function, and it needs to specify the accuracy of the selected model. In this invention, the reward refers to the absolute prediction error of the source domain prediction model.
[0058] Discount factor γ. The discount factor γ∈[0,1) represents the weight of future performance. If only one prediction is needed and order information is ignored, then γ can be set to 0. In this invention, this parameter can be set to 0.
[0059] In this embodiment of the invention, the TS-DDQN algorithm is used to select the RL agent for the source prediction model. Considering that the MDP setting M has a discrete action space A = {1,2,…,N}, where action i corresponds to the i-th source domain prediction model, i∈{1,2,…,N}, DDQN is selected as the backbone RL algorithm.
[0060] Regarding the model structure in the RL agent, the encoder module in Autoformer is first used to extract time series features:
[0061] v t =encoder(x t )
[0062] in It is an input time series of length T. It is the extracted time series embedding, where d s and d v This represents the dimensions of the input time series and the hidden representation. The optimal Q-function is then approximated as:
[0063] Q θ (s,a)=MaxPooling(v t W θ
[0064] Where in embedding v t Add one-dimensional max pooling to the time series, where state s is the input time series x. t , It outputs the weight matrix, with an output dimension d. o It equals the number of prediction models in the source domain.
[0065] The metric for the selected source domain prediction model is argmax. a Q θ (s,a), the model is trained by minimizing the temporal difference (TD) error:
[0066]
[0067] In this embodiment, the target network is used in accordance with the above formula. To evaluate argmax a′ Q θ The next state-action pair of (s′,a′).
[0068] In this embodiment of the invention, some source domain prediction models are learned through a GRU network. At this point, the original short-term power load prediction task in the target domain can be transformed into a model selection task. For example... Figure 4 As shown, assuming there are three types of source domain power load data, i.e., three source domain power load datasets, there are corresponding three source domain prediction models. The reinforcement learning agent selects the second source prediction model f2(·) to predict y2(t)=f2(x) from these models. t Given a time series input x with timestamp t. t The appropriate source domain prediction model is selected for prediction through an RL agent.
[0069] In this embodiment of the invention, an RL agent is used to select the source domain prediction model. However, in practice, there may be a lack of sufficient data to train the RL agent. Furthermore, since the TS-DDQN algorithm often suffers from overfitting, it may be impossible to learn a robust RL agent using limited training samples on the target domain dataset. To address this issue, this patent proposes a two-stage training method. In the first stage, the RL agent is pre-trained on a dataset consisting of a validation set and a test set on the source domain power load data; that is, the time-series dual-depth Q-network is pre-trained using the source domain power load data. In the second stage, fine-tuning is performed on the test set of the target domain power load dataset; that is, the time-series dual-depth Q-network is fine-tuned using the target domain power load data. The process is as follows: Figure 5 As shown.
[0070] The two-stage training method involves combining all validation and test sets from each source domain to create a large training set, typically consisting of training, validation, and test sets. Since these validation and test sets are invisible to the source prediction model during training, they can be safely used as new training sets to pre-train the RL agent. However, because the size of the validation and test sets in the source domains can be very large, leading to excessively long pre-training times and consequently long overall training times, this invention introduces a time-series data length metric, l. α , l α Defined as: the average time-series data length of the prediction model obtained in the source domain. The time-series data length for the next pre-training is 2*θ times the time-series data length of the previous pre-training. As long as the time-series data length for pre-training does not reach l... max Then the size of θ is dynamically adjusted. This process is repeated cyclically. If the length of the pre-training time series data exceeds l... max Then the length of the pre-trained time series data remains l. max The length of the nth pre-trained time series data remains unchanged, i.e., l. β The length is:
[0071]
[0072] Where θ is a length index factor that changes according to the length of the time series data from the previous pre-training, and θ ≥ 0.4, l max The initial pre-training time series data length l α It is twice as long. Since pre-training is very sensitive to the length of time series data, a smaller length index factor is set to reduce the time consumed by pre-training.
[0073] Given an input time series, the goal of an RL agent is to select the most suitable source prediction model with the lowest prediction error. To achieve this, the RL agent needs a relatively complete understanding of how each source prediction model performs with different types of time series inputs. Therefore, the RL agent should have good adaptability. As seen in the MDP setup, Let represent the probability of the next possible state s′ given any state s and action a. It is independent of the selected action. This ability greatly simplifies the adaptation problem, allowing adaptive strategies to be employed without worrying that the reinforcement learning agent only collects low-quality samples.
[0074] 104. Based on the multi-core maximum mean difference algorithm, the source domain power load data and the target domain power load data are dynamically aligned, and the model parameters of the selected source domain prediction model are adjusted.
[0075] In this embodiment of the invention, the Multiple-Kernel Maximum Mean Discrepancy (MK-MMD) algorithm is used to dynamically align the distribution of source domain power load data and target domain power load data. This improves the efficiency and accuracy of target domain load prediction by utilizing both source and target domain datasets. While it's necessary to fully utilize source domain information, a distribution gap exists between the target and source domains; therefore, data, network structures, and parameters cannot be directly copied. How to effectively use source domain data, appropriately adjust the network structure, maximize data value, and prevent negative transfer is a key issue in establishing a transfer learning model. Problems to be solved include how to transfer deep learning-based data mining networks, how to adjust network structure and parameters, and how to select source domain data. However, not all data can be utilized; negative impacts from source domain data must be avoided during the transfer process.
[0076] MMD (Mean Difference of Means) plays a crucial role in transfer learning as a standard for measuring data discrepancy. Its principle is to find a continuous function *f* in the sample space, calculate the mean of the function values of samples from different distributions corresponding to *f*, and then calculate the difference between these two means as the mean difference between the two distributions corresponding to *f*. The goal is to find an *f* that maximizes this mean difference; the maximum mean difference is the MMD of the two distributions. This section presents the corresponding transfer learning-based power load forecasting model for power systems, specifically focusing on methods based on deep reinforcement learning and transfer learning. The process is as follows: Figure 6 As shown:
[0077] Since the data distribution of the training samples in the source domain and the training samples in the target domain may differ, the prediction performance of the existing prediction model in the target domain may decrease. In order to achieve the transfer and adaptation of the existing prediction model in the target domain, it is first necessary to obtain the data in the target domain and train the prediction model in the source domain. This part has been completed in the previous stage.
[0078] The source and target domains exhibit distributional differences in the new space, namely, differences in marginal distribution and conditional distribution. In migration component analysis, the marginal distributions of the source and target domains differ, P(x...). s )≠p(x t However, there exists a mapping. Then the source domain and the target domain can be mapped to obtain... and The data distribution after mapping A common approach is to use MMD (Multi-Kernel Distance) to find a transformation function that maps data from two domains together into a high-dimensional Hilbert space. In other words, given different kernel functions, different MMD distances can be calculated. In practical applications, the problem lies in how to choose a suitable kernel function from among many available options. To address this issue, this patent employs the MK-MMD method, which treats the current kernel K as a combination of different kernel functions and weights the multiple kernels with a weight of β. i In addressing the practical migration problem between the source domain power load and target domain power load prediction models of this invention, a kernel function that better suits the current situation is selected through weight adaptive selection.
[0079] Meanwhile, to address the issue of discrepancies between marginal and conditional distributions, a quantitative initial prediction strategy is added to the dynamic and quantitative adaptation of marginal and conditional distributions, increasing the applicability of dynamic distribution adaptation. The importance of marginal and conditional distributions is adaptively adjusted according to each specific task, and this is formally expressed as:
[0080] D(D s D t )=(1-μ)D(P(x s ),P(x t ))+μWD(P(y s |x s ),P(y t |x t ))
[0081] Here, μ ranges from 0 to 1. The closer μ is to 0, the stronger the marginal distribution D(P(x)). s ),P(x t The influence of conditional distribution D(P(y)) is relatively large. s |x s ),P(yt |x t The influence of μ is relatively small; correspondingly, when μ approaches 1, it indicates a marginal distribution D(P(x)). s ),P(x t The influence of )) is relatively small, and the conditional distribution D(P(y) s |x s ),P(y t |x t The impact of pseudo-labels is relatively large. Because the labels of samples in the target domain are pseudo-labels, an iterative pseudo-label refinement strategy combined with a weight update strategy is used to reduce the impact of pseudo-labels and improve the accuracy of pseudo-labels. W represents the weight of samples in the target domain. Therefore, the difference in importance between the marginal distribution and the conditional distribution can be dynamically adjusted using a dynamic parameter μ combined with the iteratively assigned weights.
[0082] Importantly, all samples in the target domain are unlabeled, so directly calculating the conditional distribution is not feasible. However, the model trained in the previous step can be used to approximate the conditional distribution. Using the source domain prediction model from the previous step to obtain sample labels for the target domain is not entirely reliable, but it can be improved iteratively. As the number of iterations increases and the accuracy of the iterative model improves, the weight of the target domain sample labels in the conditional distribution is gradually increased using W, based on the depth of the iterations and the improvement in accuracy. This implements an iterative pseudo-label refinement strategy. The iterative update of the sample label weight W can be expressed as:
[0083]
[0084] Among them, W t W represents the weight of the target domain power load data in the t-th iteration process, T represents the total number of iterations, and W represents the weight of the target domain power load data. t-1 σ represents the weight of the target domain power load data in the (t-1)th iteration process, where σ is the weight index factor and its default value is 0.25.
[0085] The balance factor μ in the above formula can be considered as a parameter, thus requiring calculation. However, calculating μ is not straightforward. In Balanced Distribution Adaptation (BDA), μ is calculated by setting a step size. Generally, the step size is set to 0.1, and then the optimal value is searched through continuous iteration. When measuring between domains, the A-distance method is used. Its definition is to build a linear classifier to distinguish the error between two domains. Therefore, the formula is expressed as follows:
[0086] d A (D s D t )=2(1-2∈(h))
[0087] Where ∈(h) represents the linear classifier h used to distinguish the source domain D. s and target domain D t The calculated distance is represented by d. m To represent the distance between conditional distributions using A-distance, we use... Let c represent the category. Then μ can be calculated using the following formula:
[0088]
[0089] Where, d m d represents the distance between the source domain power load data and the target domain power load data corresponding to the linear classifier. c The conditional distribution distance for category c is represented by c, where C represents the total number of categories. When c∈{0,1,…,C}, the difference in conditional distribution between the source and target domains is calculated. Therefore, the marginal and conditional distributions in the source and target domains are combined through category c.
[0090] Since the feature distribution may change after each estimation of the conditional distribution, this estimation must be performed in each iteration of the dynamically adaptive distribution. Through iterative solution, the eigenvectors corresponding to the smallest d eigenvalues are obtained, which is the mapping matrix A. Mapping matrix A then maps the source and target domain data to a subspace. A prediction model is obtained by training the source domain power load data to predict the target domain power load data.
[0091] 105. Transfer the adjusted source domain prediction model parameters to the power load target domain prediction model based on GRU network.
[0092] In this embodiment of the invention, since step 104 has already used the MK-MMD algorithm to dynamically distribute and align the source domain power load data and the target domain power load data, and at the same time adjusted the model parameters of the selected source domain prediction model, the adjusted model parameters can be used as the corresponding parameters of the target domain prediction model.
[0093] 106. Input the preprocessed target domain power load data and the corresponding power fusion features into the target domain prediction model, and output the prediction results of the target domain power load data in the short term.
[0094] In this embodiment of the invention, the target domain power load data can be preprocessed using the same process as in step 101. The preprocessed target domain power load data and related power fusion features are then input into the target domain prediction model to obtain the short-term prediction results of the target domain power load data.
[0095] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include ROM, RAM, disk, or optical disk, etc.
[0096] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A method for short-term power load forecasting based on deep reinforcement learning and transfer learning, characterized in that, The method includes the following steps: Multiple sets of different source domain power load data and target domain power load data are acquired, and the source domain power load data and the target domain power load data are preprocessed respectively. Multiple power load source domain prediction models based on GRU networks were trained using preprocessed multiple sets of source domain power load data and corresponding power fusion features. The preprocessed target domain power load data is used as input, and the source domain prediction model is selected based on a time series dual-depth Q network. The source domain prediction model based on time-series dual-depth Q-network selection includes: optimizing the time-series dual-depth Q-network using a two-stage method; pre-training the time-series dual-depth Q-network using source domain power load data; and fine-tuning the time-series dual-depth Q-network using target domain power load data. The step of pre-training the time-series-based dual-depth Q-network using source domain power load data includes: the length of the time-series data for the next pre-training iteration is equal to the length of the time-series data for the previous pre-training iteration. Times; as long as the length of the pre-training time series data does not reach Then adjust dynamically. The size of the pre-training time series data is determined, and this process is repeated cyclically. If the length of the pre-training time series data exceeds [a certain value], [the process continues]. Then the length of the pre-trained time series data remains unchanged. The length of the time series data in the nth pre-training iteration remains unchanged. The length is: in, It is a length index factor that changes based on the length of the time series data from the previous pre-training, and , The initial pre-training time series data length 2 times; , Indicates the number of pre-training iterations; The multi-core maximum mean difference algorithm is used to dynamically align the power load data of the source domain and the power load data of the target domain, and the model parameters of the selected source domain prediction model are adjusted. The model parameters of the adjusted source domain prediction model are transferred to the power load target domain prediction model based on the GRU network. The preprocessed target domain power load data and corresponding power fusion features are input into the target domain prediction model, and the prediction results of the target domain power load data in the short term are output.
2. The method for short-term power load forecasting based on deep reinforcement learning and transfer learning according to claim 1, characterized in that, The preprocessing of power load data includes filling in missing values according to a long-term period based on years or a short-term period based on weeks; and using improved load data based on historical data from the same period. The method detects and removes abnormal data.
3. The method for short-term power load forecasting based on deep reinforcement learning and transfer learning according to claim 2, characterized in that, The method employs an improved approach based on historical load data from the same period. The method for detecting abnormal data includes selecting historical power load data for the same period of the target date to obtain the historical load sequence, and then using... The method calculates the mean and standard deviation of the historical load series for the same period; calculates the standard score of the day to be measured based on the mean and standard deviation; compares the standard score with a threshold; if it exceeds the threshold, it is determined to be abnormal data.
4. The method for short-term power load forecasting based on deep reinforcement learning and transfer learning according to claim 1, characterized in that, Training a power load source domain prediction model based on a GRU model includes inputting source domain power load data into a GRU cell and inputting the power fusion features into a fully connected layer. A fusion layer is used to perform feature fusion, and the fused features are normalized to train the power load source domain prediction model based on the GRU network.
5. The method for short-term power load forecasting based on deep reinforcement learning and transfer learning according to claim 1, characterized in that, The multi-core maximum mean difference algorithm for dynamic distribution alignment of source and target domain power load data includes adding a quantitative initial estimation strategy to the dynamic and quantitative adaptation of marginal and conditional distributions, increasing the applicability of dynamic distribution adaptation; reducing the impact of pseudo-labels and improving their accuracy through iterative pseudo-label refinement and weight update strategies; and dynamically adjusting the difference in importance between marginal and conditional distributions using dynamic parameters combined with iteratively allocated weights, which is expressed as: This indicates the distribution distance between the source domain power load data and the target domain power load data. Indicates dynamic parameters. The value range is from 0 to 1. If The closer a value is to 0, the more marginal the distribution. The impact is relatively large, and the conditional distribution is significant. The impact is relatively small; when When the value approaches 1, it indicates a marginal distribution. The impact is relatively small, and the conditional distribution is relatively small. The impact is quite significant. This represents source domain power load data. This represents the target domain's power load data. This indicates the source domain power load data label. The label represents the target domain power load data; W represents the weight of the target domain power load data.
6. The method for short-term power load forecasting based on deep reinforcement learning and transfer learning according to claim 5, characterized in that, The weights of the target domain power load data change according to the number of iterations, and are expressed as follows: in, This represents the weight of the target domain power load data during the t-th iteration. Indicates the total number of iterations. This represents the weight of the target domain power load data during the (t-1)th iteration. This is the weighting factor, with a default value of 0.
25.
7. The method for short-term power load forecasting based on deep reinforcement learning and transfer learning according to claim 5, characterized in that, The formula for calculating the dynamic parameters is as follows: in, This represents the distance between the source domain power load data and the target domain power load data corresponding to the linear classifier. This represents the conditional distribution distance for category c. This indicates the total number of categories.