Short-term power load prediction method based on hybrid neural network model

Abstract

The invention belongs to the technical field of deep neural networks, and discloses a short-term power load prediction method based on a hybrid neural network model, and the method comprises the steps: 1, obtaining an original power load time sequence; 2, performing adaptive decomposition on the original power load time sequence to obtain an intrinsic mode function (IMF) component; step 3, for the plurality of IMF components obtained after decomposition, fusing meteorological features and peak period features to obtain a fusion time sequence, then performing feature extraction and joint prediction, and constructing a multi-modal prediction model; and step 4, intelligently optimizing the key parameters of the multi-modal prediction model, outputting optimal hyper-parameters, and performing optimal parameter setting on the multi-modal prediction model to obtain a final prediction result. According to the method, the accuracy and robustness of prediction are effectively improved, and the adaptability of the model to non-stationary and multi-scale energy time sequences is effectively improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep neural networks, and particularly relates to a short-term power load prediction method based on a hybrid neural network model. BACKGROUND

[0002] In the process of energy system operation and dispatch, accurate prediction of time series data such as power load is an important basis for ensuring the safe and stable operation of the power grid, optimizing energy allocation, and improving economic efficiency. However, in the prior art, a combination model such as SSA-BiLSTM-MA is often used for energy time series prediction. The SSA-BiLSTM-MA method utilizes the modeling capability of the deep learning model (BiLSTM) for time series dependency to some extent, and automatically adjusts the hyperparameters through SSA; the attention mechanism (MA) is used to further improve the expression capability of the model for important time information; this type of method has achieved certain results in some energy prediction tasks and has good engineering applicability.

[0003] However, the SSA-BiLSTM-MA method still has the following key technical problems to be solved in practical application:

[0004] 1. Insufficient adaptability to non-stationary and multi-scale energy time series

[0005] The existing method usually directly inputs the original complex energy time series into the deep learning model for prediction without effective decomposition and preprocessing. Due to the characteristics of non-stationarity, multi-frequency components, and strong random noise interference of energy data, it is difficult for a traditional single deep network to fully extract multi-level time series features, resulting in limited prediction accuracy.

[0006] 2. Lack of multi-scale and local feature extraction mechanism

[0007] The existing technology usually relies on BiLSTM (bidirectional long short-term memory network) and other recurrent neural networks for time series modeling, which can capture long-range dependencies, but lacks effective perception of key information such as local mutations and periodic characteristics, and does not introduce convolution and other local feature extraction modules, making it difficult to adapt to the multi-scale and multi-modal change patterns in energy data.

[0008] 3. Poor effect of hyperparameter optimization method

[0009] The existing model usually uses SSA (Sparrow Search Algorithm) and other traditional intelligent optimization algorithms to optimize the hyperparameters of BiLSTM such as learning rate, iteration number, and hidden layer node number, but SSA has problems such as being prone to local optimization, weak global search capability, and slow convergence speed, making it difficult to obtain a better combination of hyperparameters and limiting the improvement of the overall performance of the model. SUMMARY

[0010] In order to solve the problems of insufficient local feature extraction capability, poor hyperparameter optimization effect and weak module collaboration of the energy time series prediction model in the prior art in response to non-stationarity and multi-scale characteristics, the present application provides a short-term power load prediction method based on a hybrid neural network model, which constructs a multi-stage collaborative and end-to-end energy prediction method through deep fusion of variable variation mode decomposition, time series convolution network, bidirectional adaptive long short-term memory network, improved attention mechanism and intelligent optimization algorithm.

[0011] In order to achieve the above purpose, the present application is realized by the following technical scheme:

[0012] The present application is a short-term power load prediction method based on a hybrid neural network model, which is realized by a short-term power load prediction method based on a hybrid neural network model. The hybrid neural network model is an SVMD-TLPSO-TCN-BiALSTM-MA hybrid neural network model. The short-term power load prediction method specifically includes the following steps:

[0013] Step 1, collecting original power load data and normalizing the original power load data, and obtaining original power load time series according to a sliding window;

[0014] Step 2, using variable variation mode decomposition (SVMD) to adaptively decompose the original power load time series to obtain a plurality of relatively stationary intrinsic mode function (IMF) components;

[0015] Step 3, constructing a TCN-BiALSTM-MA multi-modal prediction model, fusing meteorological features and peak period features to obtain a fused time series for the plurality of intrinsic mode function (IMF) components obtained after decomposition, and then performing feature extraction and joint prediction, wherein the TCN-BiALSTM-MA multi-modal prediction model includes a time series convolution network layer (TCN), a bidirectional adaptive long short-term memory network layer (BiALSTM) and an improved attention mechanism layer (MA);

[0016] Time series convolution network layer (TCN): extract local key features and multi-scale features in the fused time series to obtain local multi-scale features;

[0017] Bidirectional adaptive long short-term memory network layer (BiALSTM): perform bidirectional global time series modeling according to the local multi-scale features output by the time convolution network layer, and dynamically adjust the forgetting gate threshold value according to whether the local multi-scale features are peak period features to obtain global time series dependent features;

[0018] Improved attention mechanism (MA) layer: attention weighted fusion of global time series dependent features to obtain final load prediction value;

[0019] Step 4, the key parameters of the TCN-BiALSTM-MA multi-modal prediction model are intelligently optimized by using the TLPSO algorithm, the particle initialization parameters are obtained, the first prediction result is obtained after iteration, the iteration number, learning rate, and two hidden layer node numbers (i.e. hidden layer dimension) of the bidirectional adaptive long short-term memory network layer of the TCN-BiALSTM-MA multi-modal prediction model training are iterated until the set iteration number or the convergence result is output, the optimal hyperparameters are output, and the final prediction result is obtained after the TCN-BiALSTM-MA multi-modal prediction model is set with the optimal parameters.

[0020] A further improvement of the present application is that step 2 utilizes variable variation modal decomposition (SVMD) to adaptively decompose the original power load time series, specifically including the following steps:

[0021] Step 2.1, calculate the window volatility: set the window parameter M, which is the number of all original time series, and the data f window Calculate the window mean μ window And the window standard deviation σ window :

[0022]

[0023] Where i is a specific power load data index within the window, i∈[1,M];

[0024] Step 2.2, calculate the sliding window volatility vaof the power load data:

[0025]

[0026] Step 2.3, dynamically adjust the penalty factor aaccording to the sliding window volatility va:

[0027] α new =α base ·(1+k·va)

[0028] Where a base is the initial penalty factor of the VMD algorithm (usually 2000), a new is the penalty factor after dynamic adjustment according to the sliding window volatility vaand the adjustment coefficient k; when va<0.05, k is 0.1, when 0.05≤va≤0.15, k is 0.15, and when va>0.15, k is 0.3,

[0029] Step 2.4, bring the dynamically adjusted penalty factor a new into the VMD algorithm to decompose the original power load time series and obtain K intrinsic mode function (IMF) components uk , the center frequency ω k , the formula is:

[0030]

[0031] Wherein, f is the original power load time series (input data to be decomposed), u k is the multiple intrinsic mode function IMF components obtained by SVMD decomposition, corresponding to each component after decomposition of the original power load sequence, ω k is the center frequency, δ(t) is the Dirac function, is the signal gradient.

[0032] Further improvement of the application is that the bidirectional adaptive long short-term memory network layer (BiALSTM) comprises:

[0033] Input sequence: the local multi-scale feature obtained after processing by the time convolution network is taken as the initial input;

[0034] Embedding layer: the local multi-scale feature is embedded and processed, and the local multi-scale feature is converted into a local multi-scale vector more suitable for subsequent network processing, laying a foundation for subsequent operations such as feature extraction;

[0035] Dynamic threshold generator: according to whether the local multi-scale feature is in the peak period or not, a dynamic forgetting gate threshold is generated, which is used for subsequent adjustment of the forgetting degree of information by the LSTM unit, and the memory of key period information is enhanced;

[0036] Forward LSTM unit: starting from the starting point of the local multi-scale feature, sequentially passing through the input gate, the forgetting gate, the cell state, and the output gate, a forward hidden state sequence is obtained;

[0037] Reverse LSTM unit: backtracking from the end point of the local multi-scale feature, also passing through the input gate, the forgetting gate, the cell state, and the output gate, a reverse hidden state sequence is obtained;

[0038] Bidirectional output splicing: the outputs of the forward LSTM unit and the reverse LSTM unit are spliced, the forward hidden state sequence and the reverse hidden state sequence are fused, a global time sequence dependent feature is obtained, and output to the attention layer.

[0039] Further improvement of the application is that the specific process of obtaining the global time sequence dependent feature by the bidirectional adaptive long short-term memory network layer (BiALSTM) comprises the following steps:

[0040] Step 3.1.1, receiving the local multi-scale feature obtained after processing by the time convolution network and mapping it to the dimension d required by the bidirectional adaptive long short-term memory network layer (BiALSTM) inA new feature sequence is obtained. in

[0041] Step 3.1.2: Design a dynamic forget gate: Let the timestamp characteristic at time t be s. t When the time is peak period, s t The value is 1 during off-peak hours and 0 during off-peak hours.

[0042] Step 3.1.3: Calculate the dynamic forgetting gate threshold: θt = θt max ·s t +θt min ·(1-s t ), θt max θt is the forgetting threshold during peak hours. min This is the forgetting threshold, excluding peak periods;

[0043] During peak hours, i.e. t When the threshold is 1, the forgetting threshold is more conservative, resulting in less forgetting of historical information and a greater focus on remembering characteristics of peak periods.

[0044] Off-peak hours (s) t When the threshold is 0, the forgetting threshold is more lenient, allowing for the forgetting of redundant history and the retention of features specific to the current period.

[0045] Step 3.1.4: Filter the original forget gate output f using the dynamic forget gate threshold θt. t That is, the adaptive forget gate f is obtained by comparing the dynamic forget gate threshold θt. t ′:

[0046] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0047] Where σ is the sigmoid function, W f Let h be the weight matrix. t-1 x is the hidden state of the previous time step. t For the current input, b f For bias;

[0048] When f t ≥θ t Then retain the original forget gate output at that moment, i.e., f. t ′=f t ;

[0049] When f t <θ t Then forced forgetting is f t ′=0;

[0050] Step 3.1.5, bidirectional global time sequence modeling: forward LSTM unit (LSTM forward ) processes the feature sequence X from the local multi-scale feature starting point t = 1 to the end point t = T FC , and the reverse LSTM unit (LSTM backward ) processes the feature sequence X from the local multi-scale feature end point t = T to the starting point t = 1 FC , and the forward and reverse hidden state sequences are obtained through the update of the cell state c t , and the update formula of the cell state c

[0051]

[0052] Where, is the element product, i t is the input gate output, c t is the current cell state; b c is the bias term when updating the cell state;

[0053] Hidden state calculation:

[0054]

[0055] h t = o t ⊙ tanh (c t )

[0056] Where, is the activation function, W o is the weight matrix of the output gate, b o is the bias term of the output gate, and h t is the current time hidden state;

[0057] Step 3.1.6, bidirectional output splicing: the forward hidden state sequence H and the reverse hidden state sequence H are spliced according to the time to obtain the global time sequence dependent feature Each is the splicing of and , and the splicing is input into the improved attention mechanism layer.

[0058] Further improvement of the application is that the improved attention mechanism layer (MA) performs attention weighted fusion on the global time sequence dependent feature to obtain the final load prediction value, and the improved attention mechanism layer sets different attention weight matrices according to the historical load features and meteorological features in the global time sequence dependent feature, specifically including:

[0059] Step 3.2.1, the global time sequence dependent feature H BiALSTM is input into the improved attention mechanism layer H BiALSTM= [h1,... h t ];

[0060] Step 3.2.2, calculate attention weight matrix W BiALSTM according to global timing dependency feature H a and historical load feature W b , where a

[0061] W a = [i,j] = w1 + N(0, 0.01)

[0062] W b = [i,j] = w2 + N(0, 0.01)

[0063] where w1, w2 represent initial scalar values, take w1 > w2, so that the initial weight of weather feature W a is higher, accelerating key feature learning, Gaussian noise N(0, 0.01) avoids weight homogenization;

[0064] Step 3.2.3, attention weight calculation:

[0065] e t = v T · tanh(W a · h t + b)

[0066] Normalized weight

[0067]

[0068] where e t is the attention weight at time t, v, b is the attention parameter; e k is the attention weight represented by the kth moment, exp(e t ) means taking the exponential of e t ; exp(e k ) is the exponential of e k ;

[0069] Step 3.2.4, prediction output weighted fusion

[0070]

[0071] where, where S is the context vector obtained by weighted summation of hidden state h BiALSTM in H t at time t, and the final prediction output sequence y is obtained through S:

[0072] y = W o · S + b o ​

[0073] wherein W o is the weight matrix of the output gate, b o is the output layer bias term;

[0074] Step 3.2.5, recombining the final output sequence obtained according to different intrinsic modal components to obtain a final load prediction value.

[0075] The further improvement of the present application is that the step 4 specifically comprises the following steps:

[0076] Step 4.1, setting optimization key parameters and particle population information: for the hyperparameters to be optimized: the number of iterations of the TCN-BiALSTM-MA multi-modal prediction model training, the learning rate, the number of nodes of the two hidden layers of BiALSTM, setting the population size, the maximum number of iterations, and the value range of the parameters to be optimized, each particle represents a set of hyperparameter combinations to be optimized, and the optimization hyperparameter combination range is as follows:

[0077]

[0078] wherein T max is the maximum number of iterations of the multi-modal prediction model training, lr is the learning rate, and h is the number of nodes of the two hidden layers of BiALSTM;

[0079] The particle coding format is as follows:

[0080] X i = [T max,i , lr i , h 1,i , h 2,i ]

[0081] wherein each X i represents a set of hyperparameter combinations;

[0082] Step 4.2, particle population initialization: the hyperparameter combinations of the particles are initialized by using Tent mapping, the population diversity is enhanced, and the algorithm is prevented from falling into a local area too early, an initial set of hyperparameters is generated after the particle population is initialized, and then the hyperparameter combinations are used for prediction:

[0083] Z n+1 = 2·min(Z n , 1-Z n ), Z0∈(0, 1), Z0≠0.5

[0084] x ij = L j + Z n ·(U j -L j )

[0085] wherein Z n is a random number representing 0-1, x ij represents the value of the j-th dimension of the i-th particle, L j , U j respectively represent the maximum and minimum values of the parameter combination;

[0086] Step 4.3, fitness evaluation: to comprehensively evaluate the feasibility of the TCN-BiALSTM-MA multi-modal prediction model, the mean absolute percentage error (MAPE) is selected as the fitness value of the particle; the smaller the fitness value, the better the performance of the parameter combination:

[0087]

[0088] wherein, is the predicted value, y k is the true value, and n represents the number of samples;

[0089] Step 4.4, particle position update and reverse learning strategy: calculate the dynamic inertia weight, and generate a reverse solution by lens imaging reverse learning, dynamically adjust the speed and position of the particle according to the current iteration stage, until the maximum iteration number T total or convergence is reached;

[0090] The dynamic inertia weight is as follows:

[0091]

[0092] wherein t is the current iteration number, T total is the total number of iterations of parameter search, and ω(t) represents the inertia weight at the t-th iteration;

[0093] The lens imaging reverse learning generates a reverse solution, and retains the solution with better fitness, and the formula is as follows:

[0094] x ij ′ = L j + U j - x ij

[0095] wherein x ij represents the value of the j-th dimension of the i-th particle, L j , U j respectively represent the maximum and minimum values of the parameter combination;

[0096] Speed and position update, dynamically adjust the speed and position of the particle according to the current iteration stage, obtain the parameter combination represented by the next particle, and the speed update and position formula are as follows:

[0097] v ij (t+1) = ω(t) · v ij (t) + c1 · r1 · (pbest ij -x ij (t) + c2 · r2 · (gbest j -x ij (t)

[0098] x ij (t+1) = x ij (t) + x ij (t+1)

[0099] where c1 = c2 = 2, representing cognitive factor and social factor respectively, r1, r2 ~ U(0, 1) are uniform random numbers, pbest ij is the historical optimal of the j-th dimension of particle i, gbest j is the value of the j-th dimension parameter of the global optimal particle;

[0100] For the boundary, the following processing is performed to ensure that the parameters do not exceed the set range:

[0101]

[0102] Step 4.5, output the optimal parameter combination: set the hyperparameter combination represented by the updated particle to the TCN-BiALSTM-MA multi-modal prediction model, calculate the fitness, and after multiple iterations to the maximum number of iterations or the TLPSO algorithm converges, that is, the difference between the calculated fitness and the previous fitness is not more than 0.05%, output the optimal hyperparameter combination (i.e. the optimal particle) with the best fitness in single prediction, to guide the subsequent model training and prediction process.

[0103] The beneficial effects of the present application are:

[0104] The present application first uses SVMD (variable variation modal decomposition) to split the original complex energy time series into multiple relatively stable intrinsic modal functions (IMF), and dynamically adjusts the penalty factor parameter according to the fluctuation characteristics of the time series during the splitting process. This preprocessing method significantly reduces the non-stationarity and multi-frequency interference of the original data, so that the subsequent TCN and BiALSTM modules can more accurately extract local and global features under each mode. Compared with the prior art of directly inputting the original sequence, the present application enhances the model's adaptability to complex time series characteristics, and is more suitable for power load prediction, effectively improving the accuracy and robustness of the prediction, and effectively improving the model's adaptability to non-stationary and multi-scale energy time series.

[0105] The application extracts local features of each IMF component decomposed by SVMD by introducing TCN (temporal convolutional network), BiALSTM and improved MA attention mechanism, makes up for the deficiency of traditional BiLSTM and other recurrent networks in capturing short-term mutation response, and the expansion convolution structure and parallel computing capability of TCN enable it to efficiently extract multi-scale and locally relevant time series features, thereby improving the modeling capability of the model for key information such as load mutation and periodic fluctuation, which is difficult to achieve by traditional single recurrent network. The application enhances the perception ability of the model for local features and multi-scale patterns.

[0106] The application adopts TLPSO (particle swarm optimization algorithm based on Tent mapping initialization and lens imaging reverse learning) to intelligently optimize key hyperparameters in the model such as the number of iterations, learning rate, BiALSTM hidden layer node number, etc. Compared with traditional optimization algorithms such as SSA, TLPSO has stronger global search ability, faster convergence speed and better ability to jump out of local optimum, can automatically find a more suitable combination of hyperparameters, significantly optimizes the combination quality of model hyperparameters, and improves the prediction performance and generalization ability. BRIEF DESCRIPTION OF DRAWINGS

[0107] Figure 1 is a schematic diagram of the hybrid neural network model of the application.

[0108] Figure 2 is a schematic diagram of the temporal convolutional network layer (TCN) of the application.

[0109] Figure 3 is a schematic diagram of the bidirectional adaptive long short-term memory network layer (BiALSTM) of the application.

[0110] Figure 4 is a schematic diagram of the improved attention mechanism (MA) layer of the application. DETAILED DESCRIPTION

[0111] The embodiments of the application will be described below with reference to the drawings. For the purpose of clear illustration, many practical details will be described in the following description. However, it should be understood that these practical details should not be used to limit the application. That is, in some embodiments of the application, these practical details are not necessary. In addition, for the purpose of simplifying the drawings, some conventional structures and components will be shown in the drawings in a simple schematic manner.

[0112] As Figure 1As shown, the present application is a short-term power load prediction method based on a hybrid neural network model, which is realized by a hybrid neural network model, and the hybrid neural network model is an SVMD-TLPSO-TCN-BiALSTM-MA hybrid neural network model. The short-term power load prediction method specifically comprises the following steps:

[0113] Step 1, collecting original power load data, and normalizing the original power load data, and obtaining original power load time series according to a sliding window;

[0114] The present application takes the power load data and key meteorological features of a certain area in southern China from 2017 to 2020 as input, i.e. minimum temperature, maximum temperature, average temperature, and rainfall. The data sampling interval is 15 minutes, and there are 87552 load points. The training set and the test set are divided according to 8:2. The original data is first normalized, such as Min-Max standardization, to improve the numerical stability of model training; input-output sample pairs are constructed according to a sliding window for supervised training.

[0115] Step 2, using variable variation mode decomposition (SVMD) to adaptively decompose the original power load time series to obtain a plurality of relatively stable intrinsic mode function (IMF) components. The variable variation mode decomposition (SVMD) combines the traditional VMD decomposition method and the adaptive adjustment of the penalty factor according to the power load sliding window fluctuation rate. By VMD decomposing the sequence, the decomposition complexity of the non-stationary random sequence can be effectively reduced. Since the original power load data has the characteristics of strong nonlinearity and instability, directly using it as input data will have high prediction deviation. Therefore, VMD is used to preprocess the prediction data to improve the prediction accuracy. However, the traditional VMD variation model adopts a fixed penalty factor, which cannot dynamically match the volatility of the data. The SVMD variable variation mode decomposition can dynamically adjust the parameters according to the volatility characteristics of the data without human intervention. Specifically, the variable variation mode decomposition (SVMD) is used to adaptively decompose the original power load time series, which specifically comprises the following steps:

[0116] Step 2.1, calculate the window fluctuation rate: set the window parameter M as the number of all original time series, and calculate the window fluctuation rate va of the window data f window Calculate the window mean μ window And the window standard deviation σ window :

[0117]

[0118] Wherein, i is a specific power load data index in the window, i∈[1,M];

[0119] Step 2.2, calculate the sliding window fluctuation rate va of the power load data:

[0120]

[0121] Step 2.3: Dynamically adjust the penalty factor a based on the sliding window volatility va:

[0122] α new =α base ·(1+k·va)

[0123] Where, α base The initial penalty factor for the VMD algorithm is 2000 in this embodiment, α. new This is the penalty factor dynamically adjusted based on the sliding window volatility va and the adjustment coefficient k; when va < 0.05, k is 0.1, when 0.05 ≤ va ≤ 0.15, k is 0.15, and when va > 0.15, k is 0.3.

[0124] Step 2.4: Adjust the dynamically adjusted penalty factor α new By incorporating the VMD algorithm, the original power load time series is decomposed to obtain K intrinsic mode function (IMF) components u. k The center frequency ω corresponding to the intrinsic mode function (IMF) components k The formula is:

[0125]

[0126] Where f is the original power load time series, i.e., the input data to be decomposed, u k For the multiple intrinsic mode functions (IMF) components obtained by SVMD decomposition, corresponding to the components after decomposition of the original power load sequence, ω k Let δ(t) be the center frequency, and δ(t) be the Dirac function. This represents the signal gradient.

[0127] Step 3: Construct the TCN-BiALSTM-MA multimodal prediction model. For the multiple intrinsic mode functions (IMF) components obtained after decomposition, fuse meteorological features and peak period features to obtain a fused time series. Then, feature extraction and joint prediction are performed. The TCN-BiALSTM-MA multimodal prediction model includes a temporal convolutional network layer (TCN), a bidirectional adaptive long short-term memory network layer (BiALSTM), and an improved attention mechanism layer (MA).

[0128] like Figure 2 As shown, the Temporal Convolutional Network (TCN) layer extracts local key features and multi-scale features from the fused time series, making up for the problem that traditional recurrent networks (such as BiLSTM) are not good at capturing local information such as "short-term mutations and periodic fluctuations", and obtains local multi-scale features.

[0129] BiALSTM: Bidirectional Adaptive Long Short-Term Memory Network Layer: Bidirectional global temporal modeling is performed on the basis of the local multi-scale features output by the time convolution network layer, and the forgetting gate threshold is dynamically adjusted according to whether the local multi-scale features are in a peak period, so as to obtain a global temporal dependence feature; the core role of the BiALSTM is to model the global dependence relationship across time scales on the basis of the local features of the TCN, and to enhance the information memory of the power consumption peak through a dynamic forgetting gate.

[0130] BiALSTM (Bidirectional Adaptive Long Short-Term Memory): As a recurrent neural network variant that integrates bidirectional information processing mechanism, it dynamically generates a forgetting gate threshold according to the peak period of the power load time, and the LSTM units in the forward and reverse directions are transmitted to work collaboratively - the forward unit gradually learns historical information from the beginning of the sequence, and the reverse unit captures future correlations by backtracking from the end of the sequence, thereby more comprehensively modeling the global temporal dependence relationship across different time scales in each IMF component on the basis of the local fine features extracted by the TCN. At the same time, unlike the traditional BiLSTM, the forgetting gate threshold is dynamically adjusted according to whether the time series is in a peak period, to enhance the memory of key period information. As shown in Figure 3 The BiALSTM includes:

[0131] Input sequence: The local multi-scale features obtained after processing by the time convolution network are used as the initial input;

[0132] Embedding layer: The local multi-scale features are embedded and converted into local multi-scale vectors that are more suitable for subsequent network processing, laying a foundation for subsequent operations such as feature extraction;

[0133] Dynamic threshold generator: A dynamic forgetting gate threshold is generated according to whether the local multi-scale features are in a peak period, which is used to adjust the degree of forgetting of information by the LSTM unit and enhance the memory of key period information;

[0134] Forward LSTM unit: Starting from the beginning of the local multi-scale features, the input gate, forgetting gate, cell state, and output gate are sequentially passed through to obtain a forward hidden state sequence;

[0135] Reverse LSTM unit: Backtracking from the end of the local multi-scale features, the input gate, forgetting gate, cell state, and output gate are also sequentially passed through to obtain a reverse hidden state sequence;

[0136] Bi-directional output concatenation: the outputs of the forward LSTM unit and the backward LSTM unit are concatenated to fuse the forward hidden state sequence and the backward hidden state sequence to obtain global time-dependent features, which are output to the attention layer.

[0137] The specific process of obtaining global time-dependent features by the bidirectional adaptive long short-term memory network layer (BiALSTM) includes the following steps:

[0138] Step 3.1.1, receiving the local multi-scale features obtained after processing by the time convolution network and mapping them to the dimension d required by the bidirectional adaptive long short-term memory network layer (BiALSTM) in to obtain a new feature sequence wherein

[0139] Step 3.1.2, designing a dynamic forgetting gate: let the timestamp feature at time t be s t When the time is the peak period, i.e. 8:00-22:00, s t is 1, and the off-peak period is 0.

[0140] Step 3.1.3, calculating the dynamic forgetting gate threshold: θt=θt max ·s t +θt min ·(1-s t ), θt max is the forgetting gate threshold in the peak period, θt min is the forgetting gate threshold except in the peak period, the peak period θt is 0.7, and the rest is 0.5

[0141] When the peak period, i.e. s t is 1, the forgetting gate is more conservative, less forgetting historical information, and focuses on remembering the peak period features.

[0142] When the off-peak period, i.e. s t is 0, the forgetting gate is more relaxed, more forgetting redundant history, and remembering the current period features.

[0143] Step 3.1.4, filtering the original forgetting gate output f t with the dynamic forgetting gate threshold θt, i.e. through the dynamic forgetting gate threshold θt comparison to obtain the adaptive forgetting gate f t ':

[0144] f t =σ(W f ·[h t-1 ,x t ]+b f )

[0145] wherein, σ is the sigmoid function, W fLet h be the weight matrix. t-1 x is the hidden state of the previous time step. t For the current input, b f For bias;

[0146] When f t ≥θ t Then retain the original forget gate output at that moment, i.e., f. t ′=f t

[0147] When f t <θ t Then forced forgetting is f t ′=0;

[0148] Step 3.1.5, Bidirectional Global Timing Modeling: Forward LSTM Unit (LSTM) forward Process the feature sequence X from the local multi-scale feature starting point (t=1) to the ending point (t=T). FC Inverse LSTM unit (LSTM) backward From the local multi-scale feature endpoint, i.e., t=T, to the feature sequence X starting at t=1 FC The hidden state sequence is obtained by updating the cell state, and the cell state c t The update formula is:

[0149]

[0150] Where ⊙ represents the element-wise product, i t For input gate output, c t b is the current cell state. c This is a bias term used during cell state updates;

[0151] Hidden state calculation:

[0152]

[0153] h t =o t ⊙ tanh (c t )

[0154] Where σ is the activation function, W o Let b be the weight matrix of the output gate. o h is the bias term for the output gate. t The current state is hidden.

[0155] Step 3.1.6, Bidirectional Output Concatenation: Concatenate the forward hidden state sequence H and the reverse hidden state sequence H By concatenating time sequences, we obtain the global temporal dependency features. Each is and are concatenated, and the concatenated features are input into the improved attention mechanism layer. As shown in Figure 4 , the improved attention mechanism (MA) layer: attention-weighted fusion of global time-dependent features to obtain the final load prediction value. The application initializes the domain prior weight of the traditional attention mechanism layer to cope with load history features and weather features. The improved attention mechanism layer (MA) attention-weighted fusion of global time-dependent features to obtain the final load prediction value, the improved attention mechanism layer sets different attention weight matrices according to the historical load features and weather features in the global time-dependent features, which specifically includes:

[0156] Step 3.2.1, inputting the global time-dependent feature H BiALSTM into the improved attention mechanism layer H BiALSTM = [h1, …h t ];

[0157] Step 3.2.2, according to the weather feature W BiALSTM and the historical load feature W b in the global time-dependent feature H a , calculate the attention weight matrix W a , wherein

[0158] W a = [i,j] = w1+N(0,0.01)

[0159] W b = [i,j] = w2+N(0,0.01)

[0160] wherein w1, w2 represent initial scalar values, take w1> w2, so that the initial weight of the weather feature W a is higher, which accelerates the learning of key features, and the Gaussian noise N(0,0.01) avoids weight homogenization;

[0161] Step 3.2.3, attention weight calculation:

[0162] e t = v T ·tanh(W a ·h t +b)

[0163] normalized weight

[0164]

[0165] wherein e t is the attention weight at time t, v, b is the attention parameter; e krepresents the attention weight at the k-th time point, exp(e t ) denotes taking the exponential of e t ; exp(e k ) is the exponential of e k ;

[0166] Step 3.2.4, prediction output weighted fusion

[0167]

[0168] wherein, wherein S is the context vector obtained by weighted summation of the hidden state h BiALSTM at time t of the H t network, and the output sequence y is finally predicted by S:

[0169] y = W o · S + b o

[0170] wherein, W o is the weight matrix of the output gate, and b o is the output layer bias term;

[0171] Step 3.2.5, the final load prediction value is obtained by recombining the final output sequences obtained according to different intrinsic modal components.

[0172] Step 4, the key parameters of the TCN-BiALSTM-MA multi-modal prediction model are intelligently optimized by using the TLPSO algorithm, the particle initialization parameters are obtained, and after the first prediction result is obtained, the iteration number, the learning rate, and the two hidden layer node numbers (i.e. the hidden layer dimension) of the BiALSTM layer of the TCN-BiALSTM-MA multi-modal prediction model training are iterated until the set iteration number or the convergence result, and the optimal hyperparameters are output. After the optimal parameters of the TCN-BiALSTM-MA multi-modal prediction model are set, the final prediction result is obtained.

[0173] The application adopts TLPSO (particle swarm optimization algorithm based on Tent mapping initialization and lens imaging reverse learning) to automatically optimize the key hyperparameters in the model. This optimization process effectively improves the optimization quality of the hyperparameter combination by enhancing population diversity, improving global search ability, and jumping out of local optimal ability. The optimized key parameters include: the iteration number of model training; the learning rate; and the two hidden layer node numbers of BiALSTM, i.e. the hidden layer dimension.

[0174] Specifically, step 4 specifically includes the following steps:

[0175] Step 4.1, setting optimization key parameters and particle population information: for the hyperparameters to be optimized: the number of iterations of TCN-BiALSTM-MA multi-modal prediction model training, learning rate, two hidden layer node numbers of BiALSTM, set the population size of particle swarm, the maximum number of iterations, the value range of the parameters to be optimized, each particle represents a set of hyperparameter combinations to be optimized, and the optimization hyperparameter combination range is as follows:

[0176]

[0177] wherein, T max is the maximum number of iterations, lr is the learning rate, and h is the two hidden layer node numbers of BiALSTM.

[0178] The hyperparameter combination range of the embodiment of the application is as follows:

[0179]

[0180] The particle coding format is as follows:

[0181] X i =[T max,i ,lr i ,h 1,i ,h 2,i ]

[0182] wherein, each X i represents a set of hyperparameter combinations;

[0183] Step 4.2, particle population initialization: the hyperparameter combinations of the particles are initialized by using Tent mapping, the population diversity is enhanced, and the algorithm is prevented from falling into a local area too early. After the particle population is initialized, an initial set of hyperparameters is generated, and then the hyperparameter combinations are used for prediction:

[0184] Z n+1 =2·min(Z n ,1-Z n ),Z0∈(0,1),Z0≠0.5

[0185] x ij =L j +Z n ·(U j -L j )

[0186] wherein, Z n represents a random number from 0 to 1, x ij represents the value of the jth dimension of the ith particle (for integer parameters, rounding is required), L j and U j represent the maximum and minimum values of the hyperparameter combinations respectively.

[0187] Step 4.3, fitness evaluation: to comprehensively evaluate the feasibility of the TCN-BiALSTM-MA multi-modal prediction model, the mean absolute percentage error (MAPE) is selected as the fitness value of the particle; the smaller the fitness value, the better the performance of the group of parameters:

[0188]

[0189] wherein, is the predicted value, y k is the true value, and n represents the number of samples;

[0190] Step 4.4, particle position update and reverse learning strategy: calculate the dynamic inertia weight, and generate a reverse solution by lens imaging reverse learning, dynamically adjust the speed and position of the particle according to the current iteration stage, until the maximum iteration number T total or convergence,

[0191] The dynamic inertia weight is as follows:

[0192]

[0193] wherein, ω max = 0.9, ω min = 0.4, t is the current iteration number, T total is the total iteration number of hyperparameter search, ω(t) represents the inertia weight at the tth iteration, which controls the degree of "retaining previous speed" of the particle. In the early stage of iteration, i.e., t is small, ω(t) is large, and the particle inertia is large, and the global search ability is strong; in the later stage of iteration, i.e., t is close to T total , ω(t) is small, and the particle inertia is small, and the local fine search ability is strong.

[0194] The lens imaging reverse learning generates a reverse solution, and retains the solution with better fitness, and the formula is as follows:

[0195] x ij ′ = L j + U j - x ij

[0196] wherein, x ij represents the value of the jth hyperparameter of the ith group of hyperparameters, L j , U j respectively represent the maximum and minimum values of the hyperparameter combination, x ij ′ represents the reverse solution, and the better solution is calculated according to the fitness formula, and the better solution is retained.

[0197] Velocity and position update, dynamically adjust the velocity and position of the particle according to the current iteration stage, get the next particle represents the combination of hyperparameters, velocity update and position formula as follows:

[0198] v ij (t+1) = ω(t)·v ij (t) + c1·r1·(pbest ij -x ij (t)) + c2·r2·(gbest j -x ij (t))

[0199] x ij (t+1) = x ij (t) + x ij (t+1)

[0200] Where, c1 = c2 = 2, respectively represent the cognitive factor and social factor, r1, r2 ~ U(0, 1) are uniform random numbers, pbest ij is the historical optimal of the j-th dimension of particle i, gbest j is the value of the j-th dimension of the global optimal particle parameter;

[0201] For the boundary, the following processing is carried out to ensure that the parameters do not exceed the set range:

[0202]

[0203] Step 4.5, output the optimal parameter combination: set the hyperparameter combination represented by the updated particle to the TCN-BiALSTM-MA multi-modal prediction model, calculate the fitness, and after multiple iterations to the maximum iteration number or TLPSO algorithm converges, that is, the difference between the calculated fitness and the previous one is not more than 0.05%, output the fitness optimal hyperparameter combination, that is, the optimal particle, to guide the subsequent model training and prediction process.

[0204] Root mean square error (RMSE), mean absolute percentage error (MAPE) and determination coefficient (R 2 ) are selected as evaluation indexes to observe the prediction accuracy.

[0205] The following formula is calculated:

[0206]

[0207] In the formula, n is the number of prediction points, y i , are the actual value, average value and predicted value of the load data respectively.

[0208] The above merely describes the embodiments of the present application, but is not intended to limit the present application. The present application can have various changes and modifications for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A short-term power load forecasting method based on a hybrid neural network model, characterized in that: The short-term power load forecasting method is implemented using a hybrid neural network model, specifically the SVMD-TLPSO-TCN-BiALSTM-MA hybrid neural network model. The short-term power load forecasting method includes the following steps: Step 1: Collect raw power load data and normalize the raw power load data to obtain the raw power load time series; Step 2: Adaptively decompose the original power load time series using variant variational mode decomposition (SVMD) to obtain multiple relatively stable intrinsic mode function (IMF) components; Step 3: Construct the TCN-BiALSTM-MA multimodal prediction model. For the multiple intrinsic mode functions (IMF) components obtained after decomposition, fuse meteorological features and peak period features to obtain a fused time series. Then, feature extraction and joint prediction are performed. The TCN-BiALSTM-MA multimodal prediction model includes a temporal convolutional network layer (TCN), a bidirectional adaptive long short-term memory network layer (BiALSTM), and an improved attention mechanism layer (MA). Temporal Convolutional Network (TCN): Extracts local key features and multi-scale features from the fused time series to obtain local multi-scale features; Bidirectional Adaptive Long Short-Term Memory (BiALSTM) network layer: Bidirectional global temporal modeling is performed based on the local multi-scale features output by the temporal convolutional network layer, and the forget gate threshold is dynamically adjusted according to whether the local multi-scale features are peak periods to obtain global temporal dependent features. Improved Attention Mechanism (MA) Layer: Attention-weighted fusion of global temporal-dependent features to obtain the final load prediction value; Step 4: Utilize the TLPSO algorithm to intelligently optimize the key parameters of the TCN-BiALSTM-MA multimodal prediction model. Initialize the parameters for the particles, obtain the first prediction result, and then iterate the number of iterations, learning rate, and number of hidden layer nodes in the two hidden layers of the bidirectional adaptive long short-term memory network layer of the TCN-BiALSTM-MA multimodal prediction model until the set number of iterations or convergence result is reached. Output the optimal hyperparameters, and obtain the final prediction result after setting the optimal parameters for the TCN-BiALSTM-MA multimodal prediction model.

2. The short-term power load forecasting method based on a hybrid neural network model according to claim 1, characterized in that: Step 2, which uses variant variational mode decomposition (SVMD) to adaptively decompose the original power load time series, specifically includes the following steps: Step 2.1, Calculate window volatility: Set the window parameter M to the number of all original time series, and calculate the volatility of the window data f. window Calculate the window mean μ window and window standard deviation σ window : Where i is a specific power load data index within the window, i∈[1,M]; Step 2.2: Calculate the sliding window volatility va of the power load data: Step 2.3: Dynamically adjust the penalty factor α based on the sliding window volatility va: α new =α base ·(1+k·va) Where, α base Let α be the initial penalty factor for the VMD algorithm. new This is the penalty factor after dynamic adjustment based on the sliding window volatility va and the adjustment coefficient k; Step 2.4: Adjust the dynamically adjusted penalty factor α new By incorporating the VMD algorithm, the original power load time series is decomposed to obtain K intrinsic mode function (IMF) components u. k The center frequency ω corresponding to the intrinsic mode function (IMF) components k The formula is: Where f is the original power load time series, u k For the multiple intrinsic mode functions (IMF) components obtained by SVMD decomposition, corresponding to each component after decomposition of the original power load sequence, ω k Let δ(t) be the center frequency, and δ(t) be the Dirac function. This represents the signal gradient.

3. The short-term power load forecasting method based on a hybrid neural network model according to claim 1, characterized in that: The Bidirectional Adaptive Long Short-Term Memory (BiALSTM) network layer includes: Input sequence: The local multi-scale features obtained after processing by the temporal convolutional network are used as the initial input; Embedding layer: Embeds local multi-scale features, converting them into local multi-scale vectors; Dynamic threshold generator: Generates a dynamic forget gate threshold based on whether the local multi-scale features are at their peak. Forward LSTM unit: Starting from the local multi-scale feature origin, it sequentially passes through the input gate, forget gate, cell state, and output gate to obtain the forward hidden state sequence; Reverse LSTM unit: It traces back from the endpoint of local multi-scale features, and also passes through the input gate, forget gate, cell state, and output gate to obtain the reverse hidden state sequence; Bidirectional output concatenation: The outputs of the forward LSTM unit and the backward LSTM unit are concatenated, and the forward hidden state sequence and the backward hidden state sequence are fused to obtain the global temporal dependency feature, which is then output to the attention layer.

4. The short-term power load forecasting method based on a hybrid neural network model according to claim 1, characterized in that: The specific process by which the Bidirectional Adaptive Long Short-Term Memory (BiALSTM) network layer obtains global temporal dependency features includes the following steps: Step 3.1.1: Receive the local multi-scale features obtained after processing by the temporal convolutional network and map them to the desired dimension d of the Bidirectional Adaptive Long Short-Term Memory (BiALSTM) network layer. in , to obtain the feature sequence in Step 3.1.2: Design a dynamic forget gate: Let the timestamp characteristic at time t be s. t When the time is peak period, s t The value is 1 during off-peak hours and 0 during off-peak hours. Step 3.1.3: Calculate the dynamic forgetting gate threshold: θt = θt max ·s t +θt min ·(1-s t ), θt max θt is the forgetting threshold during peak hours. min This is the forgetting threshold, excluding peak periods; During peak hours, i.e. t When the value is 1, the characteristics of the peak memory period in the forgetting gate are observed. Off-peak hours (s) t When the value is 0, the forget gate remembers the characteristics of the current time period; Step 3.1.4: Filter the original forget gate output f using the dynamic forget gate threshold θt. t That is, the adaptive forget gate f is obtained by comparing the dynamic forget gate threshold θt. t ′: f t =σ(W f ·[h t-1 ,x t ]+b f ) Where σ is the sigmoid function, W f Let h be the weight matrix. t-1 x is the hidden state of the previous time step. t For the current input, b f For bias; When f t ≥θ t Then retain the original forget gate output at that moment, i.e., f. t ′=f t When f t <θ t Then forced forgetting is f t ′=0; Step 3.1.5, Bidirectional Global Timing Modeling: Forward LSTM Unit (LSTM) forward Process the feature sequence X from the starting point (t=1) to the ending point (t=T) of the local multi-scale features. FC Inverse LSTM unit (LSTM) backward From the local multi-scale feature endpoint, i.e., t=T, to the feature sequence X starting at t=1 FC The forward and reverse hidden state sequences are obtained by updating the cell state, and the cell state c t The update formula is: Where ⊙ represents the element-wise product, i t For input gate output, c t b is the current cell state. c This is a bias term used during cell state updates; Hidden state calculation: h t = no t ⊙ fishy (c t ) Where σ is the activation function, W o Let b be the weight matrix of the output gate. o h is the bias term for the output gate. t The current state is hidden. Step 3.1.6, Bidirectional Output Concatenation: Concatenate the forward hidden state sequence and reverse hidden state sequence By concatenating time sequences, we obtain the global temporal dependency features. Each yes and The concatenation is then fed into the improved attention mechanism layer.

5. The short-term power load forecasting method based on a hybrid neural network model according to claim 4, characterized in that: The improved attention mechanism layer (MA) performs attention-weighted fusion of global temporal dependency features to obtain the final load forecast. The improved attention mechanism layer sets different attention weight matrices based on historical load features and meteorological features within the global temporal dependency features, specifically including: Step 3.2.1: Transfer the global temporal dependency feature H BiALSTM H as input to the improved attention mechanism layer BiALSTM =[h1,…h t ]; Step 3.2.2: Based on the global temporal dependency feature H BiALSTM Meteorological characteristics W a and historical load characteristics W b Calculate the attention weight matrix W a ,in W a =[i,j]=w1+N(0,0.01) IN b =[i,j]=w2+N(0,0.01) Where w1 and w2 represent the initial scalar values, and N(0,0.01) is Gaussian noise; Step 3.2.3, Attention Weight Calculation: e t =v T tanh(W) a h t +b) Normalized weights Among them, e t Let be the attention weights at time t, and v and b be the attention parameters; e k Let exp(e) represent the attention weight at time k. t ) refers to e t Take the exponent; exp(e k ) for e k Take the exponent; Step 3.2.4: Weighted Fusion of Predicted Outputs Where S is through H BiALSTM The hidden state h at time t t The context vector is obtained by weighted summation, and the output sequence y is finally predicted through S. y=W o ·S+b o Among them, W o It is the weight matrix of the output gate, b o It is the output layer bias term; Step 3.2.5: Reassemble the final output sequence obtained from different intrinsic mode components to obtain the final load prediction value.

6. The short-term power load forecasting method based on a hybrid neural network model according to claim 1, characterized in that: Step 4 specifically includes the following steps: Step 4.1: Set key optimization parameters and particle swarm information: For the hyperparameters to be optimized, namely the number of iterations, learning rate, and number of two hidden layer nodes of the TCN-BiALSTM-MA multimodal prediction model, set the swarm size, maximum number of iterations, and value range of the parameters to be optimized. Each particle represents a set of hyperparameter combinations to be optimized. The range of optimized hyperparameter combinations is shown below: Among them, T max is the maximum number of iterations for training the multimodal prediction model, lr is the learning rate, and h is the number of nodes in the two hidden layers of BiALSTM. The particle encoding format is as follows: X i =[T max,i ,lr i ,h 1,i ,h 2,i ] Where each X i Represents a set of hyperparameter combinations; Step 4.2, Particle Swarm Initialization: The hyperparameter combination of particles is initialized using the Tent mapping. After particle swarm initialization, an initial set of hyperparameters is generated, and then prediction is performed using the hyperparameter combination. WITH n+1 =2 min(Z n ,1-Z n ),Z0∈(0,1),Z0≠0.5 x ij =L j +Z n ·(U j -L j ) Among them, Z n x represents a random number between 0 and 1. ij L represents the value of the j-th dimension hyperparameter of the i-th particle. j U j These represent the maximum and minimum values ​​of the hyperparameter combination, respectively. Step 4.3, Fitness Evaluation: To comprehensively evaluate the feasibility of the TCN-BiALSTM-MA multimodal prediction model, the mean absolute percentage error (MAPE) is selected as the fitness value for this particle; the smaller the fitness value, the better the performance of this set of parameters. in, For the predicted value, y k The true value is given, and n represents the number of samples. Step 4.4, Particle Position Update and Backward Learning Strategy: Calculate dynamic inertial weights and generate a backward solution through lens imaging back learning. Dynamically adjust the particle velocity and position according to the current iteration stage until the maximum number of iterations T is reached. total Or convergence: The dynamic inertia weights are as follows: Where t is the current iteration number, T total Let ω(t) represent the total number of iterations for the hyperparameter search, and let ω(t) represent the inertia weight at the t-th iteration. Lens imaging uses inverse learning to generate inverse solutions, retaining the solutions with better fitness, as shown in the following formula: x ij ′ =L j +U j -x ij Where, x ij L represents the value of the j-th dimension hyperparameter of the i-th particle. j U j These represent the maximum and minimum values ​​of the hyperparameter combination, respectively. Velocity and position are updated by dynamically adjusting the particle's velocity and position based on the current iteration stage to obtain the hyperparameter combination represented by the next particle. The velocity update and position formulas are shown below: v ij (t+1)=ω(t)·v ij (t)+c1·r1·(pbest ij -x ij (t))+c2·r2·(gbest j -x ij (t)) x ij (t+1)=x ij (t)+x ij (t+1) Where c1=c2=2, representing cognitive and social factors respectively, r1,r2~U(0,1) are uniform random numbers, and pbest ij For particle i in the j-th dimension, gbest j The value of the j-th dimension parameter of the globally optimal particle; For boundaries, the following processing is performed to ensure that the parameters do not exceed the set range: Step 4.5: Output the optimal parameter combination: Set the parameters of the TCN-BiALSTM-MA multimodal prediction model using the updated hyperparameter combination represented by the particle, calculate the fitness, and after multiple iterations until the maximum number of iterations or the TLPSO algorithm converges (i.e., the difference between the calculated fitness values ​​does not exceed 0.05%), output the optimal particle to guide the subsequent training and prediction process of the model.

Images