A small sample load forecasting method based on hybrid neural network and generative adversarial

Abstract

The present invention discloses a small sample load forecasting method based on a hybrid neural network and generative adversarial methods. The method comprises the following steps: collecting historical user electricity consumption data and corresponding weather and date features; constructing a load feature input matrix; extracting high-dimensional load features of historical users and new users, and sending them into a load forecasting neural network for pre-training; fixing the source domain load feature extractor parameters, and constructing a target domain load feature extractor, and updating the target domain feature extractor parameters; connecting the target domain feature extractor G2 obtained through adversarial training with the pre-trained load forecasting neural network, and applying it to the load forecasting of new users in the target domain. The present invention makes full use of a large amount of historical load knowledge already available in the source domain through the transfer of knowledge between different domains, effectively solving the problem of insufficient day-ahead load forecasting accuracy when electricity consumption data of new users in the power system is insufficient, and giving full play to the value of historical load data.

Description

Technical Field

[0001] The present invention relates to the technical field of power load forecasting, and in particular to a small sample load forecasting method based on a hybrid neural network and generative adversarial networks. Background Art

[0002] In recent years, with the development of the national economy and the improvement of social living standards, the demand for electricity across all industries has been continuously increasing. Rational energy utilization and maintaining a balance between power grid supply and demand and that of various sectors are major livelihood issues. Accurate power load forecasting is not only a key means to achieve economical operation and rational dispatch of power grids, but also a crucial component of power system management. Currently, thanks to advances in computing power, various AI-based deep learning methods are being applied to power load forecasting. High-precision forecasts not only ensure the economical and safe operation of power systems, but also provide a solid foundation for a large number of industrial and commercial users to rationally plan their electricity usage and optimize energy conservation. While data-driven deep learning methods have achieved promising results in power load forecasting, particularly for short-term load forecasting with forecast horizons ranging from a few days to a few weeks, they still have some shortcomings. Existing deep learning prediction models are typically trained extensively on data from existing users in the power system. When new industrial and commercial users enter the system, deep learning-based methods are incapable of predicting load changes due to their limited operational time and limited available data. At the same time, the different working modes of new users will lead to different data distribution differences, which will reduce the generalization ability of the original deep prediction model and even cause significant deviations. Therefore, how to transfer the original prediction knowledge to new users in the power system and enhance the generalization ability of the existing prediction model under different power consumption modes is an urgent problem to be solved. At present, relevant research has explored similar problems. For example, a method for generating load scenarios for integrated energy systems based on generative adversarial networks (CN111950868A) in the prior art uses generative adversarial networks to learn the probability distribution of existing load time series data to generate different real scenarios of integrated energy systems to solve the problem of lack of real data. The existing method and system for IES incomplete data load forecasting based on C-GAN transfer learning (CN111191835A) addresses the problem of incomplete data in integrated energy systems by expanding the samples through a C-GAN (conditional generative adversarial network). The above methods all focus on the structural construction of the transfer learning model and the characteristics of small sample data themselves, and do not consider the characteristic differences in sample distribution between different fields. Summary of the Invention

[0003] The present invention addresses the shortcomings and problems in the existing technology, and takes into account the similarities and differences in the distribution of sample features between different fields, and proposes a small-sample load forecasting method based on a hybrid neural network and generative adversarial methods. It overcomes the shortcomings of traditional artificial intelligence methods when new industrial and commercial users appear in the power system, and solves the problem of different data features caused by different energy consumption modes of new users in a small sample. The existing model can be retrained through the transfer learning method to achieve high-precision load forecasting under user electricity consumption data in a small sample, providing an effective solution for improving the generalization ability of the load forecasting model.

[0004] The purpose of the present invention is achieved by at least one of the following technical solutions.

[0005] A small sample load forecasting method based on a hybrid neural network and generative adversarial model includes the following steps:

[0006] S1. Identify historical users with similar electricity usage behavior to the new user, and use smart meters to collect historical user electricity usage data and corresponding weather and date characteristics;

[0007] S2. Construct a load characteristic input matrix and determine the lag length of the source domain user historical data input into the prediction model;

[0008] S3. Construct a load feature extractor, use the load feature extractor to extract high-dimensional load features of historical users and new users, and feed the high-dimensional load features of historical users into the load forecasting neural network for pre-training until the load forecasting neural network converges;

[0009] S4. Fix the parameters of the source domain load feature extractor G1 and construct the target domain load feature extractor G2. Introduce a generative adversarial network to align the distribution of the source domain load and the target domain load in high-dimensional space, calculate the Wasserstein distance between the source domain and target domain distributions, and use it to update the parameters of the target domain feature extractor.

[0010] S5. Connect the target domain feature extractor G2 obtained by adversarial training with the pre-trained load prediction neural network. At this time, G2 has learned the consistent features of the source and target domains, and the target domain load features always belong to the subcategory of the source domain load features with sufficient data volume. Therefore, the prediction network with good prediction ability for this type of features can be applied to the load prediction of new users in the target domain.

[0011] Furthermore, in step S1, the method used to determine historical users with similar electricity usage behaviors to the newly added user is the edit distance on real sequence (EDR). Unlike traditional measurement methods, the edit distance on real sequence can measure the similarity of sequence vectors of different lengths, as follows:

[0012] S1.1. Set the historical user load data as the source domain data to be selected, and the newly added user load data as the target domain data. Assume that the load characteristic data of the source domain and the target domain to be selected are vectors V sou 、V tar , whose dimensions are m and n respectively, and an empty matrix Mat of size (m+1)×(n+1) is generated by initialization;

[0013] S1.2. Initialize the first row and first column of the matrix Mat, Mat[0][b] = b, Mat[a][0] = a, a = 0, 1 ... m, b = 0, 1 ... n;

[0014] S1.3. Calculate the distance between each element in the matrix Mat in turn where v sou (c) represents the value of the cth load characteristic element in the source domain, v tar (e) represents the value of the e-th load characteristic element in the target area;

[0015] S1.4, if d(v sou (c),v tar (e))≤ε, let ρ=1, otherwise ρ=0, ε is the threshold, and ρ is the penalty factor;

[0016] S1.5. Fill in the element values of the matrix Mat according to the iterative formula Mat[a][b]=min{Mat[a][b+1]+1,Mat[a+1][b]+1,Mat[a][b]+ρ};

[0017] S1.6. Increase a and b by 1 respectively, and repeat steps S1.3-S1.5 until all elements in the matrix Mat are traversed; return the last element Mat[m][n] in the matrix Mat, then the vector V sou and V tar The similarity between Similarity γ(V sou ,V tar )The historical user data sequence that is less than the set threshold is the source domain data.

[0018] Furthermore, in step S1, the smart meter collects historical user electricity consumption data as active power data, with a sampling interval of 30 minutes;

[0019] Weather characteristics include seven dimensions of meteorological characteristics: temperature, light intensity, humidity, air pressure, wind direction, wind speed, and visibility, with a sampling interval of 30 minutes;

[0020] Date features include weekdays and holidays.

[0021] Furthermore, in step S2, a load feature input matrix is constructed as a two-dimensional feature matrix of size d×k, where d is the number of historical load days to be determined, and k is the load feature dimension, including weather features and date features, a total of 9 dimensions;

[0022] Methods used to determine the lag length of source domain user historical data input into the prediction model include grey correlation analysis.

[0023] Furthermore, in step S3, a load feature extractor is constructed as a two-dimensional convolutional neural network, including a convolution layer for extracting nonlinear features, a pooling layer for reducing the dimension of features, an activation function, a batch normalization layer for preventing overfitting, a random dropout layer, and a fully connected layer;

[0024] The high-dimensional load feature is the output feature of the load feature extractor, which includes the first convolution layer Conv2d1, the first pooling layer Maxpool1, the second convolution layer Conv2d2, the second pooling layer Maxpool2, the third convolution layer Conv2d3, the first fully connected layer Dense1 and the second fully connected layer Dense2, which are connected in sequence.

[0025] Furthermore, the load forecasting neural network is a long short-term memory (LSTM) network. Its internal structure includes two LSTM layers. Two fully connected layers are connected after the two LSTM layers to change the output dimension to the required dimension. The LSTM network can memorize the historical information of the time series, avoiding the problem of sequence information loss.

[0026] Pre-training involves dividing 80% of source domain users into a training set and 20% into a test set. The constructed d×k feature input matrix is input into the load feature extractor and connected to the load forecasting neural network. The output label is the electric load at 48 o'clock on the day to be predicted, in kW. The parameters of the load feature extractor and the load forecasting neural network are updated using the stochastic gradient update method.

[0027] Furthermore, in step S4, the structures of the target domain load feature extractor G2 and the source domain load feature extractor G1 are the same as the structure of the load feature extractor constructed in step S3, but the weight parameters are initialized to a normal distribution with a mean of 0 and a variance of 1.

[0028] Furthermore, in step S4, the alignment training process of the generative adversarial network is specifically as follows:

[0029] S4.1. Obtain the load characteristic matrix F of the source domain and the target domain respectively sou and F tar , respectively input the source domain load feature extractor G1 and the target domain load feature extractor G2, and obtain the nonlinear high-dimensional features H sou=G1(x sou θ sou ) and H tar =G2(x tar θ tar ); where x sou ,θ sou 、H sou They represent the original sample data of the source domain input load, the network parameters of the source domain load feature extractor G1, and the high-dimensional features of the source domain load sample extracted by the source domain load feature extractor G1; x tar ,θ tar 、H tar They represent the original sample data of the load input into the target domain, the network parameters of the target domain load feature extractor G2, and the high-dimensional features of the target domain load sample extracted by the target domain load feature extractor G2 respectively;

[0030] The Wasserstein distance is used to measure the difference in high-dimensional feature distribution between the source and target domains, namely WGAN (Wasserstein generative adversarial network, WGAN). The solution process of the Wasserstein distance is as follows:

[0031]

[0032] Among them, Π(H sou ,H tar ) represents the high-dimensional feature H of the source domain sou and the target domain high-dimensional features H tar The joint distribution of W(H sou ,H tar ) is the Wasserstein distance between high-dimensional features; in all possible joint distributions, the expected infimum of the sample distance is the Wasserstein distance between them;

[0033] Among them, the improvement of the present invention is that the optimal transmission theory is used to solve the approximate Wasserstein distance, and the result can be obtained by performing fewer iterations. The solution is fast and can meet the real-time requirements of load forecasting. The optimal transmission theory is used to add a regularization term for relaxation, and the Wasserstein distance W(H sou ,H tar ) performs the following conversion:

[0034]

[0035] Among them, m sou 、n tar Represent the number of high-dimensional feature samples in the source domain and the number of high-dimensional feature samples in the target domain respectively; Cij H sou The i-th feature is transferred to H tar The cost used by the j-th high-dimensional feature in , the cost distance is expressed as Euclidean distance; μ represents the regularization coefficient; h(T) represents the regularization term; T represents the transmission plan; T ij Indicates that from H sou The i-th load characteristic in is transmitted to the target domain H tar The amount of data of the j-th load characteristic in sou ,H tar ) is converted into the optimal transmission plan T best The solution of

[0036] Optimal transmission plan T best The solution is performed using the near-end point iteration method, and the iterative solution expression is:

[0037]

[0038] Where t represents the number of iterations; β represents the change step size; h(T) represents the entropy function form of the transmission plan T; T best The solution can be converted into the following form through the sinkhorn algorithm:

[0039]

[0040] Among them, T ijbest Indicates that from H sou The i-th feature is transferred to H tar The optimal amount of data for the jth feature of R; R is the intermediate matrix; R′ is the transpose of the intermediate matrix R; R ij Indicates that from H sou The i-th feature is transferred to H tar The intermediate cost of the jth feature; ε is the coefficient of variation between (0,1); C ij H sou The i-th feature in is transferred to H tar The cost of the j-th high-dimensional feature in p; (t) and q (t) is the intermediate vector of the current iteration number; p (t+1) and q (t+1) is the intermediate vector of the next iteration, p (t) and q (t) Initialized to the inverse of the input feature size and u and v are two matrices whose elements are all 1;

[0041] S4.2. Calculate the Wasserstein distance W(H sou ,H tar), in order to enhance the high-dimensional features H of the source and target domains sou and H tar Internal alignment, introducing local loss L fea :

[0042]

[0043] Where M is the number of convolutional layers in the target domain load feature extractor G2 and the source domain load feature extractor G1; G 1f (x sou ) represents the output feature of the f-th convolution layer of G1; G 2f (x tar ) represents the output feature of the f-th layer convolution of G2; the total loss Loss total Expressed as:

[0044]

[0045] Among them, α1 and α2 are weight coefficients; during training, the back propagation algorithm is used to continuously update the generator and discriminator;

[0046] For the generator, it is hoped to generate target domain features with similar distribution as much as possible, so the total loss Loss should be minimized. total , Then the parameter update method of generator G2 is:

[0047]

[0048] in, is the target domain load feature extractor G2 parameter; is the learning rate; is the gradient of the target domain load feature extractor G2;

[0049] For the discriminator, it is hoped to learn the consistency between the source domain and the target domain, and to maximize the total loss, L D =-Loss total , then the update method of the discriminator parameters is:

[0050]

[0051] Among them, θ D is the network parameter of the discriminator D; α D is the learning rate; is the gradient of D; repeat the training steps in steps S4.1 and S4.2 until the network converges;

[0052] When training a generative adversarial network, the generator hopes to make the target domain generated data as close as possible to the source domain data distribution, while the discriminator is responsible for distinguishing the target domain generated data from the source domain real data. After continuous adversarial games, G2 is finally able to learn the consistent features of the source and target domains. Since the target domain features always belong to the subclass of the source domain, the prediction network with good predictive ability for this type of features can be applied to the target domain load forecasting.

[0053] Furthermore, in step S4.2, the discriminator D is composed of a three-layer fully connected neural network, and the first and second fully connected neural network layers are both activated using the ReLu() activation function.

[0054] Furthermore, the loss function used in the load forecasting neural network is the mean square error loss function.

[0055] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0056] The present invention proposes a small-sample user load forecasting method based on a hybrid neural network and generative adversarial methods, which achieves deep automatic extraction of load characteristics through a two-dimensional convolutional neural network. At the same time, the introduction of a generative adversarial network reduces the differences in the distribution of different electricity consumption characteristics brought about by new users in the power system, allowing the original trained prediction model to be transferred to the new users for load forecasting, avoiding the additional cost of training the model from scratch. In addition, the present invention uses the proximal endpoint iteration method to solve the Wasserstein distance, ensuring the instability of the parameters during the training of the generative adversarial network, and can be used for on-site prediction, which is of great significance and social value. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a flowchart of the steps of a small sample load prediction method based on a hybrid neural network and generative adversarial methods in an embodiment of the present invention.

[0058] Figure 2a 、 Figure 2b and Figure 2c These are diagrams of source domain selection results in different embodiments of the present invention.

[0059] Figure 3a 、 Figure 3b and Figure 3c They are respectively diagrams of the lag correlation results in different embodiments of the present invention.

[0060] Figure 4 2 is a structural diagram of a feature extractor in an embodiment of the present invention.

[0061] Figure 5 This is a structural diagram of an LSTM unit in an embodiment of the present invention.

[0062] Figure 6 This is a graph showing the change in WGAN adversarial training loss in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described below with reference to specific embodiments.

[0064] Example:

[0065] The simulation examples in this paper are based on real user load data from four industrial parks, A, B, C, and D, in a certain region, covering the period from January 1, 2011, to December 31, 2012. Parks A, B, and C are designated as source domains, while three randomly selected 15-day power loads from Park D are designated as target domains. In each of the three examples, the simulations were validated using three datasets. The experimental platform was an Intel(R) Xeon(R) @ 2.20 GHz, a Tesla P100 GPU, 16 GB of memory, Python 3 as the programming language, and PyTorch 1.5 as the deep learning framework.

[0066] A small sample load prediction method based on hybrid neural network and generative adversarial Figure 1 As shown, the following steps are included:

[0067] S1. Identify historical users with similar electricity usage behavior to the new user, and use smart meters to collect historical user electricity usage data and corresponding weather and date characteristics;

[0068] The method used to identify historical users with similar electricity usage behaviors to the new user is the edit distance on real sequence (EDR). Unlike traditional measurement methods, the edit distance on real sequence can measure the similarity of sequence vectors of different lengths. The details are as follows:

[0069] S1.1. Set the historical user load data as the source domain data to be selected, and the newly added user load data as the target domain data. Assume that the load characteristic data of the source domain and the target domain to be selected are vectors V sou 、V tar , whose dimensions are m and n respectively, and an empty matrix Mat of size (m+1)×(n+1) is generated by initialization;

[0070] In this embodiment, the dimension sizes of 13, 14, 15, 16, and 17 are randomly selected, and the historical user data to be selected is randomly divided approximately according to the target domain data length days; that is, m = 13, 14, 15, 16, 17, n = 15;

[0071] S1.2. Initialize the first row and first column of the matrix Mat, Mat[0][b] = b, Mat[a][0] = a, a = 0, 1 ... m, b = 0, 1 ... n;

[0072] S1.3. Calculate the distance between each element in the matrix Mat in turn where v sou (c) represents the value of the cth load characteristic element in the source domain, v tar (e) represents the value of the e-th load characteristic element in the target area;

[0073] S1.4, if d(v sou (c),v tar (e))≤ε, let ρ=1, otherwise ρ=0, ε is the threshold, and ρ is the penalty factor;

[0074] S1.5. Fill in the element values of the matrix Mat according to the iterative formula Mat[a][b]=min{Mat[a][b+1]+1,Mat[a+1][b]+1,Mat[a][b]+ρ};

[0075] S1.6. Increase a and b by 1 respectively, and repeat steps S1.3-S1.5 until all elements in the matrix Mat are traversed; return the last element Mat[m][n] in the matrix Mat, then the vector V sou and V tar The similarity between Similarity γ(V sou ,V tar ) is less than the set threshold value of the historical user data sequence is the source domain data, in this embodiment, the threshold is set to 0.5. The source domain selection results are shown in Figure 2a 、 Figure 2b 、 Figure 2c shown.

[0076] The smart meter collects historical user electricity consumption data as active power data, with a sampling interval of 30 minutes;

[0077] Weather characteristics include seven dimensions of meteorological characteristics: temperature, light intensity, humidity, air pressure, wind direction, wind speed, and visibility, with a sampling interval of 30 minutes;

[0078] Date features include weekdays and holidays.

[0079] S2. Construct a load characteristic input matrix and determine the lag length of the source domain user historical data input into the prediction model;

[0080] The load feature input matrix is constructed as a two-dimensional feature matrix of size d×k, where d is the number of historical load days to be determined and k is the load feature dimension, including weather features and date features, totaling 9 dimensions.

[0081] The method used to determine the lag length of the source domain user historical data input into the prediction model includes the grey correlation analysis method. When the correlation degree decreases, the length is determined as the turning point. The final lag length is determined to be 3, and the lag correlation degree results are as follows: Figure 3a 、 Figure 3b 、 Figure 3c shown.

[0082] S3. Construct a load feature extractor, use the load feature extractor to extract high-dimensional load features of historical users and new users, and feed the high-dimensional load features of historical users into the load forecasting neural network for pre-training until the load forecasting neural network converges;

[0083] The load feature extractor is constructed as a two-dimensional convolutional neural network, including a convolution layer for extracting nonlinear features, a pooling layer for feature dimensionality reduction, an activation function, a batch normalization layer to prevent overfitting, a random dropout layer, and a fully connected layer;

[0084] The high-dimensional load feature is the output feature of the load feature extractor, such as Figure 4 As shown, the load feature extractor includes a first convolutional layer Conv2d1, a first pooling layer Maxpool1, a second convolutional layer Conv2d2, a second pooling layer Maxpool2, a third convolutional layer Conv2d3, a first fully connected layer Dense1, and a second fully connected layer Dense2, which are connected in sequence;

[0085] In this embodiment, the input of the load feature extractor is a load feature input matrix of size batch×144×9, where batch is the batch size; in the first convolutional layer Conv2d1, there are 8 convolution kernels of size 5×5, and the activation function is the ReLU activation function; in the first pooling layer Maxpool1, the pooling kernel size is 2×2; in the second convolutional layer Conv2d2, there are 16 convolution kernels of size 3×3, and the activation function is the ReLU activation function; in the second pooling layer Maxpool2, the pooling kernel size is The convolutional kernel is 2×2. The third convolutional layer, Conv2d3, has 64 convolution kernels of 3×3 size, followed by a batch normalization layer (BatchNorm) to prevent overfitting. The first fully connected layer, Dense1, has 64 neurons and uses the ReLU activation function. To prevent overfitting, a dropout layer with a dropout rate of 0.2 is added. The second fully connected layer, Dense2, has 200 neurons and a dropout layer with a dropout rate of 0.2 to prevent overfitting. After the load feature extractor, the output high-dimensional load features are of size batch×4×4×64.

[0086] The load forecasting neural network is a long short-term memory (LSTM) network, such as Figure 5 As shown in the figure, its internal structure includes two LSTM layers, with hidden layer neuron sizes of 50 and 60 respectively. Two fully connected layers are spliced after the two LSTM layers to change the output dimension to the required dimension. The number of neurons in the fully connected layers is 86 and 48 respectively. The LSTM network can memorize the historical information of the time series, avoiding the problem of sequence information loss.

[0087] Pre-training involves dividing 80% of source domain users into a training set and 20% into a test set. The constructed d×k feature input matrix is input into the load feature extractor and connected to the load forecasting neural network. The output label is the electric load at 48 o'clock on the day to be predicted, in kW. The parameters of the load feature extractor and the load forecasting neural network are updated using the stochastic gradient update method.

[0088] S4. Fix the parameters of the source domain load feature extractor G1 and construct the target domain load feature extractor G2. Introduce a generative adversarial network to align the distribution of the source domain load and the target domain load in high-dimensional space, calculate the Wasserstein distance between the source domain and target domain distributions, and use it to update the parameters of the target domain feature extractor.

[0089] The structures of the target domain load feature extractor G2 and the source domain load feature extractor G1 are the same as those of the load feature extractor constructed in step S3, but the weight parameters are initialized to a normal distribution with a mean of 0 and a variance of 1. The alignment training process of the generative adversarial network is as follows:

[0090] S4.1. Obtain the load characteristic matrix F of the source domain and the target domain respectively sou and F tar , respectively input the source domain load feature extractor G1 and the target domain load feature extractor G2, and obtain the nonlinear high-dimensional features H sou =G1(x sou θ sou ) and H tar =G2(x tar θ tar ); the size is batch×4×4×64; where x sou ,θ sou 、H sou They represent the original sample data of the source domain input load, the network parameters of the source domain load feature extractor G1, and the high-dimensional features of the source domain load sample extracted by the source domain load feature extractor G1; x tar ,θ tar 、H tar They represent the original sample data of the load input into the target domain, the network parameters of the target domain load feature extractor G2, and the high-dimensional features of the target domain load sample extracted by the target domain load feature extractor G2 respectively;

[0091] The Wasserstein distance is used to measure the difference in high-dimensional feature distribution between the source and target domains, namely WGAN (Wasserstein generative adversarial network, WGAN). The solution process of the Wasserstein distance is as follows:

[0092]

[0093] Among them, ∏(H sou ,H tar ) represents the high-dimensional feature H of the source domain sou and the target domain high-dimensional features H tar The joint distribution of W(H sou ,H tar ) is the Wasserstein distance between high-dimensional features; in all possible joint distributions, the expected infimum of the sample distance is the Wasserstein distance between them;

[0094] Among them, the improvement of the present invention is that the optimal transmission theory is used to solve the approximate Wasserstein distance, and the result can be obtained by performing fewer iterations. The solution is fast and can meet the real-time requirements of load forecasting. The optimal transmission theory is used to add a regularization term for relaxation, and the Wasserstein distance W(H sou ,H tar ) performs the following conversion:

[0095]

[0096] Among them, m sou 、n tar Represent the number of high-dimensional feature samples in the source domain and the number of high-dimensional feature samples in the target domain respectively; C ij H sou The i-th feature in is transferred to H tar The cost used by the j-th high-dimensional feature in , the cost distance is expressed as Euclidean distance; μ represents the regularization coefficient; h(T) represents the regularization term; T represents the transmission plan; T ij Indicates that from H sou The i-th load characteristic in is transmitted to the target domain H tar The amount of data of the j-th load characteristic in sou ,H tar ) is converted into the optimal transmission plan T best The solution of

[0097] Optimal transmission plan T bestThe solution is performed using the near-end point iteration method, and the iterative solution expression is:

[0098]

[0099] Where t represents the number of iterations; β represents the change step size; h(T) represents the entropy function form of the transmission plan T; T best The solution can be converted into the following form through the sinkhorn algorithm:

[0100]

[0101] Among them, T ijbest Indicates that from H sou The i-th feature is transferred to H tar The optimal amount of data for the jth feature of R; R is the intermediate matrix; R′ is the transpose of the R matrix; R ij Indicates that from H sou The i-th feature is transferred to H tar The intermediate cost of the jth feature; ε is the coefficient of variation between (0,1); C ij H sou The i-th feature in is transferred to H tar The cost of the j-th high-dimensional feature in p; (t) and q (t) is the intermediate vector of the current iteration number; p (t+1) and q (t+1) is the intermediate vector of the next iteration, p (t) and q (t) Initialized to the inverse of the input feature size and u and v are two matrices whose elements are all 1;

[0102] S4.2. Calculate the Wasserstein distance W(H sou ,H tar ), in order to enhance the high-dimensional features H of the source and target domains sou and H tar Internal alignment, introducing local loss L fea :

[0103]

[0104] Where M is the number of convolutional layers in the target domain load feature extractor G2 and the source domain load feature extractor G1; G 1f (x sou ) represents the output feature of the f-th convolution layer of G1; G 2f (x tar ) represents the output feature of the f-th layer convolution of G2; the total loss Loss total Expressed as:

[0105]

[0106] Among them, α1 and α2 are weight coefficients; during training, the back propagation algorithm is used to continuously update the generator and discriminator;

[0107] For the generator, it is hoped to generate target domain features with similar distribution as much as possible, so the total loss Loss should be minimized. total , Then the parameter update method of generator G2 is:

[0108]

[0109] in, is the target domain load feature extractor G2 parameter; is the learning rate; is the gradient of the target domain load feature extractor G2;

[0110] For the discriminator, it is hoped to learn the consistency between the source domain and the target domain, and to maximize the total loss, L D =-Loss total , then the update method of the discriminator parameters is:

[0111]

[0112] Among them, θ D is the network parameter of the discriminator D; D is the learning rate; is the gradient of D; repeat the training steps in steps S4.1 and S4.2 until the network converges;

[0113] When training the generative adversarial network, the generator hopes to make the target domain generated data as close as possible to the source domain data distribution, while the discriminator is responsible for distinguishing the target domain generated data from the source domain real data. After continuous adversarial games, G2 is finally able to learn the consistent features of the source and target domains. Since the target domain features always belong to the subclass of the source domain, the prediction network with good predictive ability for this type of features can be applied to the target domain load forecasting. The final prediction result is the load change at 48 o'clock the next day in the target domain.

[0114] The loss function used in the load forecasting neural network is the mean square error loss function.

[0115] The discriminator D is composed of a three-layer fully connected neural network, and the first and second fully connected neural network layers are activated using the ReLu() activation function.

[0116] S5. Connect the target domain feature extractor G2 obtained by adversarial training with the pre-trained load prediction neural network. At this time, G2 has learned the consistent features of the source and target domains, and the target domain load features always belong to the subcategory of the source domain load features with sufficient data volume. Therefore, the prediction network with good prediction ability for this type of features can be applied to the load prediction of new users in the target domain.

[0117] The prediction evaluation index uses the mean absolute percentage error y MAPE (MAPE, mean absolute percent error), its expression is as follows:

[0118]

[0119] Among them, x real (h) represents the actual value of the load at the hth moment; x pred (h) represents the predicted value of the load at the hth moment; N is the number of predictions; y MAPE The smaller it is, the better the prediction performance of the model.

[0120] The training loss curve and prediction results are as follows Figure 6 , as shown in Table 1. Table 1 is a comparison of prediction results of different prediction algorithms, where the model that has undergone feature distribution alignment is called a model that has undergone migration. The comparison prediction algorithms are mainstream load forecasting models such as CNN-LSTM with migration without local loss, CNN-LSTM without migration, LSTM, and support vector regression (SVR). The experimental data sets are the three small sample data sets mentioned above. It can be seen from the table that the small sample short-term power load forecasting method based on hybrid neural network and generative adversarial proposed in the present invention has higher prediction accuracy on all data sets compared with other traditional artificial intelligence models.

[0121] Table 1 Prediction results of different prediction algorithms

[0122]

[0123] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made based on the shape and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A small sample load forecasting method based on hybrid neural network and generative adversarial learning, characterized in that: The following steps are involved: S1. Identify historical users with similar electricity usage behavior to the new user, and use smart meters to collect historical user electricity usage data and corresponding weather and date characteristics; S2. Construct a load characteristic input matrix and determine the lag length of the source domain user historical data input into the prediction model; S3. Construct a load feature extractor, use the load feature extractor to extract high-dimensional load features of historical users and new users, and feed the high-dimensional load features of historical users into the load forecasting neural network for pre-training until the load forecasting neural network converges; construct the load feature extractor as a two-dimensional convolutional neural network, including a convolution layer for extracting nonlinear features, a pooling layer for feature dimensionality reduction, an activation function, a batch normalization layer to prevent overfitting, a random dropout layer, and a fully connected layer; The high-dimensional load feature is the output feature of the load feature extractor, which includes the first convolution layer Conv2d1, the first pooling layer Maxpool1, the second convolution layer Conv2d2, the second pooling layer Maxpool2, the third convolution layer Conv2d3, the first fully connected layer Dense1, and the second fully connected layer Dense2, which are connected in sequence; The load forecasting neural network is a long short-term memory (LSTM) network. Its internal structure consists of two LSTM layers. Two fully connected layers are connected after the two LSTM layers to change the output dimension to the required dimension. The LSTM network can memorize the historical information of the time series, avoiding the problem of sequence information loss. Pre-training involves dividing 80% of source users into a training set and 20% into a test set. The constructed d×k feature input matrix is then fed into the load feature extractor and connected to the load forecasting neural network. The output label is the electric load at 48 o'clock on the day to be predicted, in kW. The parameters of the load feature extractor and the load forecasting neural network are updated using the stochastic gradient update method. S4. Fix the parameters of the source domain load feature extractor G1 and construct the target domain load feature extractor G2. Introduce a generative adversarial network to align the distribution of the source domain load and the target domain load in high-dimensional space, calculate the Wasserstein distance between the source domain and target domain distributions, and use it to update the parameters of the target domain feature extractor. The structures of the target domain load feature extractor G2 and the source domain load feature extractor G1 are the same as those of the load feature extractor constructed in step S3, but the weight parameters are initialized to a normal distribution with a mean of 0 and a variance of 1; S5. Connect the target domain feature extractor G2 obtained through adversarial training with the pre-trained load forecasting neural network and apply it to the load forecasting of new users in the target domain.

2. The method for small sample load forecasting based on hybrid neural network and generative adversarial learning according to claim 1, characterized in that: In step S1, the method used to determine historical users with similar electricity usage behaviors to the newly added user is the real sequence edit distance method, which is as follows: S1.

1. Set the historical user load data as the source domain data to be selected, and the newly added user load data as the target domain data. Assume that the load characteristic data of the source domain and the target domain to be selected are vectors V sou 、V tar , whose dimensions are m and n respectively, and an empty matrix Mat of size (m+1)×(n+1) is generated by initialization; S1.

2. Initialize the first row and first column of the matrix Mat, Mat[0][b] = b, Mat[a][0] = a, a = 0, 1 ... m, b = 0, 1 ... n; S1.

3. Calculate the distance between each element in the matrix Mat in turn where v sou (c) represents the value of the cth load characteristic element in the source domain, v tar (e) represents the value of the e-th load characteristic element in the target area; S1.4, if d(v sou (c),v tar (e))≤ε, let ρ=1, otherwise ρ=0, ε is the threshold, and ρ is the penalty factor; S1.

5. Fill in the element values of the matrix Mat according to the iterative formula Mat[a][b]=min{Mat[a][b+1]+1,Mat[a+1][b]+1,Mat[a][b]+ρ}; S1.

6. Increase a and b by 1 respectively, and repeat steps S1.3-S1.5 until all elements in the matrix Mat are traversed; return the last element Mat[m][n] in the matrix Mat, then the vector V sou and V tar The similarity between Similarity γ(V sou ,V tar )The historical user data sequence that is less than the set threshold is the source domain data.

3. The method for small sample load forecasting based on hybrid neural network and generative adversarial learning according to claim 1, characterized in that: In step S1, the smart meter collects historical user electricity consumption data as active power data, with a sampling interval of 30 minutes; Weather characteristics include seven dimensions of meteorological characteristics: temperature, light intensity, humidity, air pressure, wind direction, wind speed, and visibility, with a sampling interval of 30 minutes; Date features include weekdays and holidays.

4. The method for small sample load forecasting based on hybrid neural network and generative adversarial learning according to claim 1, characterized in that: In step S2, a load feature input matrix is constructed as a two-dimensional feature matrix of size d×k, where d is the number of historical load days to be determined and k is the load feature dimension, including weather features and date features, totaling 9 dimensions; Methods used to determine the lag length of source domain user historical data input into the prediction model include grey correlation analysis.

5. The small sample load forecasting method based on hybrid neural network and generative adversarial learning according to any one of claims 1 to 4, characterized in that: In step S4, the alignment training process of the generative adversarial network is as follows: S4.

1. Obtain the load characteristic matrix F of the source domain and the target domain respectively sou and F tar , respectively input the source domain load feature extractor G1 and the target domain load feature extractor G2, and obtain the nonlinear high-dimensional features H sou =G1(x sou θ sou ) and H tar =G2(x tar θ tar ); where x sou ,θ sou 、H sou They represent the original sample data of the source domain input load, the network parameters of the source domain load feature extractor G1, and the high-dimensional features of the source domain load sample extracted by the source domain load feature extractor G1; x tar ,θ tar 、H tar They represent the original sample data of the load input into the target domain, the network parameters of the target domain load feature extractor G2, and the high-dimensional features of the target domain load sample extracted by the target domain load feature extractor G2 respectively; The Wasserstein distance is used to measure the difference in high-dimensional feature distribution between the source domain and the target domain. The solution process of the Wasserstein distance is as follows: Among them, Π(H sou ,H tar ) represents the high-dimensional feature H of the source domain sou and the target domain high-dimensional features H tar The joint distribution of W(H sou ,H tar ) is the Wasserstein distance between high-dimensional features; in all possible joint distributions, the expected infimum of the sample distance is the Wasserstein distance between them; The optimal transmission theory is used to add regularization terms for relaxation, and the wasserstein distance W(H sou ,H tar ) performs the following conversion: Among them, m sou 、n tar Represent the number of high-dimensional feature samples in the source domain and the number of high-dimensional feature samples in the target domain respectively; C ij H sou The i-th feature in is transferred to H tar The cost used by the j-th high-dimensional feature in , the cost distance is expressed as Euclidean distance; μ represents the regularization coefficient; h(T) represents the regularization term; T represents the transmission plan; T ij Indicates that from H sou The i-th load characteristic in is transmitted to the target domain H tar The amount of data of the j-th load characteristic in sou ,H tar ) is converted into the optimal transmission plan T best The solution of Optimal transmission plan T best The solution is performed using the near-end point iteration method, and the iterative solution expression is: Where t represents the number of iterations; β represents the change step size; h(T) represents the entropy function form of the transmission plan T; T best The solution is converted into the following form through the sinkhorn algorithm: Among them, T ijbest Indicates that from H sou The i-th feature is transferred to H tar The optimal amount of data for the jth feature of ; R is the intermediate matrix; R' is the transpose of the intermediate matrix R; R ij Indicates that from H sou The i-th feature is transferred to H tar The intermediate cost of the jth feature; ε is the coefficient of variation between (0,1); C ij H sou The i-th feature in is transferred to H tar The cost of the j-th high-dimensional feature in p; (t) and q (t) is the intermediate vector of the current iteration number; p (t+1) and q (t+1) is the intermediate vector of the next iteration, p (t) and q (t) Initialized to the inverse of the input feature size and u and v are two matrices whose elements are all 1; S4.

2. Calculate the Wasserstein distance W(H sou ,H tar ), in order to enhance the high-dimensional features H of the source and target domains sou and H tar Internal alignment, introducing local loss L fea : Where M is the number of convolutional layers in the target domain load feature extractor G2 and the source domain load feature extractor G1; G 1f (x sou ) represents the output feature of the f-th convolution layer of G1; G 2f (x tar ) represents the output feature of the f-th layer convolution of G2; the total loss Loss total Expressed as: Among them, α1 and α2 are weight coefficients; during training, the back propagation algorithm is used to continuously update the generator and discriminator; For the generator, it is hoped to generate target domain features with similar distribution as much as possible, so the total loss Loss should be minimized. total , Then the parameter update method of generator G2 is: in, is the target domain load feature extractor G2 parameter; is the learning rate; is the gradient of the target domain load feature extractor G2; For the discriminator, it is hoped to learn the consistency between the source domain and the target domain, and to maximize the total loss, L D =-Loss total , then the update method of the discriminator parameters is: i D =θ D +a D ▽Loss total ; Among them, θ D is the network parameter of the discriminator D; D is the learning rate; ▽L D is the gradient of D; repeat the training steps in steps S4.1 and S4.2 until the network converges.

6. The method for small sample load forecasting based on hybrid neural network and generative adversarial learning according to claim 5, characterized in that: In step S4.2, the discriminator D is composed of a three-layer fully connected neural network, and the first and second fully connected neural network layers are activated using the ReLu() activation function.

7. The method for small sample load forecasting based on hybrid neural network and generative adversarial learning according to claim 5, characterized in that: The loss function used in the load forecasting neural network is the mean square error loss function.

Images