Source-load interval prediction method based on combination of SWLSTM and ensemble learning

Abstract

The invention discloses a source-load interval prediction method based on the combination of SWLSTM and integrated learning, and the method comprises the following steps: S1, obtaining an original data set, carrying out the dimension reduction of the original data set based on local linear embedding, and carrying out the clustering of the data set after the dimension reduction based on a k-means algorithm, and obtaining a feature day data set; s2, training each base learner based on the feature day data set, respectively calculating Q statistical values among the results of each base learner, and screening out a base learner pair with the minimum Q statistical value as a base learner model; s3, taking the base learner model as a first-layer prediction model of an integrated learning framework, taking the SWLSTM neural network as a second-layer prediction model, and introducing a confidence interval to obtain prediction uncertainty information; and S4, based on the point prediction result and the uncertainty prediction result obtained by prediction in the step S3, evaluating the prediction result by adopting a point prediction evaluation index and an interval prediction index respectively.

Description

Technical Field

[0001] The present invention relates to the technical field of source-load interval prediction, and more specifically to a source-load interval prediction method based on the combination of SWLSTM and ensemble learning. Background Art

[0002] The existing source-load prediction research methods mainly include model method, Monte Carlo random simulation method, and artificial intelligence method. The simulation method usually uses mathematical modeling and simulation methods to simulate real behaviors, and the prediction results are not credible. The data-driven artificial intelligence method uses real load data to achieve high-precision source-load prediction; it has considerable reliability. Traditional artificial intelligence methods such as support vector machines, BP neural network and other algorithms have certain nonlinear fitting capabilities and parameter learning capabilities; however, their training models require a large number of data samples and have high requirements for data; at the same time, the model complexity is high, and the training process is prone to overfitting, and the training time is long. Long short-term memory neural networks (LSTM) can process the time series characteristics of data, and EV charging characteristics and photovoltaic output have certain time series characteristics, so LSTM can be used to realize source-load power prediction. Gated recurrent unit (GRU) is a variant of LSTM. Compared with the LSTM network model, GRU couples the forget gate with the input gate, retains only the input and output gates, and reduces the training time of the system. Inspired by GRU, Zhang proposed a shared weight long short-term memory network (SWLSTM), which reduces the weight matrix of LSTM and controls the weight matrices of all gate structures by the same gate; therefore, SWLSTM can reduce the number of weights that need to be optimized in the network, thereby reducing the network training time without reducing the accuracy of the model.

[0003] The publication number is CN115222137A, and the name is A method for short-term interval prediction of microgrid load based on GWO-LSSVM. It discloses: S1. Data input: input load-related data, including meteorological elements, power consumption characteristics data, such as holidays, weekends, special events, etc., load and other information, S2. Data preprocessing: integrate and summarize data from multiple sources, mainly data from multiple files or databases, to analyze data characteristics at different levels. The short-term interval prediction method for microgrid load based on GWO-LSSVM combines MIC and grey correlation method to screen similar load sequences, improves the prediction validity and accuracy, and establishes a short-term prediction model for load power based on GWO-LSSVM, solves the problem of training model parameter optimization, and improves the prediction accuracy. Finally, the interval prediction method of error fitting is used to solve the uncertainty problem of load point prediction error, and the prediction interval coverage and prediction interval width are used to reflect the accuracy of the prediction interval.

[0004] The publication number is CN113657660A, and the name is a heat source load prediction method based on substation load and heating network lag model. The method includes collecting operating data and corresponding weather data of each heating station in a first time interval and a second time interval within a preset historical time period, selecting valid data to form a sample data set; extracting feature correlation of dynamic data, preprocessing the extracted dynamic data and static data, and dividing them into a training set and a test set; constructing a deep learning model for load prediction, inputting the training set into the deep learning model for learning to obtain load prediction data for the second time interval, and based on the heat load prediction data and the test set, measuring the prediction ability of the current deep learning model, and if it meets the requirements, inputting the test set into the deep learning model to obtain load prediction data for each heating station in the future time period; predicting the load of the heat source in the future time period according to the load prediction data and the constructed heating network lag model.

[0005] Although the above methods can achieve source load prediction, most of the above articles use a single model for prediction. The single model has randomness in the training process and homogeneity in the training results, so its generalization ability is insufficient. In recent years, the literature has further improved the prediction accuracy of the model by combining prediction models. However, the above methods are the same type of algorithms using different parameters for prediction, which cannot reflect the differences between different prediction algorithms. Therefore, an ensemble learning method has emerged in everyone's vision. Ensemble learning combines several base learner models to complete the learning task, which can fully explore the differences between different learning algorithms and learn from each other's strengths and weaknesses, thereby improving the overall prediction accuracy. Most of the existing source load predictions consider the point prediction accuracy, and there are few methods to obtain prediction uncertainty information.

[0006] Therefore, how to provide a new method based on the combination of SWLSTM-Stacking and interval prediction method to realize the prediction of uncertainty information and use confidence interval (CI) to obtain the prediction credibility boundary value is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0007] In view of this, the present invention provides a source-load interval prediction method based on the combination of SWLSTM and ensemble learning, aiming to solve the above technical problems.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] A source-load interval prediction method based on SWLSTM combined with ensemble learning includes the following steps:

[0010] Step S1, obtaining an original data set, performing dimensionality reduction processing on the original data set based on local linear embedding, and clustering the reduced-dimensional data set based on the k-means algorithm to obtain a feature day data set;

[0011] Step S2: training each base learner based on the characteristic day data set, calculating the Q statistics between the results of each base learner, and selecting the base learner pair with the smallest Q statistics as the base learner model;

[0012] Step S3: Use the base learner model as the first-layer prediction model of the integrated learning framework, use the SWLSTM neural network as the second-layer prediction model, and introduce the confidence interval to obtain prediction uncertainty information;

[0013] Step S4: Based on the prediction uncertainty information predicted in step S3, the prediction results are evaluated using a point prediction evaluation index and an interval prediction index.

[0014] Furthermore, the step S1 specifically includes:

[0015] Step S11, obtain the original data set, pre-process the original data set, use the 3σ criterion to eliminate abnormal data, and use the root mean square method to fill the data:

[0016]

[0017] Where N is the number of data samples, j i is the data point at the i-th moment; if |j i -μ|>3σ, then judge j i Abnormal, remove it from the data set;

[0018] Step S12: Reduce the dimension of the preprocessed data set.i , first select l sample centers and construct the neighborhood matrix W, using W ib Represents data point j i With neighboring point j b The weight between

[0019] Reconstruct the point by combining the neighboring points of the random combination, use the weight matrix W and the local linear relationship to minimize the error, recalculate the weight matrix M, and use M ib Represents data point j i By the neighborhood point j b Weights of linear representation;

[0020] Construct a low-dimensional matrix Y where each row of y i is the corresponding data point j i The low-dimensional representation of the matrix Y is optimized by minimizing the following objective function:

[0021]

[0022] Convert the M matrix into a positive definite matrix and take the eigenvector corresponding to the first d and the smallest non-zero eigenvalue as the data point j i The value of , where d is the dimension of the target low-dimensional space;

[0023] Step S13: randomly select k points as initial cluster centers, expressed as C = {c1, c2, ..., c k}, calculate for each data point j i The distance d(c k ,j i ), the data point j i Assigned to the cluster C with the closest cluster center k In the above example, we calculate each cluster C separately. k The mean value of all data in k , and the mean m k As the new cluster center, repeat the above steps until the change of the cluster center is less than the threshold or reaches the preset number of iterations.

[0024] Step S14: Obtain a characteristic day data set.

[0025] Furthermore, the step S2 specifically includes:

[0026] Step S21, using the characteristic day data set as a training set, pre-training the base learner model, the base learner model includes: random forest RF, gradient boosted decision tree GBDT, support vector machine SVM, decision tree DTR, gradient boosted regression GBR, extreme learning tree ET and adaptive boosting algorithm ADA;

[0027] Step S22: Calculate the correlation between the base learner models based on the Q statistic, and select the base learner pair with the smallest Q statistic as the base learner model.

[0028] Furthermore, the training process of the random forest RF and gradient boosted decision tree GBDT model is:

[0029] Random Forest RF model training:

[0030] Let N l is the number of data in the training data set, and the number of input features n is randomly selected from the training data set with replacement l , the total number is N l times, as the training set of the decision tree, where n l Much smaller than N l ; Then select q features from all the features as candidate features for decision tree division; select specific features respectively and generate decision trees recursively; repeat the above steps to build multiple decision trees;

[0031] The gradient boosting decision tree GBDT training method is:

[0032] The mean of the target variable is used as the initial predicted value of the target variable, and the difference between each sample and the observed value, i.e., the residual, is calculated. Then, the residual is used as the target variable to generate a regression tree. The goal of this decision tree is to fit the current residual as much as possible. Then, the newly generated decision tree is weighted and accumulated to the existing model, and the predicted value is updated.

[0033] Among them, weighting is to use a smaller learning rate to control the contribution of each model;

[0034] Repeat the above steps to iterate and build more decision trees. Each decision tree corrects the deviation of model prediction by fitting the residual of the previous round of iteration. Finally, all the generated decision trees are weighted and added to the initial prediction value to form the final integrated model.

[0035] Furthermore, the support vector machine (SVM) model training method is:

[0036] Construct training data set {(j i ,y i ),i=1,2,…,N},j i ,y i are training data and output prediction data respectively, N is the number of data samples, so the mathematical model of SVM is expressed as:

[0037] y i =f(j i )=w·φ(j i )+b

[0038] Among them, w is the normal vector, b is the displacement term, φ(j i ) is j i In order to obtain w, b, the following equation is satisfied:

[0039]

[0040] Among them, C is the regularization coefficient, that is, the slack variable, and its dual problem is:

[0041]

[0042] Among them, φ(j i ) and φ(j k ) are j i and j k The corresponding mapping value;

[0043] Introduce kernel function to reduce the difficulty of calculation:

[0044] k=(j i , j k )=〈φ(j i ),φ(j k )〉=φ(j i ) T φ(j k ).

[0045] Furthermore, the training process of the decision tree DTR and the gradient boosting regression GBR is:

[0046] Decision tree DTR training:

[0047] Select a feature as the partitioning condition based on the original data set. For the selected feature, set a threshold or condition to divide the data set into two subsets and minimize the variance of the target variable in each subset. Repeat the above steps on the subsets until the preset stop criteria are met. Set a prediction value on each leaf node. For new sample points, traverse the tree nodes step by step from the root node according to the partitioning condition to find the corresponding leaf node, and return the prediction value of the leaf node as the output.

[0048] Gradient boosting regression GBR training:

[0049] First, the mean of the target variable is used as the initial prediction value of the target variable, the residual of the current model is calculated, and the residual is used as the target variable to train a new weak regressor. Secondly, the weight of the model is determined according to the prediction results and learning rate of the new model to minimize the overall loss function; the prediction results are updated, and finally the above steps are repeated until the loss function converges or reaches the predetermined number of times. The prediction value function is:

[0050]

[0051] Among them, y i is the predicted value of the i-th sample, M is the number of weak regressors, h(j i ,θ i ) is the prediction result of the jth weak regressor, x i is the feature vector of the i-th sample, θ i is the parameter of the jth weak regression model; w j is the weight of the jth weak regression model, and C is a constant term.

[0052] Furthermore, the extreme learning tree ET training process is:

[0053] Perform feature selection. For each decision tree node, randomly select a subset from the complete set of input features. For each feature, randomly select a partition threshold, and calculate the information gain and Gini coefficient after partitioning the feature using the threshold.

[0054] Implement node partitioning, use the above selected features and thresholds to partition the data set into smaller subsets, calculate the purity or impurity metric for each subset, and select the partition that can minimize the purity or impurity as the best partition;

[0055] Construct a decision tree model and repeat the above steps until the stopping condition is met, such as reaching the maximum tree depth or the number of samples in the node is less than the set threshold. Each decision tree is a randomized decision tree. The randomness of feature selection and node division makes each tree have a different structure. In the process of achieving the prediction result, for regression problems, the prediction results of each decision tree are weighted averaged to obtain the final regression result. For classification problems, voting is used to perform majority voting based on the classification results of each decision tree to obtain the final classification result.

[0056] Furthermore, the training process of the adaptive boosting algorithm ADA is as follows:

[0057] Initialize the sample, set the weight of each sample data to an equal value, select a weak classifier as the base classifier, iterate the training, and for each round of iteration, train a weak classifier based on the current training data weight, calculate the classification accuracy of the weak classifier, and use it as the weight of the weak classifier;

[0058] Update the weight of the training data. According to the classification error rate, increase the weight of the wrongly classified samples and reduce the weight of the correctly classified samples. Repeat multiple rounds of iterations until the preset number of iterations is reached or the classification accuracy meets the requirements.

[0059] For integrated prediction, for regression problems, the prediction results of each weak classifier are weighted averaged to obtain the final regression result. For classification problems, the classification results of each weak classifier are weighted voted to obtain the final classification result.

[0060] Furthermore, the step S3 specifically includes:

[0061] Step S31, calculate the shared gate and information status:

[0062]

[0063] Among them, W h is the weight corresponding to the hidden layer output at the previous moment, W x is the weight corresponding to the current input, b is the weight bias term, x t is the input value at the current moment, h t-1 is the hidden state of the previous moment, σ is the sigmoid activation function;

[0064] Step S32: Update cell status:

[0065] C t =f t *C t-1 +(1-f t )*a t

[0066] Step S33, calculate the hidden layer output:

[0067] h t =f t **tamC t

[0068] Step S34: output of prediction results:

[0069] y t =σ(z t )=σ(W y ·h t +b y )

[0070] Step S35: construct optimization target:

[0071]

[0072] Among them, E t is the prediction error at time t, Y t is the observed value at time t;

[0073] Step S36: Update the SWLSTM model based on Adam, specifically:

[0074]

[0075] Among them, p i and v i are the first-order moment estimation and the second-order moment estimation of the parameters, respectively. and are the modified first-order moment estimate and second-order moment estimate respectively, η is the learning rate, W i is the weight matrix, representing [W h ,W x ,b] or [W y ,b y ];

[0076] Step S37, calculate the prediction result through forward propagation, and then update the weight through back propagation, which is one round of update; a total of m rounds of iterations are required, and each round divides the training set into n batches for training, and each batch completes one update;

[0077] Step S38: Introduce confidence interval to obtain prediction uncertainty information.

[0078] Furthermore, the step S4 specifically includes:

[0079] Step S41: Use point prediction evaluation index and interval prediction index to evaluate the prediction results:

[0080]

[0081] in, is the predicted value, f i is the actual value, is the average value, SS res SS tol are the residual sum of squares and the grand mean, respectively;

[0082] Step S42: Evaluate the effectiveness of the source-load error model, using the normalized average width of the prediction interval and the prediction interval coverage as evaluation indicators of the confidence interval of the error model:

[0083]

[0084] Among them, N' is the number of error samples, L is the prediction error value, and f upper 、f lower They are respectively the upper and lower bounds of the interval sampled at the corresponding moment of the error prediction model.

[0085] The above technical solution includes at least the following technical effects:

[0086] It can be seen from the above technical solutions that the present invention discloses a source-load interval prediction method based on the combination of SWLSTM and ensemble learning. Compared with the prior art, the present invention adopts locally linear embedding (LLE) to improve the k-means algorithm to achieve fast time series feature selection; in order to improve the problem that the generalization performance of a single model is insufficient and the combined prediction cannot reflect the differences between prediction algorithms, it is considered to use a SWLSTM neural network model to implement a multi-model fusion source-load prediction algorithm under the Stacking ensemble learning framework; in view of the correlation problem of the base learner model of the Stacking ensemble learning model, the Q statistic is used to select the base learner; finally, in view of the fact that point prediction cannot provide prediction uncertainty information, the confidence prediction interval is introduced in combination with the ensemble learning framework to achieve interval prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0088] Figure 1 The present invention is a flow chart of a source-load interval prediction method based on a shared weight long short-term memory network combined with a Stacking integrated model.

[0089] Figure 2 This is a schematic diagram of a SWLSTM structure of the present invention.

[0090] Figure 3 A schematic diagram of the framework structure of a Stacking fusion model of the present invention.

[0091] Figure 4 The figure is a schematic diagram of the effect of combining a local linear embedding with a k-means (LLE-k-means) clustering algorithm according to the present invention.

[0092] Figure 5 This is a schematic diagram of the Q statistic result of a base learner of the present invention.

[0093] Figure 6 This is a schematic diagram of an integrated learning deterministic prediction comparison of the present invention.

[0094] Figure 7 A schematic diagram of a multi-model fusion interval prediction comparison of the present invention DETAILED DESCRIPTION

[0095] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0096] See attached Figure 1-7 Embodiment 1 of the present invention discloses a source-load interval prediction method based on the combination of SWLSTM and ensemble learning, characterized in that it includes the following steps:

[0097] Step S1, obtaining an original data set, performing dimensionality reduction processing on the original data set based on local linear embedding, and clustering the reduced-dimensional data set based on the k-means algorithm to obtain a feature day data set;

[0098] Step S2: training each base learner based on the characteristic day data set, calculating the Q statistics between the results of each base learner, and selecting the base learner pair with the smallest Q statistics as the base learner model;

[0099] Step S3: Figure 3 It is a framework of the Stacking fusion model provided in an embodiment of the present invention, which uses the base learner model as the first-layer prediction model of the integrated learning framework, uses the SWLSTM neural network as the second-layer prediction model, and introduces the confidence interval to obtain prediction uncertainty information;

[0100] Step S4: Based on the prediction uncertainty information predicted in step S3, the prediction results are evaluated using a point prediction evaluation index and an interval prediction index.

[0101] Embodiment 2, to further optimize the above technical solution, step S1 specifically includes:

[0102] Step S11, obtain the original data set, pre-process the original data set, use the 3σ criterion to eliminate abnormal data, and use the root mean square method to fill the data:

[0103]

[0104] Where N is the number of data samples, j i is the data point at the i-th moment; if |j i -μ|>3σ, then judge j i Abnormal, remove it from the data set;

[0105] Step S12: Reduce the dimension of the preprocessed data set. i , first select l sample centers and construct the neighborhood matrix W, using W ib Represents data point ji With neighboring point j b The weight between

[0106] Reconstruct the point by combining the neighboring points of the random combination, use the weight matrix W and the local linear relationship to minimize the error, recalculate the weight matrix M, and use M ib Represents data point j i By the neighborhood point j b Weights of linear representation;

[0107] Construct a low-dimensional matrix Y where each row of y i is the corresponding data point j i The low-dimensional representation of the matrix Y is optimized by minimizing the following objective function:

[0108]

[0109] Convert the M matrix into a positive definite matrix and take the eigenvector corresponding to the first d and the smallest non-zero eigenvalue as the data point j i The value of , where d is the dimension of the target low-dimensional space;

[0110] Step S13: randomly select k points as initial cluster centers, expressed as C = {c1, c2, ..., c k}, calculate for each data point j i The distance d(c k ,j i ), the data point j i Assigned to the cluster C with the closest cluster center k In the above example, we calculate each cluster C separately. k The mean value of all data in k , and the mean m k As the new cluster center, repeat the above steps until the change of the cluster center is less than the threshold or reaches the preset number of iterations.

[0111] Step S14: Obtain a characteristic day data set.

[0112] To further optimize the above technical solution, step S2 specifically includes:

[0113] Step S21, using the feature day data set as a training set, pre-training the base learner model, the base learner model includes: random forest RF, gradient boosted decision tree GBDT, support vector machine SVM, decision tree DTR, gradient boosted regression GBR, extreme learning tree ET and adaptive boosting algorithm ADA;

[0114] Step S22: Calculate the correlation between the base learner models based on the Q statistic, and select the base learner pair with the smallest Q statistic as the base learner model.

[0115] To further optimize the above technical solution, the training process of the random forest RF and gradient boosted decision tree GBDT model is as follows:

[0116] Random Forest RF model training:

[0117] Let N l is the number of data in the training data set, and the number of input features n is randomly selected from the training data set with replacement l , the total number is N l times, as the training set of the decision tree, where n l Much smaller than N l ; Then select q features from all the features as candidate features for decision tree division; select specific features respectively and generate decision trees recursively; repeat the above steps to build multiple decision trees;

[0118] The gradient boosting decision tree GBDT training method is:

[0119] The mean of the target variable is used as the initial predicted value of the target variable, and the difference between each sample and the observed value, i.e., the residual, is calculated. Then, the residual is used as the target variable to generate a regression tree. The goal of this decision tree is to fit the current residual as much as possible. Then, the newly generated decision tree is weighted and accumulated to the existing model, and the predicted value is updated.

[0120] Among them, weighting is to use a smaller learning rate to control the contribution of each model;

[0121] Repeat the above steps to iterate and build more decision trees. Each decision tree corrects the deviation of model prediction by fitting the residual of the previous round of iteration. Finally, all the generated decision trees are weighted and added to the initial prediction value to form the final integrated model.

[0122] To further optimize the above technical solution, the support vector machine SVM model training method is:

[0123] Construct training data set {(j i ,y i ),i=1,2,…,N},j i ,y i are training data and output prediction data respectively, N is the number of data samples, so the mathematical model of SVM is expressed as:

[0124] y i =f(j i )=w·φ(j i )+b

[0125] Among them, w is the normal vector, b is the displacement term, φ(j i ) is ji In order to obtain w, b, the following equation is satisfied:

[0126]

[0127] Among them, C is the regularization coefficient, that is, the slack variable, and its dual problem is:

[0128]

[0129] Among them, φ(j i ) and φ(j k ) are j i and j k The corresponding mapping value;

[0130] Introduce kernel function to reduce the difficulty of calculation:

[0131] k=(j i , j k )=〈φ(j i ),φ(j k )〉=φ(j i ) T φ(j k ).

[0132] To further optimize the above technical solution, the decision tree DTR and gradient boosting regression GBR training process is:

[0133] Decision tree DTR training:

[0134] Select a feature as the partitioning condition based on the original data set. For the selected feature, set a threshold or condition to divide the data set into two subsets and minimize the variance of the target variable in each subset. Repeat the above steps on the subsets until the preset stop criteria are met. Set a prediction value on each leaf node. For new sample points, traverse the tree nodes step by step from the root node according to the partitioning condition to find the corresponding leaf node, and return the prediction value of the leaf node as the output.

[0135] Gradient boosting regression GBR training:

[0136] First, the mean of the target variable is used as the initial prediction value of the target variable, the residual of the current model is calculated, and the residual is used as the target variable to train a new weak regressor. Secondly, the weight of the model is determined according to the prediction results and learning rate of the new model to minimize the overall loss function; the prediction results are updated, and finally the above steps are repeated until the loss function converges or reaches the predetermined number of times. The prediction value function is:

[0137]

[0138] Among them, yi is the predicted value of the i-th sample, M is the number of weak regressors, h(j i ,θ i ) is the prediction result of the jth weak regressor, x i is the feature vector of the i-th sample, θ i is the parameter of the jth weak regression model; w j is the weight of the jth weak regression model, and C is a constant term.

[0139] To further optimize the above technical solution, the extreme learning tree ET training process is as follows:

[0140] Perform feature selection. For each decision tree node, randomly select a subset from the complete set of input features. For each feature, randomly select a partition threshold, and calculate the information gain and Gini coefficient after partitioning the feature using the threshold.

[0141] Implement node partitioning, use the above selected features and thresholds to partition the data set into smaller subsets, calculate the purity or impurity metric for each subset, and select the partition that can minimize the purity or impurity as the best partition;

[0142] Construct a decision tree model and repeat the above steps until the stopping condition is met, such as reaching the maximum tree depth or the number of samples in the node is less than the set threshold. Each decision tree is a randomized decision tree. The randomness of feature selection and node division makes each tree have a different structure. In the process of achieving the prediction result, for regression problems, the prediction results of each decision tree are weighted averaged to obtain the final regression result. For classification problems, voting is used to perform majority voting based on the classification results of each decision tree to obtain the final classification result.

[0143] To further optimize the above technical solution, the training process of the adaptive boosting algorithm ADA is as follows:

[0144] Initialize the sample, set the weight of each sample data to an equal value, select a weak classifier as the base classifier, iterate the training, and for each round of iteration, train a weak classifier based on the current training data weight, calculate the classification accuracy of the weak classifier, and use it as the weight of the weak classifier;

[0145] Update the weight of the training data. According to the classification error rate, increase the weight of the wrongly classified samples and reduce the weight of the correctly classified samples. Repeat multiple rounds of iterations until the preset number of iterations is reached or the classification accuracy meets the requirements.

[0146] For integrated prediction, for regression problems, the prediction results of each weak classifier are weighted averaged to obtain the final regression result. For classification problems, the classification results of each weak classifier are weighted voted to obtain the final classification result.

[0147] In order to further optimize the above technical solution, Figure 2 : is a SWLSTM structure diagram provided by an embodiment of the present invention, and step S3 specifically includes:

[0148] Step S31, calculate the shared gate and information status:

[0149]

[0150] Among them, W h is the weight corresponding to the hidden layer output at the previous moment, W x is the weight corresponding to the current input, b is the weight bias term, x t is the input value at the current moment, h t-1 is the hidden state of the previous moment, σ is the sigmoid activation function;

[0151] Step S32: Update cell status:

[0152] C t =f t *C t-1 +(1-f t )*a t

[0153] Step S33, calculate the hidden layer output:

[0154] h t =f t **tanC t

[0155] Step S34: output of prediction results:

[0156] y t =σ(z t )=σ(W y ·h t +b y )

[0157] Step S35: construct optimization target:

[0158]

[0159] Among them, E t is the prediction error at time t, Y t is the observed value at time t;

[0160] Step S36: Update the SWLSTM model based on Adam, specifically:

[0161]

[0162] Among them, p i and v i are the first-order moment estimation and the second-order moment estimation of the parameters, respectively. and are the modified first-order moment estimate and second-order moment estimate respectively, η is the learning rate, W i is the weight matrix, representing [W h ,W x ,b] or [W y ,b y ];

[0163] Step S37, calculate the prediction result through forward propagation, and then update the weight through back propagation, which is one round of update; a total of m rounds of iterations are required, and each round divides the training set into n batches for training, and each batch completes one update;

[0164] Step S38: Introduce confidence interval to obtain prediction uncertainty information.

[0165] To further optimize the above technical solution, S4 specifically includes:

[0166] Step S41: Use point prediction evaluation index and interval prediction index to evaluate the prediction results:

[0167]

[0168] in, is the predicted value, f i is the actual value, is the average value, SS res SS tol are the residual sum of squares and the grand mean, respectively;

[0169] Step S42: Evaluate the effectiveness of the source-load error model, using the normalized average width of the prediction interval and the prediction interval coverage as evaluation indicators of the confidence interval of the error model:

[0170]

[0171] Among them, N' is the number of error samples, L is the prediction error value, and f upper 、f lower They are respectively the upper and lower bounds of the interval sampled at the corresponding moment of the error prediction model.

[0172] Embodiment 3, Figure 1The flowchart of the source-load interval prediction method based on the shared weight long short-term memory network and the Stacking integrated model of the present invention is shown, which specifically includes the following steps:

[0173] (1) Collect source load data and preprocess them using the 3σ criterion.

[0174] (2) A feature day selection scheme combining local linear embedding (LLE) and k-means clustering algorithm is adopted to obtain the prediction data set. Figure 4 Shown is the effect diagram of the source-load LLE-k-means clustering algorithm of the present invention.

[0175] (3) Based on the feature data set, a source-load interval prediction model based on a shared weight long short-term memory network and a stacking integrated model was built; in order to select a suitable base learner, the Q statistic was used to evaluate the effect of the base learner; Figure 5 The figure shows the Q statistic result of the base learner of the present invention; Table 1 is a comparison table of the deterministic prediction results of the base learner of the present invention; GBDT, ET, RF and DTR are selected as the base learner models of the present invention according to the above indicators.

[0176] Table 1 Comparison of prediction effects of base learners

[0177]

[0178] (4) Figure 6 This is a comparison chart of ensemble learning deterministic prediction provided by the embodiment of the invention. From the results, it can be seen that the ensemble learning prediction performance is far better than other models. Figure 7 is a comparison chart of multi-model fusion interval prediction provided by the embodiment of the present invention, and Table 2 is a comparison of interval prediction. In summary, the SWLSTM-Stacking fusion model can achieve accurate interval prediction. From the results, the prediction interval has a smaller PINAW and a larger CP value, indicating that the prediction interval can include most of the observation points in the smallest possible interval width. Therefore, the method proposed by the present invention is superior to the traditional artificial intelligence method.

[0179] Table 2 Comparison of interval predictions

[0180]

[0181] The present invention provides a source load interval prediction method based on a shared weight long short-term memory network combined with a Stacking integrated model. Through a feature day selection scheme combining a local linear embedding with a k-means (LLE-k-means) clustering algorithm, the establishment of source load prediction models under different time scales and scenarios can be achieved; SWLSTM-Stacking can significantly enhance the interpretability of the model, and has higher accuracy and less network training time.

[0182] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0183] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A source-load interval prediction method based on SWLSTM combined with ensemble learning, characterized in that: The following steps are involved: Step S1, obtaining an original data set, performing dimensionality reduction processing on the original data set based on local linear embedding, and clustering the reduced-dimensional data set based on the k-means algorithm to obtain a feature day data set; Step S2: training each base learner based on the characteristic day data set, calculating the Q statistics between the results of each base learner, and selecting the base learner pair with the smallest Q statistics as the base learner model; Step S3: Use the base learner model as the first-layer prediction model of the integrated learning framework, use the SWLSTM neural network as the second-layer prediction model, and introduce the confidence interval to obtain prediction uncertainty information; Step S4: Based on the prediction uncertainty information predicted in step S3, the prediction results are evaluated using a point prediction evaluation index and an interval prediction index.

2. According to claim 1, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The step S1 specifically includes: Step S11, obtain the original data set, pre-process the original data set, use the 3σ criterion to eliminate abnormal data, and use the root mean square method to fill the data: Where N is the number of data samples, j i is the data point at the i-th moment; if |j i -μ|>3σ, then judge j i Abnormal, remove it from the data set; Step S12: Reduce the dimension of the preprocessed data set. i , first select l sample centers and construct the neighborhood matrix W, using W ib Represents data point j i With neighboring point j b The weight between Reconstruct the point by combining the neighboring points of the random combination, use the weight matrix W and the local linear relationship to minimize the error, recalculate the weight matrix M, and use M ib Represents data point j i By the neighborhood point j b Weights of linear representation; Construct a low-dimensional matrix Y where each row of y i is the corresponding data point j i The low-dimensional representation of the matrix Y is optimized by minimizing the following objective function: Convert the M matrix into a positive definite matrix and take the eigenvector corresponding to the first d and the smallest non-zero eigenvalue as the data point j i The value of , where d is the dimension of the target low-dimensional space; Step S13: randomly select k points as initial cluster centers, expressed as C = {c1, c2, ..., c k }, calculate for each data point j i The distance d(c k ,j i ), the data point j i Assigned to the cluster C with the closest cluster center k In the above example, we calculate each cluster C separately. k The mean value of all data in k , and the mean m k As the new cluster center, repeat the above steps until the change of the cluster center is less than the threshold or reaches the preset number of iterations. Step S14: Obtain a characteristic day data set.

3. According to claim 1, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The step S2 specifically includes: Step S21, using the characteristic day data set as a training set, pre-training the base learner model, the base learner model includes: random forest RF, gradient boosted decision tree GBDT, support vector machine SVM, decision tree DTR, gradient boosted regression GBR, extreme learning tree ET and adaptive boosting algorithm ADA; Step S22: Calculate the correlation between the base learner models based on the Q statistic, and select the base learner pair with the smallest Q statistic as the base learner model.

4. According to claim 3, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The training process of the random forest RF and gradient boosted decision tree GBDT model is: Random Forest RF model training: Let N l is the number of data in the training data set, and the number of input features n is randomly selected from the training data set with replacement l , the total number is N l times, as the training set of the decision tree, where n l Much smaller than N l ; Then select q features from all the features as candidate features for decision tree division; Select specific features respectively and generate decision trees recursively; repeat the above steps to build multiple decision trees; The gradient boosting decision tree GBDT training method is: The mean of the target variable is used as the initial predicted value of the target variable, and the difference between each sample and the observed value, i.e., the residual, is calculated. Then, the residual is used as the target variable to generate a regression tree. The goal of this decision tree is to fit the current residual as much as possible. Then, the newly generated decision tree is weighted and accumulated to the existing model, and the predicted value is updated. Among them, weighting is to use a smaller learning rate to control the contribution of each model; Repeat the above steps to iterate and build more decision trees. Each decision tree corrects the deviation of model prediction by fitting the residual of the previous round of iteration. Finally, all the generated decision trees are weighted and added to the initial prediction value to form the final integrated model.

5. According to claim 3, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The support vector machine SVM model training method is: Construct training data set {(j i ,y i ),i=1,2,…,N},j i ,y i are training data and output prediction data respectively, N is the number of data samples, so the mathematical model of SVM is expressed as: y i =f(j i )=w·φ(j i )+b Among them, w is the normal vector, b is the displacement term, φ(j i ) is j i In order to obtain w, b, the following equation is satisfied: Among them, C is the regularization coefficient, that is, the slack variable, and its dual problem is: Among them, φ(j i ) and φ(j k ) are j i and j k The corresponding mapping value; Introduce kernel function to reduce the difficulty of calculation: k=(j i ,j k )=<φ(j i ),φ(j k )>=φ(j i ) T φ(j k )。 6. According to claim 3, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The decision tree DTR and the gradient boosting regression GBR training process are: Decision tree DTR training: Select a feature as the partitioning condition based on the original data set. For the selected feature, set a threshold or condition to divide the data set into two subsets and minimize the variance of the target variable in each subset. Repeat the above steps on the subsets until the preset stop criteria are met. Set a prediction value on each leaf node. For new sample points, traverse the tree nodes step by step from the root node according to the partitioning condition to find the corresponding leaf node, and return the prediction value of the leaf node as the output. Gradient boosting regression GBR training: First, the mean of the target variable is used as the initial prediction value of the target variable, the residual of the current model is calculated, and the residual is used as the target variable to train a new weak regressor. Secondly, the weight of the model is determined according to the prediction results and learning rate of the new model to minimize the overall loss function; the prediction results are updated, and finally the above steps are repeated until the loss function converges or reaches the predetermined number of times. The prediction value function is: Among them, y i is the predicted value of the i-th sample, M is the number of weak regressors, h(j i ,θ i ) is the prediction result of the jth weak regressor, x i is the feature vector of the i-th sample, θ i is the parameter of the jth weak regression model; w j is the weight of the jth weak regression model, and C is a constant term.

7. According to claim 3, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The extreme learning tree ET training process is: Perform feature selection. For each decision tree node, randomly select a subset from the complete set of input features. For each feature, randomly select a partition threshold, and calculate the information gain and Gini coefficient after partitioning the feature using the threshold. Implement node partitioning, use the above selected features and thresholds to partition the data set into smaller subsets, calculate the purity or impurity metric for each subset, and select the partition that can minimize the purity or impurity as the best partition; Construct a decision tree model and repeat the above steps until the stopping condition is met, such as reaching the maximum tree depth or the number of samples in the node is less than the set threshold. Each decision tree is a randomized decision tree. The randomness of feature selection and node division makes each tree have a different structure. In the process of achieving the prediction result, for regression problems, the prediction results of each decision tree are weighted averaged to obtain the final regression result. For classification problems, voting is used to perform majority voting based on the classification results of each decision tree to obtain the final classification result.

8. According to claim 3, a source-load interval prediction method based on SWLSTM combined with ensemble learning is characterized in that: The training process of the adaptive boosting algorithm ADA is as follows: Initialize the sample, set the weight of each sample data to an equal value, select a weak classifier as the base classifier, iterate the training, and for each round of iteration, train a weak classifier based on the current training data weight, calculate the classification accuracy of the weak classifier, and use it as the weight of the weak classifier; Update the weight of the training data. According to the classification error rate, increase the weight of the wrongly classified samples and reduce the weight of the correctly classified samples. Repeat multiple rounds of iterations until the preset number of iterations is reached or the classification accuracy meets the requirements. For integrated prediction, for regression problems, the prediction results of each weak classifier are weighted averaged to obtain the final regression result. For classification problems, the classification results of each weak classifier are weighted voted to obtain the final classification result.

9. The source-load interval prediction method based on SWLSTM combined with ensemble learning according to claim 1 is characterized in that: The step S3 specifically includes: Step S31, calculate the shared gate and information status: Among them, W h is the weight corresponding to the hidden layer output at the previous moment, W x is the weight corresponding to the current input, b is the weight bias term, x t is the input value at the current moment, h t-1 is the hidden state of the previous moment, σ is the sigmoid activation function; Step S32: Update cell status: C t =f t *C t-1 +(1-f t )*a t Step S33, calculate the hidden layer output: h t =f t *tanC t Step S34: output of prediction results: y t =σ(z t )=σ(W y ·h t +b y ) Step S35: construct optimization target: Among them, E t is the prediction error at time t, Y t is the observed value at time t; Step S36: Update the SWLSTM model based on Adam, specifically: Among them, p i and v i are the first-order moment estimation and the second-order moment estimation of the parameters, respectively. and are the modified first-order moment estimate and second-order moment estimate respectively, η is the learning rate, W i is the weight matrix, representing [W h ,W x ,b] or [W y ,b y ]; Step S37, calculate the prediction result through forward propagation, and then update the weight through back propagation, which is one round of update; a total of m rounds of iterations are required, and each round divides the training set into n batches for training, and each batch completes one update; Step S38: Obtain prediction uncertainty information based on the confidence interval.

10. The source-load interval prediction method based on SWLSTM combined with ensemble learning according to claim 1, characterized in that: The step S4 specifically includes: Step S41: Use point prediction evaluation index and interval prediction index to evaluate the prediction results: in, is the predicted value, f i is the actual value, is the average value, SS res SS tol are the residual sum of squares and the grand mean, respectively; Step S42: Evaluate the effectiveness of the source-load error model, using the normalized average width of the prediction interval and the prediction interval coverage as evaluation indicators of the confidence interval of the error model: Among them, N' is the number of error samples, L is the prediction error value, and f upper 、f lower They are respectively the upper and lower bounds of the interval sampled at the corresponding moment of the error prediction model.

Images