Power supply power prediction method based on bidirectional GRU combined with attention mechanism

Abstract

A power supply power prediction method based on a bidirectional GRU combined with an attention mechanism, which collects real-time data, converts text data in the real-time data into a numerical vector through one-hot encoding, calculates the weight of each data feature in the numerical vector through a feature weight mechanism, measures the importance of the feature by combining the weight and the numerical vector, extracts key features and cleanses noise data in a large amount of heterogeneous data, improves the accuracy of subsequent modeling, realizes multidimensional extraction of important features in the input data through a multi-layer attention mechanism, enhances the model's ability to capture the context connection of time series related data, and adopts a bidirectional GRU network based on a coding and decoding structure to realize the prediction of distributed power supply power. The application can not only measure the importance of features by combining weights and numerical vectors, but also realize the prediction of distributed power supply power through a bidirectional GRU network combined with an attention mechanism.

Description

Technical Field

[0001] This invention relates to a power supply prediction method, and more particularly to a power supply prediction method based on a bidirectional GRU power supply that incorporates an attention mechanism. Background Technology

[0002] Distributed energy resources offer advantages such as high energy efficiency, environmental friendliness, and grid resilience, making them a crucial component of future power systems. However, distributed power sources heavily rely on renewable energy sources (such as solar and wind power), and their generation capacity exhibits uncontrollable characteristics such as intermittency and volatility, leading to unstable power quality. As modern society becomes increasingly reliant on electricity supply, power outages or power quality fluctuations can severely impact critical sectors such as industrial production and data centers; therefore, the demand for stable and reliable power quality in distributed energy systems is growing.

[0003] To address this challenge, power quality forecasting technology, as one of the effective means to ensure power quality stability, achieves efficient prediction of future power quality by analyzing the fluctuation characteristics of distributed energy resources. Based on the prediction results, it rationally allocates power resources and improves power quality stability by reducing the imbalance between power supply and demand. Therefore, researching efficient power quality forecasting methods is not only an important way to improve the operational stability of distributed energy systems, but also a necessary means to ensure the normal operation of critical infrastructure and reduce economic losses caused by power outages.

[0004] Existing technologies typically involve optimizing and filtering multi-dimensional time-series data to obtain the correlation between various feature parameter sets and wind power; based on the feature correlation, the optimal feature parameter set is selected as the model feature input to solve the linear model, thereby predicting the power generation.

[0005] Although this prediction method can improve prediction accuracy by optimizing and filtering multidimensional input parameters, it still has the following drawbacks:

[0006] 1. Power quality is affected by a variety of factors. These data are not only diverse in dimensions and complex in source, but also have different correlations and time series among different types of data. Existing technologies often cannot handle the heterogeneity of data, resulting in low model accuracy.

[0007] 2. The operating status of a power system is affected by many factors such as load changes, equipment failures, and the external environment, exhibiting obvious dynamic characteristics. However, fluctuations in power quality are often not linear, and simple linear prediction models are unable to capture the complex behavior of the system, resulting in low prediction accuracy.

[0008] The information disclosed in this background section is intended only to enhance understanding of the overall background of this application and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of existing technologies, such as the inability to handle data heterogeneity leading to low model accuracy, and the difficulty of linear prediction models in capturing the complex behavior of the system, resulting in low prediction accuracy. This invention provides a bidirectional GRU power supply prediction method that can handle data heterogeneity to improve model accuracy, and capture the complex behavior of the system through a multi-layer attention mechanism to achieve higher prediction accuracy.

[0010] To achieve the above objectives, the technical solution of the present invention is:

[0011] A bidirectional GRU-based power supply prediction method incorporating an attention mechanism, the prediction method comprising:

[0012] S1. Collect real-time data, convert the text data in the real-time data into numerical vectors through one-hot encoding, calculate the weight of each data feature in the numerical vector through the feature weight mechanism, and combine the weights with the numerical vector to calculate the weighted numerical vector.

[0013] S2. The important features in the weighted numerical vector are extracted by using a multi-layer attention mechanism at two different time scales, namely days and hours, to obtain the data features extracted by the multi-layer attention mechanism.

[0014] S3. Based on the data features extracted by the multi-layer attention mechanism and the numerical data in the real-time data, a bidirectional GRU network based on the encoding and decoding structure is used to predict the power supply of the distributed power source.

[0015] In step S1, the real-time data includes numerical data and text data. The text data includes historical environmental data and estimated future environmental data, which include meteorological conditions, time-related characteristics, and power company discount plans. One-hot encoding is used to convert the text data into a numerical vector matrix, including:

[0016] ;

[0017] In the above formula, the numerical vector matrix consists of M rows, and each row contains N features. for The rth line;

[0018] The feature weighting mechanism comprises two fully connected layers. The first layer of the neural network uses an exponential linear unit as the activation function to handle the negative part of the input vector, as follows:

[0019] ;

[0020] ;

[0021] In the above formula, This is the output of the first layer of the neural network using the feature weighting mechanism. The weights of the first layer of the neural network are the feature weighting mechanism. This is the proportionality coefficient. The bias of the first layer of the neural network for the feature weighting mechanism;

[0022] The output of the activation function The input is fed into the second layer of the feature weighting mechanism neural network, and the softmax function is used to calculate the weight of each element in the input X, expressed as:

[0023] ;

[0024] In the above formula, Let j be the weight of the j-th element. The output of the second layer of the neural network for the feature weighting mechanism The j-th element, and These represent the weights and biases of the second layer of the neural network in the feature weighting mechanism, respectively.

[0025] Weight vector and input features Perform a dot product to obtain a weighted numerical vector. , represented as ,in This represents the matrix dot product operation; It includes historical environmental data from the previous T days and estimated environmental data for the next day, with each day including... Hours.

[0026] In S2, For the historical data of day t, let Indicates the period on day t Hourly historical environmental data, Indicates the next day Estimated values ​​of hourly environmental data;

[0027] Calculate historical data for day t With future data The Pearson correlation coefficient and the correlation coefficient quantifying the data characteristics between two days are expressed as follows:

[0028] ;

[0029] In the above formula, and They represent and The element in the i-th row and j-th column, and They represent and The mean of the elements in the j-th column;

[0030] Based on the correlation coefficient, the softmax function is further used to calculate the feature weights of the historical data on day t, expressed as:

[0031] ;

[0032] Merging historical data and future environmental data for the same time period, and representing it as... Then, it is fed into a two-layer fully connected network with a multi-layer attention mechanism to obtain the correlation coefficient, where the first layer uses ELU as the activation function and the second layer uses softmax as the activation function, as shown below:

[0033] ;

[0034] In the above formula, Let be the attention coefficient at hour h on day t and the correlation coefficient with the attention coefficient at hour h in the future. The output of the second layer of the multi-layer attention mechanism The Middle There are elements, among which and These are the network weights in the first and second layers of the multi-layer attention mechanism, respectively. and These represent the biases in the first and second layers of the multi-layer attention mechanism, respectively. This is data from the previous T days. For a day Data for one hour.

[0035] In step S3, the numerical data includes historical power supply data. A bidirectional GRU network based on an encoding / decoding structure is used to capture the complex relationship between historical and future data. The bidirectional GRU network consists of two GRUs. The GRUs are used to capture the dependencies between contextual information in the input sequence through a gating mechanism. Each GRU includes an update gate and a reset gate. The update gate is used to determine how much information from the previous hidden state should be retained in the current hidden state, expressed as:

[0036] ;

[0037] In the above formula, A larger value indicates that more state information from the previous time step is retained in the current state. For the sigmoid function, For the input of the GRU module, The hidden state for time step u-1 of the output sequence contains information about the time step, where... and These are the update gate network weights and biases, respectively. The weight matrix from the hidden state to the gate;

[0038] The reset gate is used to control the influence of the previous hidden state when calculating the candidate hidden state, and is expressed as follows:

[0039] ;

[0040] In the above formula, For resetting the network output, and These represent resetting the gate network weights and biases, respectively. The weight matrix from the hidden state to the re-encoder gate;

[0041] Based on the reset gate, the candidate states of GRU The calculation formula includes:

[0042] ;

[0043] In the above formula, This represents the network weights after the refactoring gate adjustment. Let be the weight matrix from the hidden state to the candidate hidden state after the re-gate adjustment. The hyperbolic tangent activation function is used to compute candidate hidden states. Combined with the update gate, the hidden state is determined. It can be represented as:

[0044] ;

[0045] The input sequence of the bidirectional GRU network is simultaneously input into two GRU networks in both forward and reverse directions to compute the hidden state, including:

[0046] ;

[0047] ;

[0048] In the above formula, and These represent the forward hidden state and the reverse hidden state, respectively. The forward hidden state and the reverse hidden state are then integrated to obtain the final hidden state. ;

[0049] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. Using a fully connected layer with the Softplus activation function, hidden information is transformed into future information. Hourly power supply prediction , is represented as:

[0050] ;

[0051] In the above formula, and These represent the network weights and biases in the fully connected layer, respectively.

[0052] In S3, the bidirectional GRU network based on the encoding and decoding structure includes an Encoder network and a Decoder network, both of which are composed of a bidirectional GRU network;

[0053] The input to the Encoder network is a combination of historical environmental data and historical power supply data. ,Will The input sequence is fed into the Encoder network, which encodes the input sequence into a fixed-length context vector. The calculation process can be represented as follows:

[0054] ;

[0055] In the above formula, The hidden state of the input sequence at time step u. Represents the input sequence The uth line in the middle, For encoding functions;

[0056] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. By combining the data features extracted from the multi-layer attention mechanism in S2 and and intermediate semantic vectors Obtaining hierarchical attention vectors ,in This represents the historical hidden state of the Encoder network at hour h on day t;

[0057] hierarchical attention vector With future environmental data As input to the Decoder network to obtain the output sequence, including:

[0058] ;

[0059] In the above formula, This represents the hidden state at time step u-1 of the output sequence. This represents the decoding function.

[0060] A power supply prediction system based on bidirectional GRU with attention mechanism is provided. The system is used to execute the power supply prediction method based on bidirectional GRU with attention mechanism as described above, and specifically includes: a data processing module, a multi-layer attention module and a prediction module.

[0061] The data processing module is used to collect real-time data, convert text data in real-time data into numerical vectors through one-hot encoding, calculate the weight of each data feature in the numerical vector through a feature weighting mechanism, and combine the weights with the numerical vector to calculate a weighted numerical vector.

[0062] The multi-layer attention module is used to extract important features from the weighted numerical vector at two different time scales, day and hour, through a multi-layer attention mechanism, to obtain the data features extracted by the multi-layer attention mechanism.

[0063] The prediction module is used to predict the power supply of distributed power sources by using data features extracted based on a multi-layer attention mechanism and numerical data in real-time data, and adopts a bidirectional GRU network based on an encoding and decoding structure.

[0064] In the data processing module, the real-time data includes numerical data and text data. The text data includes historical environmental data and estimated future environmental data, which include meteorological conditions, time-related characteristics, and power company discount plans. One-hot encoding is used to convert the text data into a numerical vector matrix, including:

[0065] ;

[0066] In the above formula, the numerical vector matrix consists of M rows, and each row contains N features. for The rth line;

[0067] The feature weighting mechanism comprises two fully connected layers. The first layer of the neural network uses an exponential linear unit as the activation function to handle the negative part of the input vector, as follows:

[0068] ;

[0069] ;

[0070] In the above formula, This is the output of the first layer of the neural network using the feature weighting mechanism. The weights of the first layer of the neural network are the feature weighting mechanism. This is the proportionality coefficient. The bias of the first layer of the neural network for the feature weighting mechanism;

[0071] The output of the activation function The input is fed into the second layer of the feature weighting mechanism neural network, and the softmax function is used to calculate the weight of each element in the input X, expressed as:

[0072] ;

[0073] In the above formula, Let j be the weight of the j-th element. The output of the second layer of the neural network for the feature weighting mechanism The j-th element, and These represent the weights and biases of the second layer of the neural network in the feature weighting mechanism, respectively.

[0074] Weight vector and input features Perform a dot product to obtain a weighted numerical vector. , represented as ,in This represents the matrix dot product operation; It includes historical environmental data from the previous T days and estimated environmental data for the next day, with each day including... Hours;

[0075] In the multi-layer attention module For the historical data of day t, let Indicates the period on day t Hourly historical environmental data, Indicates the next day Estimated values ​​of hourly environmental data;

[0076] Calculate historical data for day t With future data The Pearson correlation coefficient and the correlation coefficient quantifying the data characteristics between two days are expressed as follows:

[0077] ;

[0078] In the above formula, and They represent and The element in the i-th row and j-th column, and They represent and The mean of the elements in the j-th column;

[0079] Based on the correlation coefficient, the softmax function is further used to calculate the feature weights of the historical data on day t, expressed as:

[0080] ;

[0081] Merging historical data and future environmental data for the same time period, and representing it as... Then, it is fed into a two-layer fully connected network with a multi-layer attention mechanism to obtain the correlation coefficient, where the first layer uses ELU as the activation function and the second layer uses softmax as the activation function, as shown below:

[0082] ;

[0083] In the above formula, Let be the attention coefficient at hour h on day t and the correlation coefficient with the attention coefficient at hour h in the future. The output of the second layer of the multi-layer attention mechanism The Middle There are elements, among which and These are the network weights in the first and second layers of the multi-layer attention mechanism, respectively. and These represent the biases in the first and second layers of the multi-layer attention mechanism, respectively. This is data from the previous T days. Data for Th hours contained in a day;

[0084] In the prediction module, the numerical data includes historical power supply data. A bidirectional GRU network based on an encoding / decoding structure is used to capture the complex relationship between historical and future data. The bidirectional GRU network consists of two GRUs. The GRUs are used to capture the dependencies between contextual information in the input sequence through a gating mechanism. Each GRU includes an update gate and a reset gate. The update gate is used to determine how much information from the previous hidden state should be retained in the current hidden state, expressed as:

[0085] ;

[0086] In the above formula, A larger value indicates that more state information from the previous time step is retained in the current state. For the sigmoid function, For the input of the GRU module, The hidden state for time step u-1 of the output sequence contains information about the time step, where... and These are the update gate network weights and biases, respectively. The weight matrix from the hidden state to the gate;

[0087] The reset gate is used to control the influence of the previous hidden state when calculating the candidate hidden state, and is expressed as follows:

[0088] ;

[0089] In the above formula, For resetting the network output, and These represent resetting the gate network weights and biases, respectively. The weight matrix from the hidden state to the re-encoder gate;

[0090] Based on the reset gate, the candidate states of GRU The calculation formula includes:

[0091] ;

[0092] In the above formula, This represents the network weights after the refactoring gate adjustment. Let be the weight matrix from the hidden state to the candidate hidden state after the re-gate adjustment. The hyperbolic tangent activation function is used to compute candidate hidden states. Combined with the update gate, the hidden state is determined. It can be represented as:

[0093] ;

[0094] The input sequence of the bidirectional GRU network is simultaneously input into two GRU networks in both forward and reverse directions to compute the hidden state, including:

[0095] ;

[0096] ;

[0097] In the above formula, and These represent the forward hidden state and the reverse hidden state, respectively. The forward hidden state and the reverse hidden state are then integrated to obtain the final hidden state. ;

[0098] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. Using a fully connected layer with the Softplus activation function, hidden information is transformed into future information. Hourly power supply prediction , is represented as:

[0099] ;

[0100] In the above formula, and These represent the network weights and biases in the fully connected layer, respectively.

[0101] The predicted module includes an Encoder network and a Decoder network in the bidirectional GRU network based on the encoding and decoding structure. Both the Encoder network and the Decoder network are composed of bidirectional GRU networks.

[0102] The input to the Encoder network is a combination of historical environmental data and historical power supply data. ,Will The input sequence is fed into the Encoder network, which encodes the input sequence into a fixed-length context vector. The calculation process can be represented as follows:

[0103] ;

[0104] In the above formula, The hidden state of the input sequence at time step u. Represents the input sequence The uth line in the middle, For encoding functions;

[0105] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. By combining data features extracted from the multi-layer attention mechanism in the multi-layer attention module and and intermediate semantic vectors Obtaining hierarchical attention vectors ,in This represents the historical hidden state of the Encoder network at hour h on day t;

[0106] hierarchical attention vector With future environmental data As input to the Decoder network to obtain the output sequence, including:

[0107] ;

[0108] In the above formula, This represents the hidden state at time step u-1 of the output sequence. This represents the decoding function.

[0109] A power prediction device based on bidirectional GRU that incorporates an attention mechanism, characterized in that it includes a memory and a processor, wherein the memory is used to store computer program code and transmit the computer program code to the processor;

[0110] The processor is configured to execute the aforementioned bidirectional GRU power supply prediction method incorporating an attention mechanism, according to instructions in the computer program code.

[0111] A computer program product includes a computer program, characterized in that the computer program is executed by a processor of the aforementioned bidirectional GRU power supply prediction method incorporating an attention mechanism.

[0112] A computer-storable medium storing a computer program, characterized in that the computer program is executed by a processor using the aforementioned bidirectional GRU power supply prediction method incorporating an attention mechanism.

[0113] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0114] 1. In this invention, a power prediction method based on a bidirectional GRU power supply incorporating an attention mechanism, after collecting real-time data, the text data in the real-time data is converted into numerical vectors through one-hot encoding. Then, a feature weighting mechanism is used to calculate the weight of each data feature in the numerical vector. By combining the weights and numerical vectors, the importance of the features is measured. This method extracts key features from large amounts of heterogeneous data, cleans up noisy data, and improves the accuracy of subsequent modeling, thereby enhancing model accuracy. Therefore, this design can effectively improve model accuracy by combining weights and numerical vectors to measure feature importance.

[0115] 2. In this invention, a bidirectional GRU-based power supply prediction method incorporating an attention mechanism is proposed. This method utilizes a multi-layer attention mechanism to extract important features from the input data from multiple dimensions, enhancing the model's ability to capture the contextual relationships of time-series related data. Furthermore, a bidirectional GRU network based on an encoding / decoding structure is employed to predict the power supply of distributed power sources. Therefore, this design can effectively improve prediction accuracy by using a bidirectional GRU network incorporating an attention mechanism to predict the power supply of distributed power sources. Attached Figure Description

[0116] Figure 1 This is a flowchart of the method described in this invention.

[0117] Figure 2 This is a schematic diagram of the planning concept of the method described in this invention.

[0118] Figure 3 This is a diagram showing the predicted power supply effect of the method and the comparative scheme described in Example 3 of the present invention.

[0119] Figure 4 This is a structural diagram of the system described in this invention.

[0120] Figure 5 This is a structural diagram of the device described in this invention. Detailed Implementation

[0121] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0122] Example 1:

[0123] See Figures 1 to 2 A power supply prediction method based on bidirectional GRUs that incorporates an attention mechanism, the prediction method comprising:

[0124] S1. Collect real-time data, convert the text data in the real-time data into numerical vectors through one-hot encoding, calculate the weight of each data feature in the numerical vector through the feature weight mechanism, and combine the weights with the numerical vector to calculate the weighted numerical vector.

[0125] S2. The important features in the weighted numerical vector are extracted by using a multi-layer attention mechanism at two different time scales, namely days and hours, to obtain the data features extracted by the multi-layer attention mechanism.

[0126] S3. Based on the data features extracted by the multi-layer attention mechanism and the numerical data in the real-time data, a bidirectional GRU network based on the encoding and decoding structure is used to predict the power supply of the distributed power source.

[0127] In step S1, the real-time data includes numerical data and text data. The text data includes historical environmental data and estimated future environmental data, which include meteorological conditions, time-related characteristics, and power company discount plans. One-hot encoding is used to convert the text data into a numerical vector matrix, including:

[0128] ;

[0129] In the above formula, the numerical vector matrix consists of M rows, and each row contains N features. for The rth line;

[0130] The feature weighting mechanism comprises two fully connected layers. The first layer of the neural network uses an exponential linear unit as the activation function to handle the negative part of the input vector, as follows:

[0131] ;

[0132] ;

[0133] In the above formula, This is the output of the first layer of the neural network using the feature weighting mechanism. The weights of the first layer of the neural network are the feature weighting mechanism. This is the proportionality coefficient. The bias of the first layer of the neural network for the feature weighting mechanism;

[0134] The output of the activation function The input is fed into the second layer of the feature weighting mechanism neural network, and the softmax function is used to calculate the weight of each element in the input X, expressed as:

[0135] ;

[0136] In the above formula, Let j be the weight of the j-th element. The output of the second layer of the neural network for the feature weighting mechanism The j-th element, and These represent the weights and biases of the second layer of the neural network in the feature weighting mechanism, respectively.

[0137] Weight vector and input features Perform a dot product to obtain a weighted numerical vector. , represented as ,in This represents the matrix dot product operation; It includes historical environmental data from the previous T days and estimated environmental data for the next day, with each day including... Hours.

[0138] In S2, For the historical data of day t, let Indicates the period on day t Hourly historical environmental data, Indicates the next day Estimated values ​​of hourly environmental data;

[0139] Calculate historical data for day t With future data The Pearson correlation coefficient and the correlation coefficient quantifying the data characteristics between two days are expressed as follows:

[0140] ;

[0141] In the above formula, and They represent and The element in the i-th row and j-th column, and They represent and The mean of the elements in the j-th column;

[0142] Based on the correlation coefficient, the softmax function is further used to calculate the feature weights of the historical data on day t, expressed as:

[0143] ;

[0144] Merging historical data and future environmental data for the same time period, and representing it as... Then, it is fed into a two-layer fully connected network with a multi-layer attention mechanism to obtain the correlation coefficient, where the first layer uses ELU as the activation function and the second layer uses softmax as the activation function, as shown below:

[0145] ;

[0146] In the above formula, Let be the attention coefficient at hour h on day t and the correlation coefficient with the attention coefficient at hour h in the future. The output of the second layer of the multi-layer attention mechanism The Middle There are elements, among which and These are the network weights in the first and second layers of the multi-layer attention mechanism, respectively. and These represent the biases in the first and second layers of the multi-layer attention mechanism, respectively. This is data from the previous T days. For a day Data for one hour.

[0147] In step S3, the numerical data includes historical power supply data. A bidirectional GRU network based on an encoding / decoding structure is used to capture the complex relationship between historical and future data. The bidirectional GRU network consists of two GRUs. The GRUs are used to capture the dependencies between contextual information in the input sequence through a gating mechanism. Each GRU includes an update gate and a reset gate. The update gate is used to determine how much information from the previous hidden state should be retained in the current hidden state, expressed as:

[0148] ;

[0149] In the above formula, A larger value indicates that more state information from the previous time step is retained in the current state. For the sigmoid function, For the input of the GRU module, The hidden state for time step u-1 of the output sequence contains information about the time step, where... and These are the update gate network weights and biases, respectively. The weight matrix from the hidden state to the gate;

[0150] The reset gate is used to control the influence of the previous hidden state when calculating the candidate hidden state, and is expressed as follows:

[0151] ;

[0152] In the above formula, For resetting the network output, and These represent resetting the gate network weights and biases, respectively. The weight matrix from the hidden state to the re-encoder gate;

[0153] Based on the reset gate, the candidate states of GRU The calculation formula includes:

[0154] ;

[0155] In the above formula, This represents the network weights after the refactoring gate adjustment. Let be the weight matrix from the hidden state to the candidate hidden state after the re-gate adjustment. The hyperbolic tangent activation function is used to compute candidate hidden states. Combined with the update gate, the hidden state is determined. It can be represented as:

[0156] ;

[0157] The input sequence of the bidirectional GRU network is simultaneously input into two GRU networks in both forward and reverse directions to compute the hidden state, including:

[0158] ;

[0159] ;

[0160] In the above formula, and These represent the forward hidden state and the reverse hidden state, respectively. The forward hidden state and the reverse hidden state are then integrated to obtain the final hidden state. ;

[0161] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. Using a fully connected layer with the Softplus activation function, hidden information is transformed into future information. Hourly power supply prediction , is represented as:

[0162] ;

[0163] In the above formula, and These represent the network weights and biases in the fully connected layer, respectively.

[0164] In S3, the bidirectional GRU network based on the encoding and decoding structure includes an Encoder network and a Decoder network, both of which are composed of a bidirectional GRU network;

[0165] The input to the Encoder network is a combination of historical environmental data and historical power supply data. ,Will The input sequence is fed into the Encoder network, which encodes the input sequence into a fixed-length context vector. The calculation process can be represented as follows:

[0166] ;

[0167] In the above formula, The hidden state of the input sequence at time step u. Represents the input sequence The uth line in the middle, For encoding functions;

[0168] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. By combining the data features extracted from the multi-layer attention mechanism in S2 and and intermediate semantic vectors Obtaining hierarchical attention vectors ,in This represents the historical hidden state of the Encoder network at hour h on day t;

[0169] hierarchical attention vector With future environmental data As input to the Decoder network to obtain the output sequence, including:

[0170] ;

[0171] In the above formula, This represents the hidden state at time step u-1 of the output sequence. This represents the decoding function.

[0172] Example 2:

[0173] A power supply prediction system based on bidirectional GRU with attention mechanism is provided. The system is used to execute the power supply prediction method based on bidirectional GRU with attention mechanism as described in Embodiment 1. Specifically, it includes: a data processing module, a multi-layer attention module, and a prediction module.

[0174] The data processing module is used to collect real-time data, convert text data in real-time data into numerical vectors through one-hot encoding, calculate the weight of each data feature in the numerical vector through a feature weighting mechanism, and combine the weights with the numerical vector to calculate a weighted numerical vector.

[0175] In the data processing module, the real-time data includes numerical data and text data. The text data includes historical environmental data and estimated future environmental data, which include meteorological conditions, time-related characteristics, and power company discount plans. One-hot encoding is used to convert the text data into a numerical vector matrix, including:

[0176] ;

[0177] In the above formula, the numerical vector matrix consists of M rows, and each row contains N features. for The rth line;

[0178] The feature weighting mechanism comprises two fully connected layers. The first layer of the neural network uses an exponential linear unit as the activation function to handle the negative part of the input vector, as follows:

[0179] ;

[0180] ;

[0181] In the above formula, This is the output of the first layer of the neural network using the feature weighting mechanism. The weights of the first layer of the neural network are the feature weighting mechanism. This is the proportionality coefficient. The bias of the first layer of the neural network for the feature weighting mechanism;

[0182] The output of the activation function The input is fed into the second layer of the feature weighting mechanism neural network, and the softmax function is used to calculate the weight of each element in the input X, expressed as:

[0183] ;

[0184] In the above formula, Let j be the weight of the j-th element. The output of the second layer of the neural network for the feature weighting mechanism The j-th element, and These represent the weights and biases of the second layer of the neural network in the feature weighting mechanism, respectively.

[0185] Weight vector and input features Perform a dot product to obtain a weighted numerical vector. , represented as ,in This represents the matrix dot product operation; It includes historical environmental data from the previous T days and estimated environmental data for the next day, with each day including... Hours;

[0186] The multi-layer attention module is used to extract important features from the weighted numerical vector at two different time scales, day and hour, through a multi-layer attention mechanism, to obtain the data features extracted by the multi-layer attention mechanism.

[0187] In the multi-layer attention module For the historical data of day t, let Indicates the period on day t Hourly historical environmental data, Indicates the next day Estimated values ​​of hourly environmental data;

[0188] Calculate historical data for day t With future data The Pearson correlation coefficient and the correlation coefficient quantifying the data characteristics between two days are expressed as follows:

[0189] ;

[0190] In the above formula, and They represent and The element in the i-th row and j-th column, and They represent and The mean of the elements in the j-th column;

[0191] Based on the correlation coefficient, the softmax function is further used to calculate the feature weights of the historical data on day t, expressed as:

[0192] ;

[0193] Merging historical data and future environmental data for the same time period, and representing it as... Then, it is fed into a two-layer fully connected network with a multi-layer attention mechanism to obtain the correlation coefficient, where the first layer uses ELU as the activation function and the second layer uses softmax as the activation function, as shown below:

[0194] ;

[0195] In the above formula, Let be the attention coefficient at hour h on day t and the correlation coefficient with the attention coefficient at hour h in the future. The output of the second layer of the multi-layer attention mechanism The Middle There are elements, among which and These are the network weights in the first and second layers of the multi-layer attention mechanism, respectively. and These represent the biases in the first and second layers of the multi-layer attention mechanism, respectively. This is data from the previous T days. For a day Hourly data;

[0196] The prediction module is used to predict the power supply of distributed power sources by using data features extracted based on a multi-layer attention mechanism and numerical data in real-time data, and adopts a bidirectional GRU network based on an encoding and decoding structure.

[0197] In the prediction module, the numerical data includes historical power supply data. A bidirectional GRU network based on an encoding / decoding structure is used to capture the complex relationship between historical and future data. The bidirectional GRU network consists of two GRUs. The GRUs are used to capture the dependencies between contextual information in the input sequence through a gating mechanism. Each GRU includes an update gate and a reset gate. The update gate is used to determine how much information from the previous hidden state should be retained in the current hidden state, expressed as:

[0198] ;

[0199] In the above formula, A larger value indicates that more state information from the previous time step is retained in the current state. For the sigmoid function, For the input of the GRU module, The hidden state for time step u-1 of the output sequence contains information about the time step, where... and These are the update gate network weights and biases, respectively. The weight matrix from the hidden state to the gate;

[0200] The reset gate is used to control the influence of the previous hidden state when calculating the candidate hidden state, and is expressed as follows:

[0201] ;

[0202] In the above formula, For resetting the network output, and These represent resetting the gate network weights and biases, respectively. The weight matrix from the hidden state to the re-encoder gate;

[0203] Based on the reset gate, the candidate states of GRU The calculation formula includes:

[0204] ;

[0205] In the above formula, This represents the network weights after the refactoring gate adjustment. Let be the weight matrix from the hidden state to the candidate hidden state after the re-gate adjustment. The hyperbolic tangent activation function is used to compute candidate hidden states. Combined with the update gate, the hidden state is determined. It can be represented as:

[0206] ;

[0207] The input sequence of the bidirectional GRU network is simultaneously input into two GRU networks in both forward and reverse directions to compute the hidden state, including:

[0208] ;

[0209] ;

[0210] In the above formula, and These represent the forward hidden state and the reverse hidden state, respectively. The forward hidden state and the reverse hidden state are then integrated to obtain the final hidden state. ;

[0211] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. Using a fully connected layer with the Softplus activation function, hidden information is transformed into future information. Hourly power supply prediction , is represented as:

[0212] ;

[0213] In the above formula, and These represent the network weights and biases in the fully connected layer, respectively.

[0214] The predicted module includes an Encoder network and a Decoder network in the bidirectional GRU network based on the encoding and decoding structure. Both the Encoder network and the Decoder network are composed of bidirectional GRU networks.

[0215] The input to the Encoder network is a combination of historical environmental data and historical power supply data. ,Will The input sequence is fed into the Encoder network, which encodes the input sequence into a fixed-length context vector. The calculation process can be represented as follows:

[0216] ;

[0217] In the above formula, The hidden state of the input sequence at time step u. Represents the input sequence The uth line in the middle, For encoding functions;

[0218] The intermediate semantic matrix is ​​obtained by summing up the hidden states at all time points. By combining data features extracted from the multi-layer attention mechanism in the multi-layer attention module and and intermediate semantic vectors Obtaining hierarchical attention vectors ,in This represents the historical hidden state of the Encoder network at hour h on day t;

[0219] hierarchical attention vector With future environmental data As input to the Decoder network to obtain the output sequence, including:

[0220] ;

[0221] In the above formula, This represents the hidden state at time step u-1 of the output sequence. This represents the decoding function.

[0222] Example 3:

[0223] This embodiment simulates the scheme described in Embodiment 1 and the comparative scheme to illustrate the effect of this design. The comparative scheme is a conventional power supply prediction scheme based on long short-term memory network.

[0224] The simulation data in this embodiment is based on the ISO-NE dataset, which is the data set of the grid operators in the New England region of the northeastern United States, responsible for the reliable operation of the regional power system, the management of the electricity market, and energy planning.

[0225] The comparative scheme directly merges quantified historical environmental data, future environmental data, and historical power supply data as input, and uses a typical Long Short-Term Memory (LSTM) network to predict future power supply. Figure 3 As can be seen, compared with the comparative scheme, the power supply value predicted by this design is closer to the actual value. Therefore, the prediction accuracy of the proposed scheme is better than that of the comparative scheme.

[0226] Example 4:

[0227] See Figure 4 A power prediction device based on bidirectional GRU power supply, incorporating an attention mechanism, is characterized by comprising a memory and a processor, wherein the memory is used to store computer program code and transmit the computer program code to the processor;

[0228] The processor is configured to execute, according to instructions in the computer program code, the bidirectional GRU power supply prediction method incorporating an attention mechanism as described in Embodiment 1.

[0229] See Figure 5 A power prediction device based on bidirectional GRU power supply, incorporating an attention mechanism, is characterized by comprising a memory and a processor, wherein the memory is used to store computer program code and transmit the computer program code to the processor;

[0230] The processor is configured to execute, according to instructions in the computer program code, the bidirectional GRU power supply prediction method incorporating an attention mechanism as described in Embodiment 1.

[0231] A computer program product includes a computer program, characterized in that the computer program is executed by a processor as described in Example 1, which is a bidirectional GRU power supply prediction method incorporating an attention mechanism.

[0232] A computer-storable medium storing a computer program, characterized in that the computer program is executed by a processor as described in Embodiment 1, which is a bidirectional GRU power supply prediction method incorporating an attention mechanism.

[0233] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included within the scope of protection set forth in the claims.

Claims

1. A power supply power prediction method based on bidirectional GRU combined with attention mechanism, characterized in that: The prediction method comprises: S1, collecting real-time data, converting the text data in the real-time data into a numerical vector through one-hot encoding, calculating the weight of each data feature in the numerical vector through a feature weight mechanism, and calculating a weighted numerical vector based on the weight and the numerical vector; S2, extracting important features in the weighted numerical vector through a multi-layer attention mechanism at two different time scales of days and hours, respectively, to obtain data features extracted by the multi-layer attention mechanism; S3, based on the data features extracted by the multi-layer attention mechanism and the numerical data in the real-time data, a bidirectional GRU network based on an encoding-decoding structure is used to realize the prediction of distributed power supply power; The bidirectional GRU network based on the encoding-decoding structure comprises an Encoder network and a Decoder network, both of which are composed of bidirectional GRU networks; The input to the Encoder network is a combination of historical environmental data and historical power supply data. ,Will The input sequence is fed into the Encoder network, which encodes the input sequence into a fixed-length context vector. The calculation process can be represented as follows: ； In the above formulae, is the hidden state for time step u of the input sequence, denotes the input sequence the u-th row in, is an encoding function; The hidden states at all times are summarized to obtain an intermediate semantic matrix , by combining the data features extracted in S2 through the multi-layer attention mechanism and and the intermediate semantic vector obtain a hierarchical attention vector , wherein represents the historical hidden state in the Encoder network about the h hour of the t day; applying a hierarchical attention vector with future environment data as input to the decoder network to obtain an output sequence, comprising: ； In the above formulae, denotes the hidden state of the output sequence at time step u-1, denotes the decoding function. 2.The power supply power prediction method based on bidirectional GRU combined with attention mechanism according to claim 1, characterized in that: In S1, the real-time data includes numerical data and text data, the text data includes historical environmental data and estimated future environmental data, the historical environmental data and the estimated future environmental data include meteorological conditions, time-related features, and power company discount plans, the text data is converted into a numerical vector matrix using one-hot encoding, including: ； In the above formula, the numerical vector matrix includes M rows, each row including N features, For the rth row of The feature weight mechanism includes two fully connected networks, wherein the first layer neural network of the feature weight mechanism uses an exponential linear unit as an activation function to process the negative part of the input vector, denoted as: ； ； In the above formula, is an output of the first layer neural network of the feature weight mechanism, is a weight of the first layer neural network of the feature weight mechanism, is a proportionality coefficient, is a bias of the first layer neural network of the feature weight mechanism; The output of the activation function The input is fed into the second layer of the feature weighting mechanism neural network, and the softmax function is used to calculate the weight of each element in the input X, expressed as: ； In the above formula, is the weight of the jth element, is the output of the second layer neural network of the feature weight mechanism is the jth element in the middle, and respectively represent the weight and bias of the second layer neural network of the feature weight mechanism; The weight vector and the input features are dot multiplied to obtain a weighted numerical vector , denoted as , where denotes a matrix dot multiplication operation; contains the historical environmental data of the previous T days and the estimated values of the environmental data in the next day, where each day includes hours. 3.The power supply power prediction method based on bidirectional GRU combined with attention mechanism according to claim 2, characterized in that: In S2, For the historical data of day t, let Indicates the period on day t Hourly historical environmental data, Indicates the next day An estimated value of hourly environmental data, with each row including N features; Computing historical data for day t Pearson correlation coefficient with future data Pearson correlation coefficient with future data ； In the above formulae, denote denote denote denote the element in the i-th row and j-th column of matrix denote denote denote denote the average of the elements in the j-th column of matrix Based on the correlation coefficient, the feature weight of the historical data of the t-th day is further calculated by combining a softmax function, denoted as: ； The historical data and the data of the same time period in the future environmental data are merged, denoted as , and then input into a two-layer fully connected network of the multi-layer attention mechanism to obtain the correlation coefficient, wherein the first layer adopts ELU as the activation function, and the second layer adopts softmax as the activation function, denoted as: ； In the above formula, is the attention coefficient of the th hour of the t day and the correlation coefficient of the future h hour, is the output of the second layer neural network of the multi-layer attention mechanism th element in the middle wherein and are the network weights in the first and second layer neural networks of the multi-layer attention mechanism, respectively, and are the biases in the first and second layer neural networks of the multi-layer attention mechanism, respectively, is the data of the previous T days, is the data containing hours within a day. 4.The power supply power prediction method based on bidirectional GRU combined with attention mechanism according to claim 3, characterized in that: In S3, the numerical data includes historical power supply data, the bidirectional GRU network based on the encoding-decoding structure is used to capture the complex relationship between the historical data and the future data, the bidirectional GRU network is composed of two GRUs, the GRU is used to capture the dependency relationship between the context information in the input sequence through the gating mechanism, the GRU includes an update gate and a reset gate, the update gate is used to determine how much information from the previous hidden state needs to be retained in the current hidden state, denoted as: ； The greater the value of represents the more the state information of the previous time step is preserved in the current state, is a sigmoid function, is the input of the GRU module, is the hidden state of the time step u-1 of the output sequence, which contains the information of the time step, where and are the update gate network weight and bias, respectively, is the weight matrix of the hidden state to the gate; The reset gate is used to control the influence of the previous hidden state in calculating the candidate hidden state, denoted as: ； In the above formula, to reset the gate network output, and are the reset gate network weights and biases, respectively, is the weight matrix from hidden state to the reset gate. candidate state of a GRU based on a reset gate The computational formula includes: ； In the above equation, denotes the network weights after the reset gate adjustment, is a weight matrix from the hidden state to the candidate hidden state after the reset gate adjustment, is a hyperbolic tangent activation function used to compute the candidate hidden state in conjunction with the update gate, hidden state may be represented as: ； The input sequence of the bidirectional GRU network is simultaneously input into the two GRU networks in a forward and reverse manner to calculate the hidden state, including: ； ； In the above formula, and respectively represent a forward hidden state and a backward hidden state, and the forward hidden state and the backward hidden state are integrated to obtain a final hidden state ; The hidden states of all time are summarized to obtain an intermediate semantic matrix , using a fully connected layer with a Softplus activation function, to convert the hidden information into a future hour power supply power prediction , denoted as: ； In the above formula, and respectively represent the network weights and biases in the fully connected layer.

5. A power supply power prediction system based on bidirectional GRU combined with attention mechanism, characterized in that, The system is used to execute the power supply power prediction method based on bidirectional GRU combined with attention mechanism as claimed in any one of claims 1 to 4, and specifically comprises a data processing module, a multi-layer attention module, and a prediction module. The data processing module is used for collecting real-time data, converting text data in the real-time data into a numerical vector through one-hot encoding, calculating the weight of each data feature in the numerical vector through a feature weight mechanism, and calculating a weighted numerical vector in combination with the weight and the numerical vector; The multi-layer attention module is used for extracting important features in the weighted numerical vector through a multi-layer attention mechanism in two different time scales of days and hours respectively, to obtain data features extracted by the multi-layer attention mechanism; The prediction module is used for predicting the power supply of the distributed power supply based on the data features extracted by the multi-layer attention mechanism and the numerical data in the real-time data, and adopting a bidirectional GRU network based on a coding and decoding structure. In the prediction module, the bidirectional GRU network based on the coding and decoding structure includes an Encoder network and a Decoder network, and both the Encoder network and the Decoder network are composed of a bidirectional GRU network. The input of the Encoder network is the integration of historical environmental data and historical supply data , which is input into the Encoder network, and the Encoder network encodes the input sequence into a fixed-length context vector, and the calculation process can be represented as: ​ ； In the above formulae, is the hidden state for time step u of the input sequence, denotes the input sequence the u-th row in, is an encoding function; The hidden states at all times are summarized to obtain an intermediate semantic matrix data features extracted by combining the multi-layer attention mechanisms in the multi-layer attention module and and the intermediate semantic vector obtaining a hierarchical attention vector wherein represents a historical hidden state in the Encoder network about the h hour of the t day; applying a hierarchical attention vector with future environment data as input to a decoder network to obtain an output sequence, comprising: ； In the above formulae, denotes the hidden state of the output sequence at time step u-1, denotes the decoding function.

6. The power supply power prediction system based on a bidirectional GRU combined with an attention mechanism according to claim 5, characterized in that: In the data processing module, the real-time data includes numerical data and text data, the text data includes historical environmental data and estimated future environmental data, the historical environmental data and the estimated future environmental data include meteorological conditions, time-related features, and power company discount plans, the text data is converted into a numerical vector matrix through one-hot encoding, and the numerical vector matrix includes: ； In the above formula, the numerical vector matrix includes M rows, each row including N features, For the rth row of The feature weight mechanism includes two fully connected networks, wherein the first layer neural network of the feature weight mechanism adopts an exponential linear unit as an activation function to process the negative part of the input vector, and is expressed as: ； ； In the above formula, is an output of the first layer neural network of the feature weight mechanism, is a weight of the first layer neural network of the feature weight mechanism, is a proportionality coefficient, is a bias of the first layer neural network of the feature weight mechanism; The output of the activation function is The second layer neural network is input to the feature weight mechanism, and the weight of each element in the input X is calculated using a softmax function, denoted as: ； In the above formula, is the weight of the jth element, is the output of the second layer neural network of the feature weight mechanism is the jth element in the middle, and respectively represent the weight and bias of the second layer neural network of the feature weight mechanism; The weight vector and the input features are dot multiplied to obtain a weighted numerical vector , denoted as , where denotes a matrix dot multiplication operation; contains the historical environmental data of the previous T days and the estimated values of the environmental data in the next day, where each day includes hours; In the multi-layer attention module, Let denote the historical data in the t-th day, and denote the estimated value of the environmental data in the t-th day, and Computing historical data for day t Pearson correlation coefficient with future data and quantifying the correlation coefficient between the data features of the two days, denoted as: ； In the above formulae, denotes denotes denotes the element in the i-th row and j-th column of denotes denotes denotes the mean of the elements in the j-th column; Based on the correlation coefficient, the feature weight of the historical data of the t-th day is further calculated in combination with a softmax function, and is expressed as: ； The historical data and the data of the same time period in the future environmental data are merged, denoted as , and then input into a two-layer fully connected network of the multi-layer attention mechanism to obtain the correlation coefficient, wherein the first layer adopts ELU as the activation function, and the second layer adopts softmax as the activation function, denoted as: ； In the above formula, is the attention coefficient of the th hour of the t day and the correlation coefficient of the future h hour, is the output of the second layer neural network of the multi-layer attention mechanism is the th element in the wherein and are the network weights in the first and second layer neural networks of the multi-layer attention mechanism, respectively, and are the biases in the first and second layer neural networks of the multi-layer attention mechanism, respectively, is the data of the previous T days, is the data of Th hours contained in a day; In the prediction module, the numerical data includes historical power supply data, the bidirectional GRU network based on the coding and decoding structure is used to capture the complex relationship between the historical data and the future data, the bidirectional GRU network is composed of two GRUs, the GRU is used to capture the dependency relationship between the context information in the input sequence through a gating mechanism, the GRU includes an update gate and a reset gate, the update gate is used to determine how much information from the previous hidden state needs to be retained in the current hidden state, and is expressed as: ； The greater the value of represents the more the state information of the previous time step is preserved into the current state, is a sigmoid function, is the input of the GRU module, is the hidden state of the output sequence at time step u-1, which contains the information of the time step, where and are the update gate network weights and biases, respectively, is the weight matrix of the hidden state to the gate; The reset gate is used to control the influence of the previous hidden state in calculating the candidate hidden state, and is expressed as: ； In the above formula, is the output of the reset gate network, is the output of the reset gate network, are the weights and bias of the reset gate network, respectively, is the weight matrix of the hidden state to the reset gate. candidate state of a GRU based on a reset gate the computational formula includes: ； In the above equation, denotes the network weights after the reset gate adjustment, is the weight matrix from the hidden state to the candidate hidden state after the reset gate adjustment, is the hyperbolic tangent activation function used to compute the candidate hidden state in conjunction with the update gate, hidden state can be represented as: ； The input sequence of the bidirectional GRU network is simultaneously input into the two GRU networks to calculate the hidden state in a forward and reverse manner, and includes: ； ； In the above formula, and respectively represent a forward hidden state and a backward hidden state, and the forward hidden state and the backward hidden state are integrated to obtain a final hidden state ; The hidden states of all time are summarized to obtain an intermediate semantic matrix , a fully connected layer with a Softplus activation function, to convert the hidden information into a power supply power prediction for the next hours , denoted as: ； In the above formula, and respectively represent the network weights and biases in the fully connected layer.

7. A power supply power prediction device based on a bidirectional GRU combined with an attention mechanism, characterized by, The memory is used for storing computer program codes and transmitting the computer program codes to the processor; The processor is used for executing the power supply power prediction method based on a bidirectional GRU combined with an attention mechanism according to any one of claims 1 to 4 according to the instructions in the computer program codes.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to execute the power supply power prediction method based on a bidirectional GRU combined with an attention mechanism according to any one of claims 1 to 4.

9. A computer storable medium having stored therein a computer program, characterized in that, The computer program is executed by a processor to perform the power supply power prediction method based on the bidirectional GRU combined with the attention mechanism as claimed in any one of claims 1 to 4.

Images