Load forecasting method and device based on area profile, and electronic equipment

Abstract

The present invention discloses a load prediction method and device based on substation portrait, and electronic equipment, and relates to the technical field of power load prediction. The method includes: preprocessing the collected substation data, dividing the substation data into multiple substation portraits by using a clustering algorithm, extracting the load data of the time series, inputting the load data into the input layer, and inputting the output sequence of the input layer into the position coding layer for position coding, obtaining the position coding output sequence, inputting the position coding output sequence into the convolution layer and the long short-term memory layer, obtaining feature data, inputting the feature data into the multi-head attention layer, performing a linear transformation on it to obtain a data vector, calculating the periodic position coding of each head, obtaining the multi-head attention layer output vector, inputting the output vector into the fully connected layer, performing a dimensional transformation on it, and obtaining a prediction result. The present invention solves the technical problem that the periodic characteristics of load data cannot be effectively identified in the related art, resulting in reduced prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of electric load forecasting, and in particular to a load forecasting method and device based on substation image, and electronic equipment. Background Art

[0002] The distribution transformer substation is a key link in the power system that connects the high-voltage transmission network and the low-voltage user network. Its function is to convert high-voltage electricity into low-voltage electricity suitable for home and small commercial facilities. In the power system, the substation is not only a node for power distribution, but also a center for data collection and monitoring. It can monitor the power consumption of users in real time and collect a large amount of power consumption data. Power load forecasting is the core part of power system operation and management. It is of great significance in power dispatching, power generation planning, power consumption planning, power demand management and power system planning. By analyzing historical power consumption data and predicting power demand at a certain time in the future, power companies can adjust power generation plans in advance, optimize the allocation of power resources, and avoid imbalances in power supply and demand, which lead to economic losses and environmental damage.

[0003] In the related art, the existing power load forecasting strategies are usually statistical analysis strategies, which include time series analysis strategies and linear regression strategies. The time series analysis strategy makes predictions by analyzing the patterns and trends of historical load data, and the linear regression strategy makes predictions based on the linear relationship between historical loads and influencing factors. However, these strategies are difficult to capture the complex periodic change characteristics in power load data. Power load data is affected by many factors, including weather conditions, working days and holidays, seasonal changes, etc. These factors will affect the periodic change characteristics of power load data on different time scales. However, since the model parameters of the statistical analysis strategy are constant at all time points, it is difficult to accurately capture the long-term and short-term periodic change characteristics of power load data, which makes the existing power load forecasting strategies limited in terms of the accuracy and reliability of power load forecasting.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a load prediction method and device, and electronic equipment based on substation profiles, so as to at least solve the technical problem in the related art that the periodic characteristics of load data cannot be effectively identified, resulting in reduced prediction accuracy.

[0006] According to one aspect of an embodiment of the present invention, a load prediction method based on substation portraits is provided, including: preprocessing the collected substation data, dividing the preprocessed substation data into multiple substation portraits using a clustering algorithm, and extracting the time series load data of each substation portrait; inputting the load data into the input layer of the target neural network, and inputting the output sequence of the input layer into the position coding layer of the target neural network, performing position coding processing in the position coding layer, and obtaining a position coding output sequence; inputting the position coding output sequence into the convolution layer and the long short-term memory layer of the target neural network to obtain feature data, wherein the long short-term memory layer is used to capture the data dependency relationship of the load data; inputting the feature data into the multi-head attention layer of the target neural network, performing linear transformation on the feature data to obtain multiple data vectors, calculating the periodic position coding of each head in the multi-head attention layer, and obtaining the output vector of the multi-head attention layer based on the data vector and the periodic position coding; inputting the output vector into the fully connected layer of the target neural network, performing dimensionality conversion processing on the output vector, and obtaining the prediction result of the load data.

[0007] Optionally, the step of preprocessing the collected substation data includes: obtaining non-numerical data in the substation data, wherein the substation data at least includes: substation capacity, substation load, substation category, weather conditions and geographical location information; digitally processing the non-numerical data, wherein the non-numerical data includes: substation category, weather conditions and geographical location information.

[0008] Optionally, a clustering algorithm is used to cluster the substation data to obtain multiple substation portraits, including: determining the number of clusters of the clustering algorithm; based on the number of clusters, executing the clustering algorithm to divide the substation data into multiple substation clusters, each substation cluster corresponds to a substation portrait.

[0009] Optionally, the load data is input into an input layer in a target neural network, and the output sequence of the input layer is input into a position coding layer in the target neural network, and the step of performing position coding processing in the position coding layer includes: inputting the load data of the time series into the input layer in the target neural network to obtain an output sequence, wherein the target neural network includes: an input layer, a position coding layer, a convolutional layer, a long short-term memory layer, a multi-head attention layer, and a fully connected layer; inputting the output sequence into the position coding layer, and determining the dimension of the position coding vector based on the length of the output sequence and the number of layers of the target neural network; and calculating the position coding vector of each position in the output sequence according to a first formula based on the dimension of the position coding vector, and the first formula is:

[0010] ;

[0011] in, The output sequence is The position encoding vector of the position The position code of the elements, is the dimension of the position encoding vector; combine the output sequence and the position encoding vector to obtain the position encoding output sequence.

[0012] Optionally, the step of performing a linear transformation on the feature data to obtain multiple data vectors includes: obtaining a weight matrix for each head in a multi-head attention layer, wherein the weight matrix includes: a query weight matrix, a key weight matrix, and a value weight matrix; multiplying the feature data by the weight matrix to obtain a data vector, wherein the data vector includes: a query vector, a key vector, and a value vector.

[0013] Optionally, the step of calculating the periodic position code of each head in the multi-head attention layer includes: defining a different frequency for each head in the multi-head attention layer; and calculating the periodic position code of each head in the multi-head attention layer based on a second formula, the second formula being:

[0014] ;

[0015] in, The characteristic data Position in Periodic position coding on each head, For the The frequency of the head, It is the position number of the feature data.

[0016] Optionally, the step of obtaining the output vector of the multi-head attention layer based on the data vector and the periodic position encoding includes: calculating the attention score of each head in the multi-head attention layer based on a third formula, and the third formula is:

[0017] ;

[0018] in, For the The attention score of the head, For the The query vector of the head, For the The key vector of the head, is the dimension of the key vector, represents the transpose operation, For all positions in the feature data The sum of the periodic positional encodings on the heads; normalize the attention score of each head; use the attention score to weight the value vector to obtain the context vector of each head; concatenate the context vectors of all heads to obtain the output vector of the multi-head attention layer.

[0019] According to another aspect of an embodiment of the present invention, a load prediction device based on substation portraits is also provided, including: an extraction unit, used to pre-process the collected substation data, divide the pre-processed substation data into multiple substation portraits using a clustering algorithm, and extract the time series load data of each substation portrait; an encoding unit, used to input the load data into the input layer in the target neural network, and input the output sequence of the input layer into the position encoding layer in the target neural network, perform position encoding processing in the position encoding layer, and obtain a position encoding output sequence; a capture unit, used to input the position encoding output sequence into the convolution layer and the long short-term memory layer in the target neural network to obtain feature data, wherein the long short-term memory layer is used to capture the data dependency relationship of the load data; a calculation unit, used to input the feature data into the multi-head attention layer in the target neural network, perform linear transformation on the feature data to obtain multiple data vectors, calculate the periodic position coding of each head in the multi-head attention layer, and obtain the output vector of the multi-head attention layer based on the data vector and the periodic position coding; a conversion unit, used to input the output vector into the fully connected layer in the target neural network, perform dimensionality conversion processing on the output vector, and obtain the prediction result of the load data.

[0020] Optionally, the extraction unit includes: a first acquisition module, used to acquire non-numerical data in the substation data, wherein the substation data at least includes: substation capacity, substation load, substation category, weather conditions and geographical location information; a digitization module, used to digitize the non-numerical data, wherein the non-numerical data includes: substation category, weather conditions and geographical location information.

[0021] Optionally, the extraction unit further includes: a determination module for determining the number of clusters of the clustering algorithm; an execution module for executing the clustering algorithm based on the number of clusters to divide the substation data into a plurality of substation clusters, each substation cluster corresponding to a substation portrait.

[0022] Optionally, the encoding unit includes: a first input module, used to input the load data of the time series into the input layer of the target neural network to obtain an output sequence, wherein the target neural network includes: an input layer, a position encoding layer, a convolution layer, a long short-term memory layer, a multi-head attention layer and a fully connected layer; a second input module, used to input the output sequence into the position encoding layer, and determine the dimension of the position encoding vector based on the length of the output sequence and the number of layers of the target neural network; a first calculation module, based on the dimension of the position encoding vector, calculates the position encoding vector of each position in the output sequence according to a first formula, and the first formula is:

[0023] ;

[0024] in, The output sequence is The position encoding vector of the position The position code of the elements, is the dimension of the position encoding vector; a combination module is used to combine the output sequence and the position encoding vector to obtain a position encoding output sequence.

[0025] Optionally, the computing unit includes: a second acquisition module, used to obtain the weight matrix of each head in the multi-head attention layer, wherein the weight matrix includes: a query weight matrix, a key weight matrix, and a value weight matrix; a second computing module, used to multiply the feature data by the weight matrix to obtain a data vector, wherein the data vector includes: a query vector, a key vector, and a value vector.

[0026] Optionally, the calculation unit further includes: a definition module, used to define different frequencies for each head in the multi-head attention layer; a third calculation module, based on the second formula, calculating the periodic position encoding of each head in the multi-head attention layer, the second formula is:

[0027] ;

[0028] in, The characteristic data Position in Periodic position coding on each head, For the The frequency of the head, It is the position number of the feature data.

[0029] Optionally, the calculation unit further includes: a fourth calculation module, which calculates the attention score of each head in the multi-head attention layer based on the third formula, and the third formula is:

[0030] ;

[0031] in, For the The attention score of the head, For the The query vector of the head, For the The key vector of the head, is the dimension of the key vector, represents the transpose operation, For all positions in the feature data The sum of the periodic position codes on the heads; the normalization module is used to normalize the attention score of each head; the weighting module is used to weight the value vector using the attention score to obtain the context vector of each head; the splicing module is used to splice the context vectors of all heads to obtain the output vector of the multi-head attention layer.

[0032] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned load forecasting methods based on substation portraits.

[0033] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned load forecasting methods based on substation portraits.

[0034] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, which implements any step of the above-mentioned load forecasting method based on substation profile when executed by a processor.

[0035] In the present disclosure, the collected substation data is preprocessed, and a clustering algorithm is used to divide the preprocessed substation data into multiple substation portraits, and the time series load data of each substation portrait is extracted, and then the load data is input into the input layer of the target neural network, and the output sequence of the input layer is input into the position coding layer of the target neural network, and position coding processing is performed in the position coding layer to obtain a position coding output sequence, and then the position coding output sequence is input into the convolution layer and the long short-term memory layer in the target neural network to obtain feature data. The long short-term memory layer is used to capture the data dependency relationship of the load data, and then the feature data is input into the multi-head attention layer in the target neural network, and a linear transformation is performed on the feature data to obtain multiple data vectors, and the periodic position coding of each head in the multi-head attention layer is calculated. Based on the data vector and the periodic position coding, the output vector of the multi-head attention layer is obtained, and finally the output vector is input into the fully connected layer in the target neural network, and the output vector is dimensionalized to obtain the prediction result of the load data.

[0036] The present invention performs position encoding on load data so that the target neural network can distinguish information at different positions in the load data. By adopting a multi-head attention layer, the target neural network can capture the periodic change characteristics at different time scales in the load data. By combining the position encoding layer and the periodic position encoding, the prediction accuracy of the target neural network when processing time series load data with periodic characteristics is significantly improved, and the reliability of load data prediction is enhanced, thereby solving the technical problem in the related technology that the periodic characteristics of load data cannot be effectively identified, resulting in reduced prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0038] Figure 1 is a flow chart of an optional load forecasting method based on substation profile according to an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of an optional load prediction device based on a substation profile according to an embodiment of the present invention;

[0040] Figure 3 The present invention is a hardware structure block diagram of an electronic device (or mobile device) for executing a load prediction method based on a substation profile according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] It should be noted that the load prediction method based on substation portrait and the device thereof in the present disclosure can be used in the technical field of electric power load prediction. When the electric power load data is predicted based on the substation portrait, it can also be used in any field except the technical field of electric power load prediction. When the electric power load data is predicted based on the substation portrait, the present disclosure does not limit the application field of the load prediction method based on the substation portrait and the device thereof.

[0044] The following embodiments of the present invention can be applied to various systems / applications / devices for load prediction based on substation portraits. The present invention collects the load data of the time series of substation portraits, inputs the load data into the input layer of the target neural network, and inputs the output sequence of the input layer into the position coding layer of the target neural network, performs position coding processing in the position coding layer, so that the target neural network can distinguish the information at different positions in the load data, obtains feature data by inputting the position coding output sequence into the convolution layer and the long short-term memory layer in the target neural network, and then inputs the feature data into the multi-head attention layer in the target neural network, so that the target neural network can capture the periodic change characteristics of the load data on different time scales, obtains multiple data vectors by linearly transforming the feature data, calculates the periodic position coding of each head in the multi-head attention layer, obtains the output vector of the multi-head attention layer based on the data vector and the periodic position coding, and finally inputs the output vector into the fully connected layer in the target neural network, performs dimensional conversion processing on the output vector, and obtains the prediction result of the load data. The present invention combines the position coding layer with the periodic position coding, significantly improves the prediction accuracy of the target neural network when processing time series load data with periodic characteristics, and enhances the reliability of predicting the load data.

[0045] The present invention is described in detail below in conjunction with various embodiments.

[0046] Embodiment 1

[0047] According to an embodiment of the present invention, an embodiment of a load forecasting method based on substation portrait is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0048] Figure 1 is a flow chart of an optional load forecasting method based on a substation profile according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0049] Step S101, pre-processing the collected substation data, using a clustering algorithm to divide the pre-processed substation data into multiple substation portraits, and extracting the time series load data of each substation portrait.

[0050] In the power system, the substation is the key link between the high-voltage transmission network and the low-voltage user network. It can not only convert high-voltage electricity into low-voltage electricity suitable for home and small commercial facilities, but also monitor users' electricity consumption in real time and collect a large amount of electricity consumption data.

[0051] After collecting the substation data of the substation, the embodiment of the present invention pre-processes the substation data, and then uses a clustering algorithm to divide the pre-processed substation data into multiple substation clusters. These clusters are regarded as substation portraits with similar load behaviors. Each substation cluster corresponds to a substation portrait. After obtaining the substation portrait, the time series load data of each substation portrait is extracted for subsequent prediction processing. Load data refers to load information related to the power system or power equipment. Load data can reflect the power demand of the power grid, substation, and distribution station at a certain moment or a period of time. By analyzing the acquired load data and predicting the power demand at a certain moment in the future, the power company can adjust the power generation plan in advance, optimize the allocation of power resources, and avoid the imbalance of power supply and demand, which leads to economic losses and environmental damage.

[0052] Optionally, the step of preprocessing the collected substation data includes: obtaining non-numerical data in the substation data, wherein the substation data at least includes: substation capacity, substation load, substation category, weather conditions and geographical location information; digitally processing the non-numerical data, wherein the non-numerical data includes: substation category, weather conditions and geographical location information.

[0053] In an embodiment of the present invention, when preprocessing the substation data, it includes digitizing the non-numerical data in the substation data. The collected substation data include substation capacity, substation load, substation category, weather conditions and geographical location information, among which the substation category, weather conditions and geographical location information are non-numerical data. In order to enable these data to be accepted by the target neural network, these data need to be digitized. The embodiment of the present invention adopts a one-hot encoding strategy to convert the substation category, weather conditions and geographical location information into numerical data. One-hot encoding can convert these data into binary vectors, so that the non-numerical data in the substation data can also be received and processed by the target neural network.

[0054] Optionally, a clustering algorithm is used to cluster the substation data to obtain multiple substation portraits, including: determining the number of clusters of the clustering algorithm; based on the number of clusters, executing the clustering algorithm to divide the substation data into multiple substation clusters, each substation cluster corresponds to a substation portrait.

[0055] In an embodiment of the present invention, when a clustering algorithm is used to cluster the substation data, the selected clustering algorithm is the K-means clustering algorithm. The optimal number of clusters is determined by adopting the elbow rule or the silhouette coefficient strategy. Based on the number of clusters, the clustering algorithm is executed to divide the substation data into multiple substation clusters, and each substation cluster corresponds to a substation portrait.

[0056] The elbow rule changes the number of clusters, calculates the loss function, and finds an inflection point (i.e., the elbow), which is considered to be the optimal number of clusters. The silhouette coefficient strategy determines the optimal number of clusters by comparing the average distance between each data point and the points in its cluster and the average distance between the point and the points in the nearest other clusters. The embodiment of the present invention determines the optimal number of clusters, executes a clustering algorithm, and divides the preprocessed substation data into multiple substation clusters. These clusters are regarded as substation portraits with similar load behaviors, and each substation cluster corresponds to a substation portrait.

[0057] Step S102, input the load data into the input layer of the target neural network, and input the output sequence of the input layer into the position coding layer of the target neural network, perform position coding processing in the position coding layer, and obtain a position coding output sequence.

[0058] In an embodiment of the present invention, after the load data of the substation image is obtained, the load data can be input into the target neural network. The target neural network includes an input layer, a position encoding layer, a convolution layer, a long short-term memory layer, a multi-head attention layer, and a fully connected layer, wherein the input layer is used to receive the load data of the time series and transmit the output sequence of the input layer to the position encoding layer. The position encoding layer is used to perform position encoding processing on the load data, and add position encoding information to the load data of the time series, so that the target neural network can understand the position information of any element in the load data, thereby improving the recognition ability of the periodic features in the load data and improving the accuracy of load prediction.

[0059] It should be noted that before the target neural network processes real-time load data, it is necessary to use historical load data to train the target neural network. By applying a large amount of historical load data, the parameters of the target neural network are continuously adjusted and updated during the iteration process. Finally, when the preset convergence conditions are reached, the iteration process is stopped, the target neural network is trained, and it is used for load forecasting of real-time load data.

[0060] Optionally, the load data is input into an input layer in a target neural network, and the output sequence of the input layer is input into a position coding layer in the target neural network, and the step of performing position coding processing in the position coding layer includes: inputting the load data of the time series into the input layer in the target neural network to obtain an output sequence, wherein the target neural network includes: an input layer, a position coding layer, a convolutional layer, a long short-term memory layer, a multi-head attention layer, and a fully connected layer; inputting the output sequence into the position coding layer, and determining the dimension of the position coding vector based on the length of the output sequence and the number of layers of the target neural network; and calculating the position coding vector of each position in the output sequence according to a first formula based on the dimension of the position coding vector, and the first formula is:

[0061] ;

[0062] in, The output sequence is The position encoding vector of the position The position code of the elements, is the dimension of the position encoding vector; combine the output sequence and the position encoding vector to obtain the position encoding output sequence.

[0063] In the embodiment of the present invention, when position encoding is performed on the load data, the load data is first input into the input layer of the target neural network, and then the output sequence of the input layer is input into the position encoding layer, and the load data is position encoded in the position encoding layer. When performing the position encoding process, the dimension of the position encoding vector is first determined according to the length of the output sequence and the number of layers of the target neural network, and then based on the dimension of the position encoding vector, the position encoding vector of each position in the output sequence is calculated according to the first formula.

[0064] The load data is a time series. The output sequence obtained after the load data is processed by the input layer is still a time series. There are many elements in the time series, and each element has a position. The embodiment of the present invention calculates a dimension according to each position. The position encoding vector of The size of is set by itself, usually according to the requirements of the embodiment. Each element in the position encoding vector of each position is calculated according to the first formula, in which, is the position index of the output sequence, is the element index of the position coding vector. It can be seen from the first formula that in the even-numbered positions of the position coding vector, the element is a sine value, and in the odd-numbered positions of the position coding vector, the element is a cosine value. After calculating the position coding vector for each position, the output sequence and the position coding vector are combined to obtain a position-coded output sequence. It should be noted that the calculation strategy of the first formula makes each position in the output sequence have a unique position coding vector, and makes the position coding value of each element in each position between -1 and 1. The embodiment of the present invention performs position coding on the load data so that the target neural network can distinguish the information at different positions in the load data, improve the recognition ability of the target neural network for the periodic features in the load data, and enhance the accuracy of load prediction.

[0065] Step S103, inputting the position encoding output sequence into the convolution layer and the long short-term memory layer in the target neural network to obtain feature data, wherein the long short-term memory layer is used to capture the inter-data dependency of the load data.

[0066] In an embodiment of the present invention, after the load data is processed by position coding, a position coding output sequence is obtained, and the position coding output sequence is sequentially input into the convolution layer and the long short-term memory layer to obtain feature data, wherein the convolution layer can extract local feature data of the position coding output sequence, and the long short-term memory layer can capture the dependency between data.

[0067] The long short-term memory layer is called the LSTM layer. The long short-term memory layer can selectively remember specific information fragments in the input sequence and forget other unimportant information. Through this strategy, the long short-term memory layer can remember the key information in the input sequence. Even if this information is far apart in the input sequence, the long short-term memory layer can effectively use this information to perform load forecasting tasks. In an embodiment of the present invention, the long short-term memory layer can recognize and remember important patterns in the position encoding output sequence, even if these patterns occur at distant time points. For example: the long short-term memory layer can recognize patterns in which load peaks occur before or after a specific time, and remember these patterns for use in the task of predicting future load peaks.

[0068] The output features of the long short-term memory layer are hidden states and cell states. In the embodiment of the present invention, the hidden state can be regarded as the understanding of the long short-term memory layer for the position coding output sequence, which integrates the state information of the previous moment and the input information of the current moment. The hidden state can be regarded as a highly abstract representation of the position coding output sequence, which contains important historical information about the position coding output sequence. The cell state is responsible for preserving and controlling the flow of information to ensure that important information is not easily forgotten by the long short-term memory layer. The embodiment of the present invention uses the hidden state as the output feature of the long short-term memory layer, that is, the feature data. The feature data will be passed to the multi-head attention layer for the subsequent load forecasting steps.

[0069] Step S104: input the feature data into the multi-head attention layer in the target neural network, perform linear transformation on the feature data to obtain multiple data vectors, calculate the periodic position code of each head in the multi-head attention layer, and obtain the output vector of the multi-head attention layer based on the data vector and the periodic position code.

[0070] In an embodiment of the present invention, after obtaining the feature data output by the long short-term memory layer, the feature data is input into the multi-head attention layer for processing. The multi-head attention layer obtains different attention distributions of the input sequence by running multiple independent attention mechanisms in parallel, thereby more comprehensively capturing the potential multiple semantic associations in the input sequence.

[0071] In the multi-head attention layer, the feature data is first linearly transformed to obtain multiple data vectors, and then the periodic position coding of each head in the multi-head attention layer is calculated. Based on the data vector and the periodic position coding, the attention score of each head is calculated, and finally the output vector of the multi-head attention layer is obtained. The output vector will be input into the last layer of the target neural network, the fully connected layer. After being processed by the fully connected layer, the prediction result of the load data can be obtained. By considering the periodic position coding in the calculation process of the attention score, it is not only possible to distinguish the information at different positions of the feature data, but also to take into account the periodic change characteristics in the time series, so that the target neural network can better predict the complex periodic changes of the load data.

[0072] Optionally, the step of performing a linear transformation on the feature data to obtain multiple data vectors includes: obtaining a weight matrix for each head in a multi-head attention layer, wherein the weight matrix includes: a query weight matrix, a key weight matrix, and a value weight matrix; multiplying the feature data by the weight matrix to obtain a data vector, wherein the data vector includes: a query vector, a key vector, and a value vector.

[0073] In the embodiment of the present invention, when performing a linear transformation on the feature data, it is first necessary to obtain the weight matrix of each head in the multi-head attention layer. In the multi-head attention layer, each head has an independent weight matrix, and the weight matrix includes a query weight matrix, a key weight matrix, and a value weight matrix. By multiplying the weight matrix with the feature data, the data vector of each head is obtained, and the data vector of each head includes a query vector, a key vector, and a value vector. The calculation formula for linear transformation is:

[0074] ;

[0075] in, For the The query vector of the head, is the feature data output by the long short-term memory layer, For the The query weight matrix, key vector and value vector of each head are obtained through the same calculation strategy.

[0076] Optionally, the step of calculating the periodic position code of each head in the multi-head attention layer includes: defining a different frequency for each head in the multi-head attention layer; and calculating the periodic position code of each head in the multi-head attention layer based on a second formula, the second formula being:

[0077] ;

[0078] in, The characteristic data Position in Periodic position coding on each head, For the The frequency of the head, It is the position number of the feature data.

[0079] In the embodiment of the present invention, when calculating the periodic position encoding of each head in the multi-head attention layer, it is first necessary to define different frequencies for each head in the multi-head attention layer. Different frequencies correspond to different time scales, so that the target neural network can notice the periodic features at different frequencies. In the embodiment of the present invention, the attention mechanism is divided into three heads, and the frequency of each head is set to:

[0080] Head 1: Daily periodicity, ;

[0081] Head 2: Weekly periodicity, ;

[0082] Head 3: Monthly periodicity, ;

[0083] In the above frequency setting, Corresponding to the daily time scale, Corresponding to the weekly time scale, Corresponding to the monthly time scale. By setting different frequencies for each head in the multi-head attention layer, the target neural network can simultaneously focus on the short-term and long-term periodic patterns in the load data, capture the periodic characteristics at different frequencies, and improve the precision and accuracy of load forecasting.

[0084] After defining different frequencies for each head in the multi-head attention layer, the periodic position encoding of each head in the multi-head attention layer is calculated according to the second formula for subsequent calculation of the attention score. The embodiment of the present invention considers the periodic position encoding in the calculation process of the attention score, and can identify the periodic change characteristics in the load data while distinguishing different position information in the load data, so that the target neural network can better predict the complex periodic changes of the load data.

[0085] Optionally, the step of obtaining the output vector of the multi-head attention layer based on the data vector and the periodic position encoding includes: calculating the attention score of each head in the multi-head attention layer based on a third formula, and the third formula is:

[0086] ;

[0087] in, For the The attention score of the head, For the The query vector of the head, For the The key vector of the head, is the dimension of the key vector, represents the transpose operation, For all positions in the feature data The sum of the periodic positional encodings on the heads; normalize the attention score of each head; use the attention score to weight the value vector to obtain the context vector of each head; concatenate the context vectors of all heads to obtain the output vector of the multi-head attention layer.

[0088] In the embodiment of the present invention, after obtaining the data vector and the periodic position encoding, the attention score of each head in the multi-head attention layer is calculated according to the third formula. It can be seen in the third formula that the calculation of the attention score only uses the query vector and the key vector in the data vector. After calculating the attention score for the query vector and the corresponding key vector of each head, the embodiment of the present invention performs weighted processing on the value vector in the data vector. In the third formula, For all positions in the feature data The sum of the periodic position codes on the heads, when the feature data has When the position for to The sum of .

[0089] The attention score reflects the correlation between the query vector and the key vector, which determines the weight of the value vector in the context vector. After the attention score is calculated, it needs to be normalized, and then the normalized attention score is used to perform weighted summation on the value vector. The function is used to normalize the attention scores to ensure that the sum of the attention scores of all heads is 1. The normalized attention scores are used to weight the value vector to generate the context vector of each attention head. The normalization and weighting process can be expressed as:

[0090] ;

[0091] in, For the The attention score of the head, For the The value vector of the head, For the Finally, the context vectors of all attention heads are concatenated to obtain the output vector of the multi-head attention layer. The output vector will be input into the fully connected layer of the target neural network. After being processed by the fully connected layer, the prediction result of the load data can be obtained.

[0092] Step S105, input the output vector into the fully connected layer in the target neural network, perform dimension conversion processing on the output vector, and obtain the prediction result of the load data.

[0093] In the embodiment of the present invention, the context vectors of all attention heads are concatenated to obtain the output vector of the multi-head attention layer, and the output vector is dimensionally converted in the fully connected layer so that the output vector is converted into the output dimension required by the embodiment of the present invention. The concatenation and dimension conversion process can be expressed as:

[0094] ;

[0095] in, is the output vector of the fully connected layer, For the The context vector of each head, is the weight matrix, Represents a concatenation operation, wherein the weight matrix is ​​a matrix that transforms the output vector of the multi-head attention layer into the required vector space, and is usually set according to the requirements of the embodiment. After the output vector of the multi-head attention layer is processed by the fully connected layer, the output vector of the fully connected layer is obtained, which is the prediction result of the target neural network for the input load data.

[0096] The steps provided by the above-mentioned load prediction method based on substation portrait are as follows: preprocessing the collected substation data, using a clustering algorithm to divide the preprocessed substation data into multiple substation portraits, extracting the time series load data of each substation portrait, and then inputting the load data into the input layer of the target neural network, and inputting the output sequence of the input layer into the position coding layer of the target neural network, performing position coding processing in the position coding layer to obtain a position coding output sequence, and then inputting the position coding output sequence into the convolution layer and the long short-term memory layer in the target neural network to obtain feature data. The long short-term memory layer is used to capture the data dependency relationship of the load data, and then inputting the feature data into the multi-head attention layer in the target neural network, performing linear transformation on the feature data to obtain multiple data vectors, calculating the periodic position coding of each head in the multi-head attention layer, and obtaining the output vector of the multi-head attention layer based on the data vector and the periodic position coding. Finally, the output vector is input into the fully connected layer in the target neural network, and the output vector is dimensionalized to obtain the prediction result of the load data.

[0097] The embodiment of the present invention performs position encoding on the load data so that the target neural network can distinguish the information at different positions in the load data. By adopting a multi-head attention layer, the target neural network can capture the periodic change characteristics at different time scales in the load data. By combining the position encoding layer and the periodic position encoding, the prediction accuracy of the target neural network when processing time series load data with periodic characteristics is significantly improved, and the reliability of load data prediction is enhanced, thereby solving the technical problem in the related art that the periodic characteristics of the load data cannot be effectively identified, resulting in reduced prediction accuracy.

[0098] The following is a detailed description in conjunction with another embodiment.

[0099] Embodiment 2

[0100] A load prediction device based on substation profile provided in this embodiment includes multiple implementation units, and each implementation unit corresponds to each implementation step in the above-mentioned embodiment 1.

[0101] Figure 2 is a schematic diagram of an optional load prediction device based on a substation profile according to an embodiment of the present invention, such as Figure 2 As shown, the load prediction device based on the substation portrait may include: an extraction unit 20, an encoding unit 21, a capture unit 22, a calculation unit 23, and a conversion unit 24.

[0102] The extraction unit 20 is used to pre-process the collected substation data, divide the pre-processed substation data into multiple substation images using a clustering algorithm, and extract the time series load data of each substation image;

[0103] The encoding unit 21 is used to input the load data into the input layer of the target neural network, and input the output sequence of the input layer into the position encoding layer of the target neural network, perform position encoding processing in the position encoding layer, and obtain the position encoding output sequence;

[0104] A capture unit 22 is used to input the position encoding output sequence into the convolution layer and the long short-term memory layer in the target neural network to obtain feature data, wherein the long short-term memory layer is used to capture the dependency relationship between the load data;

[0105] A computing unit 23 is used to input the feature data into a multi-head attention layer in a target neural network, perform a linear transformation on the feature data to obtain a plurality of data vectors, calculate a periodic position code of each head in the multi-head attention layer, and obtain an output vector of the multi-head attention layer based on the data vector and the periodic position code;

[0106] The conversion unit 24 is used to input the output vector into the fully connected layer in the target neural network, perform dimension conversion processing on the output vector, and obtain the prediction result of the load data.

[0107] The above-mentioned load prediction device based on substation portrait can pre-process the collected substation data through the extraction unit 20, divide the pre-processed substation data into multiple substation portraits by using a clustering algorithm, extract the time series load data of each substation portrait, and then input the load data into the input layer of the target neural network through the encoding unit 21, and input the output sequence of the input layer into the position coding layer of the target neural network, perform position coding processing in the position coding layer to obtain the position coding output sequence, and then input the position coding output sequence into the convolution layer and the long short-term memory layer in the target neural network through the capture unit 22 to obtain feature data. The long short-term memory layer is used to capture the data dependency relationship of the load data, and then input the feature data into the multi-head attention layer in the target neural network through the calculation unit 23, perform linear transformation on the feature data to obtain multiple data vectors, calculate the periodic position coding of each head in the multi-head attention layer, and obtain the output vector of the multi-head attention layer based on the data vector and the periodic position coding. Finally, the output vector is input into the fully connected layer in the target neural network through the conversion unit 24, and the output vector is dimensionally converted to obtain the prediction result of the load data. This embodiment performs position encoding on the load data so that the target neural network can distinguish information at different positions in the load data. By adopting a multi-head attention layer, the target neural network can capture the periodic change characteristics of the load data at different time scales. By combining the position encoding layer and the periodic position encoding, the prediction accuracy of the target neural network when processing time series load data with periodic characteristics is significantly improved, and the reliability of load data prediction is enhanced, thereby solving the technical problem in the related technology that the periodic characteristics of the load data cannot be effectively identified, resulting in reduced prediction accuracy.

[0108] Optionally, the extraction unit 20 includes: a first acquisition module, used to acquire non-numerical data in the substation data, wherein the substation data at least includes: substation capacity, substation load, substation category, weather conditions and geographical location information; a digitization module, used to digitize the non-numerical data, wherein the non-numerical data includes: substation category, weather conditions and geographical location information.

[0109] Optionally, the extraction unit 20 further includes: a determination module for determining the number of clusters of the clustering algorithm; an execution module for executing the clustering algorithm based on the number of clusters to divide the substation data into a plurality of substation clusters, each substation cluster corresponding to a substation portrait.

[0110] Optionally, the encoding unit 21 includes: a first input module, used to input the load data of the time series into the input layer of the target neural network to obtain an output sequence, wherein the target neural network includes: an input layer, a position encoding layer, a convolution layer, a long short-term memory layer, a multi-head attention layer and a fully connected layer; a second input module, used to input the output sequence into the position encoding layer, and determine the dimension of the position encoding vector based on the length of the output sequence and the number of layers of the target neural network; a first calculation module, based on the dimension of the position encoding vector, calculates the position encoding vector of each position in the output sequence according to a first formula, and the first formula is:

[0111] ;

[0112] in, The output sequence is The position encoding vector of the position The position code of the elements, is the dimension of the position encoding vector; a combination module is used to combine the output sequence and the position encoding vector to obtain a position encoding output sequence.

[0113] Optionally, the computing unit 23 includes: a second acquisition module, used to obtain the weight matrix of each head in the multi-head attention layer, wherein the weight matrix includes: a query weight matrix, a key weight matrix, and a value weight matrix; a second computing module, used to multiply the feature data by the weight matrix to obtain a data vector, wherein the data vector includes: a query vector, a key vector, and a value vector.

[0114] Optionally, the calculation unit 23 further includes: a definition module, used to define different frequencies for each head in the multi-head attention layer; a third calculation module, based on the second formula, calculating the periodic position encoding of each head in the multi-head attention layer, the second formula is:

[0115] ;

[0116] in, The characteristic data Position in Periodic position coding on each head, For the The frequency of the head, It is the position number of the feature data.

[0117] Optionally, the calculation unit 23 further includes: a fourth calculation module, which calculates the attention score of each head in the multi-head attention layer based on the third formula, and the third formula is:

[0118] ;

[0119] in, For the The attention score of the head, For the The query vector of the head, For the The key vector of the head, is the dimension of the key vector, represents the transpose operation, For all positions in the feature data The sum of the periodic position codes on the heads; the normalization module is used to normalize the attention score of each head; the weighting module is used to weight the value vector using the attention score to obtain the context vector of each head; the splicing module is used to splice the context vectors of all heads to obtain the output vector of the multi-head attention layer.

[0120] The above-mentioned load prediction device based on substation portrait can also include a processor and a memory. The above-mentioned extraction unit 20, encoding unit 21, capture unit 22, calculation unit 23, conversion unit 24, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0121] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and load forecasting for power stations can be performed by adjusting kernel parameters.

[0122] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0123] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the load forecasting methods based on substation portraits in the above-mentioned embodiment 1.

[0124] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the load forecasting method based on substation portrait of any one of the above-mentioned embodiments one.

[0125] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the load forecasting method based on substation profile described in each embodiment of the present application.

[0126] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the load forecasting method based on substation profile described in each embodiment of the present application are implemented.

[0127] Figure 3 1 is a hardware structure block diagram of an electronic device (or mobile device) for executing a load forecasting method based on a substation profile according to an embodiment of the present invention. Figure 3 As shown, the electronic device may include one or more ( Figure 3 302a, 302b, ..., 302n are used to illustrate) processor (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 304 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 3 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 3 More or fewer components as shown, or with Figure 3 Different configurations are shown.

[0128] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0129] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0134] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A load forecasting method based on substation profile, characterized in that: include: Preprocessing the collected substation data, dividing the preprocessed substation data into a plurality of substation portraits using a clustering algorithm, and extracting the time series load data of each substation portrait; Inputting the load data into an input layer of a target neural network, and inputting an output sequence of the input layer into a position coding layer of the target neural network, performing position coding processing at the position coding layer, and obtaining a position coding output sequence; Inputting the position encoding output sequence into a convolutional layer and a long short-term memory layer in the target neural network to obtain feature data, wherein the long short-term memory layer is used to capture the inter-data dependency of the load data; Input the feature data into a multi-head attention layer in the target neural network, perform linear transformation on the feature data to obtain multiple data vectors, calculate the periodic position code of each head in the multi-head attention layer, and obtain an output vector of the multi-head attention layer based on the data vector and the periodic position code; The step of calculating the periodic position encoding of each head in the multi-head attention layer includes: defining a different frequency for each head in the multi-head attention layer; and calculating the periodic position encoding of each head in the multi-head attention layer based on a second formula, wherein the second formula is: ,in, The characteristic data Position in Periodic position coding on each head, For the The frequency of the head, is the position serial number of the feature data; The output vector is input into the fully connected layer in the target neural network, and the output vector is dimensionally transformed to obtain the prediction result of the load data.

2. The load forecasting method according to claim 1, characterized in that: The steps of preprocessing the collected area data include: Acquire non-numerical data in the substation data, wherein the substation data at least includes: substation capacity, substation load, substation category, weather conditions and geographical location information; The non-numeric data is digitally processed, wherein the non-numeric data includes: the station area category, the weather conditions and geographical location information.

3. The load forecasting method according to claim 1, characterized in that: The step of clustering the substation data using a clustering algorithm to obtain multiple substation portraits includes: Determining the number of clusters of the clustering algorithm; Based on the number of clusters, the clustering algorithm is executed to divide the station area data into a plurality of station area clusters, each of the station area clusters corresponding to a station area portrait.

4. The load forecasting method according to claim 1, characterized in that: The step of inputting the load data into an input layer of a target neural network, and inputting an output sequence of the input layer into a position encoding layer of the target neural network, and performing position encoding processing in the position encoding layer comprises: Inputting the load data of the time series into the input layer of the target neural network to obtain an output sequence, wherein the target neural network includes: the input layer, the position encoding layer, the convolution layer, the long short-term memory layer, the multi-head attention layer and the fully connected layer; Inputting the output sequence into the position encoding layer, and determining the dimension of the position encoding vector based on the length of the output sequence and the number of layers of the target neural network; Based on the dimension of the position encoding vector, the position encoding vector of each position in the output sequence is calculated according to a first formula, wherein the first formula is: ; in, The output sequence The position encoding vector of the position The position code of the elements, The dimension of the position encoding vector; The output sequence and the position encoding vector are combined to obtain the position encoding output sequence.

5. The load forecasting method according to claim 1, characterized in that: The step of performing linear transformation on the feature data to obtain a plurality of data vectors comprises: Obtain a weight matrix for each head in the multi-head attention layer, wherein the weight matrix includes: a query weight matrix, a key weight matrix, and a value weight matrix; The feature data is multiplied by the weight matrix to obtain the data vector, wherein the data vector includes: a query vector, a key vector, and a value vector.

6. The load forecasting method according to claim 5, characterized in that: The step of obtaining the output vector of the multi-head attention layer based on the data vector and the periodic position encoding comprises: Based on the third formula, the attention score of each head in the multi-head attention layer is calculated, and the third formula is: ; in, For the The attention score of the head, For the The query vector of the head, For the The key vector of the head, is the dimension of the key vector, represents the transpose operation, For all positions in the feature data The sum of the periodic position codes on each head; Normalizing the attention score of each head; Using the attention score to perform weighted processing on the value vector to obtain a context vector for each head; The context vectors of all heads are concatenated to obtain the output vector of the multi-head attention layer.

7. A load forecasting device based on a substation profile, characterized in that: include: An extraction unit, used to pre-process the collected substation data, divide the pre-processed substation data into a plurality of substation portraits using a clustering algorithm, and extract the time series load data of each substation portrait; An encoding unit, used for inputting the load data into an input layer in a target neural network, and inputting an output sequence of the input layer into a position encoding layer in the target neural network, performing position encoding processing in the position encoding layer, and obtaining a position encoding output sequence; A capture unit, used for inputting the position encoding output sequence into a convolution layer and a long short-term memory layer in the target neural network to obtain feature data, wherein the long short-term memory layer is used for capturing the inter-data dependency of the load data; A computing unit, configured to input the feature data into a multi-head attention layer in the target neural network, perform a linear transformation on the feature data to obtain a plurality of data vectors, calculate a periodic position code of each head in the multi-head attention layer, and obtain an output vector of the multi-head attention layer based on the data vector and the periodic position code; The calculation unit includes: a third calculation module, which calculates the periodic position encoding of each head in the multi-head attention layer based on a second formula, and the second formula is: ,in, The characteristic data Position in Periodic position coding on each head, For the The frequency of the head, is the position serial number of the feature data; The conversion unit is used to input the output vector into the fully connected layer in the target neural network, perform dimension conversion processing on the output vector, and obtain the prediction result of the load data.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the load forecasting method based on substation profile according to any one of claims 1 to 6.

9. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the load forecasting method based on substation portrait as described in any one of claims 1 to 6.

Images