Electricity price prediction method and system based on multiple attention mechanisms and convolutional network

Abstract

The invention provides an electricity price prediction method and system based on a multi-head attention mechanism and a convolutional network, and the method comprises the steps: obtaining model training data which is time series data; the model training data is input into an electricity price prediction algorithm model for training, the electricity price prediction algorithm model is of an encoder-decoder structure, and an encoder and a decoder are each composed of a network layer formed by connecting a multi-head attention mechanism and a time convolution network; and obtaining training results of electricity price prediction, and carrying out prediction value splicing to obtain corresponding electricity price prediction results in multiple time periods. According to the method, a series of preprocessing processes such as cleaning, feature extraction, dimensionality reduction and normalization are carried out on historical power related data, the obtained data are input into an electricity price prediction algorithm model based on a multi-attention mechanism and a time convolution network, and accurate electricity price prediction values in a plurality of time periods in the future can be calculated.

Description

Technical Field

[0001] The present invention relates to the technical field of power systems, and in particular to an electricity price prediction method and system based on a multi-head attention mechanism and a convolutional network. Background Art

[0002] In order to explore the operation mechanism of the power market, assist power market transaction decision-making, and improve power dispatch efficiency, industry-related personnel have conducted in-depth research on the problem of electricity price forecasting in recent years. Among them, the particularity of the electricity price forecasting problem is that it is affected by many factors and does not have a certain overall change trend.

[0003] Generally speaking, the fluctuation of electricity prices is not only affected by the relevant characteristics of the power system such as fuel prices, power generation of various power generation methods, electricity demand, and grid load, but also by environmental conditions such as temperature, precipitation, wind conditions, and light intensity. At the same time, the changes in electricity prices are cyclical within a certain time range. At present, electricity price forecasting mainly adopts time series analysis methods based on statistical models, neural network model methods, and time-frequency domain prediction methods based on wavelet analysis theory. However, current research methods are often unable to accurately explore the different impacts of historical data in different time periods on the prediction target under the input conditions of multiple variables. Summary of the invention

[0004] The purpose of the present invention is to provide an electricity price prediction method and system based on a multi-head attention mechanism and a convolutional network, so as to solve the problem of inaccurate electricity price prediction in the prior art.

[0005] In a first aspect, the present application provides an electricity price prediction method based on a multi-head attention mechanism and a convolutional network, the method comprising:

[0006] Acquire model training data, wherein the training data is time series data;

[0007] Inputting the model training data into the electricity price prediction algorithm model for training, wherein the electricity price prediction algorithm model is an encoder-decoder structure, and the encoder and the decoder are both composed of a network layer connected by a multi-head attention mechanism and a temporal convolutional network;

[0008] The training results of electricity price prediction are obtained and the predicted values ​​are spliced ​​to obtain the corresponding electricity price prediction results under multiple time periods.

[0009] In a possible implementation of the present application, the obtaining of model training data specifically includes:

[0010] Acquire raw data, the raw data including power generation information and environmental information, wherein the power generation information at least includes power generation source, power generation cost and grid load, and the environmental information at least includes weather type, air pressure information, temperature information and humidity information;

[0011] The model training data is obtained by performing data preprocessing using a preprocessing module based on the original data.

[0012] In a possible implementation of the present application, it is characterized in that the step of performing data preprocessing based on the original data using a preprocessing module to obtain the model training data specifically includes:

[0013] Performing data cleaning based on the raw data, including removing missing values ​​and outliers, and aligning data items from different sources according to collection time;

[0014] The cleaned raw data is concatenated, which specifically includes encoding discrete feature data to obtain continuous features, and the encoding method includes one-hot encoding;

[0015] Based on the principal component analysis method, the original data after cleaning and information splicing is reduced in dimension;

[0016] The original data after dimensionality reduction and the electricity price data are normalized to obtain the model training data.

[0017] In a possible implementation of the present application, the inputting the model training data into the electricity price prediction algorithm model for training specifically includes:

[0018] The encoder receives the training data for learning to obtain a first time domain autocorrelation parameter of the training data in the encoder;

[0019] Based on the target data in the training data received by the decoder, learning is performed to obtain a second time domain autocorrelation parameter of the training data in the decoder;

[0020] Performing cross-correlation calculation based on the first time domain autocorrelation parameter and the second time domain autocorrelation parameter to obtain a current period training result of the decoder;

[0021] The training result of the current time period of the decoder is used as the input parameter of the decoder in the next time period, and the training results corresponding to multiple time periods are obtained by repeated iterative calculation.

[0022] In a possible implementation of the present application, the learning based on receiving the training data by the encoder to obtain the first time domain autocorrelation parameter of the training data in the encoder specifically includes:

[0023] The training data is received based on the encoder, and the input vector of the multi-head self-attention mechanism is obtained by encoding in combination with the position encoding, wherein sine encoding is used for elements at even positions, and cosine encoding is used for elements at odd positions;

[0024] Based on a multi-head self-attention mechanism, the input vector is distributed to different representation spaces for parallel learning;

[0025] The learning result based on the multi-head self-attention mechanism is used as the input of the temporal convolutional network, and the first time domain autocorrelation parameter is obtained based on the output of the temporal convolutional network.

[0026] In a possible implementation of the present application, the learning based on receiving the target data in the training data by the decoder to obtain the second time domain autocorrelation parameter of the training data in the decoder specifically includes:

[0027] Based on the training data or historical training results received by the decoder, encoding is performed in combination with position encoding to obtain an input vector of the multi-head self-attention mechanism, wherein sine encoding is used for elements at even positions, and cosine encoding is used for elements at odd positions;

[0028] Based on a multi-head self-attention mechanism, the input vector is distributed to different representation spaces for parallel learning;

[0029] The learning result based on the multi-head self-attention mechanism is used as the input of the temporal convolutional network, and the second time domain autocorrelation parameter is obtained based on the output of the temporal convolutional network.

[0030] In a possible implementation of the present application, performing cross-correlation calculation based on the first time domain autocorrelation parameter and the second time domain autocorrelation parameter to obtain a current period training result of the decoder specifically includes:

[0031] Obtaining an input vector of a multi-head mutual attention mechanism in a decoder based on the first time-domain autocorrelation parameter and the second time-domain autocorrelation parameter;

[0032] Based on a multi-head mutual attention mechanism, the input vector is distributed to different representation spaces for parallel learning;

[0033] The learning result based on the multi-head mutual attention mechanism is used as the input of the temporal convolutional network, and the training result of the current period is obtained based on the output of the temporal convolutional network.

[0034] In a second aspect, the present application provides an electricity price prediction system based on a multi-head attention mechanism and a convolutional network, the system comprising:

[0035] An acquisition module, used to acquire model training data, wherein the training data is time series data;

[0036] A training module, used for inputting the model training data into the electricity price prediction algorithm model for training, wherein the electricity price prediction algorithm model is an encoder-decoder structure, and the encoder and the decoder are both composed of a network layer connected by a multi-head attention mechanism and a temporal convolutional network;

[0037] A prediction module is used to obtain the training results of electricity price prediction and perform prediction value splicing to obtain the corresponding electricity price prediction results under multiple time periods. In a third aspect, the present application provides a computer-readable storage medium as mentioned above, on which a computer program is stored, which, when executed by a processor, implements the electricity price prediction method based on a multi-head attention mechanism and a convolutional network.

[0038] In a fourth aspect, the present application provides an electronic device as mentioned above, comprising: a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to load and execute the computer program so that the electronic device executes the electricity price prediction method based on a multi-head attention mechanism and a convolutional network.

[0039] As described above, the electricity price prediction method and system based on the multi-head attention mechanism and convolutional network of the present invention, through a series of preprocessing processes such as cleaning, feature extraction, dimensionality reduction, and normalization of historical electricity-related data, inputs the obtained data into the electricity price prediction algorithm model based on the multi-head attention mechanism and temporal convolutional network, and can calculate the accurate prediction value of electricity price for several time periods in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Shown is a schematic diagram of method steps of an electricity price prediction method based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0041] Figure 2 Shown is a schematic diagram of method steps of an electricity price prediction method based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0042] Figure 3 Shown is a schematic diagram of method steps in another embodiment of the electricity price prediction method based on multi-head attention mechanism and convolutional network of the present invention;

[0043] Figure 4 Shown is a schematic diagram of method steps of an electricity price prediction method based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0044] Figure 5 Shown is a flow chart of the training of an algorithm model for electricity price prediction in one embodiment of the electricity price prediction method based on a multi-head attention mechanism and a convolutional network of the present invention;

[0045] Figure 6 Shown is a schematic diagram of method steps of an electricity price prediction method based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0046] Figure 7 Shown is a schematic diagram of a residual connection structure of an electricity price prediction method based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0047] Figure 8 Shown is a schematic diagram of method steps of an electricity price prediction method based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0048] Fig. 9 Shown is a schematic diagram of the structure of an electricity price prediction system based on a multi-head attention mechanism and a convolutional network in one embodiment of the present invention;

[0049] Fig.10 Shown is a schematic structural diagram of an electronic device in an embodiment of the present invention.

[0050] Component number description

[0051] Steps S102 to S206

[0052] Steps S202 to S204

[0053] Steps S302 to S308

[0054] Steps S402 to S408

[0055] Steps S602 to S606

[0056] Steps S802 to S806

[0057] 90 Electricity price prediction system based on multi-head attention mechanism and convolutional network

[0058] 91 Get Module

[0059] 92 Processing Module

[0060] 93 Conversion Module DETAILED DESCRIPTION

[0061] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0062] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0063] The core idea of ​​electricity price prediction is to establish a model based on the main conditions that affect electricity prices and calculate the changes in future electricity prices based on historical data. The electricity price prediction method based on multi-head attention mechanism and convolutional network proposed in this application learns and calculates the degree of correlation between long-term and short-term inputs and target outputs through multi-head attention mechanism, and uses time convolutional network to further learn the correlation between local positions in time series, so as to obtain more accurate future electricity price prediction values.

[0064] Among them, the multi-head attention mechanism was first proposed in the Transformer model. Its design purpose is to learn the dynamic relationship between the original text and the translation in machine translation problems. It can be widely used in a series of sequence data correlation calculation problems. The input of the multi-head attention mechanism is a sequence. Its core idea is multi-head splitting, that is, dividing the input data into multiple parts in dimension and performing parallel attention calculation. Its purpose is to learn the correlation of data in different representation spaces.

[0065] The multi-head attention mechanism learns the association between each position vector in the input sequence through the scaled dot product operation, with the aim of capturing the global dependency between the input sequences and calculating the representation of the output sequence. The attention learning process can be summarized as mapping the input Query (Q), Key (K) and Value (V) vectors into the output sequence, where the dot product is first used to calculate the relevant weights of Q and K, and then V is weighted according to the relevant weights to obtain the output result. The specific calculation process is as follows:

[0066] For the input Q, K, V sequences, they are first transformed in dimension through a trainable weight matrix, and then their respective transformed outputs are evenly split into multiple parts in dimension. Each part corresponds to an attention head, which represents a separate learning representation space. Each attention head learns in parallel, and their learning processes do not affect each other. The learning process of each attention head is expressed as:

[0067] head i =Attention(QW i Q ,KW i K ,VW i V );W iQ , W i K , W i V is the weight matrix. In each attention head, the scaled dot product attention algorithm is used, which performs a dot product operation between Q and K to learn the relevance weight. The specific calculation formula is:

[0068]

[0069] To meet the dimensional requirements of the dot product operation, K needs to be transposed. To prevent the dot product operation result from being too large and affecting optimization, the dot product result also needs to be divided by the scaling factor. where d k is the dimension of the position vector. The dot product result obtained at this time is the correlation matrix of Q and K, and its element in the i-th row and j-th column represents the correlation score between the i-th position vector in Q and the j-th position vector in K. In order to obtain the correlation weight, Softmax normalization is required on this basis. Finally, the dot product weighting of each position vector in V is performed according to the obtained weight to obtain the result of the attention operation. Finally, the attention calculation results of each head are reassembled according to the following formula to obtain the final output of the attention mechanism:

[0070] MultiHead(Q,K,V)=Concat(head1,head2,…,head n )W;

[0071] Where W is a trainable weight matrix.

[0072] The multi-head attention mechanism includes self-attention and mutual attention. In self-attention, the input Q, K, and V vectors are in the same sequence, and the attention mechanism calculates the autocorrelation between the position vectors of the input sequence. In mutual attention, Q represents input sequence one, and K and V are represented by input sequence two. The attention mechanism calculates the mutual correlation between the position vectors of the two sequences.

[0073] As for the temporal convolutional network, the basic unit of the temporal convolutional network is the one-dimensional convolution layer. One-dimensional convolution (Conv1D) is a convolutional neural network used to process sequence data. It evolved from the traditional two-dimensional convolutional network that processes spatial structure data such as images and topology. The receptive field of the temporal convolution is expanded along the sequence direction, and its design purpose is to learn the intrinsic relationship between each adjacent sequence vector.

[0074] The difference between one-dimensional convolution and traditional two-dimensional convolution lies in the size of the convolution kernel. Each one-dimensional convolution kernel needs to cover the entire position vector dimension. Specifically, for a size of R T×dThe input data has a sequence length of T and a position vector dimension of d, so the size of the corresponding one-dimensional convolution kernel is R. t×d , where t is the length of the convolution kernel, indicating that the convolution receptive field covers t adjacent position vectors. If k convolution kernels are set, the vector dimension of the output result is k. By stacking one-dimensional convolution layers, the receptive field of the sequence relationship operation can be expanded, and the long-term correlation of the vector can be finally mined. This multi-layer stacked one-dimensional convolution network is a temporal convolution network.

[0075] See also Figure 1 In one embodiment of the invention, the electricity price prediction method based on multi-head attention mechanism and convolutional network of the present invention comprises the following steps:

[0076] Step S102, obtaining model training data, wherein the training data is time series data;

[0077] Step S104, inputting the model training data into the electricity price prediction algorithm model for training;

[0078] Step S106, obtaining the training results of the electricity price prediction and performing prediction value splicing to obtain the corresponding electricity price prediction results in multiple time periods.

[0079] It should be noted that, in this embodiment, the electricity price prediction algorithm model is an encoder-decoder structure, and the encoder and the decoder are both composed of a network layer connected by a multi-head attention mechanism and a temporal convolutional network. In practical applications, the output of the encoder can be used as the input of the decoder. By obtaining the model training data, it is input into the electricity price prediction algorithm model for training. After obtaining the training results, the predicted values ​​are spliced ​​to obtain the corresponding electricity price prediction results for multiple time periods.

[0080] Further, in one embodiment of the invention, if Figure 2 As shown, obtaining model training data includes the following steps:

[0081] Step S202, obtaining original data, wherein the original data includes power generation information and environmental information;

[0082] Step S204: Preprocess the original data using a preprocessing module to obtain the model training data.

[0083] It should be noted that, in this embodiment, in the electricity price prediction scenario, in addition to the electricity price data, the data set used for electricity price prediction also includes the original data, wherein the power generation information at least includes the power generation source, power generation cost, grid load and power generation data, the power generation source includes various energy sources (fossil fuels, wind energy, hydropower and nuclear energy, etc.), and the environmental information includes at least environmental attribute information such as weather type, air pressure information, temperature information and humidity information.

[0084] Further, after obtaining the original data, the original data needs to be preprocessed using a preprocessing module to obtain the model training data. In one embodiment of the invention, Figure 3 As shown, the data is preprocessed by using a preprocessing module based on the original data to obtain the model training data, which specifically includes:

[0085] Step S302, performing data cleaning based on the original data, which specifically includes removing missing values ​​and outliers, and aligning data items from different sources according to the collection time;

[0086] Step S304, concatenating the cleaned raw data, which specifically includes encoding discrete feature data to obtain continuous features, and the encoding method includes one-hot encoding;

[0087] Step S306, reducing the dimension of the original data after cleaning and information splicing based on the principal component analysis method;

[0088] Step S308, normalizing the original data after dimensionality reduction and the electricity price data to obtain the model training data.

[0089] It should be noted that in this embodiment, in order to turn the above raw data into trainable input in the neural network, the raw data needs to be preprocessed in advance, and the process is as follows: first, the power generation information and weather condition information in the raw data are cleaned, missing values ​​and outliers are removed, and the number of data entries from different sources is aligned according to the acquisition time, and then the two parts of information are spliced ​​together, and the attribute discrete features such as weather type are one-hot encoded to become continuous features suitable for neural networks. At this time, the data attributes are more and the feature dimensions are higher, which increases the complexity of the algorithm calculation, so the principal component analysis (PCA) and other methods are used for dimensionality reduction. Finally, the processed data is spliced ​​with the electricity price data, and normalized to the (0, 1) interval to match the neural network input, and finally the input time series data of the electricity price prediction algorithm model is obtained.

[0090] Specifically, after obtaining the model training data, the model training data needs to be input into the electricity price prediction algorithm model for training, such as Figure 4 As shown, the specific steps include:

[0091] Step S402, performing learning based on the encoder receiving the training data to obtain a first time domain autocorrelation parameter of the training data in the encoder;

[0092] Step S404, learning based on the target data in the training data received by the decoder to obtain a second time domain autocorrelation parameter of the training data in the decoder;

[0093] Step S406, performing cross-correlation calculation based on the first time domain autocorrelation parameter and the second time domain autocorrelation parameter to obtain a current period training result of the decoder;

[0094] Step S408, using the training result of the decoder in the current period as an input parameter of the decoder in the next period, and repeatedly iterating and calculating to obtain training results corresponding to multiple periods.

[0095] It should be noted that the electricity price prediction algorithm model described in the above embodiment is an encoder-decoder structure, which means that the algorithm model is based on the iterative output structure of the encoder-decoder. The encoder-decoder contains a multi-head attention mechanism layer and a time convolution network layer to learn the correlation between time series data. Specifically, Figure 5 As shown, it is a flowchart of the learning and training of the electricity price prediction algorithm model, wherein, based on the encoder receiving the training data, the autocorrelation of the time domain is preliminarily learned, and the initial input of the decoder is the last segment of data in the time series. The autocorrelation of the decoder input is first calculated, and then the cross-correlation with the decoder output is calculated. The output result of the decoder can be obtained as the electricity price prediction result of the current time period. At the same time, the set of prediction results of all previous time periods will be used as the input of the decoder in the prediction process of the next time period. The decoder learning process is repeated and iterated multiple times to finally obtain the electricity price prediction values ​​of multiple time periods.

[0096] Specifically, in one embodiment of the invention, Figure 6 As shown, based on the encoder receiving the training data for learning to obtain the first time domain autocorrelation parameter of the training data in the encoder, specifically includes the following steps:

[0097] Step S602, receiving the training data based on the encoder, and encoding it in combination with the position encoding to obtain an input vector of the multi-head self-attention mechanism;

[0098] Step S604, allocating the input vector to different representation spaces for parallel learning based on a multi-head self-attention mechanism;

[0099] Step S606: The learning result based on the multi-head self-attention mechanism is used as the input of the temporal convolutional network, and the first time domain autocorrelation parameter is obtained based on the output of the temporal convolutional network.

[0100] It should be noted that given the input time series data (i.e. training data) X = *x1,x2,...,x T}, which contains feature vectors of T time periods in total. First, position encoding information is introduced. The reason is that the multi-head attention mechanism is different from the traditional recurrent neural network. The input data does not flow in a certain order during the operation, which will lead to the loss of sequence information. Adding position encoding on the basis of the original sequence data is equivalent to adding the absolute position information of each feature vector, so that the multi-head attention model has the ability to extract the input order relationship. The mathematical formula of position encoding is as follows:

[0101]

[0102] Among them, pos represents the position of the feature vector in the sequence, and i represents the position of the target element in the feature vector. Sine coding is used for elements in even positions, and cosine coding is used for elements in odd positions. The offset of the coding function between different sequence positions can be approximated as a linear change.

[0103] The data after adding position encoding is first learned by the multi-head attention mechanism. The multi-head attention mechanism distributes the input vector to different representation spaces for parallel learning. In the multi-head attention layer, each node vector is split into h attention heads. After completing the calculation of each attention head, the results are concatenated. The overall learning process is expressed as:

[0104]

[0105] The encoder only involves learning the autocorrelation of time series data, and the input Q, K, and V vectors are all X. i Q , W i K , W i V , W O is a trainable weight matrix. In the actual learning process, the multi-head attention mechanism may have gradient vanishing or gradient exploding problems. Therefore, Add&Norm operations are often required after the multi-head attention calculation to improve learning efficiency. Figure 7As shown in the figure, Add is added to realize residual connection, and the multi-head attention calculation result is added to the original input to avoid the gradient vanishing problem. Norm is added to introduce layer normalization, which normalizes the mean and variance of each neuron input tensor, so that the data is scaled to the fastest convergence range.

[0106] The learning result of the multi-head attention mechanism is used as the input of the temporal convolutional network (TCN). The temporal convolutional network consists of two one-dimensional convolutional layers. Its design idea is to integrate adjacent temporal position vectors into the same receptive field and further explore the connection between local adjacent time periods through convolution operations. The process is defined as:

[0107] TCN(M) = ReLU(Conv1D (2) (M,w));

[0108] Where M is the learning result of the multi-head attention mechanism, w is the parameter of the one-dimensional convolutional layer, and the output of the multi-head attention mechanism and the temporal convolutional network needs to go through the Add&Norm operation. Its calculation process is as follows, and finally H is taken as the learning output of the encoder:

[0109]

[0110] Furthermore, the above-mentioned embodiment describes the learning based on the encoder receiving the training data to obtain the first time domain autocorrelation parameter of the training data in the encoder. In this embodiment, it will be described that the learning is based on the decoder receiving the target data in the training data to obtain the second time domain autocorrelation parameter of the training data in the decoder. Among them, since the basic model structure of multi-head attention mechanism - Add&Norm operation - temporal convolutional network - Add&Norm operation has been constructed in the encoder, this structure continues to be used in the decoder, and therefore the specific details are not repeated here.

[0111] Further, after obtaining the first time domain autocorrelation parameter and the second time domain autocorrelation parameter, a cross-correlation calculation needs to be performed to obtain the training result of the current period, such as Figure 8 As shown, the specific steps include:

[0112] Step S802, obtaining an input vector of a multi-head mutual attention mechanism in a decoder based on the first time-domain autocorrelation parameter and the second time-domain autocorrelation parameter;

[0113] Step S804, allocating the input vector to different representation spaces for parallel learning based on a multi-head mutual attention mechanism;

[0114] Step S806: The learning result based on the multi-head mutual attention mechanism is used as the input of the temporal convolutional network, and the training result of the current period is obtained based on the output of the temporal convolutional network.

[0115] It should be noted that the reference Figure 5 It can be seen that the first time-domain autocorrelation parameter of the encoder is used as the input vector of the second multi-head mutual attention mechanism on the decoder side. Correspondingly, the second time-domain autocorrelation parameter of the decoder is also used as the input vector of the second multi-head decoder on the decoder side. Then, based on the multi-head mutual attention mechanism, the input vector can be distributed to different representation spaces for parallel learning, and then the learning result of the multi-head mutual attention mechanism is used as the input of the temporal convolutional network, and the training result of the current period is obtained based on the output of the temporal convolutional network.

[0116] Furthermore, the above embodiment illustrates that the training results of the electricity price prediction are obtained and the prediction values ​​are spliced ​​to obtain the corresponding electricity price prediction results in multiple time periods. Specifically, the electricity price prediction values ​​P = *p1, p2, ..., p in the future multiple time periods can be realized through iterative calculation. t} output, where for the future time period n, the output forecast values ​​of each time period *p1,p2,...,p n-1}(The initial input is the last segment x in the original time series data T ) is used as the decoder input and position encoding is added. First, autocorrelation learning is performed, and the learning result is used as the K, V vector of the multi-head mutual attention mechanism, while the Q vector takes the encoder output H. The mutual attention mechanism is used to learn the correlation between the historical data sequence and the future prediction value, and the corresponding output result is taken as the output of the decoder, which is the electricity price prediction value p under this step. n-1 , the calculation process of the decoder is iterated n times, and the predicted values ​​of each time can be concatenated to obtain the target output P.

[0117] An embodiment of the present application also provides an electricity price prediction system based on a multi-head attention mechanism and a convolutional network. The electricity price prediction system based on a multi-head attention mechanism and a convolutional network can implement the electricity price prediction method based on a multi-head attention mechanism and a convolutional network described in the present application. However, the implementation device of the electricity price prediction method based on a multi-head attention mechanism and a convolutional network described in the present application includes but is not limited to the structure of the electricity price prediction system based on a multi-head attention mechanism and a convolutional network listed in the present embodiment. All structural deformations and replacements of the prior art made according to the principles of the present application are included in the protection scope of the present application.

[0118] See also Fig. 9 In one embodiment, this embodiment provides an electricity price prediction system 90 based on a multi-head attention mechanism and a convolutional network, the system comprising:

[0119] An acquisition module 91 is used to acquire model training data, wherein the training data is time series data;

[0120] A training module 92 is used to input the model training data into the electricity price prediction algorithm model for training, wherein the electricity price prediction algorithm model is an encoder-decoder structure, and the encoder and the decoder are both composed of a network layer connected by a multi-head attention mechanism and a temporal convolutional network;

[0121] The prediction module 93 is used to obtain the training results of the electricity price prediction and perform prediction value splicing to obtain the corresponding electricity price prediction results in multiple time periods.

[0122] Since the specific implementation of this embodiment corresponds to the aforementioned method embodiment, the same details will not be repeated here, and those skilled in the art should also understand that Fig. 9 The division of the modules in the embodiment is merely a division of logical functions, and in actual implementation, all or part of them can be integrated into one or more physical entities, and these modules can be implemented entirely in the form of software called by processing elements, or entirely in the form of hardware, or some modules can be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware.

[0123] See also Fig.10 This embodiment provides an electronic device. Specifically, the electronic device includes at least a memory and a processor connected via a bus, wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to execute all or part of the steps in the aforementioned method embodiment.

[0124] In summary, the present invention performs a series of preprocessing processes such as cleaning, feature extraction, dimensionality reduction, and normalization on historical electricity-related data, and inputs the obtained data into an electricity price prediction algorithm model based on a multi-head attention mechanism and a temporal convolutional network, so as to calculate the accurate prediction values ​​of electricity prices for several time periods in the future.

[0125] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.

[0126] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0127] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0128] The present application embodiment also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the method for implementing the above embodiment can be completed by a program to instruct the processor, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid-state hard disk (SSD)), etc.

[0129] The present application embodiment may also provide a computer program product, the computer program product including one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer or data center.

[0130] When the computer program product is executed by a computer, the computer executes the method described in the above method embodiment. The computer program product may be a software installation package, and when the above method is required, the computer program product may be downloaded and executed on a computer.

[0131] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0132] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A method for predicting electricity prices based on a multi-head attention mechanism and a convolutional network, characterized in that: include: Acquire model training data, wherein the training data is time series data; Inputting the model training data into the electricity price prediction algorithm model for training, wherein the electricity price prediction algorithm model is an encoder-decoder structure, and the encoder and the decoder are both composed of a network layer connected by a multi-head attention mechanism and a temporal convolutional network; The training results of electricity price prediction are obtained and the predicted values ​​are spliced ​​to obtain the corresponding electricity price prediction results under multiple time periods.

2. The electricity price prediction method based on multi-head attention mechanism and convolutional network according to claim 1 is characterized in that: The obtaining of model training data specifically includes: Acquire raw data, the raw data including power generation information and environmental information, wherein the power generation information at least includes power generation source, power generation cost and grid load, and the environmental information at least includes weather type, air pressure information, temperature information and humidity information; The model training data is obtained by performing data preprocessing using a preprocessing module based on the original data.

3. The electricity price prediction method based on multi-head attention mechanism and convolutional network according to claim 2 is characterized in that: The method of performing data preprocessing based on the original data using a preprocessing module to obtain the model training data specifically includes: Performing data cleaning based on the raw data, including removing missing values ​​and outliers, and aligning data items from different sources according to collection time; The cleaned raw data is concatenated, which specifically includes encoding discrete feature data to obtain continuous features, and the encoding method includes one-hot encoding; Based on the principal component analysis method, the original data after cleaning and information splicing is reduced in dimension; The original data after dimensionality reduction and the electricity price data are normalized to obtain the model training data.

4. The electricity price prediction method based on multi-head attention mechanism and convolutional network according to claim 1 is characterized in that: The inputting the model training data into the electricity price prediction algorithm model for training specifically includes: The encoder receives the training data for learning to obtain a first time domain autocorrelation parameter of the training data in the encoder; Based on the target data in the training data received by the decoder, learning is performed to obtain a second time domain autocorrelation parameter of the training data in the decoder; Performing cross-correlation calculation based on the first time domain autocorrelation parameter and the second time domain autocorrelation parameter to obtain a current period training result of the decoder; The training result of the current time period of the decoder is used as the input parameter of the decoder in the next time period, and the training results corresponding to multiple time periods are obtained by repeated iterative calculation.

5. The electricity price prediction method based on multi-head attention mechanism and convolutional network according to claim 4 is characterized in that: The receiving the training data based on the encoder for learning to obtain a first time domain autocorrelation parameter of the training data in the encoder specifically includes: The training data is received based on the encoder, and the input vector of the multi-head self-attention mechanism is obtained by encoding in combination with the position encoding, wherein sine encoding is used for elements at even positions, and cosine encoding is used for elements at odd positions; Based on a multi-head self-attention mechanism, the input vector is distributed to different representation spaces for parallel learning; The learning result based on the multi-head self-attention mechanism is used as the input of the temporal convolutional network, and the first time domain autocorrelation parameter is obtained based on the output of the temporal convolutional network.

6. The electricity price prediction method based on multi-head attention mechanism and convolutional network according to claim 5 is characterized in that: The learning based on receiving the target data in the training data by the decoder to obtain the second time domain autocorrelation parameter of the training data in the decoder specifically includes: Based on the training data or historical training results received by the decoder, encoding is performed in combination with position encoding to obtain an input vector of the multi-head self-attention mechanism, wherein sine encoding is used for elements at even positions, and cosine encoding is used for elements at odd positions; Based on a multi-head self-attention mechanism, the input vector is distributed to different representation spaces for parallel learning; The learning result based on the multi-head self-attention mechanism is used as the input of the temporal convolutional network, and the second time domain autocorrelation parameter is obtained based on the output of the temporal convolutional network.

7. The electricity price prediction method based on multi-head attention mechanism and convolutional network according to claim 6 is characterized in that: The performing cross-correlation calculation based on the first time domain autocorrelation parameter and the second time domain autocorrelation parameter to obtain the current period training result of the decoder specifically includes: Obtaining an input vector of a multi-head mutual attention mechanism in a decoder based on the first time-domain autocorrelation parameter and the second time-domain autocorrelation parameter; Based on a multi-head mutual attention mechanism, the input vector is distributed to different representation spaces for parallel learning; The learning result based on the multi-head mutual attention mechanism is used as the input of the temporal convolutional network, and the training result of the current period is obtained based on the output of the temporal convolutional network.

8. An electricity price prediction system based on multi-head attention mechanism and convolutional network, characterized in that: include: An acquisition module, used to acquire model training data, wherein the training data is time series data; A training module, used for inputting the model training data into the electricity price prediction algorithm model for training, wherein the electricity price prediction algorithm model is an encoder-decoder structure, and the encoder and the decoder are both composed of a network layer connected by a multi-head attention mechanism and a temporal convolutional network; The prediction module is used to obtain the training results of electricity price prediction and perform prediction value splicing to obtain the corresponding electricity price prediction results under multiple time periods.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the electricity price prediction method based on a multi-head attention mechanism and a convolutional network as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; wherein the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the electricity price prediction method based on a multi-head attention mechanism and a convolutional network as described in any one of claims 1 to 7.

Images