Power consumption prediction method and system based on attention mechanism fusion frequency enhancement

Abstract

This application discloses a method, system, device, and medium for long-term power consumption prediction based on an attention mechanism and frequency enhancement. A deep network structure model is constructed based on the attention mechanism and frequency enhancement. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected sequentially. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module. Power consumption is predicted based on this deep network structure model. The power consumption prediction method provided by this invention incorporates an attention mechanism into the Transformer neural network to address the characteristics of power consumption data, making the model's prediction results more stable. The frequency-enhanced hybrid attention module enhances the model's noise resistance in the frequency domain, thereby improving the model's feature learning ability and the accuracy of long-term prediction of sequence data. An iterative prediction strategy is adopted, making multiple predictions to achieve long-term prediction.

Description

Technical Field

[0001] This application relates to the field of power consumption prediction technology, and in particular to a power consumption prediction method, system, device and medium based on attention mechanism fusion frequency enhancement. Background Technology

[0002] For power consumption prediction methods, time series prediction methods are generally used, including both traditional methods and deep learning methods.

[0003] Traditional methods use mathematical statistics and other techniques to predict future values ​​of data. Examples include autoregressive (AR) models, moving average (MA) models, and autoregressive moving average (ARMA) models. However, these models are only suitable for data with stable distributions. The ARIMA model combines differencing methods with previous models, requiring only that the data after multiple differencing be stable for long-term predictions, but this is still a demanding requirement for the data.

[0004] Deep learning methods employ multi-layered nonlinear operators to predict future values. Artificial Neural Networks (ANNs) and Convolutional Neural Networks (CNNs) lack good interpretability and do not fully consider the temporal correlations between time series, thus their results are difficult to gain widespread acceptance. Recurrent Neural Networks (RNNs) models reuse parameters within the neural network, inputting time series data in chronological order. This considers the temporal relationships of the time series and provides the neural network with continuous input data and parameter updates. However, RNN models overemphasize trend prediction for time series data. The parameters in an RNN network change with the input sequence, making the model more sensitive to the latest input data and less dependent on historical data. However, many time series data not only exhibit short-term trends but also long-term periodicity. Long Short-Term Memory (LSTM) models independently store long-term and short-term features from historical data through gating units, thereby generating prediction results with both short-term trends and long-term periodicity. Bidirectional Recurrent Neural Networks (Bi-RNNs) can be used to capture the influence of subsequent data on preceding data in text sequence data. The gated recurrent unit (GRU) optimizes the structure of LSTM, reducing the computational cost per node.

[0005] Decision-making in power grid systems requires long-term and accurate power consumption forecasts as support. Therefore, how to predict power consumption stably over the long term is an urgent problem to be solved. However, traditional time series forecasting methods have low accuracy, and deep learning methods lack stability. Summary of the Invention

[0006] This application provides a long-term power consumption prediction method based on attention mechanism fusion frequency enhancement to solve the technical problem that the stability and accuracy of existing power consumption prediction methods cannot be balanced, thereby improving the accuracy and stability of long-term power consumption prediction.

[0007] To address the aforementioned technical problems, in a first aspect, this application provides a long-term power consumption prediction method based on attention mechanism fusion frequency enhancement, the method comprising: Historical power transformer data is obtained, and the historical power transformer data is processed to obtain a sample dataset; A deep network structure model is constructed based on the integration of frequency enhancement with an attention mechanism. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module. After the sample dataset is positionally encoded, a first sequence data is obtained. The first sequence data is then input into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain first multivariate data. The first multivariate data is input into the frequency-enhanced hybrid attention module. The first multivariate data is subjected to self-attention processing and frequency-enhanced channel attention processing respectively, and then weighted averaged to output the second multivariate data. Obtain the power transformer data to be predicted, obtain the label data from the sample dataset, combine the label data and the power transformer data to be predicted and obtain the second sequence data through position encoding; The second sequence data and the second multivariate data are input into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0008] Preferably, encoding the sample dataset into a first sequence of data includes: The sample dataset is input into the location encoding module, which embeds location information into each data point in the sample dataset to obtain the first sequence of data.

[0009] Preferably, the first sequence data is subjected to multi-head attention and feedforward neural network learning to obtain first multivariate data, including: After the first sequence data is input into the first multi-head attention module for calculation, residual linking and regularization are performed to obtain the first attention level of each position in the first sequence data relative to other positions. Based on the first level of attention, the first sequence data is input into the first feedforward neural network for calculation, and then residual linking and regularization are performed to obtain the first multivariate data.

[0010] Preferably, the first multi-head attention module includes 8 sets of combined parameter matrices, and each set of combined parameter matrices includes three parameter matrices; The first feedforward neural network includes two feedforward neural network layers, and the number of neurons in the feedforward neural network layers is the same as the length of the first sequence data.

[0011] Preferably, the first multivariate data is subjected to self-attention processing and frequency-enhanced channel attention processing, including: The first multivariate data is input into the self-attention module to obtain the correlation relationship of the data sequences in the first multivariate data; The global average pooling operation in the channel attention mechanism is replaced by discrete cosine transform to obtain a frequency-enhanced channel attention module; The first multivariate data is input into the frequency enhancement channel attention module to obtain the correlation between different features in the first multivariate data.

[0012] Preferably, the output of the second multivariate data includes: The second multivariate data is obtained by weighted averaging the correlation between the data sequences and the correlation between the different features.

[0013] Preferably, the step of inputting the second sequence data and the second multivariate data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted includes: After the second sequence data is input into the second multi-head attention module for calculation, residual linking and regularization are performed to obtain the second attention level of each position in the second sequence data relative to other positions; Based on the second attention, the second sequence data and the second multivariate data are input into the third multi-head attention module for calculation, and then residual linking and regularization are performed to obtain the third attention of the sequence data corresponding to the power transformer data to be predicted relative to other positions in the first time series data and the second time series data. Based on the third level of attention, the second time series data and the second multivariate data are input into the second feedforward neural network for calculation, and then residual linking and regularization are performed to obtain the third multivariate data. The third multivariate data is input into the fully connected layer and the softmax layer for calculation to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0014] Secondly, this application also provides a long-term power consumption prediction system based on attention mechanism fusion frequency enhancement, the system comprising: a sample data acquisition unit, a model building unit, an encoding processing unit, a frequency enhancement fusion attention processing unit, a sequence data acquisition unit to be predicted unit, and a decoding processing unit; The sample data acquisition unit is used to acquire historical power transformer data and process the historical power transformer data to obtain a sample dataset. The model building unit is used to construct a deep network structure model based on the attention mechanism and frequency enhancement. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module. The encoding processing unit is used to encode the sample dataset by position to obtain the first sequence data, and input the first sequence data into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain the first multivariate data. The frequency-enhanced fusion attention processing unit is used to input the first multivariate data into the frequency-enhanced hybrid attention module, perform self-attention processing and frequency-enhanced channel attention processing on the first multivariate data respectively, and then perform a weighted average to output the second multivariate data. The data acquisition unit for the sequence to be predicted is used to acquire data of the power transformer to be predicted, acquire label data from the sample dataset, combine the label data and the data of the power transformer to be predicted and obtain the second sequence data through position encoding; The decoding processing unit is used to input the second sequence data and the second multi-element data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0015] Thirdly, this application also provides a computer device, which includes a memory, a processor, and a transceiver connected to each other via a bus; the memory is used to store a set of computer program instructions and data, and to transmit the stored data to the processor, and the processor executes the program instructions stored in the memory to perform the method described above.

[0016] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed, implements the method described above.

[0017] This application provides a method, system, device, and medium for predicting power consumption based on attention mechanism and frequency enhancement. The method involves acquiring historical power transformer data, processing the historical power transformer data to obtain a sample dataset, constructing a deep network structure model based on attention mechanism and frequency enhancement, and the deep network structure model including an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module. The sample dataset is position-encoded to obtain a first sequence data. The first sequence data is input to the encoder for multi-head attention and feedforward neural network learning to obtain first multivariate data. The first multivariate data is input to the frequency-enhanced hybrid attention module, where self-attention processing and frequency-enhanced channel attention processing are performed separately, followed by weighted averaging to output second multivariate data. Power transformer data to be predicted is acquired, and label data is obtained from the sample dataset. The label data and the power transformer data to be predicted are combined and position-encoded to obtain the second sequence data. The second sequence data and the second multivariate data are input to the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted. This invention provides a power consumption prediction method based on attention mechanism and frequency enhancement. It incorporates an attention mechanism into the Transformer neural network to address the characteristics of power consumption data, making the model's prediction results more stable. The frequency-enhanced hybrid attention module strengthens the model's noise resistance in the frequency domain, thereby improving the model's feature learning ability and the accuracy of long-term prediction of sequence data. An iterative prediction strategy is employed, achieving long-term prediction through multiple predictions. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the steps of a power consumption prediction method based on attention mechanism fusion frequency enhancement provided in a preferred embodiment of this application; Figure 2 This is a schematic diagram of a deep network structure model based on attention mechanism fusion frequency enhancement provided in a preferred embodiment of this application; Figure 3 This is a schematic diagram of the structure of FAM provided in a preferred embodiment of this application; Figure 4 This is a schematic diagram of the steps of a power consumption decoding and prediction method provided in a preferred embodiment of this application; Figure 5 This is a schematic diagram of a power consumption prediction system based on attention mechanism fusion frequency enhancement provided in a preferred embodiment of this application; Figure 6 This is a schematic diagram of a computer device provided in a preferred embodiment of this application. Detailed Implementation

[0019] The embodiments of this application are described in detail below with reference to the accompanying drawings. The embodiments are provided for illustrative purposes only and should not be construed as limiting the scope of this application. The accompanying drawings are for reference and illustration only and do not constitute a limitation on the scope of patent protection of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] To address the technical problem of the inability to simultaneously achieve stability and accuracy in existing power consumption prediction methods, this application provides a power consumption prediction method based on attention mechanism fusion frequency enhancement, thereby improving the accuracy and stability of long-term power consumption prediction.

[0021] Please see Figure 1 In an embodiment of this application, a long-term power consumption prediction method based on attention mechanism fusion frequency enhancement is provided, the method comprising: S1. Obtain historical power transformer data, and process the historical power transformer data to obtain a sample dataset.

[0022] S2. Construct a deep network structure model based on the attention mechanism and frequency enhancement. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module.

[0023] S3. After the sample dataset is positionally encoded, a first sequence data is obtained. The first sequence data is then input into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain first multivariate data.

[0024] S4. Input the first multivariate data into the frequency-enhanced hybrid attention module, perform self-attention processing and frequency-enhanced channel attention processing on the first multivariate data respectively, and then perform weighted averaging to output the second multivariate data.

[0025] S5. Obtain the power transformer data to be predicted, obtain the label data from the sample dataset, combine the label data and the power transformer data to be predicted and obtain the second sequence data through position encoding.

[0026] S6. Input the second sequence data and the second multivariate data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0027] In this application, as Figure 2As shown, a deep network structure model based on a deep learning network model and incorporating frequency-enhanced attention (FEHAT) is constructed. This deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The encoder includes a first multi-head attention module, a first feedforward neural network, and a residual connection and regularization module, used to convert sequence data of length n into multi-dimensional data of dimension m. For computational efficiency, m is generally set to an integer power of 2; in this application, m is 512.

[0028] To obtain historical power transformer data, this application uses two power stations in two different regions of the same province in China as sampling points to collect two years of data on power transformers at the power stations. The data includes the recording date of the data points, oil temperature, and six different types of external load values.

[0029] Historical power transformer data is preprocessed to remove outliers. The length of the time step is obtained according to different time resolutions. In this application, 96 historical power transformer data points are used as the sample dataset at a 15-minute resolution, i.e., 96 × 15 minutes, which is 4 days.

[0030] In this application, 96 consecutive historical power transformer data points are used as a sample dataset. The sample dataset is input into a location encoding module for location encoding, embedding location information into each data point to obtain a first sequence of data. The length of the first sequence of data is set to n, which is 96 in this application. In practical applications, n can be set according to specific needs.

[0031] The first sequence data is input into the encoder. First, the first sequence data is input into the first multi-head attention module. The multi-head attention modules involved in this application are all used to calculate the attention level of each position in the sequence to other positions, and each includes 8 sets of combined parameter matrices. Each set of combined parameter matrices includes three parameter matrices, named respectively. , , The dimensions of these three parameter matrices are all m×n. Multiplying the first sequence of data by each of the three parameter matrices yields Q (query matrix), K (key matrix), and V (value matrix). Based on Q, K, and V, the attention of the current query matrix to the historical data in the first sequence of data is calculated using the following formula: ( )V.

[0032] Furthermore, residual linking and regularization are performed on the calculation results of the first multi-head attention module to obtain the first attention level of each position in the first sequence data relative to other positions.

[0033] Next, based on the initial level of attention, the first sequence of data is input into the first feedforward neural network. This first feedforward neural network consists of two feedforward layers, each with the same number of neurons as the length of the first sequence of data. The internal parameter matrices of these two feedforward neural network layers are of dimension n×n. This design facilitates the subsequent linking of residuals, ensuring that each layer of the encoder generates an output vector of dimension n.

[0034] Furthermore, residual linking and regularization are applied to the computation results of the first feedforward neural network to obtain the first multivariate data of dimension m.

[0035] In this application, a frequency-enhanced hybrid attention module (FAM) is also included between the decoder and encoder, such as... Figure 3 The diagram shows the structure of the FAM module, which reduces the frequency domain uncertainty introduced by the encoder to the first time-series data. The core task of the FAM module is to perform a more detailed and comprehensive model of the first time-series data by fusing channel attention and self-attention mechanisms. Channel attention helps capture the correlations between different features, while self-attention helps capture the dependencies within the sequence. By combining these two attention methods, the FAM module can gain a more comprehensive understanding of the intrinsic structure of the time-series data.

[0036] Traditional attention mechanisms typically use global average pooling as a standard compression method. However, simply applying global average pooling in the time dimension can lead to insufficient information extraction, or even information loss. In time series tasks, global average pooling focuses more on the low-frequency space, resulting in insufficient or even missing information in the high-frequency space. To enhance information extraction capabilities in the frequency domain, Discrete Cosine Transform (DCT) is introduced to replace global average pooling in channel attention mechanisms. DCT can represent high-frequency information well while concentrating energy in the low-frequency region, and preserves the channel self-attention mechanism to capture information in the low-frequency domain. By combining the self-attention mechanism with the frequency-enhanced channel attention mechanism, the model's ability to simultaneously acquire effective temporal information in both low-frequency and high-frequency spaces is enhanced. The frequency-enhanced hybrid attention module can significantly improve the model's ability to model time series data, especially in long-term prediction tasks involving electricity consumption, because frequency domain information often contains key periodic features that can provide strong support for long-term prediction.

[0037] Furthermore, a weighted average is performed on the correlation between the data sequences and the correlation between the different features to obtain the second multivariate data.

[0038] In this application, the second sequence data and the second multivariate data are input into a decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted, such as... Figure 4 As shown, it includes the following steps: S601. After inputting the second sequence data into the second multi-head attention module for calculation, residual linking and regularization are performed to obtain the second attention level of each position in the second sequence data relative to other positions.

[0039] S602. Based on the second attention, the second sequence data and the second multivariate data are input into the third multi-head attention module for calculation, and residual linking and regularization are performed to obtain the third attention of the sequence data corresponding to the power transformer data to be predicted relative to other positions in the first time series data and the second time series data.

[0040] S603. Based on the third level of attention, the second time series data and the second multivariate data are input into the second feedforward neural network for calculation, and then residual linking and regularization are performed to obtain the third multivariate data.

[0041] S604. Input the third multivariate data into the fully connected layer and the softmax layer for calculation to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0042] In this application, the decoder and encoder have similar structures. During the decoding process, a masked multi-head attention operation needs to be performed on the decoder's input, which is implemented through a second multi-head attention module. The decoder's input includes not only second multivariate data but also second sequence data. This second sequence data is obtained by combining the power transformer data to be predicted with a portion of historical power transformer data from the sample dataset. In this application, the last 48 historical power transformer data from the sample dataset are used as label data, and the label data is combined with the power transformer data to be predicted to form the second sequence data.

[0043] The second sequence of data is input into a second multi-head attention module. The matrix dimension of the second multi-head attention module is the same as the length of the second sequence of data, and it records the positional information of the label data and the data to be predicted. In this matrix of the second multi-head attention module, the elements corresponding to the label data are set to 1, while the elements corresponding to the prediction positions are set to 0. The purpose of this matrix is ​​to restrict the model to using only the label data for prediction during the decoding process, thereby ensuring the accuracy of the prediction.

[0044] After the second sequence data is processed by the second multi-head attention module, residual linking and regularization are performed to obtain the second attention level of each position in the second sequence data relative to other positions.

[0045] Based on the second degree of attention, the second sequence data and the second multivariate data output by the decoder are input into the third multi-head attention module for calculation. The calculation results are subjected to residual linking and regularization to obtain the third degree of attention of the sequence data corresponding to the power transformer data to be predicted relative to other positions in the first time series data and the second time series data.

[0046] The multi-head attention module calculation formula for the decoder is the same as that for the encoder, but the W obtained from the encoder is used in the decoder. Query W Key and W Value The training results of the parameters.

[0047] Based on the third level of attention, the second time series data and the second multivariate data are input into the second feedforward neural network for calculation. The calculation results are then subjected to residual linking and regularization to obtain the third multivariate data.

[0048] Finally, the third multivariate data is input into the fully connected layer and the softmax layer for calculation to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0049] In this application, as shown in Table 1, the components of the variables and predictive variables are used. Recording time and six different types of external load values ​​are used as factors for power consumption prediction, and transformer oil temperature is used as the prediction target. Transformer oil temperature can effectively reflect the operating condition of the power transformer. By predicting the transformer oil temperature, the ultimate load capacity of the power transformer can be predicted, while simultaneously avoiding unnecessary waste.

[0050] Table 1. Composition of variables and predictor variables

[0051] In the traditional Transformer architecture, the length of the data to be predicted is the same as the length of the training data, i.e., both are n. However, when the required prediction length exceeds n, traditional models cannot handle it directly. To solve this problem, an iterative prediction strategy is adopted. The decoder can automatically extract the corresponding length from the power transformer data to be predicted based on the length n and perform iterative prediction. Through a sliding window operation, multiple prediction results of length n steps are concatenated to obtain the prediction result with the desired prediction step length, enabling long-term prediction of power consumption.

[0052] The model was evaluated through experiments and compared with state-of-the-art methods, including the traditional time series forecasting method ARIMA and deep learning-based methods such as LSTM, ConvLSTM, Transformer, and Informer. The mean squared error (MSE) and mean absolute error (MAE) were primarily used as the evaluation metrics for model prediction performance; smaller error values ​​indicate better model performance.

[0053] The model was built using an NVIDIA GTX 3090Ti GPU and the code was developed using Python and the PyTorch framework. The model uses the Adam optimizer with an initial learning rate of approximately 0.0001, which changes with the number of iterations using the following formula: , Here, `step` represents the current training iteration, and `epoch` represents the total number of training iterations. Since the dataset is small, a total of 100 iterations are performed in this iteration.

[0054] The dataset used is a publicly available open-source power transformer dataset, which contains 6 variables and 1 prediction variable. The dataset is sampled every 15 minutes and contains 69,680 training sample points, which are divided into training and test sets in a 7:3 ratio. Table 2 shows the comparative experimental results of the 6 methods. It can be seen that the deep network structure model (FEHAT) proposed in this application outperforms the other models in both error types on the power consumption prediction dataset, achieving good prediction results.

[0055] Table 2. Comparative experimental results of the six methods

[0056] As shown in Table 2, 1) When the original data changes, the ARIMA model continues to use the previous distribution for prediction, thus the error gradually accumulates and increases. When the original data remains stable, ARIMA can achieve better results. LSTM has lower error at shorter prediction step lengths, but as the step length increases, the error accumulates continuously, resulting in a higher final error value. LSTM forms positive feedback on the result during prediction, causing the prediction distribution to change too quickly, resulting in poor final results. The Transformer model maintains lower error due to its spatiotemporal attention mechanism. The Informer model does not perform as well as the Transformer at short prediction step lengths, but as the prediction step length increases, its advantage in extracting long-period features gradually becomes apparent, and the error growth is more stable. 2) Compared with the Transformer model without the FAM module, the network proposed in this paper achieves better results, proving the effectiveness of the FAM module. 3) The deep network structure model (FEHAT) provided in this application shows the best performance at various prediction step lengths, and the loss value increases the least in long-term prediction. Compared with all the comparison methods, the long-term power consumption prediction method based on frequency enhancement fusion attention mechanism provided by the method itself has excellent prediction performance and is particularly suitable for long-term prediction tasks of user power consumption.

[0057] The power consumption prediction method based on frequency enhancement fusion based on attention mechanism proposed in this application involves: acquiring historical power transformer data; processing the historical power transformer data to obtain a sample dataset; constructing a deep network structure model based on frequency enhancement fusion based on attention mechanism; the deep network structure model includes an encoder, a frequency enhancement hybrid attention module, and a decoder connected in sequence; the frequency enhancement hybrid attention module includes a self-attention module and a frequency enhancement channel attention module; encoding the sample dataset by position to obtain a first sequence data; inputting the first sequence data into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain first multivariate data; inputting the first multivariate data into the frequency enhancement hybrid attention module, performing self-attention processing and frequency enhancement channel attention processing on the first multivariate data respectively, and then performing a weighted average to output second multivariate data; acquiring power transformer data to be predicted; obtaining label data from the sample dataset; combining the label data and the power transformer data to be predicted and encoding by position to obtain the second sequence data; inputting the second sequence data and the second multivariate data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted. This invention provides a power consumption prediction method based on attention mechanism and frequency enhancement. It incorporates an attention mechanism into the Transformer neural network to address the characteristics of power consumption data, making the model's prediction results more stable. The frequency-enhanced hybrid attention module strengthens the model's noise resistance in the frequency domain, thereby improving the model's feature learning ability and the accuracy of long-term prediction of sequence data. An iterative prediction strategy is employed, achieving long-term prediction through multiple predictions.

[0058] Accordingly, such as Figure 5 As shown, according to the power consumption prediction method based on attention mechanism fusion frequency enhancement, this embodiment of the invention also provides a power consumption prediction system based on attention mechanism fusion frequency enhancement. The system includes: a sample data acquisition unit 1, a model building unit 2, an encoding processing unit 3, a frequency enhancement fusion attention processing unit 4, a sequence data acquisition unit 5, and a decoding processing unit 6. The sample data acquisition unit 1 is used to acquire historical power transformer data and process the historical power transformer data to obtain a sample dataset.

[0059] The model building unit 2 is used to build a deep network structure model based on the attention mechanism and frequency enhancement. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module.

[0060] The encoding processing unit 3 is used to encode the sample dataset by position to obtain the first sequence data, and input the first sequence data into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain the first multivariate data.

[0061] The frequency-enhanced fusion attention processing unit 4 is used to input the first multivariate data into the frequency-enhanced fusion attention module, perform self-attention processing and frequency-enhanced channel attention processing on the first multivariate data respectively, and then perform a weighted average to output the second multivariate data.

[0062] The data acquisition unit 5 for the sequence to be predicted is used to acquire data of the power transformer to be predicted, acquire label data from the sample dataset, combine the label data and the data of the power transformer to be predicted and obtain the second sequence data through position encoding.

[0063] The decoding processing unit 6 is used to input the second sequence data and the second multi-element data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

[0064] For specific limitations regarding a long-term power consumption prediction system based on a frequency-enhanced fusion attention mechanism, please refer to the above-described limitations regarding a long-term power consumption prediction method based on a frequency-enhanced fusion attention mechanism, which will not be repeated here. Those skilled in the art will recognize that the various modules and steps described in conjunction with the embodiments disclosed in this application can be implemented in hardware, software, or a combination of both. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0065] like Figure 6 As shown in the figure, an embodiment of the present invention provides a computer device including a memory, a processor, and a transceiver, which are connected to each other via a bus; the memory is used to store a set of computer program instructions and data, and transmits the stored data to the processor; the processor executes the program instructions stored in the memory to perform the steps of the above-described long-term power consumption prediction method based on attention mechanism fusion frequency enhancement.

[0066] The memory may include volatile memory or non-volatile memory, or both; the processor may be a central processing unit, a microprocessor, an application-specific integrated circuit, a programmable logic device, or a combination thereof. By way of example, but not limitation, the programmable logic device described above may be a complex programmable logic device, a field-programmable gate array, a general-purpose array logic, or any combination thereof.

[0067] In addition, memory can be a physically independent unit or integrated with the processor.

[0068] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have the same component arrangement.

[0069] In one embodiment, a computer-readable storage medium is provided for storing one or more computer programs, the one or more computer programs including program code, which, when run on a computer, is used to perform the steps of the above-described attention mechanism-based frequency enhancement-based long-term power consumption prediction method.

[0070] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD), etc.

[0071] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed, it can include the processes of the embodiments of the above methods.

[0072] This embodiment provides a method, system, computer device, and storage medium for long-term power consumption prediction based on attention mechanism fusion frequency enhancement, which addresses the technical problem that existing power consumption prediction methods cannot simultaneously achieve both stability and accuracy. The method of this application includes: acquiring historical power transformer data; processing the historical power transformer data to obtain a sample dataset; constructing a deep network structure model based on attention mechanism and frequency enhancement; the deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence; the frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module; encoding the sample dataset by position to obtain a first sequence data; inputting the first sequence data into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain first multivariate data; inputting the first multivariate data into the frequency-enhanced hybrid attention module; performing self-attention processing and frequency-enhanced channel attention processing on the first multivariate data respectively, and then performing weighted averaging to output second multivariate data; acquiring power transformer data to be predicted; acquiring label data from the sample dataset; combining the label data and the power transformer data to be predicted and encoding by position to obtain second sequence data; inputting the second sequence data and the second multivariate data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted. This invention provides a power consumption prediction method based on attention mechanism and frequency enhancement. It incorporates an attention mechanism into the Transformer neural network to address the characteristics of power consumption data, making the model's prediction results more stable. The frequency-enhanced hybrid attention module strengthens the model's noise resistance in the frequency domain, thereby improving the model's feature learning ability and the accuracy of long-term prediction of sequence data. An iterative prediction strategy is employed, achieving long-term prediction through multiple predictions.

[0073] The embodiments described above are merely preferred embodiments of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the technical principles of this invention, and these improvements and substitutions should also be considered within the scope of protection of this application. Therefore, the scope of protection of this patent application should be determined by the scope of the claims.

Claims

1. A power consumption prediction method based on attention mechanism fusion frequency enhancement, characterized in that, The method includes: Historical power transformer data is obtained, and the historical power transformer data is processed to obtain a sample dataset; A deep network structure model is constructed based on the integration of frequency enhancement with an attention mechanism. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module. After the sample dataset is positionally encoded, a first sequence data is obtained. The first sequence data is then input into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain first multivariate data. The first multivariate data is input into the frequency-enhanced hybrid attention module. The first multivariate data is subjected to self-attention processing and frequency-enhanced channel attention processing respectively, and then weighted averaged to output the second multivariate data. The first multivariate data is subjected to self-attention processing and frequency-enhanced channel attention processing, including: The first multivariate data is input into the self-attention module to obtain the correlation relationship of the data sequences in the first multivariate data; The global average pooling operation in the channel attention mechanism is replaced by discrete cosine transform to obtain a frequency-enhanced channel attention module; The first multivariate data is input into the frequency enhancement channel attention module to obtain the correlation between different features in the first multivariate data; The output of the second multivariate data includes: The second multivariate data is obtained by weighting the correlation between the data sequences and the correlation between the different features. Obtain the power transformer data to be predicted, extract the historical power transformer data from the sample dataset as label data, combine the label data and the power transformer data to be predicted and obtain the second sequence data through location encoding; The second sequence data and the second multivariate data are input into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted; The step of inputting the second sequence data and the second multivariate data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted includes: After the second sequence data is input into the second multi-head attention module for calculation, residual linking and regularization are performed to obtain the second attention level of each position in the second sequence data relative to other positions; Based on the second attention, the second sequence data and the second multivariate data are input into the third multi-head attention module for calculation, and then residual linking and regularization are performed to obtain the third attention of the sequence data corresponding to the power transformer data to be predicted relative to other positions in the first sequence data and the second sequence data. Based on the third level of attention, the second sequence data and the second multivariate data are input into the second feedforward neural network for calculation, and then residual linking and regularization are performed to obtain the third multivariate data. The third multivariate data is input into the fully connected layer and the softmax layer for calculation to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

2. The power consumption prediction method based on attention mechanism fusion frequency enhancement as described in claim 1, characterized in that, Encoding the sample dataset into a first sequence of data includes: The sample dataset is input into the location encoding module, which embeds location information into each data point in the sample dataset to obtain the first sequence of data.

3. The power consumption prediction method based on attention mechanism fusion frequency enhancement as described in claim 1, characterized in that, Multi-head attention and feedforward neural network learning are performed on the first sequence data to obtain the first multivariate data, including: After the first sequence data is input into the first multi-head attention module for calculation, residual linking and regularization are performed to obtain the first attention level of each position in the first sequence data relative to other positions. Based on the first level of attention, the first sequence data is input into the first feedforward neural network for calculation, and then residual linking and regularization are performed to obtain the first multivariate data.

4. The power consumption prediction method based on attention mechanism fusion frequency enhancement as described in claim 3, characterized in that, The first multi-head attention module includes 8 sets of combined parameter matrices, and each set of combined parameter matrices includes three parameter matrices; The first feedforward neural network includes two feedforward neural network layers, and the number of neurons in the feedforward neural network layers is the same as the length of the first sequence data.

5. A power consumption prediction system based on attention mechanism fusion frequency enhancement, characterized in that, The system includes: a sample data acquisition unit, a model building unit, an encoding processing unit, a frequency-enhanced fusion attention processing unit, a sequence data acquisition unit to be predicted, and a decoding processing unit; The sample data acquisition unit is used to acquire historical power transformer data and process the historical power transformer data to obtain a sample dataset. The model building unit is used to construct a deep network structure model based on the attention mechanism and frequency enhancement. The deep network structure model includes an encoder, a frequency-enhanced hybrid attention module, and a decoder connected in sequence. The frequency-enhanced hybrid attention module includes a self-attention module and a frequency-enhanced channel attention module. The encoding processing unit is used to encode the sample dataset by position to obtain the first sequence data, and input the first sequence data into the encoder to perform multi-head attention and feedforward neural network learning on the first sequence data to obtain the first multivariate data. The frequency-enhanced fusion attention processing unit is used to input the first multivariate data into the frequency-enhanced hybrid attention module, perform self-attention processing and frequency-enhanced channel attention processing on the first multivariate data respectively, and then perform a weighted average to output the second multivariate data. The first multivariate data is subjected to self-attention processing and frequency-enhanced channel attention processing, including: The first multivariate data is input into the self-attention module to obtain the correlation relationship of the data sequences in the first multivariate data; The global average pooling operation in the channel attention mechanism is replaced by discrete cosine transform to obtain a frequency-enhanced channel attention module; The first multivariate data is input into the frequency enhancement channel attention module to obtain the correlation between different features in the first multivariate data; The output of the second multivariate data includes: The second multivariate data is obtained by weighting the correlation between the data sequences and the correlation between the different features. The data acquisition unit for the sequence to be predicted is used to acquire power transformer data to be predicted, obtain historical power transformer data from the sample dataset as tag data, combine the tag data and the power transformer data to be predicted and obtain second sequence data through position encoding; The decoding processing unit is used to input the second sequence data and the second multi-element data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted; The step of inputting the second sequence data and the second multivariate data into the decoder to obtain the power consumption prediction result corresponding to the power transformer data to be predicted includes: After the second sequence data is input into the second multi-head attention module for calculation, residual linking and regularization are performed to obtain the second attention level of each position in the second sequence data relative to other positions; Based on the second attention, the second sequence data and the second multivariate data are input into the third multi-head attention module for calculation, and then residual linking and regularization are performed to obtain the third attention of the sequence data corresponding to the power transformer data to be predicted relative to other positions in the first sequence data and the second sequence data. Based on the third level of attention, the second sequence data and the second multivariate data are input into the second feedforward neural network for calculation, and then residual linking and regularization are performed to obtain the third multivariate data. The third multivariate data is input into the fully connected layer and the softmax layer for calculation to obtain the power consumption prediction result corresponding to the power transformer data to be predicted.

6. A computer device, characterized in that: The computer device includes a memory, a processor, and a transceiver connected to each other via a bus; the memory stores a set of computer program instructions and data, and transmits the stored data to the processor, which executes the program instructions stored in the memory to perform the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1 to 4.

Images