A method and device for power load forecasting based on adaptive sparse attention network

Through the adaptive sparse attention network, the characteristics of the power load signal are captured in the power load prediction, and the problems of insufficient anti-interference and timing in the prior art are solved, thereby achieving higher power load prediction accuracy and adaptability.

CN118312736BActive Publication Date: 2024-12-03GUANGDONG MECHANICAL & ELECTRICAL COLLEGE
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410409709.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-07
Publication Date
2024-12-03
Estimated Expiration
2044-04-07

AI Technical Summary

Technical Problem

The existing power load prediction methods have shortcomings in anti-interference ability, learning signal timing and prediction accuracy, especially in the face of external interference, weather conditions and user behavior randomness.

Method used

Adopting an adaptive sparse attention network method, the characteristics of power load signals are captured through adaptive sparse space, channels and batch attention modules, enhance anti-interference ability and local outlier tolerance, and reduce the time impact, ultimately improving Accuracy of power load prediction.

Benefits of technology

It significantly improves the anti-interference ability and accuracy of power load prediction, enhances the adaptability to power load changes, reduces the time impact, and shows higher prediction accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118312736B_ABST
    Figure CN118312736B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for power load prediction based on an adaptive sparse attention network, which involves deep learning, obtaining power load signals; embedding and encoding the time characteristics, position information and morphological features in the power load signals; establishing an adaptive sparse spatial attention module to improve the anti-interference ability of the network model for power load prediction; establishing an adaptive sparse channel attention module to enhance the tolerance of the network model for local outliers in power load prediction; establishing an adaptive sparse batch attention module to reduce the time impact of the network model on power load prediction; and outputting the predicted power load signal through a multi-layer perceptron. The present invention also discloses a power load prediction device based on an adaptive sparse attention network. The present invention effectively enhances the anti-interference ability of the power load and the timing of the learning signal, and improves the adaptability of the network to power load changes and the accuracy of power load prediction.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to deep learning, and more specifically, to a method and device for power load forecasting based on an adaptive sparse attention network. Background Art

[0002] Power load forecasting is of great significance to the development of smart grids, and can provide safe and economical guarantees for service areas, especially developed areas. In addition, efficient and stable power load forecasting can provide adaptive electricity price strategies and optimized power supply planning for power systems. For example, every 1% increase in forecast error will result in an additional 10 million pounds in power system operation. Therefore, in the future smart grid, accurate power load forecasting is very necessary for power system management.

[0003] In recent years, power load forecasting has attracted widespread attention, and various power load forecasting methods have been reported, such as linear regression, fuzzy logic methods, support vector machines, radial basis function neural networks, wavelet neural networks, and machine learning methods. However, these methods show low sensitivity in power load forecasting due to their own limitations. Thanks to the rapid development of computer technology, deep convolutional neural networks have performed well in many fields such as computer vision, medical diagnosis, and sequence prediction. Unlike traditional power load forecasting methods, deep learning-based methods tend to integrate complex tasks directly into convolutional neural networks instead of providing them with preprocessing features. Due to their powerful feature extraction capabilities, convolutional neural networks have also been widely used in power load forecasting. For example, researchers designed a deep residual network model for predicting short-term power load forecasting, which can integrate domain knowledge and use different neural networks to understand special tasks. In addition, deep belief networks have also been applied to predict short-term power load. In addition, graph neural networks are used to capture the potential spatial dependencies of different residences to solve the problem of short-term residential load forecasting. These deep learning-based methods solve power load forecasting by extracting features or learning corresponding dependencies, which is more effective than traditional methods.

[0004] Although these deep learning-based methods have achieved good results, the practical application of power load forecasting is still challenging due to the complexity of this task. First, the power load signal is easily affected by external interference, such as weather conditions, temperature factors, etc., which leads to large fluctuations in conventional power load signals. Secondly, the power load consumption varies greatly in different periods. For example, there are significant differences in users' monthly (or daily) power load consumption, which has an adverse effect on the prediction performance. Third, users' behavior is random, such as family reunions, urgent business orders, etc., which leads to a large number of outliers in the power load signal. These three problems make power load forecasting still a challenging task. Summary of the invention

[0005] The technical problem to be solved by the present invention is to address the deficiencies in the prior art and provide a method and device for power load prediction based on an adaptive sparse attention network, which effectively enhances the anti-interference ability of the power load and the timing of the learning signal, and further improves the network's adaptability to power load changes and the accuracy of power load prediction.

[0006] The present invention provides a method for predicting power load based on an adaptive sparse attention network, comprising the following steps:

[0007] S1: Obtain power load signal;

[0008] S2: obtaining time characteristics, position information and morphological features in the power load signal, and embedding and encoding the time characteristics, position information and morphological features to obtain a composite embedded feature vector;

[0009] S3: Establishing an adaptive sparse spatial attention module, which captures the detail changes in the composite embedded feature vector caused by the external interference of the power load to improve the anti-interference ability of the system;

[0010] S4: establishing an adaptive sparse channel attention module, and enhancing tolerance to local outliers by learning feature information of local outliers of power load in composite embedded feature vectors through the adaptive sparse channel attention module;

[0011] S5: establishing an adaptive sparse batch attention module, and modeling the dependency of the power load timestamps in the composite embedded feature vector by the adaptive sparse batch attention module;

[0012] S6: The output feature maps of the adaptive sparse spatial attention module, the adaptive sparse channel attention module, and the adaptive sparse batch attention module are superimposed and sent to a multi-layer perceptron for calculation to output a predicted power load signal.

[0013] As a further improvement, step S2 is specifically as follows:

[0014] The power load signal is encoded by a morphological encoder to extract time characteristics, location information and morphological features; the location information is mapped into a word embedding vector by a position encoder; the time characteristics are embedded by minutes, hours, days, weeks and months by a time encoder, and then all the results are added to obtain a composite timestamp embedding vector; the morphological features, word embedding vectors and composite timestamp embedding vectors are superimposed to obtain a composite embedding feature vector.

[0015] Furthermore, the morphological encoder is composed of a one-dimensional convolutional layer and a tanh function; the expression of the tanh function is:

[0016]

[0017] In the formula, e x is the natural exponential function; e -x is the natural exponential inverse function; x is the power load signal.

[0018] Further, the position encoder is defined as:

[0019]

[0020] Where D K is the feature dimension sequence; d is D K The length of P k Represents the position sequence, obtained from the initialized sequence [1,2,…,N]; N represents the maximum length of the input electric load signal.

[0021] As a further improvement, in step S3, the adaptive sparse spatial attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened in the spatial dimension to expand it into a first high-order one-dimensional tensor; it is optimized by adaptively related eigenvalues ​​to determine μ in the spatial dimension; the first high-order one-dimensional tensor multiplied by the learnable parameter α in the first high-order one-dimensional tensor is adaptively sampled, and then multiplied with the flattened feature space K to generate a first feature attention map; the maximum mean sparsity measurement is used to measure the first high-order one-dimensional tensor. The first feature attention map and the flattened V are processed, and then the attention map is activated by the Softmax function to obtain the first output feature map, and the first output feature map is transmitted to the next linear layer.

[0022] As a further improvement, in step S4, the adaptive sparse channel attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened in the feature mapping channel dimension to expand it into a second high-order one-dimensional tensor; it is optimized by adaptively related eigenvalues ​​to determine μ in the spatial dimension; the second high-order one-dimensional tensor multiplied by the scalar β learned in the training process in the second high-order one-dimensional tensor is adaptively sampled, and then multiplied with the flattened feature space K to generate a second feature attention map; the maximum mean sparsity measurement is used to measure the second high-order one-dimensional tensor. The second feature attention map and the flattened V are processed, and then the attention map is activated by the Softmax function to obtain the second output feature map, and the second output feature map is transmitted to the next linear layer.

[0023] As a further improvement, in step S5, the adaptive sparse batch attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened on the feature map batch dimension to expand it into a third high-order one-dimensional tensor; it is optimized by adaptively related eigenvalues ​​to determine μ on the spatial dimension; the third high-order one-dimensional tensor multiplied by the attention weight y learned by the batch timestamp in the third high-order one-dimensional tensor is adaptively sampled, and then multiplied with the flattened feature space K to generate a third feature attention map; the maximum mean sparsity measurement is used The third feature attention map and the flattened V are processed, and then the attention map is activated by the Softmax function to obtain a third output feature map, and the third output feature map is transmitted to the next linear layer.

[0024] Furthermore, the maximum mean sparsity measurement It is expressed by the following formula:

[0025]

[0026] In the formula, q i is the query value; k j is the key value; L K is the length of the key value;

[0027] The μ satisfies:

[0028]

[0029] Where μ is the maximum sparse number of query values; Cor represents the correlation coefficient matrix; Eig represents the eigenvalue; Sort represents descending sort; and γ is the ratio threshold.

[0030] As a further improvement, the multilayer perceptron calculates the superimposed output feature map by the following formula:

[0031] y=wx+b,

[0032] Where y is the predicted power load signal; w is the weight of the neural network; b is the bias of the neural network; and x is the output feature map after superposition.

[0033] A power load forecasting device based on an adaptive sparse attention network, comprising:

[0034] A signal encoding module is used to embed and encode the time characteristics, position information and morphological characteristics of the acquired power load signal to obtain a composite embedded feature vector;

[0035] An adaptive sparse spatial attention module, used to capture the detail changes in the composite embedded feature vector caused by external interference of the power load to improve the anti-interference ability of the system;

[0036] Adaptive sparse channel attention module, which is used to learn the feature information of local outliers of power load in composite embedded feature vector to enhance the tolerance to local outliers;

[0037] An adaptive sparse batch attention module to model the dependencies of power load timestamps in the composite embedded feature vector;

[0038] The multi-layer perceptron module is used to superimpose the output feature maps of the adaptive sparse spatial attention module, the adaptive sparse channel attention module, and the adaptive sparse batch attention module, and send them into the multi-layer perceptron for calculation to output the predicted power load signal.

[0039] Beneficial effects:

[0040] The advantages of the present invention are: the present invention effectively enhances the anti-interference ability of the power load and the timing of the learning signal through the design of adaptive sparse modules based on space, channel and batch, and further improves the adaptability of the network to power load changes and the accuracy of power load prediction. Among them, the proposed adaptive sparse space attention module can improve the anti-interference ability of the network model for power load prediction; the proposed adaptive sparse channel attention module can enhance the tolerance of the network model for local outliers in power load prediction; the proposed adaptive sparse batch attention module can reduce the time impact of the network model on power load prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a method for power load forecasting based on an adaptive sparse network attention network provided by an embodiment of the present invention;

[0042] Figure 2 It is another flow chart of a method for power load forecasting based on an adaptive sparse network attention network provided by an embodiment of the present invention;

[0043] Figure 3 It is a comparison chart of MAE and MSE between the existing method tested based on CAEEC data set and the method provided by the embodiment of the present invention;

[0044] Figure 4It is a comparison chart of MAE and MSE of the existing method tested based on the UCI data set and the method provided by the embodiment of the present invention;

[0045] Figure 5 It is a comparison chart of MAE and MSE of the existing method and the method provided by the embodiment of the present invention tested on the REFIT data set;

[0046] Figure 6 It is a comparison chart of MAE and MSE of the existing method and the method provided by the embodiment of the present invention tested on the USD data set;

[0047] Figure 7 It is a schematic diagram of the modular structure of a power load prediction device based on an adaptive sparse attention network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the embodiments, but it does not constitute any limitation to the present invention. Any limited number of modifications made by anyone within the scope of the claims of the present invention are still within the scope of the claims of the present invention.

[0049] See also Figure 1 and Figure 2 The present invention provides a method for power load forecasting based on an adaptive sparse attention network, which specifically comprises the following steps:

[0050] S1: Obtain power load signal.

[0051] The specific approach is to access the real-time database of the power grid and obtain the power load signal of a fixed time period before the current moment.

[0052] S2: Obtain the time characteristics, position information and morphological features in the power load signal, and embed the time characteristics, position information and morphological features to obtain a composite embedded feature vector, and provide the composite embedded feature vector to the network model for learning and modeling, wherein the morphological features in the power load signal are extracted by encoding through a morphological encoder. The morphological encoder consists of a one-dimensional convolutional layer and a tanh function. The tanh function is used to limit the output of the morphological encoder to [-1,1]. The expression of the tanh function is:

[0053]

[0054] In the formula, e x is the natural exponential function; e -x is the natural exponential inverse function; x is the power load signal.

[0055] The position information, i.e. the coordinate position, is mapped to a word embedding vector through a position encoder composed of a Sin function. The position encoder is defined as:

[0056]

[0057] Where D K is the feature dimension sequence; d is D K The length of P k Represents the position sequence, obtained from the initialized sequence [1,2,…,N]; N represents the maximum length of the input electric load signal.

[0058] The time characteristics are encoded by the time encoder formed by the query table, that is, the timestamp in the power load signal, and embedded by minute, hour, day, week, and month, and then all the results are added to obtain a composite timestamp embedding vector. The morphological features, word embedding vectors, and composite timestamp embedding vectors are superimposed to form a composite embedding feature vector.

[0059] S3: An adaptive sparse spatial attention module is established to improve the system's anti-interference ability by capturing the detailed changes in the composite embedded feature vector caused by external interference of the power load.

[0060] Specifically, the adaptive sparse spatial attention module mainly uses the linear layer of the neural network to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened in the spatial dimension to expand it into a high-order one-dimensional tensor. It is optimized by adaptively related eigenvalues ​​to determine μ in the spatial dimension. For the high-order one-dimensional tensor, the high-order one-dimensional tensor multiplied by α is adaptively sampled, and then multiplied by the flattened K to generate a feature attention map. Among them, α is a learnable parameter that allows the network to rely on local information first, and then gradually learn global cues and assign more weight to global cues. Measured by the maximum mean sparsity The feature attention map and the flattened V are processed, the attention map is activated through the Softmax function, and the map activation result is passed to the next linear layer.

[0061] In this embodiment, after being processed by the scheme, the module can not only improve the anti-interference ability by capturing the detail changes in the feature mapping space, but also greatly reduce the computational complexity after sparse processing.

[0062] S4: Establish an adaptive sparse channel attention module to enhance the tolerance to local outliers by learning the feature information of local outliers of power load in the composite embedded feature vector.

[0063] Specifically, the adaptive sparse channel attention module first uses the linear layer of the neural network to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V. The feature space Q is then flattened in the feature mapping channel dimension to expand it into a high-order one-dimensional tensor. The μ in the channel dimension is determined by adaptive correlation eigenvalue optimization. After adaptively sampling the high-order one-dimensional tensor multiplied by β, it is multiplied by the flattened K to generate a feature attention map. Among them, β is a scalar learned during the training process, which can give more weight to the effective information in the channel dimension. Through the maximum mean sparsity measure M(q i ,K) processes the feature attention map and the flattened V, then activates the attention map through the Softmax function, and then transmits the map activation result to the next linear layer.

[0064] S5: Establish an adaptive sparse batch attention module to model the dependency of power load timestamps to reduce the temporal impact of the network model on power load forecasting.

[0065] Specifically, the adaptive sparse batch attention module first uses the linear layer of the neural network to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened on the feature map batch dimension to expand it into a high-order one-dimensional tensor. The μ on the batch dimension is determined by adaptive correlation eigenvalue optimization. The high-order one-dimensional tensor multiplied by y is adaptively sampled and multiplied with the flattened K to generate a feature attention map. Among them, y is the attention weight learned by batch timestamps, and the feature map can be transformed for efficient processing. Through the maximum mean sparsity measurement M(q i ,K) processes the feature attention map and the flattened V, then activates the attention map through the Softmax function, and then transmits the map activation result to the next linear layer.

[0066] In S3-S5 above, the maximum mean sparsity measure It is expressed by the following formula:

[0067]

[0068] In the formula, q i is the query value; k j is the key value; L K The length of the key value.

[0069] μ in various dimensions satisfies:

[0070]

[0071] Where μ is the maximum sparse number of query values; Cor represents the correlation coefficient matrix; Eig represents the eigenvalue; Sort represents descending sort; and γ is the ratio threshold.

[0072] The output of each attention module can be expressed as follows:

[0073]

[0074] Where K and V are key and value respectively; T represents matrix transpose; d k is the dimension of the input word vector; Q represents the maximum mean sparsity measurement M(q i ,K) acts on the feature space Q to obtain the first μ query values.

[0075] S6: The output feature maps of the adaptive sparse spatial attention module, the adaptive sparse channel attention module, and the adaptive sparse batch attention module are superimposed and sent to the multi-layer perceptron for calculation to output the predicted power load signal.

[0076] Specifically, the multi-layer perceptron calculates the superimposed output feature map using the following formula:

[0077] y=wx+b,

[0078] Where y is the predicted power load signal; w is the weight of the neural network; b is the bias of the neural network; and x is the output feature map after superposition.

[0079] This embodiment effectively enhances the anti-interference ability of the power load and the timing of the learning signal through the design of an adaptive sparse module based on space, channel, and batch, and further improves the adaptability of the network to power load changes and the accuracy of power load prediction.

[0080] Reference Figures 3 to 6 , Figures 3 to 6 They are based on the CAEEC dataset (C.Lvpeng,Y.Linfei,Y.Tao etal.,Short-term power load forecasting based on deep forest algorithm[J],Electric Power Construction,2018,39(11):42-50.) and the UCI dataset (G.Hebrail andA.Berard.Individual household electric power consumption data set[EB / OL].(2012-08-30)[2012-08-30].

[0081] https: / / archive.ics.uci.edu / ml / datasets / Individual+household+electric+power+consumption), based on the REFIT dataset (D. Murray, L. Stankovic, and V. Stankovic, An electrical load measurements dataset of united kingdom households from a two-year longitudinal study[J], Scientific data, 2017, 4(1): 1-12.), based on the USD dataset (S. Barker, A. Mishra, D. Irwin, E. Cecchet, P. Shenoy, and J. Albrecht. Smart* data set for sustainability[EB / OL]. (2017-08-01) [2017-08-01].

[0082] The MAE and MSE result histograms of the existing method tested by https: / / traces.cs.umass.edu / index.php / Smart / Smart.) and the method provided by the embodiment of the present invention.

[0083] MAE stands for Mean Square Error, which represents the average value of the absolute value of the error between the predicted power load signal and the actual power load signal; MSE stands for Mean Square Error, which represents the average value of the sum of squares of the error between the predicted power load signal and the actual power load signal. They are important indicators for measuring the performance of the prediction model and can provide a reference for model performance evaluation. The existing methods are LSTM (S. Hochreiter and J. Schmidhuber, Long short-term memory [J], Neural Computation, 1997, 9 (8): 1735–1780.), LSTNet (G. Lai, W.-C. Chang, Y. Yang, and H. Liu, Modeling long-and shortterm temporal patterns with deep neural networks [C] / / Proceedings of theInternational ACM Conference on Research and Development in InformationRetrieval(SIGIR),2018:95–104.), Transformer(A.Vaswani,N.Shazeer,N.Parmar,J.Uszkoreit,L.Jones,ANGomez, Kaiser,and I.Polosukhin,Attention is all youneed[J],Advances in Neural Information Processing Systems,2017,30:1–11.)Reformer(N.Kitaev,L.Kaiser,and A.Levskaya,Reformer:The efficient transformer[C] / / Proceedings of the International Conference on Learning Representations(ICLR),2020:1–12.), Informer(H.Zhou,S.Zhang,J.Peng,S.Zhang,J.Li,H.Xiong,andW.Zhang,Informer:Beyond efficient transformer for long sequence time-seriesforecasting[C] / / Proceedings of the AAAI conference on artificialintelligence,2021,35(12):11106–11115.).

[0084] Figure 3 The test results of the proposed method and the existing method on the CAEEC dataset are shown in Figure 1. The MSEs are 0.037, 0.028, 0.022, 0.016, and 0.018, and the MAEs are 0.178, 0.116, 0.108, 0.075, and 0.077, respectively. The MSE and MAE of the proposed method are 0.014 and 0.071, respectively, which are lower than those of other methods. It can be seen that the performance of the method provided by the embodiment of the present invention is higher than the overall performance of the existing method.

[0085] Figure 4 The test results of the proposed method and the existing method on the UCI dataset are shown in Figure 2. Their MSEs are 0.077, 0.065, 0.060, 0.055, and 0.048, and their MAEs are 0.338, 0.285, 0.214, 0.172, and 0.147, respectively. The MSE and MAE of the proposed method are 0.041 and 0.091, respectively, which are lower than those of other methods. It can be seen that the method provided by the embodiment of the present invention is more accurate in power load prediction than the existing method model.

[0086] Figure 5The test results of the proposed method and the existing method on the REFIT dataset are shown in Figure 1. The MSEs are 0.092, 0.078, 0.066, 0.057, and 0.046, and the MAEs are 0.412, 0.311, 0.288, 0.215, and 0.139, respectively. The MSE and MAE of the proposed method are 0.038 and 0.101, respectively, which are lower than those of other methods. It can be seen that the method provided by the embodiment of the present invention has higher performance than the existing method model in long-sequence power load forecasting.

[0087] Figure 6 The test results of the proposed method and the existing method on the USD dataset are shown in Figure 2. The MSEs are 0.071, 0.063, 0.056, 0.049, and 0.044, and the MAEs are 0.269, 0.185, 0.061, 0.108, and 0.093, respectively. The MSE and MAE of the proposed method are 0.034 and 0.059, respectively, which are lower than those of other methods. It can be seen that the method provided by the embodiment of the present invention is superior to the existing method model in terms of power load forecasting performance.

[0088] Reference Figure 7 ,The present invention also provides a power load prediction device based on ,adaptive attention network, which includes the following five modules.

[0089] The signal encoding module 701 is used to encode the time characteristics, location information and morphological characteristics of the power load signal to generate a composite embedding tensor; the adaptive sparse spatial attention module 702 is used to improve the anti-interference ability of the network model for power load prediction; the adaptive sparse channel attention module 703 is used to enhance the tolerance of the network model to local outliers in power load prediction; the adaptive sparse batch attention module 704 is used to reduce the time impact of the network model on power load prediction; the multi-layer perceptron module 705 is used to output the predicted power load signal.

[0090] It is understandable that if Figure 1 and Figure 2 The contents of the embodiment of the power load prediction method based on the adaptive sparse attention network shown in the figure are all applicable to the embodiment of the power load prediction device based on the adaptive sparse attention network. The functions specifically implemented by the embodiment of the power load prediction device based on the adaptive sparse attention network are the same as those in the embodiment of the power load prediction method based on the adaptive sparse attention network. Figure 1 and Figure 2 The embodiment of the power load forecasting method based on the adaptive sparse attention network shown in FIG. Figure 1 and Figure 2 The beneficial effects achieved by the embodiment of the power load forecasting method based on the adaptive sparse attention network shown are also the same.

[0091] It should be noted that the information interaction, execution process and other contents between the above-mentioned devices are based on the same concept as the method embodiment of the present invention. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0092] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0093] The above is only a preferred embodiment of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the structure of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A power load forecasting method based on adaptive sparse attention network, characterized in that: The following steps are involved: S1: Obtain power load signal; S2: obtaining time characteristics, position information and morphological features in the power load signal, and embedding and encoding the time characteristics, position information and morphological features to obtain a composite embedded feature vector; S3: Establishing an adaptive sparse spatial attention module, which captures the detail changes in the composite embedded feature vector caused by the external interference of the power load to improve the anti-interference ability of the system; S4: establishing an adaptive sparse channel attention module, and enhancing tolerance to local outliers by learning feature information of local outliers of power load in composite embedded feature vectors through the adaptive sparse channel attention module; S5: establishing an adaptive sparse batch attention module, and modeling the dependency of the power load timestamps in the composite embedded feature vector by the adaptive sparse batch attention module; S6: superimposing the output feature maps of the adaptive sparse spatial attention module, the adaptive sparse channel attention module, and the adaptive sparse batch attention module, and sending them to a multi-layer perceptron for calculation, and outputting a predicted power load signal; In step S3, the adaptive sparse spatial attention module decomposes the composite embedded feature vector using a neural network linear layer to obtain three feature spaces Q, K, and V; then flattens the feature space Q in the spatial dimension to expand it into a first high-order one-dimensional tensor; optimizes it through adaptive related eigenvalues ​​to determine μ in the spatial dimension; adaptively samples the first high-order one-dimensional tensor multiplied by the learnable parameter α in the first high-order one-dimensional tensor, and then multiplies it with the flattened feature space K to generate a first feature attention map; The maximum mean sparsity measure M(q i ,K) processing the first feature attention map and the flattened V, activating the attention map through the Softmax function to obtain a first output feature map, and transmitting the first output feature map to the next linear layer; In step S4, the adaptive sparse channel attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened in the feature mapping channel dimension to expand it into a second high-order one-dimensional tensor; it is optimized by adaptively related eigenvalues ​​to determine μ in the spatial dimension; the second high-order one-dimensional tensor multiplied by the scalar β learned in the training process in the second high-order one-dimensional tensor is adaptively sampled, and then multiplied with the flattened feature space K to generate a second feature attention map; the maximum mean sparsity measurement M(q i ,K) processing the second feature attention map and the flattened V, activating the attention map through the Softmax function to obtain a second output feature map, and transmitting the second output feature map to the next linear layer; In step S5, the adaptive sparse batch attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then the feature space Q is flattened on the feature map batch dimension to expand it into a third high-order one-dimensional tensor; it is optimized by adaptively related eigenvalues ​​to determine μ on the spatial dimension; the third high-order one-dimensional tensor multiplied by the attention weight y learned by the batch timestamp in the third high-order one-dimensional tensor is adaptively sampled, and then multiplied by the flattened feature space K to generate a third feature attention map; the maximum mean sparsity measurement is used to measure the sparsity of the third high-order one-dimensional tensor. Processing the third feature attention map and the flattened V, activating the attention map through a Softmax function to obtain a third output feature map, and transmitting the third output feature map to the next linear layer; Where μ is the maximum sparse number of query values.

2. According to claim 1, a method for power load forecasting based on an adaptive sparse attention network is characterized in that: Step S2 is specifically as follows: Encoding the electric power load signal by a morphological encoder to extract time characteristics, position information and morphological features; The position information is mapped into a word embedding vector through a position encoder; the time characteristics are embedded and encoded by minutes, hours, days, weeks, and months through a time encoder, and then all the results are added to obtain a composite timestamp embedding vector; the morphological features, word embedding vectors, and composite timestamp embedding vectors are superimposed to obtain a composite embedding feature vector.

3. The method for power load forecasting based on adaptive sparse attention network according to claim 2 is characterized in that: The morphological encoder consists of a one-dimensional convolutional layer and a tanh function; the expression of the tanh function is: In the formula, e x is the natural exponential function; e -x is the natural exponential inverse function; x is the power load signal.

4. The method for power load forecasting based on adaptive sparse attention network according to claim 2 is characterized in that: The position encoder is defined as: Where D K is a sequence of feature dimensions; d is for D K The length of P k Represents the position sequence, obtained from the initialized sequence [1,2,…,N]; N represents the maximum length of the input electric load signal.

5. The method for power load forecasting based on adaptive sparse attention network according to claim 1 is characterized in that: The maximum mean sparsity measure It is expressed by the following formula: In the formula, q i is the query value; k j is the key value; L K is the length of the key value; The μ satisfies: In the formula, Cor represents the correlation coefficient matrix; Eig represents the eigenvalue; Sort represents descending order; γ is the ratio threshold.

6. The method for power load forecasting based on adaptive sparse attention network according to claim 1, characterized in that: The multi-layer perceptron calculates the superimposed output feature map by the following formula: y=wx+b, Where y is the predicted power load signal; w is the weight of the neural network; b is the bias of the neural network; and x is the output feature map after superposition.

7. A power load forecasting device based on an adaptive sparse attention network, characterized in that: include: A signal encoding module is used to embed and encode the time characteristics, position information and morphological characteristics of the acquired power load signal to obtain a composite embedded feature vector; An adaptive sparse spatial attention module, used to capture the detail changes in the composite embedded feature vector caused by external interference of the power load to improve the anti-interference ability of the system; Adaptive sparse channel attention module, which is used to learn the feature information of local outliers of power load in composite embedded feature vector to enhance the tolerance to local outliers; An adaptive sparse batch attention module to model the dependencies of power load timestamps in the composite embedded feature vector; A multi-layer perceptron module is used to superimpose the output feature maps of the adaptive sparse spatial attention module, the adaptive sparse channel attention module, and the adaptive sparse batch attention module, and send them to the multi-layer perceptron for calculation, and output a predicted power load signal; The adaptive sparse spatial attention module decomposes the composite embedded feature vector using a neural network linear layer to obtain three feature spaces Q, K, and V; then flattens the feature space Q in the spatial dimension to expand it into a first high-order one-dimensional tensor; optimizes it through adaptive related eigenvalues ​​to determine μ in the spatial dimension; adaptively samples the first high-order one-dimensional tensor multiplied by the learnable parameter α in the first high-order one-dimensional tensor, and then multiplies it with the flattened feature space K to generate a first feature attention map; The maximum mean sparsity measure M(q i ,K) processing the first feature attention map and the flattened V, activating the attention map through the Softmax function to obtain a first output feature map, and transmitting the first output feature map to the next linear layer; The adaptive sparse channel attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then flattens the feature space Q in the feature mapping channel dimension to expand it into a second high-order one-dimensional tensor; optimizes it through adaptive correlation eigenvalues ​​to determine μ in the spatial dimension; adaptively samples the second high-order one-dimensional tensor multiplied by the scalar β learned in the training process in the second high-order one-dimensional tensor, and then multiplies it with the flattened feature space K to generate a second feature attention map; and measures M(q i ,K) processing the second feature attention map and the flattened V, activating the attention map through the Softmax function to obtain a second output feature map, and transmitting the second output feature map to the next linear layer; The adaptive sparse batch attention module uses a neural network linear layer to decompose the composite embedded feature vector to obtain three feature spaces Q, K, and V; then flattens the feature space Q on the feature map batch dimension to expand it into a third high-order one-dimensional tensor; optimizes it through adaptive correlation eigenvalues ​​to determine μ on the spatial dimension; adaptively samples the third high-order one-dimensional tensor multiplied by the attention weight y learned by batch timestamps in the third high-order one-dimensional tensor, and then multiplies it with the flattened feature space K to generate a third feature attention map; and measures M(q i ,K) processing the third feature attention map and the flattened V, activating the attention map through a Softmax function to obtain a third output feature map, and transmitting the third output feature map to the next linear layer; Where μ is the maximum sparse number of query values.

Citation Information

Patent Citations

  • Short-term load prediction method based on pattern matching

    CN115689001A

  • Refrigeration system load prediction method based on sequence-to-sequence model

    CN115982567A