Convolution embedding and local global change-based short-term load prediction method

Abstract

The invention discloses a short-term load prediction method based on convolution embedding and local and global change, and the method comprises the following steps: obtaining related historical data including power load, meteorological data and holiday and festival effects, and constructing a power load original data set based on the related historical data; sequentially performing convolution embedding, position embedding and timestamp embedding on the power load original data set to obtain a load sequence; dividing the load sequence into a local patch containing short-term information and a global patch containing long-term information; carrying out local-global change convolution on the local patch and the global patch, extracting features of different scales from the local patch and the global patch, and fusing the features to form a load fusion sequence; and constructing and training a load prediction model, and inputting the load fusion sequence into the trained load prediction model to obtain a short-term load prediction result.

Description

Technical Field

[0001] The present invention relates to the technical field of load forecasting, and in particular to a short-term load forecasting method based on convolution embedding and local-global change. Background Art

[0002] The surge in electricity demand across various sectors and the rapid development of the global power economy are driving the transformation of power systems towards digitalization, intelligence, and marketization. Therefore, high-performance short-term load forecasting has become a key component of power system planning and operation management. It is crucial for ensuring power supply reliability, optimizing resource allocation, and reducing costs. This not only ensures a high quality of life but also effectively promotes national economic development.

[0003] As power grid construction continues to expand and electricity markets continue to reform, the proportion of intermittent and volatile renewable energy on the power generation side has increased significantly, leading to more complex load variations on the demand side. This poses a serious challenge to the accuracy of traditional short-term power load forecasting methods. Therefore, developing high-precision short-term power load forecasting methods and formulating reasonable power generation plans and demand-side management plans are crucial for the stable operation and development of future power systems.

[0004] Due to the high complexity and nonlinearity of modern power loads, traditional linear regression and time series forecasting methods have difficulty explaining the underlying causes of power load changes. Especially in short-term power load forecasting tasks, which require high prediction accuracy, researchers have focused on machine learning and deep learning-based methods, such as support vector regression (SVR), random forests (RF), recurrent neural networks (RNN), convolutional neural networks (CNN), and Transformer networks, which have been widely used in the field of short-term load forecasting. However, these single-network models still face problems such as difficulty in feature extraction, inefficient extraction of nonlinear time correlations, high complexity, and the inability to capture both local and global information, which introduce unnecessary errors to short-term load forecasting. Summary of the Invention

[0005] In view of the above-mentioned prior art, the present invention provides a short-term load forecasting method based on convolution embedding and local-global change, which mainly solves the technical problems existing in the above-mentioned background technology.

[0006] To achieve the above-mentioned purpose, the technical solution of the embodiment of the present invention is implemented as follows:

[0007] A short-term load forecasting method based on convolutional embedding and local-global change, the forecasting method comprising the following steps:

[0008] Obtain relevant historical data including power load, meteorological data, and holiday effects, and construct an original power load data set based on the relevant historical data;

[0009] Performing convolution embedding, position embedding, and timestamp embedding on the original power load data set in sequence to obtain a load sequence;

[0010] dividing the load sequence into local patches containing short-term information and global patches containing long-term information;

[0011] Performing local-global variation convolution on the local patch and the global patch to extract and fuse features of different scales to form a load fusion sequence;

[0012] A load forecasting model is constructed and trained, and the load fusion sequence is input into the trained load forecasting model to obtain a short-term load forecasting result.

[0013] Optionally, the convolution embedding process includes convolution enhancement and convolution fusion. In the convolution enhancement stage, multiple factors are input into Shengweicheng Finally, a point-by-point convolution operation is used to embed it into a high-dimensional feature map along the time dimension In the convolution fusion stage, Transformed into Then point-wise group convolution is used to capture the dependencies between the load sequence and other factors, and the result is expressed as: H emb =PWConvldH enhance .

[0014] Optionally, dividing the load sequence into local patches containing short-term information and global patches containing long-term information specifically includes:

[0015] Given a scale size S, the input is represented along the time dimension L Partitioned into a 2D tensor consisting of multiple non-overlapping segments of length S It can be expressed as:

[0016] I 2D =Division(Padding(I))

[0017] Among them, Padding (I) is used to fill zero elements to expand the length L to P × S, where P represents the number of segments. For the obtained two-dimensional tensor I 2D , the feature vector of each column with a length of S contains more short-term fluctuations in the historical period, and the feature vector of each row with a length of P contains more long-term trends;

[0018] Transform the two-dimensional tensor I along the S dimension2D Divided into multiple one-dimensional local patches i∈[1,S], and sent to the subsequent local variation convolution for processing, and the two-dimensional tensor I is converted by the P dimension 2D Divided into multiple one-dimensional global patches i∈[1,P] and is sent to the subsequent global change convolution for processing.

[0019] Optionally, performing local-global change convolution on the local patch specifically includes:

[0020] Local variation convolution uses a one-dimensional convolution layer with a kernel size of S to capture local details within each patch. For each local patch The one-dimensional convolutional layer aggregates the information of all time steps along the dimension S and obtains the local detail representation after the activation function GELU

[0021]

[0022] The convolution results of all local patches are concatenated to obtain the output of the local variation convolution

[0023] Optionally, performing a local-global change convolution on the global patch specifically includes:

[0024] For a global patch set of length P Reorganize each convolution patch Obtained through deformation and fusion All Obtained by splicing

[0025] right Perform equidistant group convolution, the expression is: Where IGConv(·) represents the equally spaced group convolution, and the number of equally spaced group convolution groups is set to D. is the output of the global information representation.

[0026] Optionally, convolve the output of the local variation Output represented by global information Add them together to obtain the load fusion sequence Y.

[0027] Optionally, in the constructed load forecasting model, the loss function is an L2 loss function.

[0028] Optionally, in the constructed load forecasting model, a prediction head consisting of a flattening operation and a linear mapping is used to infer future load values from the feature representation and ultimately output a load forecast value in the short term in the future.

[0029] The beneficial effects of the present invention are as follows: the present application provides a short-term load forecasting method based on convolutional embedding and local-global changes. The two-stage process of convolutional embedding enhances the temporal feature representation of the original input, and effectively models the relationship between the load and related factors. The position and timestamp embedding also help the model better understand the sequential characteristics of the load sequence and the periodic characteristics of the long-term time. The adopted patching process enhances the ability of the proposed convolutional neural network to model local and global changes in the load sequence. The local-global change modeling mechanism can effectively capture the local details and global information of load data of different scales, and effectively improve the computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 Schematic diagram of a flow chart of a short-term load forecasting method based on convolutional embedding and local-global change in an embodiment of the present application;

[0031] Figure 2 Schematic diagram of the embedding process in the embodiment of this patent;

[0032] Figure 3 This is a schematic diagram of the patch strategy process in the embodiment of this patent.

[0033] Figure 4 This is a schematic diagram of the local-global change convolution structure in the embodiment of this patent. Figure 5 This is a schematic diagram of the finishing structure in the embodiment of this patent. DETAILED DESCRIPTION

[0034] The technical solution of the present invention is further elaborated in detail below in conjunction with the drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. In the following description, reference is made to "some embodiments", which describes a subset of all possible embodiments, but it should be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0035] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art are not described to avoid confusion with the present invention.

[0036] It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments proposed herein. On the contrary, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of the present invention to those skilled in the art. And the purpose of the terms used herein is only to describe specific embodiments and is not intended to limit the present invention. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "comprising" and / or "comprising" when used in this specification determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.

[0037] It should also be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "inner," "outer," "left," "right," and similar expressions used herein are for illustrative purposes only and do not represent the only implementation methods.

[0038] In order to fully understand the present invention, a detailed structure will be provided in the following description to illustrate the technical solution proposed by the present invention. Optional embodiments of the present invention are described in detail below. However, in addition to these detailed descriptions, the present invention may also have other implementations.

[0039] Please refer to the attached Figure 1 The present application provides a short-term load forecasting method based on convolutional embedding and local-global change, the forecasting method comprising the following steps:

[0040] S1. Obtain relevant historical data including power load, meteorological data, and holiday effects, and construct an original power load data set based on the relevant historical data;

[0041] S2. Perform convolution embedding, position embedding, and timestamp embedding on the original power load data set in sequence to obtain a load sequence;

[0042] S3, dividing the load sequence into local patches containing short-term information and global patches containing long-term information;

[0043] S4, performing local-global variation convolution on the local patch and the global patch, extracting and fusing features of different scales therefrom to form a load fusion sequence;

[0044] S5. Construct and train a load forecasting model, input the load fusion sequence into the trained load forecasting model, and obtain a short-term load forecasting result.

[0045] In the short-term load forecasting method proposed in this application, a convolution embedding method consisting of two stages, convolution enhancement and convolution fusion, is used to embed the load sequence, so as to more effectively capture the relationship between the load sequence and other factors through feature dimension increase and variable fusion. Secondly, the load sequence is divided into local patches containing short-term information and global patches containing long-term information through patch technology, which helps the model to mine the local-global information of load sequences at different scales. Finally, a local-global change convolution with a parallel structure is used to mine short-term time dependencies and long-term time dependencies and achieve fusion, so that the local-global change modeling modules at different layers can focus on the potential information of load data at different scales, significantly improving the prediction performance of the model at different prediction lengths.

[0046] Specifically, in step S1, the collected data is normalized using the zero-mean normalization method to normalize the load and related factor data of each data set to eliminate the dimensional differences between different features, thereby improving the stability of model training. The specific formula is as follows:

[0047]

[0048] where x norm Represents the normalized value, mean(X train ) and std(X train ) represents the mean and standard deviation based on the training set.

[0049] See also Figure 2 In an optional embodiment, in S2, the convolution embedding process includes convolution enhancement and convolution fusion. In order to enrich the time representation information and avoid the mixing of variable dimensions, this method uses convolution enhancement to first process the original input data, that is, in the convolution enhancement stage, by inputting multiple factors into the convolutional matrix. Shengweicheng Finally, a point-by-point convolution operation is used to embed it into a high-dimensional feature map along the time dimension In the convolution fusion stage, Transformed into Then point-wise group convolution is used to capture the dependencies between the load sequence and other factors, and the result is expressed as: H emb =PWConvldH enhance .

[0050] in, Denotes the value embedding obtained by convolutional embedding. The number of groups of point-wise grouped convolution is set to D to capture the connection between the load and correlation factors of each feature at each time step.

[0051] In the process of location embedding and timestamp embedding, learnable location embedding is used to automatically learn location information at different time steps during model training. Timestamp embedding is also added to provide global date information. For example, introducing hour embedding can help the model learn that the load on weekdays and weekends present different time patterns within a week. The input representation can be formulated as:

[0052] I=H emb +PE+TE

[0053] in represents position embedding, represents timestamp embeddings, which have the same feature dimension D.

[0054] See also Figure 3 In an optional embodiment, the load sequence is divided into local patches containing short-term information and global patches containing long-term information, specifically including:

[0055] Given a scale size S, the input is represented along the time dimension L Partitioned into a 2D tensor consisting of multiple non-overlapping segments of length S It can be expressed as:

[0056] I 2D =Division(Padding(I))

[0057] Among them, Padding (I) is used to fill zero elements to expand the length L to P × S, where P represents the number of segments. For the obtained two-dimensional tensor I 2D , the feature vector of each column with a length of S contains more short-term fluctuations in the historical period, and the feature vector of each row with a length of P contains more long-term trends;

[0058] Transform the two-dimensional tensor I along the S dimension 2D Divided into multiple one-dimensional local patches i∈[1,S], and sent to the subsequent local variation convolution for processing, and the two-dimensional tensor I is converted by the P dimension 2D Divided into multiple one-dimensional global patches i∈[1,P] and is sent to the subsequent global change convolution for processing.

[0059] See also Figure 4 In an optional embodiment, performing a local-global change convolution on the local patch specifically includes:

[0060] Local variation convolution uses a one-dimensional convolution layer with a kernel size of S to capture local details within each patch. For each local patch The one-dimensional convolutional layer aggregates the information of all time steps along the dimension S and obtains the local detail representation after the activation function GELU

[0061]

[0062] The convolution results of all local patches are concatenated to obtain the output of the local variation convolution

[0063] Furthermore, a local-global variation convolution is performed on the global patch. The global variation convolution proposes an equidistant grouped convolution based on equidistant convolution [] to capture the global information between different patches. Specifically, for a global patch set of length P Reorganize each convolution patch Obtained through deformation and fusion All Obtained by splicing

[0064] right Perform equidistant group convolution, the expression is: Where IGConv(·) represents the equally spaced group convolution, and the number of equally spaced group convolution groups is set to D. is the output of the global information representation.

[0065] The equidistant convolution uses placeholders of length S-1 to fill along the dimension S. The convolution kernel size is equal to the number of global patches S. Unlike the attention mechanism, the equidistant convolution uses a large causal convolution kernel to process the entire load sequence and uses the global information in the global patch to sequentially infer future long-term changes (trends and periodicity). In each eigenvector They are all independent time feature representations, containing long-term variation information in the load sequence. Therefore, a simple standard convolution operation is used to process the input along the dimension S. This will lead to redundant interactions between features and high computational costs, weakening the ability to model global information. Therefore, this method proposes an equidistant grouped convolution method that captures the long-term temporal dependencies between global patches of each feature by introducing a grouping mechanism, effectively enhancing the feature expression ability of the model. The process can be expressed as:

[0066]

[0067] Among them, IGConv(·) represents the equidistant group convolution, and the number of groups of the equidistant group convolution is set to D. is the global information representation of the output.

[0068] Furthermore, the global information is represented as Reshape to And use linear transformation to map dimension S to 1 to obtain global detail representation and local detail representation The load fusion sequence Y is obtained by adding them together.

[0069] See also Figure 5 In an optional embodiment, the constructed short-term power load forecasting model adopts the L2 loss function (mean square error), which calculates the average value of the square of the difference between the predicted value and the true value, quantifies the difference between the predicted value and the actual value, and enhances the learning ability of the model. The expression of the L2 loss function is:

[0070]

[0071] When training the established short-term power load forecasting model, the initial number of layers is set to 2, and the patch size of each layer is 8 and 16 respectively. During the training process, the batch size and epoch in the model are set to 32 and 30, the number of early stopping rounds is set to 10, the initial learning rate is set to 0.001, and an exponential decay strategy with a decay coefficient of 0.75 is used to automatically update the learning rate and optimize the model performance. The Adam optimization algorithm is used for model training to accelerate convergence and improve prediction performance.

[0072] Furthermore, in the constructed load forecasting model, a prediction head consisting of a flattening operation and a linear mapping is used to infer future load values from the feature representation and finally output the load forecast value in the short term in the future.

[0073] After the load forecasting model is trained, three metrics with different characteristics are selected to measure the performance of the short-term load forecasting model: mean absolute error, root mean square error, and mean absolute percentage error. Root mean square error highlights outliers with greater impact and imposes higher penalties on larger errors. Mean absolute error is more tolerant of errors of varying degrees and better reflects the gap between the predicted value and the true value. MAPE focuses more on relative error and is calculated as follows:

[0074]

[0075] Where n represents the number of predicted future time points, y i and Represent the predicted load value and actual load value respectively.

[0076] In order to further illustrate the beneficial effects of the embodiments of the present invention, the effects of the present invention are verified by simulation experiments below:

[0077] According to the forecasting requirements, historical data is used to fully consider the obvious coupling relationship between covariates (such as temperature, humidity, and festivals) and endogenous variables (load) to predict future short-term power load results. A load dataset from Panama, including hourly load values from January 3, 2015 to June 26, 2020 and 15 other related features, is selected for experiments. The training set, validation set, and test set are divided into a ratio of 7:1:2. The specific division is shown in Table 1. The classic Transformer variants Autoformer and Informer with encoder-decoder structures for time series forecasting are selected as the baseline models, and the latest SOTA model PatchTST is selected. DLinear and NLinear are selected for comparison with the proposed model. The comparison results are shown in Table 2.

[0078] Table 1 Dataset statistics

[0079] Dataset variable Time scale training set Validation set Test set Panama 16 1 hour 33442 4783 9586

[0080] According to the prediction results shown in Table 2, this application uses convolution enhancement and convolution fusion operations in convolutional embedding to enhance the time representation characteristics of the load series while capturing the influence of relevant factors such as weather and holidays, thereby improving the accuracy of load forecasting. Compared with PatchTST, the MAE, RMSE and MAPE of the model proposed in this application are improved by an average of 4.49%, 5.49% and 5.89% respectively. Compared with the best linear model NLinear, the MAE, RMSE and MAPE of our proposed model are improved by an average of 15.90%, 16.68% and 20.16% respectively. In summary, the model proposed in this application can provide reliable prediction results in short-term demand load forecasting and achieve better performance than other SOTA methods.

[0081] Table 2 Comparison of load forecast results

[0082]

[0083] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. The scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A short-term load forecasting method based on convolutional embedding and local-global change, characterized in that: The prediction method comprises the following steps: Obtain relevant historical data including power load, meteorological data, and holiday effects, and construct an original power load data set based on the relevant historical data; Performing convolution embedding, position embedding, and timestamp embedding on the original power load data set in sequence to obtain a load sequence; dividing the load sequence into local patches containing short-term information and global patches containing long-term information; Performing local-global variation convolution on the local patch and the global patch to extract and fuse features of different scales to form a load fusion sequence; A load forecasting model is constructed and trained, and the load fusion sequence is input into the trained load forecasting model to obtain a short-term load forecasting result.

2. A short-term load forecasting method based on convolutional embedding and local-global change according to claim 1, characterized in that: The convolution embedding process includes convolution enhancement and convolution fusion. In the convolution enhancement stage, multiple factors are input into Shengweicheng Finally, a point-by-point convolution operation is used to embed it into a high-dimensional feature map along the time dimension In the convolution fusion stage, Transformed into Then point-wise group convolution is used to capture the dependencies between the load sequence and other factors, and the result is expressed as: H emb =PWConvldH enhance .

3. The short-term load forecasting method based on convolutional embedding and local-global change according to claim 1 is characterized in that: Dividing the load sequence into local patches containing short-term information and global patches containing long-term information specifically includes: Given a scale size S, the input is represented along the time dimension L Partitioned into a 2D tensor consisting of multiple non-overlapping segments of length S It can be expressed as: I 2D =Division(Padding(I)) Among them, Padding (I) is used to fill zero elements to expand the length L to P × S, where P represents the number of segments. For the obtained two-dimensional tensor I 2D , the feature vector of each column with a length of S contains more short-term fluctuations in the historical period, and the feature vector of each row with a length of P contains more long-term trends; Transform the two-dimensional tensor I along the S dimension 2D Divided into multiple one-dimensional local patches i∈[1,S], and sent to the subsequent local variation convolution for processing, and the two-dimensional tensor I is converted by the P dimension 2D Divided into multiple one-dimensional global patches i∈[1,P] and is sent to the subsequent global change convolution for processing.

4. The short-term load forecasting method based on convolutional embedding and local-global change according to claim 3 is characterized in that: Performing local-global change convolution on the local patch, specifically comprising: Local variation convolution uses a one-dimensional convolution layer with a kernel size of S to capture local details within each patch. For each local patch The one-dimensional convolutional layer aggregates the information of all time steps along the dimension S and obtains the local detail representation after the activation function GELU The convolution results of all local patches are concatenated to obtain the output of the local variation convolution 5. The short-term load forecasting method based on convolutional embedding and local-global change according to claim 4 is characterized in that: Performing a local-global change convolution on the global patch, specifically comprising: For a global patch set of length P Reorganize each convolution patch Obtained through deformation and fusion All Obtained by splicing right Perform equidistant group convolution, the expression is: Where IGConv(·) represents the equally spaced group convolution, and the number of equally spaced group convolution groups is set to D. is the output of the global information representation.

6. The short-term load forecasting method based on convolutional embedding and local-global change according to claim 5 is characterized in that: The output of the local variation convolution Output represented by global information Add them together to obtain the load fusion sequence Y.

7. The short-term load forecasting method based on convolutional embedding and local-global change according to claim 6 is characterized in that: In the constructed load forecasting model, the loss function is the L2 loss function.

8. The short-term load forecasting method based on convolutional embedding and local-global change according to claim 7 is characterized in that: In the constructed load forecasting model, the prediction head consisting of flattening operation and linear mapping is used to infer future load values from the feature representation and finally output the load forecast value in the short term in the future.

Images