Load prediction method and device based on feature fusion, terminal equipment and storage medium

Abstract

The invention discloses a load prediction method and device based on feature fusion, terminal equipment and a storage medium. The method comprises the steps of obtaining to-be-processed multi-source data; wherein the to-be-processed multi-source data comprises historical load data of a first time granularity in a first preset time period, meteorological data of a second time granularity in a second preset time period and known special event information; the total duration of the first preset time period is smaller than the total duration of the second preset time period; the first time granularity is smaller than the second time granularity; inputting the to-be-processed multi-source data into a load prediction model to enable the load prediction model to extract load features, meteorological features, time features and event features, and fusing the extracted load features, meteorological features, time features and event features to obtain multi-source fusion features, and outputting load prediction data in a target prediction time period according to the multi-source fusion features. According to the invention, the accuracy of the output load prediction data in the target prediction time period can be improved.

Description

Technical Field

[0001] The present invention relates to the field of energy forecasting technology, and in particular to a load forecasting method, device, terminal equipment and storage medium based on feature fusion. Background Art

[0002] With the rapid development of smart grids and the large-scale access of renewable energy, the power load forecasting of power systems faces severe challenges due to the randomness and volatility of natural resources such as wind and light. Especially in the short-term load forecasting of power systems, there are high requirements for forecast accuracy. Accurate load forecasting is crucial to the real-time dispatching, economical and stable operation of power systems, and efficient use of renewable energy. The existing methods for forecasting short-term loads in power systems mainly include traditional statistical methods, traditional machine learning methods, integrated learning methods, and deep learning methods. These methods mainly extract single features from historical load data and then make short-term load forecasts based on the extracted features. However, the features extracted by this extraction method are single, and the influence of other factors on the load forecasting results is not considered, resulting in low load forecasting accuracy. When the load forecasting results are used in power grid dispatching, the role played by the stable operation of the auxiliary power grid is not obvious. Summary of the invention

[0003] The embodiments of the present invention provide a load forecasting method, apparatus, terminal device and storage medium based on feature fusion, which can improve the accuracy of load forecasting data output within a target forecasting period and improve the stability of power grid operation when the load forecasting data is used in power grid dispatching.

[0004] An embodiment of the present invention provides a load forecasting method based on feature fusion, comprising:

[0005] Acquire multi-source data to be processed; wherein the multi-source data to be processed includes: historical load data of a first time granularity within a first preset period, meteorological data of a second time granularity within a second preset period, and known special event information; the meteorological data includes: temperature, humidity, wind speed, and cloud cover; the known special event information includes: power outage plan information and large-scale power consumption activity information; the total duration of the first preset period is less than the total duration of the second preset period; the first time granularity is less than the second time granularity;

[0006] The multi-source data to be processed is input into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecast data within the target forecasting period according to the multi-source fusion characteristics.

[0007] Furthermore, the construction of the load forecasting model includes:

[0008] Acquire a multi-source data training sample set; wherein the multi-source data training sample set includes a plurality of multi-source data training samples, each of which includes a historical load data sample of a first time granularity, a historical meteorological data sample of a second time granularity, and a known special event information sample;

[0009] An initial load forecasting model is constructed, and the initial load forecasting model is iteratively trained with the multi-source data training samples until the loss function of the initial load forecasting model is minimized, thereby generating a load forecasting model.

[0010] Furthermore, after obtaining the multi-source data training sample set, it also includes:

[0011] Extract load sample features and meteorological sample features from the multi-source data training sample set; wherein the meteorological sample features include: temperature sample features, humidity sample features, wind speed sample features and cloud cover sample features;

[0012] According to the temperature sample characteristics and the load sample characteristics, a first interactive characteristic function characterizing the influence of temperature on load is constructed;

[0013] According to the humidity sample characteristics and the load sample characteristics, a second interactive characteristic function is constructed to characterize the influence of humidity on load;

[0014] According to the characteristics of wind speed samples and load samples, a third interactive characteristic function is constructed to characterize the influence of wind speed on load.

[0015] According to the cloud cover sample characteristics and load sample characteristics, the fourth interactive characteristic function of the component characterizing the influence of cloud cover on load is constructed;

[0016] Before iteratively training the initial load forecasting model, the first interactive characteristic function, the second interactive characteristic function, the third interactive characteristic function and the fourth interactive characteristic function are input into the initial load forecasting model.

[0017] Furthermore, the load forecasting model includes: an input layer, a feature extraction layer, a feature fusion layer and a multi-task output layer;

[0018] The step of inputting the to-be-processed multi-source data into a load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputting load forecasting data within a target forecasting period according to the multi-source fusion characteristics includes:

[0019] Inputting the multi-source data to be processed into the load forecasting model, so that the input layer receives the multi-source data to be processed, and transmits the multi-source data to be processed to the feature extraction layer, so that the feature extraction layer extracts the load characteristics, meteorological characteristics, time characteristics and event characteristics of the multi-source data to be processed;

[0020] The load characteristics, meteorological characteristics, time characteristics and event characteristics of the multi-source data to be processed are transmitted to the feature fusion layer, so that the feature fusion layer fuses the load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and the multi-source fusion characteristics are transmitted to the multi-task output layer, so that the multi-task output layer outputs the load forecast data within the target forecast period; wherein the load forecast data within the target forecast period includes: load forecast data of a first time granularity within a first forecast period and a second time granularity within a second forecast period; wherein the total duration of the first forecast period is equal to the total duration of the first preset period; and the total duration of the second forecast period is equal to the total duration of the second preset period.

[0021] Furthermore, after obtaining the multi-source data training sample set, it also includes:

[0022] Obtain historical load data samples from a multi-source data training sample set;

[0023] Performing outlier detection on the historical load data sample according to the Kalman filter algorithm to generate a first pre-processed historical load data sample;

[0024] The missing values ​​of the first preprocessed historical load data sample are filled according to the Kalman filter algorithm to generate a second preprocessed historical load data sample.

[0025] Based on the above method embodiment, the present invention provides a corresponding device embodiment;

[0026] An embodiment of the present invention provides a load prediction device based on feature fusion, including: a multi-source data acquisition module and a load prediction module;

[0027] The multi-source data acquisition module is used to acquire the multi-source data to be processed; wherein the multi-source data to be processed includes: historical load data of a first time granularity within a first preset period, meteorological data of a second time granularity within a second preset period, and known special event information; the meteorological data includes: temperature, humidity, wind speed, and cloud cover; the known special event information includes: power outage plan information and large-scale power consumption activity information; the total duration of the first preset period is less than the total duration of the second preset period; the first time granularity is less than the second time granularity;

[0028] The load forecasting module is used to input the multi-source data to be processed into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecast data within the target prediction period according to the multi-source fusion characteristics.

[0029] Furthermore, it also includes: a load forecasting model building module;

[0030] The load forecasting model building module is used to obtain a multi-source data training sample set; wherein the multi-source data training sample set includes a plurality of multi-source data training samples, each of which includes a historical load data sample of a first time granularity, a historical meteorological data sample of a second time granularity, and a known special event information sample;

[0031] An initial load forecasting model is constructed, and the initial load forecasting model is iteratively trained with the multi-source data training samples until the loss function of the initial load forecasting model is minimized, thereby generating a load forecasting model.

[0032] Furthermore, it also includes a feature interaction module;

[0033] The feature interaction module is used to extract load sample features and meteorological sample features in the multi-source data training sample set after obtaining the multi-source data training sample set; wherein the meteorological sample features include: temperature sample features, humidity sample features, wind speed sample features and cloud cover sample features;

[0034] According to the temperature sample characteristics and the load sample characteristics, a first interactive characteristic function characterizing the influence of temperature on load is constructed;

[0035] According to the humidity sample characteristics and the load sample characteristics, a second interactive characteristic function is constructed to characterize the influence of humidity on load;

[0036] According to the characteristics of wind speed samples and load samples, a third interactive characteristic function is constructed to characterize the influence of wind speed on load.

[0037] According to the cloud cover sample characteristics and load sample characteristics, the fourth interactive characteristic function of the component characterizing the influence of cloud cover on load is constructed;

[0038] Before iteratively training the initial load forecasting model, the first interactive characteristic function, the second interactive characteristic function, the third interactive characteristic function and the fourth interactive characteristic function are input into the initial load forecasting model.

[0039] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, a load forecasting method based on feature fusion described in the above-mentioned embodiment of the invention is implemented.

[0040] Another embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a load prediction method based on feature fusion described in the above-mentioned embodiment of the invention.

[0041] The following beneficial effects are achieved by implementing the present invention:

[0042] The present invention provides a load forecasting method, device, terminal device and storage medium based on feature fusion. The load forecasting method obtains the historical load data of the first time granularity in the first preset period, the meteorological data of the second time granularity in the second preset period and the known special event information to be processed multi-source data, and then inputs the multi-source data to be processed into the load forecasting model, so that the load forecasting model can extract load characteristics, meteorological characteristics, time characteristics and event characteristics, and then obtain the multi-source fusion characteristics according to the multi-extracted characteristics, and then output the load forecasting data in the target forecasting period according to the fused multi-source fusion characteristics. By inputting the multi-source data to be processed, the existing load forecasting is changed to rely only on the historical load data for prediction, and the influence of other factors on the load forecasting results is not considered, so that the load forecasting accuracy is low; further, through the load forecasting model, the multi-feature extraction and fusion of the multi-source data to be processed can be used to assist in the output of the load forecasting data in the target forecasting period when the load forecasting is carried out, so as to improve the accuracy of the load forecasting data in the output target forecasting period. Then, when the load forecasting data is used in the power grid dispatching, the stability of the power grid operation is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of a load forecasting method based on feature fusion provided by an embodiment of the present invention.

[0044] Figure 2 This is a comparison result diagram of predicted values ​​and actual values ​​of a load forecasting model based on Tramsformer architecture provided by an embodiment of the present invention and an existing LSTM load forecasting model.

[0045] Figure 3 This is a graph showing the error value comparison between the load forecasting model based on the Tramsformer architecture provided in one embodiment of the present invention and the existing LSTM load forecasting model.

[0046] Figure 4 It is a structural diagram of a load prediction device based on feature fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] like Figure 1 As shown, a load forecasting method based on feature fusion provided by an embodiment of the present invention includes:

[0049] Step S1: Acquire multi-source data to be processed; wherein the multi-source data to be processed includes: historical load data of a first time granularity within a first preset period, meteorological data of a second time granularity within a second preset period, and known special event information; the meteorological data includes: temperature, humidity, wind speed, and cloud cover; the known special event information includes: power outage plan information and large-scale power consumption activity information; the total duration of the first preset period is less than the total duration of the second preset period; the first time granularity is less than the second time granularity;

[0050] Step S2: Input the multi-source data to be processed into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecast data within the target forecasting period according to the multi-source fusion characteristics.

[0051] For step S1, obtain the multi-source data to be processed, wherein the obtained multi-source data to be processed include: historical load data with a 5-minute granularity (i.e., the first time granularity) 24 hours from the current time node (i.e., within the first preset time period); meteorological data with an hourly granularity (i.e., the second time granularity) for the next 7 days (i.e., the second time period) from the current time node; meteorological data can obtain the temperature, humidity, wind speed, and cloud cover for the next 7 days based on the forecast for the next 7 days. In addition, it is also necessary to obtain known special event information, such as power outage plan information, indicating that the load power consumption in the area involved in the power outage plan will decrease during the power outage, or decrease to 0; and information on large-scale power consumption activities can determine that the load power consumption at the location where the large-scale event is held will increase during the period based on the time and location of the large-scale event.

[0052] For step S2, the multi-source data to be processed obtained in step S1 is input into the load forecasting model, so that the load forecasting model extracts the load characteristics of historical load data, extracts the meteorological characteristics of meteorological data, extracts the event characteristics of known special event information, and the time characteristics contained in the historical load data and meteorological data; wherein the event characteristics include event location characteristics and event time characteristics; the time characteristics include: minutes, hours, days and months. The load forecasting model fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics including load characteristics, meteorological characteristics, time characteristics and event characteristics, and then outputs the load forecasting data within the target forecasting period according to the multi-source fusion characteristics. In the present invention, the load forecasting data within the target forecasting period includes: ultra-short-term load forecasting data with a 5-minute granularity for 24 hours from the current time node and short-term load forecasting data with an hourly granularity for 7 days from the current time node. These forecasting results are subjected to post-processing such as integrated forecasting and uncertainty quantification, and combined with interpretability analysis such as SHAP value, and finally presented to the dispatching personnel for use, thereby providing the dispatching personnel with more detailed and reasonable forecasting results.

[0053] It should be noted that in the invention, short-term load forecasting data refers to the forecast of the load of the power system from 1 day to 1 week from the current time node. The classification criteria are: the forecast time span is usually 1-7 days, and the forecast time granularity is hourly; through short-term load forecasting data, it is possible to update the load data of the power system once or more per hour within a week, and the accurate forecast of short-term load forecasting data has the effect of assisting in optimizing the power generation plan of the power system and improving the stability of the power grid operation during the power grid dispatching process. Ultra-short-term load forecasting data refers to the forecast of the load of the power system from 5 minutes to 24 hours from the current time node. The classification criteria are: the forecast time span is usually 5 minutes to 24 hours, and the forecast granularity is minute-level or 5-15 minutes; through ultra-short-term load forecasting data, it is possible to update the load data of the power system every 5-15 minutes within a day, and the accurate forecast of ultra-short-term load forecasting data has the beneficial effects of supporting real-time dispatching optimization, improving the utilization rate of renewable energy, enhancing the stability of the power grid, optimizing auxiliary services, and supporting demand response.

[0054] In a preferred embodiment, the load forecasting model comprises: an input layer, a feature extraction layer, a feature fusion layer and a multi-task output layer; the multi-source data to be processed is input into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecasting data within the target forecasting period according to the multi-source fusion characteristics, including: inputting the multi-source data to be processed into the load forecasting model so that the input layer receives the multi-source data to be processed, and transmits the multi-source data to the feature extraction layer, so that the feature extraction layer extracts the load characteristics of the multi-source data to be processed; The load characteristics, meteorological characteristics, time characteristics and event characteristics of the multi-source data to be processed are transmitted to the feature fusion layer, so that the feature fusion layer fuses the load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and the multi-source fusion characteristics are transmitted to the multi-task output layer, so that the multi-task output layer outputs the load forecast data within the target forecast period; wherein the load forecast data within the target forecast period includes: the load forecast data of the first time granularity within the first forecast period and the second time granularity within the second forecast period; wherein the total duration of the first forecast period is equal to the total duration of the first preset period; the total duration of the second forecast period is equal to the total duration of the second preset period.

[0055] In a preferred embodiment, the construction of the load forecasting model includes: obtaining a multi-source data training sample set; wherein the multi-source data training sample set includes a plurality of multi-source data training samples, each of the multi-source data training samples includes a historical load data sample of a first time granularity, a historical meteorological data sample of a second time granularity, and a known special event information sample; constructing an initial load forecasting model, and iteratively training the initial load forecasting model with the multi-source data training samples until the loss function of the initial load forecasting model is minimized, thereby generating a load forecasting model.

[0056] Specifically, first obtain a multi-source data training sample set; the multi-source data training sample set includes a number of multi-source data training samples, each of which includes a historical load data sample at a 5-minute granularity (i.e., the first time granularity mentioned above), a historical meteorological data sample at an hourly granularity (i.e., the second time granularity mentioned above), and a known special event information sample. The historical load data samples can be obtained from the power system, and the historical meteorological data can be obtained based on historical meteorological forecasts. The historical load data samples and historical meteorological data samples obtained can be historical load data samples and historical meteorological data samples in a historical time series, and both the historical load data samples and the historical meteorological data samples carry a timestamp for identifying the load data samples and meteorological data samples in each historical time period.

[0057] In a preferred embodiment, after obtaining a multi-source data training sample set, it also includes: obtaining historical load data samples in the multi-source data training sample set; performing outlier detection on the historical load data samples according to a Kalman filtering algorithm to generate a first preprocessed historical load data sample; performing missing value filling on the first preprocessed historical load data sample according to a Kalman filtering algorithm to generate a second preprocessed historical load data sample.

[0058] Specifically, after completing the acquisition of the multi-source data training sample set, for the historical load data samples in the multi-source data training sample set, it is necessary to perform outlier detection and missing value filling on the historical load data samples according to the Kalman filter algorithm to obtain a second pre-processed historical load data sample, and replace the historical load data samples in the multi-source data training sample with the second pre-processed historical load data sample as the basis for the subsequent training load prediction model. The Kalman filter algorithm processes the historical load data samples as follows:

[0059] x t =Fx t-1 +w t ,w t ~N(0,Q)

[0060] z t =Hx t +v t ,v t ~N(0,R)

[0061] Among them, x t is the state vector of the load data at time t; x t-1 is the state vector of the load data at time t-1; F is the load state transfer matrix; w t is the noise in the load forecasting process, which obeys the normal distribution with mean 0 and covariance Q; t is the load observation vector value at time t; H is the load observation matrix; v t is the observation noise in the load forecasting process, which obeys the normal distribution with mean 0 and covariance R.

[0062] Preferably, after completing the preprocessing of the historical load data samples, a multi-scale periodic position coding is designed, taking into account the periodic characteristics of 5 minutes, 1 hour, and 1 day, and realizing multi-scale time coding; specifically, it is obtained by the following formula:

[0063] PE(t,i)=sin(t / τ i )+cos(t / τ i )

[0064] Among them, PE(t,i) is the load time position code of the i-th dimension at time t; t is the specific time step; τi is the period of the i-th dimension, corresponding to load periods of 5 minutes, 1 hour, and 1 day respectively.

[0065] In a preferred embodiment, after obtaining a multi-source data training sample set, it also includes: extracting load sample features and meteorological sample features in the multi-source data training sample set; wherein the meteorological sample features include: temperature sample features, humidity sample features, wind speed sample features and cloud cover sample features; based on the temperature sample features and the load sample features, constructing a first interactive feature function characterizing the impact of temperature on load; based on the humidity sample features and the load sample features, constructing a second interactive feature function characterizing the impact of humidity on load; based on the wind speed sample features and the load sample features, constructing a third interactive feature function characterizing the impact of wind speed on load; based on the cloud cover sample features and the load sample features, constructing a fourth interactive feature function characterizing the impact of cloud cover on load; before iteratively training the initial load forecasting model, inputting the first interactive feature function, the second interactive feature function, the third interactive feature function and the fourth interactive feature function into the initial load forecasting model.

[0066] Specifically, for the multi-source data training sample set processed above, it is necessary to clarify the impact of each influencing factor on the load and the interaction characteristics between the factors through feature engineering. It is mainly through calculating the first-order and second-order differences of the load changes in the historical load data samples, capturing the load change trend and constructing the load inverse feature, and then using the piecewise linear function to model the nonlinear impact of each factor on the load, that is, to determine the interaction characteristic function between each factor and the load.

[0067] Before establishing the interactive feature function, first determine the eigenvalue of each sample feature after transformation according to the following formula:

[0068]

[0069] Wherein, T′ represents the transformed characteristic value, including the transformed load sample characteristics, temperature sample characteristics, humidity sample characteristics, wind speed sample characteristics and cloud cover sample characteristics; T represents the corresponding original characteristic value, including the load sample characteristics, temperature sample characteristics, humidity sample characteristics, wind speed sample characteristics and cloud cover sample characteristics before transformation; a i , b i is the slope and intercept of each segment, T i The segmentation points for quantification of each factor.

[0070] According to the above formula, the historical load data samples and historical meteorological data samples are respectively transformed nonlinearly to obtain the transformed load sample characteristics, temperature sample characteristics, humidity sample characteristics, wind speed sample characteristics and cloud cover sample characteristics. The transformed load sample characteristics are combined with the time characteristics to obtain the load sample characteristics with time characteristics, that is, the load sample characteristics with time granularity of 5 minutes, 1 hour and 1 day. Then determine the interactive relationship between the load sample characteristics and each meteorological sample characteristic, specifically, according to the temperature sample characteristics and the load sample characteristics, construct a first interactive characteristic function that characterizes the influence of temperature on the load; according to the humidity sample characteristics and the load sample characteristics, construct a second interactive characteristic function that characterizes the influence of humidity on the load; according to the wind speed sample characteristics and the load sample characteristics, construct a third interactive characteristic function that characterizes the influence of wind speed on the load; according to the cloud cover sample characteristics and the load sample characteristics, construct a fourth interactive characteristic function that characterizes the influence of cloud cover on the load. Preferably, it is also necessary to consider the influence of known special event information on the load sample characteristics, that is, further extract the event sample characteristics of the known special event information, and determine its interactive function with the load sample characteristics. Through the above processing, various factors affecting load changes and their complex interactive relationships can be comprehensively considered. Before iterative training of the initial load forecasting model, the first interactive characteristic function, the second interactive characteristic function, the third interactive characteristic function and the fourth interactive characteristic function are input into the initial load forecasting model, thereby providing richer input information for the training of the initial load forecasting model and improving the prediction accuracy of subsequent model training.

[0071] Before model training, an initial load forecasting model is first built based on the Transformer architecture, which includes an input layer, a feature extraction layer, a feature fusion layer, and a multi-task output layer. The Transformer architecture is a deep learning model architecture based on the self-attention mechanism, which can be used for natural language processing tasks and has been widely used in various sequence data processing fields.

[0072] After completing the construction of the initial load forecasting model, preferably, a multi-scale time attention mechanism is set to focus on the patterns at the scales of 5 minutes, 1 hour, and 1 day, respectively, as shown below:

[0073] MultiScaleAttn(Q,K,V)=Concat(Attn5min,Attnhour,Attn day )W O

[0074] Among them, MultiScaleAttn ​​is the multi-scale load attention function; Attn5min is the attention output of 5 minutes time scale; Attnhour is the attention output of 1 hour time scale; Attn day is the attention output on a 1-day time scale; WO Output the projection matrix for the load.

[0075] The feature fusion layer is constructed so that it can perform adaptive input fusion of large models for various input features. In the present invention, a gating mechanism is used to dynamically fuse different types of inputs:

[0076] x fused =σ(W g [x load ;x weat h er ;x time ;x event ])⊙[x load ;x weat h er ;x time ;x event ]

[0077] Among them, x fused is the input characteristic value of the integrated load after fusion, that is, the multi-source fusion feature; σ is the Sigmoid nonlinear activation function; W g is the gating weight matrix of the load feature; x load is the load characteristic; x weat h er is a meteorological feature; x time is the time feature; x event It is the event characteristic.

[0078] After constructing the feature fusion layer, it is necessary to construct the residual connection in the Transformer architecture and the normalization design of the Transformer layer, that is, to construct the multi-task output layer. The constructed multi-task output layer must ensure that the output of each Transformer block is:

[0079] H l =LayerNorm(H l-1 +MultiScaleAttn(H l-1 ))

[0080] H l =LayerNorm(H l +FFN(H l ))

[0081] Among them, H l is the load feature output of the lth layer; LayerNorm is the layer normalization function; MultiScaleAttn ​​is the multi-scale load attention function; FFN is a feedforward neural network used to further process the load features.

[0082] Based on the fine-tuned Transformer architecture, the multi-task output layer needs to be redesigned. In the present invention, two output heads are designed in the multi-task output layer, which are used for the output of short-term load forecasting data and ultra-short-term load forecasting data, respectively, as shown below:

[0083] Y ultra-short =Linear(H L )

[0084] Y short =Linear(AvgPool(H L ))

[0085] Among them, Y ultra-short It is the ultra-short-term load forecast data output, which is the load forecast data with a 5-minute time granularity for the next 24 hours; Y short It is the short-term load forecast data output, which is the load forecast data with a time granularity of 1 hour for the next 7 days; H L It is the output of the last layer of Transformer; Linear is the linear layer used to process the ultra-short-term load forecast data; AvgPool is the average pooling operation, which is used to pool the ultra-short-term load forecast data channels, and average pooling is performed independently on each channel to obtain the global average information of each channel and thus obtain the short-term load forecast data.

[0086] After completing the above steps, the construction of the initial load forecasting model is completed, and then the initial load forecasting model is iteratively trained according to the multi-source data training sample set processed above until the loss function of the initial load forecasting model is minimized, and the load forecasting model is generated.

[0087] Specifically, the historical load data samples in the multi-source data training sample set are first masked, that is, when the data of a certain period of historical load data is used to predict the load of the next N time steps, the masked self-encoder is first used to randomly mask the load value in the input sequence, and then the initial load forecasting model is allowed to perform load forecasting on the masked load value, and the prediction result is compared with the true load value of the masked part, and the corresponding loss function is obtained as follows:

[0088] Lossmae=MSE(Xmasked,X pred );

[0089]

[0090] Among them, Lossmae is the masked self-encoding loss; MSE is the mean square error function; Xmasked is the masked input; X predis the prediction output of the initial load prediction model; Lossnext is the loss difference between the predicted data and the actual data; w i is the weight of the prediction at step i; X t+i is the real load data; Forecast load data.

[0091] In the process of training the initial load forecasting model, the multi-source training data sample set is decomposed into multiple historical load data samples with a time granularity of 7 days and 1 hour, historical load data samples with a time granularity of 1 day and 5 minutes, and historical meteorological data samples with a time granularity of 7 days; it is divided into multiple groups of training input data including historical load data samples with a time granularity of 7 days and 1 hour, historical meteorological data samples with a time granularity of 7 days and special event information samples, and historical load data samples with a time granularity of 1 day and 5 minutes, historical meteorological data samples with a time granularity of 7 days and special event information samples as the input of the initial load forecasting model training. At the same time, about 30% of the load values ​​in the sequence of the input historical load data samples are randomly replaced with special mask tags, and the prediction ability of the initial load forecasting model is improved through mask language modeling. During the training process, the output of the initial load forecasting model corresponds to the load forecast data with a time granularity of 5 minutes for the next day and the load forecast data with a time granularity of hours for the next 7 days for each group of time. Through this multi-task learning strategy, combined with multi-source data input, the model can not only capture the load change patterns at different time scales, but also effectively reduce the risk of relying solely on historical load for prediction, significantly improving the prediction accuracy and generalization ability of the model.

[0092] Furthermore, the prediction accuracy of the model is adjusted, including: adopting a dynamic sampling strategy to dynamically adjust the sample weight according to the prediction difficulty and importance:

[0093] w sample =f(prediction error ,load magnitude ,time importance )

[0094] Among them, w sample is the sample weight; f is the weight calculation function; prediction error is the prediction error; load magnitude is the load size; time importance For the importance of time.

[0095] After the weight adjustment is completed, multi-task joint training is performed by combining the loss difference of ultra-short-term load forecasting, short-term load forecasting and other task requirements:

[0096] Losstotal=w1·Loss-ultrashort+w2·Loss-short+w3·Loss-more

[0097] Among them, Losstotal is the total loss value of the initial load forecasting model; w1 is the loss weight of the ultra-short-term load forecasting task; w2 is the loss weight of the short-term load forecasting task; w3 is the loss weight of other task requirements.

[0098] On the basis of the above, adversarial perturbations are added to improve the robustness of the model:

[0099]

[0100] Lossadv=Loss(θ,xadv,y)

[0101] Among them, x adv is the input after adding adversarial disturbance; ∈ is the set disturbance size; is the gradient of loss with respect to input; Lossadv is the adversarial training loss.

[0102] To improve the prediction stability of the model, use model ensemble to achieve ensemble prediction:

[0103]

[0104] Among them, y pred is the integrated prediction result; w i is the weight of each prediction model; y i is the output result of each prediction model.

[0105] Use deep ensembles and Monte Carlo Dropout to estimate prediction intervals, thereby quantifying uncertainty:

[0106]

[0107]

[0108] Among them, μ pred is the predicted mean; is the prediction variance; M is the number of models; μ m (x) is the predicted mean of the mth model for input x; is the prediction variance of the mth model for input x.

[0109] The output prediction results are processed for interpretability, multi-scale attention weights are analyzed, key time points and features are identified, and the present invention uses integrated SHAP values ​​to explain the prediction results:

[0110]

[0111] Among them, φ i is the SHAP value of feature i, SHAP m (i) is the SHAP value of feature i in the mth model.

[0112] In order to better demonstrate the effect that can be achieved by the present invention, the present invention uses the load forecasting model based on the Tramsformer architecture of the present invention and the existing LSTM load forecasting model to train the simulated 3-year hourly power system data (including time, weather, power generation and other characteristics) in a simulated large-scale renewable energy power generation access scenario, and predicts the 100-hour test set. Figure 2 and Figure 3 The results in the figure show the prediction results of 100 hours in the test set. The comparison results of the predicted values ​​and actual values ​​of the two methods are as follows: Figure 2 As shown in the figure, the error values ​​of the two methods are Figure 3 As shown, it can be seen that the load forecasting model based on the Tramsformer architecture of the present invention is accurate and effective in short-term load forecasting tasks.

[0113] Based on the above method embodiment, the present invention provides a corresponding device embodiment.

[0114] like Figure 4 As shown, an embodiment of the present invention provides a load prediction device based on feature fusion, including: a multi-source data acquisition module and a load prediction module;

[0115] The multi-source data acquisition module is used to acquire the multi-source data to be processed; wherein the multi-source data to be processed includes: historical load data of a first time granularity within a first preset period, meteorological data of a second time granularity within a second preset period, and known special event information; the meteorological data includes: temperature, humidity, wind speed, and cloud cover; the known special event information includes: power outage plan information and large-scale power consumption activity information; the total duration of the first preset period is less than the total duration of the second preset period; the first time granularity is less than the second time granularity;

[0116] The load forecasting module is used to input the multi-source data to be processed into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecast data within the target prediction period according to the multi-source fusion characteristics.

[0117] In a preferred embodiment, it also includes: a load forecasting model building module;

[0118] The load forecasting model building module is used to obtain a multi-source data training sample set; wherein the multi-source data training sample set includes a plurality of multi-source data training samples, each of which includes a historical load data sample of a first time granularity, a historical meteorological data sample of a second time granularity, and a known special event information sample;

[0119] An initial load forecasting model is constructed, and the initial load forecasting model is iteratively trained with the multi-source data training samples until the loss function of the initial load forecasting model is minimized, thereby generating a load forecasting model.

[0120] In a preferred embodiment, it also includes a feature interaction module;

[0121] The feature interaction module is used to extract load sample features and meteorological sample features in the multi-source data training sample set after obtaining the multi-source data training sample set; wherein the meteorological sample features include: temperature sample features, humidity sample features, wind speed sample features and cloud cover sample features;

[0122] According to the temperature sample characteristics and the load sample characteristics, a first interactive characteristic function characterizing the influence of temperature on load is constructed;

[0123] According to the humidity sample characteristics and the load sample characteristics, a second interactive characteristic function is constructed to characterize the influence of humidity on load;

[0124] According to the characteristics of wind speed samples and load samples, a third interactive characteristic function is constructed to characterize the influence of wind speed on load.

[0125] According to the cloud cover sample characteristics and load sample characteristics, the fourth interactive characteristic function of the component characterizing the influence of cloud cover on load is constructed;

[0126] Before iteratively training the initial load forecasting model, the first interactive characteristic function, the second interactive characteristic function, the third interactive characteristic function and the fourth interactive characteristic function are input into the initial load forecasting model.

[0127] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0128] Those skilled in the art can clearly understand that for the sake of convenience and brevity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0129] Based on the above method item embodiments, the present invention provides corresponding terminal device item embodiments.

[0130] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a load forecasting method based on feature fusion as described in any one of the present invention is implemented.

[0131] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0132] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0133] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0134] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment.

[0135] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute a load prediction method based on feature fusion as described in any one of the present inventions.

[0136] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium.

[0137] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A load forecasting method based on feature fusion, characterized in that: include: Acquire multi-source data to be processed; wherein the multi-source data to be processed includes: historical load data of a first time granularity within a first preset period, meteorological data of a second time granularity within a second preset period, and known special event information; the meteorological data includes: temperature, humidity, wind speed, and cloud cover; the known special event information includes: power outage plan information and large-scale power consumption activity information; the total duration of the first preset period is less than the total duration of the second preset period; the first time granularity is less than the second time granularity; The multi-source data to be processed is input into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecast data within the target forecasting period according to the multi-source fusion characteristics.

2. A load forecasting method based on feature fusion as claimed in claim 1, characterized in that: The construction of the load forecasting model includes: Acquire a multi-source data training sample set; wherein the multi-source data training sample set includes a plurality of multi-source data training samples, each of which includes a historical load data sample of a first time granularity, a historical meteorological data sample of a second time granularity, and a known special event information sample; An initial load forecasting model is constructed, and the initial load forecasting model is iteratively trained with the multi-source data training samples until the loss function of the initial load forecasting model is minimized, thereby generating a load forecasting model.

3. A load forecasting method based on feature fusion as claimed in claim 2, characterized in that: After obtaining the multi-source data training sample set, it also includes: Extract load sample features and meteorological sample features from the multi-source data training sample set; The meteorological sample features include: temperature sample features, humidity sample features, wind speed sample features and cloud cover sample features; According to the temperature sample characteristics and the load sample characteristics, a first interactive characteristic function characterizing the influence of temperature on load is constructed; According to the humidity sample characteristics and the load sample characteristics, a second interactive characteristic function is constructed to characterize the influence of humidity on load; According to the characteristics of wind speed samples and load samples, a third interactive characteristic function is constructed to characterize the influence of wind speed on load. According to the cloud cover sample characteristics and load sample characteristics, the fourth interactive characteristic function of the component characterizing the influence of cloud cover on load is constructed; Before iteratively training the initial load forecasting model, the first interactive characteristic function, the second interactive characteristic function, the third interactive characteristic function and the fourth interactive characteristic function are input into the initial load forecasting model.

4. The load forecasting method based on feature fusion according to claim 1, characterized in that: The load forecasting model includes: an input layer, a feature extraction layer, a feature fusion layer and a multi-task output layer; The step of inputting the to-be-processed multi-source data into a load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputting load forecasting data within a target forecasting period according to the multi-source fusion characteristics includes: Inputting the multi-source data to be processed into the load forecasting model, so that the input layer receives the multi-source data to be processed, and transmits the multi-source data to be processed to the feature extraction layer, so that the feature extraction layer extracts the load characteristics, meteorological characteristics, time characteristics and event characteristics of the multi-source data to be processed; The load characteristics, meteorological characteristics, time characteristics and event characteristics of the multi-source data to be processed are transmitted to the feature fusion layer, so that the feature fusion layer fuses the load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and the multi-source fusion characteristics are transmitted to the multi-task output layer, so that the multi-task output layer outputs the load forecast data within the target forecast period; wherein the load forecast data within the target forecast period includes: load forecast data of a first time granularity within a first forecast period and a second time granularity within a second forecast period; wherein the total duration of the first forecast period is equal to the total duration of the first preset period; and the total duration of the second forecast period is equal to the total duration of the second preset period.

5. The load forecasting method based on feature fusion according to claim 2, characterized in that: After obtaining the multi-source data training sample set, it also includes: Obtain historical load data samples from a multi-source data training sample set; Performing outlier detection on the historical load data sample according to the Kalman filter algorithm to generate a first pre-processed historical load data sample; The missing values ​​of the first preprocessed historical load data sample are filled according to the Kalman filter algorithm to generate a second preprocessed historical load data sample.

6. A load forecasting device based on feature fusion, characterized in that: include: Multi-source data acquisition module and load forecasting module; The multi-source data acquisition module is used to acquire the multi-source data to be processed; wherein the multi-source data to be processed includes: historical load data of a first time granularity within a first preset period, meteorological data of a second time granularity within a second preset period, and known special event information; the meteorological data includes: temperature, humidity, wind speed, and cloud cover; the known special event information includes: power outage plan information and large-scale power consumption activity information; the total duration of the first preset period is less than the total duration of the second preset period; the first time granularity is less than the second time granularity; The load forecasting module is used to input the multi-source data to be processed into the load forecasting model so that the load forecasting model extracts load characteristics, meteorological characteristics, time characteristics and event characteristics, and fuses the extracted load characteristics, meteorological characteristics, time characteristics and event characteristics to obtain multi-source fusion characteristics, and outputs the load forecast data within the target prediction period according to the multi-source fusion characteristics.

7. A load forecasting device based on feature fusion as claimed in claim 6, characterized in that: Also includes: Load forecasting model building module; The load forecasting model building module is used to obtain a multi-source data training sample set; wherein the multi-source data training sample set includes a plurality of multi-source data training samples, each of which includes a historical load data sample of a first time granularity, a historical meteorological data sample of a second time granularity, and a known special event information sample; An initial load forecasting model is constructed, and the initial load forecasting model is iteratively trained with the multi-source data training samples until the loss function of the initial load forecasting model is minimized, thereby generating a load forecasting model.

8. A load forecasting device based on feature fusion as claimed in claim 7, characterized in that: It also includes a feature interaction module; The feature interaction module is used to extract the load sample features and the meteorological sample features in the multi-source data training sample set after obtaining the multi-source data training sample set; The meteorological sample features include: temperature sample features, humidity sample features, wind speed sample features and cloud cover sample features; According to the temperature sample characteristics and the load sample characteristics, a first interactive characteristic function characterizing the influence of temperature on load is constructed; According to the humidity sample characteristics and the load sample characteristics, a second interactive characteristic function is constructed to characterize the influence of humidity on load; According to the characteristics of wind speed samples and load samples, a third interactive characteristic function is constructed to characterize the influence of wind speed on load. According to the cloud cover sample characteristics and load sample characteristics, the fourth interactive characteristic function of the component characterizing the influence of cloud cover on load is constructed; Before iteratively training the initial load forecasting model, the first interactive characteristic function, the second interactive characteristic function, the third interactive characteristic function and the fourth interactive characteristic function are input into the initial load forecasting model.

9. A terminal device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements a load forecasting method based on feature fusion as described in any one of claims 1 to 5.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute a load forecasting method based on feature fusion as described in any one of claims 1 to 5.

Images