Load prediction model training method, prediction method, device and electronic equipment

Abstract

The application provides a load prediction model training method, a load prediction model prediction method, a load prediction model training device, a load prediction model prediction device and electronic equipment. A sample training set is obtained, an initial load prediction model is trained based on the sample training set, a load prediction model is obtained, and the initial load prediction model comprises a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. Through the set model structure, the load prediction model can improve the accuracy of load prediction.

Description

Technical Field

[0001] This application belongs to the field of load forecasting technology, and in particular relates to a training method, forecasting method, device and electronic equipment for a load forecasting model. Background Technology

[0002] With the continuous development of smart grids and energy management, accurate load forecasting is crucial for ensuring the safe and stable operation of the power grid, optimizing resource allocation, and reducing operating costs. However, electricity load data is influenced by various factors, exhibiting complex multi-timescale characteristics. Load patterns differ significantly across different timescales; for example, hourly loads show morning and evening peaks and valleys, daily loads show differences between weekdays and weekends, and weekly or monthly loads exhibit periodic fluctuations due to seasonality, holidays, and other factors. Furthermore, load data is closely related to multiple source variables such as weather and time, and these variables have complex dependencies across multiple timescales. Traditional statistical models, based on linear assumptions, struggle to capture nonlinear changes, have weak generalization capabilities, and low prediction accuracy. Existing deep learning models, on the other hand, often focus on single-dimensional dependency modeling, failing to comprehensively characterize the complex relationships between multi-timescale load sequences and multiple source variables, resulting in load forecast accuracy that falls short of practical requirements. Summary of the Invention

[0003] This application provides a training method, prediction method, apparatus, and electronic device for a load forecasting model, which can improve the accuracy of load forecasting.

[0004] In a first aspect, embodiments of this application provide a method for training a load prediction model, comprising:

[0005] Obtain a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence;

[0006] The initial load prediction model is trained based on the sample training set to obtain a load prediction model. The initial load prediction model includes: a multi-timescale feature extraction layer, an embedding layer, a multi-scale intervariate modeling module, a inter-scale modeling module, and a prediction module. The multi-timescale feature extraction layer is used to input sample timestamp sequences, sample load sequences, and sample weather sequences, and output load sequences, weather sequences, and timestamp sequences at multiple time scales. The embedding layer is used to input load sequences, weather sequences, and timestamp sequences at multiple time scales, and output load features, weather features, and time features at multiple time scales. The multi-scale intervariate modeling module is used to input load features, weather features, time features, and global block features at multiple time scales, and output updated load features and global block features at multiple time scales. The inter-scale modeling module is used to input updated load features and global block features at multiple time scales, and output updated load features and global block features at the smallest time scale. The prediction module is used to input updated load features and global block features at the smallest time scale, and output predicted load.

[0007] In some embodiments, the embedding layer is used to divide the sample load sequence into time-scale load sequences and encode the time-scale load sequences into multiple time-scale load features through a linear layer. The embedding layer is also used to encode all time-scale sample weather variables into a single feature using a linear layer to obtain weather features. The embedding layer is also used to map target time attributes in all timestamps into features and encode the mapped features into a single feature using a linear layer to obtain time features. The embedding layer is also used to concatenate the load features at multiple time scales and the global block features and output the concatenated features to the multi-scale intervariate modeling module.

[0008] In some embodiments, the multiscale transvariate modeling module includes:

[0009] The system consists of a self-attention mechanism submodule and a cross-variable dependency modeling submodule. The self-attention mechanism submodule takes the concatenated features as input, obtains the temporal dependency between the load features and global block features at each time scale based on the self-attention mechanism, updates the load features and global block features at multiple time scales based on the dependency, inputs the updated load features at multiple time scales into the cross-scale modeling module, and outputs the updated global block features at multiple time scales into the cross-variable dependency modeling submodule. The cross-variable dependency modeling submodule outputs updated global block features at multiple time scales based on the updated global block features at multiple time scales, the time features, and the weather features.

[0010] In some embodiments, the cross-variable dependency modeling submodule is used to take the updated global block features as a query for any time scale, concatenate the weather features and time features as a key value, and use the key value as the input of the cross-attention mechanism to output the updated global block features for multiple time scales.

[0011] In some embodiments, the cross-scale modeling module uses a top-down cross-attention mechanism to exchange load features and global block features at different time scales for updated load features and global block features at multiple time scales, and outputs the load features and global block features at the smallest time scale.

[0012] In some embodiments, obtaining the sample training set includes:

[0013] Retrieve load data and weather data corresponding to the timestamp data;

[0014] The load data and weather data are preprocessed to obtain preprocessed load data and weather data;

[0015] Based on a preset sliding step size, the preprocessed load data, weather data, and timestamp data are divided into sample load sequences, sample weather sequences, and sample timestamp sequences to obtain the sample training set.

[0016] Secondly, embodiments of this application provide a prediction method, including:

[0017] Retrieve load data and weather data corresponding to historical timestamps;

[0018] The predicted load is obtained by inputting the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load prediction model trained by the method described in any one of the first aspects.

[0019] Thirdly, embodiments of this application provide a training apparatus for a load prediction model, comprising:

[0020] The first acquisition module is used to acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence;

[0021] The training module is used to train the initial load prediction model based on the sample training set to obtain the load prediction model. The initial load prediction model includes: a multi-timescale feature extraction layer, an embedding layer, a multi-scale intervariate modeling module, a cross-scale modeling module, and a prediction module. The multi-timescale feature extraction layer is used to input sample timestamp sequences, sample load sequences, and sample weather sequences, and output load sequences, weather sequences, and timestamp sequences at multiple time scales. The embedding layer is used to input load sequences, weather sequences, and timestamp sequences at multiple time scales, and output load features, weather features, and time features at multiple time scales. The multi-scale intervariate modeling module is used to input load features, weather features, time features, and global block features at multiple time scales, and output updated load features and global block features at multiple time scales. The cross-scale modeling module is used to input updated load features and global block features at multiple time scales, and output updated load features and global block features at the smallest time scale. The prediction module is used to input updated load features and global block features at the smallest time scale, and output predicted load.

[0022] Fourthly, embodiments of this application provide a prediction device, comprising:

[0023] The second acquisition module is used to acquire load data and weather data corresponding to historical timestamps;

[0024] The prediction module is used to input the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load prediction model trained by the method described in any one of the first aspects, so as to obtain the predicted load.

[0025] Fifthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the above-mentioned embodiments.

[0026] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any of the preceding claims.

[0027] In a seventh aspect, embodiments of this application provide a computer program product that, when run on a terminal device, causes an electronic device to execute any of the methods described above.

[0028] The beneficial effects of the embodiments in this application compared with the prior art are:

[0029] This application provides a method for training a load forecasting model. The method involves acquiring a sample training set, wherein each sample data in the training set includes a sample timestamp sequence and corresponding sample load and weather sequences. An initial load forecasting model is trained based on the sample training set to obtain the model. The initial load forecasting model includes a multi-timescale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module, and a forecasting module. The multi-timescale feature extraction layer takes as input sample timestamp sequences, sample load sequences, and sample weather sequences, and outputs load sequences, weather sequences, and timestamp sequences at multiple time scales. The embedding layer takes as input multiple timescale sequences... The system generates load sequences, weather sequences, and timestamp sequences at multiple time scales and outputs load features, weather features, and time features at multiple time scales. The multi-scale cross-variable modeling module is used to input load features, weather features, time features, and global block features at multiple time scales and output updated load features and global block features at multiple time scales. The cross-scale modeling module is used to input updated load features and global block features at multiple time scales and output updated load features and global block features at the smallest time scale. The prediction module is used to input updated load features and global block features at the smallest time scale and output predicted load. Through the set model structure, the load prediction model can improve the accuracy of load prediction. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic diagram illustrating the implementation process of a training method for a load forecasting model provided for the implementation of this application;

[0032] Figure 2 A schematic diagram of the framework of an initial load prediction model provided in an embodiment of this application;

[0033] Figure 3 A schematic diagram illustrating the implementation process of a training method provided in an embodiment of this application;

[0034] Figure 4 A schematic diagram illustrating the implementation process of a prediction method provided for the purposes of this application;

[0035] Figure 5 A schematic diagram of the structure of a training device for a load prediction model provided in an embodiment of this application;

[0036] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0037] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0038] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0039] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0040] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected," or "in response to detection."

[0041] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0042] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0043] Based on the problems in related technologies, this application provides a training method for a load prediction model that can be applied to electronic devices. Electronic devices may include: mobile phones, tablets, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device. Figure 1 A schematic diagram illustrating the implementation process of a training method for a load forecasting model provided for the purposes of this application is shown below. Figure 1 As shown, the training methods for the load forecasting model include:

[0044] Step S101: Obtain the sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence.

[0045] In this embodiment of the application, the sample training set is a collection containing historical load data, weather data, and corresponding timestamp data, which is used to train the load prediction model.

[0046] In this embodiment of the application, historical load data, weather data and corresponding timestamp data can be collected, preprocessed (such as eliminating outliers and missing values, and standardizing), and then the data can be divided using a sliding window to form a sample training set.

[0047] Step S102: Train the initial load prediction model based on the sample training set to obtain the load prediction model. The initial load prediction model includes: a multi-timescale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module, and a prediction module. The multi-timescale feature extraction layer is used to input sample timestamp sequences, sample load sequences, and sample weather sequences, and output load sequences, weather sequences, and timestamp sequences at multiple time scales. The embedding layer is used to input load sequences, weather sequences, and timestamp sequences at multiple time scales, and output load features, weather features, and time features at multiple time scales. The multi-scale cross-variable modeling module is used to input load features, weather features, time features, and global block features at multiple time scales, and output updated load features and global block features at multiple time scales. The cross-scale modeling module is used to input updated load features and global block features at multiple time scales, and output updated load features and global block features at the minimum time scale. The prediction module is used to input updated load features and global block features at the minimum time scale, and output predicted load.

[0048] In this embodiment of the application, the initial load forecasting model is an untrained load forecasting model framework containing multiple modules for learning the complex relationship between load data and multi-source variables (such as weather). Figure 2 A schematic diagram of the framework of an initial load forecasting model provided in this application embodiment is shown below. Figure 2 As shown, the initial load forecasting model includes: a multi-timescale feature extraction layer, an embedding layer, a multi-scale intervariate modeling module, a cross-scale modeling module, and a forecasting module. The multi-timescale feature extraction layer extracts features from time-series data (load series, weather series, timestamp series) at multiple time scales, processing the series data at different time scales through one-dimensional convolutional layers and average pooling layers. The embedding layer converts discrete time-series data (such as load values, weather variable values, and timestamps) into continuous vector representations, i.e., features. Load series use block embedding, while weather and timestamp series use variable embedding and time feature embedding, respectively. The multi-scale intervariate modeling module models the complex dependencies between load features and multi-source variable features (weather features, time features) at each time scale. This module includes a self-attention mechanism submodule and an intervariate dependency modeling submodule. The cross-scale modeling module models the complex dependencies between load features at different time scales, employing a top-down cross-scale attention mechanism to allow information from larger scales to influence feature representations at smaller scales. This module includes a cross-scale dependency modeling layer and a fully connected layer. The prediction module is used to output future load forecasts based on the feature representations learned by the model. The prediction module typically contains a linear layer and a random deactivation layer.

[0049] In this embodiment, the initial load prediction model is iteratively trained using a sample training set. Model parameters are updated via backpropagation until the model converges or reaches a specified number of iterations, thus obtaining the load prediction model. The multi-timescale feature extraction layer extracts features at different time scales from the original sequence using one-dimensional convolutional layers and average pooling layers. This multi-timescale feature extraction module can capture the changing patterns of load data at different time scales, improving the model's adaptability to complex load changes. The embedding layer converts the load sequence, weather sequence, and timestamp sequence into load features, weather features, and time features, respectively. The embedding layer can convert discrete data into continuous vector representations, facilitating the model's processing and learning of complex relationships between data. The multi-scale intervariate modeling module models the dependencies between load features and multi-source variable features at each time scale through a self-attention mechanism submodule and an intervariate dependency modeling submodule. This multi-scale intervariate modeling module can capture the complex interactions between load data and multi-source variables, improving prediction accuracy. The cross-scale modeling module interacts with load characteristics at different time scales through a top-down cross-scale attention mechanism. This enables the model to consider information transmission and influence between different time scales, enhancing its ability to predict long-term and short-term load changes. The prediction module inputs the minimum time scale load characteristics and global block characteristics learned by the model into the prediction module and outputs future load prediction values.

[0050] In this embodiment of the application, the sample training set can be configured according to a set batch size. The operation will be carried out in batches, with a total number of batches of [number missing]. .

[0051] The batch processing of the training set is based on the total number of samples in the training set and the batch size. The formula for calculating the total number of batches is as follows:

[0052]

[0053] in, This represents the total number of samples in the training set.

[0054] You can set the hyperparameters required for the initial load forecasting model, including: time-series pooling ratio. Number of time scales Patch length Number of multi-scale transvariable modeling modules Number of cross-scale modeling modules A batch of training samples is randomly selected from the training set, where each sample contains data... Power station Historical load observations at each time step Power station time step Historical observations of weather variables (temperature, irradiance, humidity, and wind speed, etc.) Power station Historical timestamps at each time step Training is performed using sample data. Updated minimum time-scale load features and global block features are obtained through multi-time-scale feature extraction layers, embedding layers, multi-scale intervariate modeling modules, and cross-scale modeling modules. The updated minimum time-scale load features are then used for training. The features are concatenated and input into the prediction module, which then outputs the predicted values. The prediction module consists of a linear layer and a dropout layer, as shown in the following formula:

[0055]

[0056] in, , This indicates the prediction time step.

[0057] In this embodiment of the application, the prediction loss of all samples in a batch can be calculated. That is, the true load observations of all samples in a batch. and the predicted values ​​of the initial load forecasting model The mean absolute error between the two. The mean absolute error is used as the prediction loss. That is, the true load observations of all samples in a batch. and the predicted values ​​of the initial load forecasting model The mean absolute error between the two values ​​is used to calculate the predicted loss, and the formula is as follows:

[0058]

[0059] After obtaining the predicted loss, based on the training loss The load forecasting model parameters are updated using the backpropagation algorithm. This involves updating the network parameters in the load forecasting model that require updating. The updated formula is as follows:

[0060]

[0061] in, The learning rate is set by the individual.

[0062] Then the training is repeated until all batches of the sample training set have participated in the model training, until the early stopping condition or the specified number of iterations is reached, thus obtaining the load prediction model.

[0063] The method provided in this application improves the accuracy of load forecasting by setting a model structure and training the resulting load forecasting model based on that model structure.

[0064] In some embodiments, see continue to see Figure 2 The embedding layer is used to divide the sample load sequence into time-scale load sequences and encode the time-scale load sequences into load features of multiple time scales through a linear layer. The embedding layer is also used to encode the sample weather variables of all time scales into a single feature through a linear layer to obtain weather features. The embedding layer is also used to map the target time attribute in all timestamps into features and encode the mapped features into a single feature through a linear layer to obtain time features. The embedding layer is also used to concatenate the load features of multiple time scales and the global block features and output the concatenated features to the multi-scale intervariate modeling module.

[0065] In this embodiment, global block features refer to global features extracted from the entire time series or dataset, which may include statistical properties of the data (such as mean, variance), trend information, etc. Sample weather variables refer to weather data related to load forecasting, such as temperature, humidity, wind speed, etc. Feature concatenation refers to concatenating multiple feature vectors along dimensional directions to form a new, higher-dimensional feature vector.

[0066] In this embodiment of the application, random parameter initialization can be used. Global block features at each time scale , As the feature dimension, Defined as time scale Load sequence, Defined as time scale Weather variable series, Defined as time scale The timestamp sequence. Therefore, historical load observations It can be represented as a time scale load sequence Historical weather variable observations It can be represented as a time scale Weather sequence Historical timestamps It can be represented as a timestamp sequence on a time scale of 1. .Will , Input the multi-timescale feature extraction layer to obtain Load characteristics at each time scale Weather characteristics Time characteristics After obtaining Load characteristics at each time scale Weather characteristics Time characteristics Then, it can be input into the embedding layer.

[0067] In this embodiment of the application, the multi-timescale feature extraction layer consists of a kernel size of A one-dimensional convolutional layer and a average pooling layer to process , and The parameters of convolutional layers and average pooling layers at different time scales are not shared. The processing formula for multi-time scale feature extraction layers is as follows:

[0068]

[0069] in, Representing time scale The load sequence on, Representing time scale Weather sequences on the screen, Representing time scale The timestamp sequence on Time scale Number of time steps The set time-series pooling ratio.

[0070] In this embodiment of the application, Load sequences at various time scales Weather sequence and timestamp sequence Input the embedding layer in the multi-scale transvariable modeling module to obtain... Load characteristics at each time scale Weather characteristics Time characteristics .

[0071] In this embodiment, the payload sequence employs block embedding, which is based on a set patch length. The load sequence is divided into segments of length [length missing]. The sequences are encoded into features using a linear layer. Weather sequences employ variable embedding, which embeds weather variables... All time steps are encoded into a single feature using a linear layer. The timestamp sequence first uses time feature embedding to linearly map the four time attributes of each time step's timestamp—Hour of Day, Day of Week, Day of Month, and Day of Year—into features. Then, variable embedding is used to encode all time steps for each time attribute into a single feature using a linear layer. The embedding layer's processing formula can be expressed as:

[0072]

[0073] in, The load features are those after block embedding. Weather features after variable embedding, For time features embedded in time features and time features embedded in variables, For feature dimensions.

[0074] The method provided in this application converts the original data into continuous feature vectors through the embedding layer, which can better capture the potential patterns and relationships in the data. The encoding process of the linear layer further extracts useful information from the data and enhances the expressive power of the features. The load sequence is divided into different time scales and features are extracted separately, so that the model can consider both short-term and long-term load change patterns at the same time. This multi-scale feature extraction method improves the model's adaptability to complex load changes.

[0075] In some embodiments, see continue to see Figure 2 The multi-scale intervariate modeling module includes a self-attention mechanism submodule and an intervariate dependency modeling submodule. The self-attention mechanism submodule is used to input the concatenated features and obtain the temporal dependency relationship between the load features and global block features at each time scale based on the self-attention mechanism. It updates the load features and global block features at multiple time scales based on the dependency relationship and inputs the updated load features at multiple time scales into the inter-scale modeling module. It outputs the updated global block features at multiple time scales into the intervariate dependency modeling submodule. The intervariate dependency modeling submodule is used to output the updated global block features at multiple time scales based on the updated global block features, time features, and weather features at multiple time scales.

[0076] In this embodiment, the multi-scale intervariate modeling module is the core component of the load forecasting model. It aims to capture the complex dependencies between load features and multi-source variables (such as weather and time) at different time scales, while integrating global information to improve forecast accuracy. The multi-scale intervariate modeling module includes a self-attention mechanism submodule and an intervariate dependency modeling submodule. The self-attention mechanism submodule is responsible for modeling the temporal dependencies between load features and global block features within a time scale; the intervariate dependency modeling submodule is responsible for modeling the intervariate dependencies between global block features and weather and time features. The self-attention mechanism submodule is based on a self-attention mechanism. By calculating the weight distribution between features, it dynamically captures the temporal dependencies between load features and global block features at different time scales. The input to the self-attention mechanism submodule is the concatenated features (load features from multiple time scales + global block features). The self-attention mechanism submodule calculates the query, key, and value matrix and obtains the correlation weights between features through a dot product attention mechanism. Based on the weights, the features are weighted and summed to update the load features and global block features. The output of the self-attention mechanism submodule is the updated load features at multiple time scales (passed to the cross-scale modeling module) and the updated global block features (passed to the cross-variable dependency modeling submodule).

[0077] Continuing with the example above, regarding the time scale Given load characteristics and global block features The self-attention mechanism submodule inputs the concatenated features of the load features and the global block features. This submodule models the temporal dependencies between all patches at each time scale, and the global block receives feature information from all patches. The processing of the self-attention mechanism submodule can be represented by the following formula:

[0078]

[0079] in, Indicates the first A multi-scale transvariable modeling module, , These represent the input load features and global block features of the self-attention submodule at layer 0, respectively. Time scale Updated load characteristics Time scale Updated global block features For feature dimension, For layer normalization.

[0080] In this embodiment, the cross-variable dependency modeling submodule is responsible for modeling the cross-variable dependencies between global block features and weather and time features, enhancing the expressive power of global features by fusing multi-source information. The inputs to the cross-variable dependency modeling submodule are: updated global block features (from the self-attention mechanism submodule), weather features (weather variables encoded through a linear layer), and time features (timestamp attributes encoded through a linear layer). The processing of the cross-variable dependency modeling submodule includes: concatenating or interacting the global block features with weather and time features (e.g., through gating or attention mechanisms), fusing information through nonlinear transformations (e.g., fully connected layers), and generating updated global block features. The output of the cross-variable dependency modeling submodule is the updated global block features (feedback to other parts of the model or used for final prediction). The cross-variable dependency modeling submodule is used to, for any time scale, use the updated global block features as a query, concatenate the weather and time features as a key-value pair, and use the key-value pair as input to the cross-attention mechanism, outputting updated global block features for multiple time scales.

[0081] Following the example above, the self-attention mechanism submodule has been updated. Global blocks at each time scale Weather features output from the embedding layer Time characteristics Input the transvariate dependency modeling submodule from the multiscale transvariate modeling module to obtain... Global block features updated at each time scale .

[0082] For time scale ,Will As a query, After feature concatenation, the concatenated features are used as keys and inputs to the cross-attention mechanism to model the dependencies between load, weather variables, and time-specific variables at each time scale, as shown in the following formula:

[0083]

[0084] in, Time scale Updated global block features Indicates the first A multi-scale transvariate modeling module.

[0085] The method provided in this application includes a self-attention mechanism submodule that adaptively learns the correlation between load features and global features at different time scales through weight allocation based on the self-attention mechanism, avoiding information loss caused by a fixed window. The cross-variable dependency modeling submodule deeply integrates weather, time features, and global features to quantify the indirect impact of external factors on the load. Global block features act as an intermediary, unifying the modeling of time-scale features (local) and weather / time features (external) to form a more complete feature representation. Multi-scale and cross-variable modeling enable the model to handle data from different time granularities and sources, adapting to complex scenarios (such as sudden weather events and holiday load fluctuations). Through the self-attention mechanism and cross-variable modeling, the model can automatically learn complex relationships between features, reducing reliance on manually designed features.

[0086] In some embodiments, see continue to see Figure 2 The cross-scale modeling module uses a top-down cross-attention mechanism to exchange load features and global block features at different time scales after updating load features and global block features at multiple time scales, and outputs the load features and global block features at the smallest time scale.

[0087] In this embodiment of the application, the updated Load characteristics at each time scale and global block features The input is fed into the cross-scale modeling module to obtain the updated minimum time-scale load features. and global block features , The number of cross-scale modeling modules is set. Given... Load characteristics at each time scale and global block features The cross-scale modeling module interacts with load features at different time scales through a top-down cross-scale attention mechanism, enabling large-scale load feature information to be passed down layer by layer to smaller scales, ultimately reaching the smallest scale load features. and global block features It can receive feature information from a larger time scale. Indicates the first The processing procedure of the cross-scale modeling module can be represented by the following formula:

[0088]

[0089] in, Time scale go through The load characteristics are updated by each cross-scale modeling module. Time scale go through The global block features are updated by each cross-scale modeling module.

[0090] In some embodiments, step S101 can be achieved through the following steps:

[0091] Step S1011: Obtain the load data and weather data corresponding to the timestamp data;

[0092] Step S1012: Preprocess the load data and weather data to obtain preprocessed load data and weather data.

[0093] In this embodiment of the application, preprocessing of load data and weather data may include: removing outliers and missing values ​​and standardizing the load data and weather data.

[0094] In this embodiment of the application, after eliminating outliers and missing values ​​in the load data and weather data, the load data and weather data can be standardized. The standardization formula is as follows:

[0095]

[0096] in, For load data of all power plants, This is the average of all loads over the training set's time span. The standard deviation of all loads over the training set time span. This is the standardized load data. Weather variables for all power plants Data, Weather variables within the training set time span The average value, Weather variables within the training set time span standard deviation For standardized weather variables The data.

[0097] Step S1013: Based on a preset sliding step size, the preprocessed load data, weather data, and timestamp data are divided into sample load sequences, sample weather sequences, and sample timestamp sequences to obtain a sample training set.

[0098] In this embodiment of the application, the time window size is determined according to the general settings in the field of load forecasting. Based on the chronological order, a sample training set is obtained by dividing the standardized load data, weather data, and original timestamp data using a fixed-length sliding step.

[0099] Based on the foregoing embodiments, this application further provides a training method.Figure 3 This is a schematic diagram illustrating the implementation process of a training method provided in an embodiment of this application, as shown below. Figure 3 As shown, it includes:

[0100] Step 301: Obtain raw load data, weather data, and timestamps.

[0101] Step S302, outlier handling.

[0102] Step S303, standardization process.

[0103] In this embodiment, outlier and missing value elimination and standardization are performed on the load and weather data, and the time span of the training set is determined. Then, the training set is obtained by dividing the processed load and weather data and the given original timestamps using a sliding window.

[0104] Step S304: Batch processing of the training set.

[0105] In this embodiment of the application, the training set is divided according to a set batch size. The operation will be carried out in batches, with a total number of batches of [number missing]. .

[0106] Step S305: Randomly select a batch of training samples from the training set.

[0107] In this embodiment of the application, a batch of training samples is randomly selected from the training set, wherein the data of each sample includes... Power station Historical load observations at each time step Power station time step Historical observations of weather variables (temperature, irradiance, humidity, and wind speed, etc.) Power station Historical timestamps at each time step .

[0108] Step S306: Input into the initial load prediction model.

[0109] In this embodiment of the application, the hyperparameters required for the initial load forecasting model can be set, including: time-series pooling ratio. Number of time scales Patch length Number of multi-scale transvariable modeling modules Number of cross-scale modeling modules Random parameter initialization Global block features at each time scale , For feature dimensions. Defined as time scale Load sequence, Defined as time scale Weather variable series, Defined as time scale The timestamp sequence. Therefore, historical load observations It can be represented as a time scale load sequence Historical weather variable observations It can be represented as a time scale Weather sequence Historical timestamps It can be represented as a timestamp sequence on a time scale of 1. .Will , Input the multi-time-scale feature extraction layer in the multi-scale transvariable modeling module to obtain... Load characteristics at each time scale Weather characteristics Time characteristics .Will Load sequences at various time scales Weather sequence and timestamp sequence Input the embedding layer in the multi-scale transvariable modeling module to obtain... Load characteristics at each time scale Weather characteristics Time characteristics Regarding the time scale , load characteristics and global block features The features are concatenated and input into the self-attention mechanism submodule of the multi-scale intervariate modeling module to obtain the updated features. Load characteristics at each time scale and global block features The self-attention mechanism submodule has been updated. Global blocks at each time scale Weather features output from the embedding layer Time characteristics Input the transvariate dependency modeling layer in the multiscale transvariate modeling module to obtain Global block features updated at each time scale The updated version Load characteristics at each time scale and global block features The input is fed into the cross-scale modeling module to obtain the updated minimum time-scale load features. and global block features , The number of cross-scale modeling modules set for step 3. The updated minimum timescale load characteristics. The features are concatenated and input into the prediction module to obtain the predicted value. .

[0110] Step S307: Obtain the prediction result.

[0111] In this embodiment of the application, the prediction module outputs the predicted value.

[0112] Step S308: Calculate the training loss.

[0113] In this embodiment of the application, the prediction loss of all samples in a batch is calculated. That is, the true load observations of all samples in a batch. and model predictions The average absolute error between them.

[0114] Step S309: Update model parameters.

[0115] In this embodiment of the application, based on training loss The model network parameters are updated using the backpropagation algorithm.

[0116] Step S310: Is training complete?

[0117] In this embodiment of the application, if the training is completed, step S311 is executed; if the training is not completed, step S305 is executed.

[0118] In this embodiment of the application, training completion includes: all batches of the training set participating in model training until the early stopping condition or a specified number of iterations is reached.

[0119] Step S311: Output the model.

[0120] The method provided in this application first preprocesses the load and weather variable data, and constructs a training set based on a sliding window partitioning according to the chronological order. Second, a multi-scale intervariate modeling module is introduced to extract multi-timescale features from the load and multi-source variable data, and utilizes an intervariate attention mechanism to interact the load and multi-source variable features at each time scale, thereby modeling intervariate dependencies at multiple time scales. Then, a cross-multi-scale modeling module is introduced, utilizing a top-down cross-scale attention mechanism to interact the load features at different time scales, thereby modeling complex dependencies of load features across different time scales. Finally, a prediction module is introduced to output the prediction results.

[0121] The method provided in this application, based on extracted multi-timescale load features and multi-source variable features, introduces a multi-scale intervariate modeling module. It utilizes an intervariate attention mechanism to interact with load features and multi-source variable features at each time scale, enabling the modeling of complex intervariate dependencies across multiple time scales. The introduction of the inter-scale modeling module, employing a top-down inter-scale attention mechanism to interact with load features at different time scales, allows for the modeling of complex dependencies of load features across different time scales.

[0122] Based on the foregoing embodiments, this application provides another prediction method that can be applied to electronic devices, including mobile phones, tablets, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device. Figure 4 A schematic diagram illustrating the implementation process of a prediction method provided for the purposes of this application is shown below. Figure 4 As shown, the prediction methods include:

[0123] Step S401: Obtain the load data and weather data corresponding to the historical timestamps;

[0124] Step S402: Input the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load prediction model trained by the method provided in the above embodiment to obtain the predicted load.

[0125] The method provided in this application embodiment can accurately predict load through a load forecasting model.

[0126] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0127] According to the foregoing embodiments, this application provides a training device for a load prediction model. The modules and units included in the device can be implemented by a processor in a computer device; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0128] This application provides a training device for a load forecasting model. Figure 5 This is a schematic diagram of the structure of a training device for a load prediction model provided in an embodiment of this application, as shown below. Figure 5 As shown, the training device 500 for the load forecasting model includes:

[0129] The first acquisition module 501 is used to acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence;

[0130] Training module 502 is used to train the initial load prediction model based on the sample training set to obtain the load prediction model. The initial load prediction model includes: a multi-timescale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module, and a prediction module. The multi-timescale feature extraction layer is used to take sample timestamp sequences, sample load sequences, and sample weather sequences as inputs and output load sequences, weather sequences, and timestamp sequences at multiple time scales. The embedding layer is used to take load sequences, weather sequences, and timestamp sequences at multiple time scales as inputs and output load features, weather features, and time features at multiple time scales. The multi-scale cross-variable modeling module is used to take load features, weather features, time features, and global block features at multiple time scales as inputs and output updated load features and global block features at multiple time scales. The cross-scale modeling module is used to take updated load features and global block feature values ​​at multiple time scales as inputs and output updated load features and global block features at the minimum time scale. The prediction module is used to take updated load features and global block features at the minimum time scale as inputs and output predicted load.

[0131] In some embodiments, the embedding layer is used to divide the sample load sequence into time-scale load sequences and encode the time-scale load sequences into multiple time-scale load features through a linear layer. The embedding layer is also used to encode all time-scale sample weather variables into a single feature using a linear layer to obtain weather features. The embedding layer is also used to map target time attributes in all timestamps into features and encode the mapped features into a single feature using a linear layer to obtain time features. The embedding layer is also used to concatenate the load features at multiple time scales and the global block features and output the concatenated features to the multi-scale intervariate modeling module.

[0132] In some embodiments, the multiscale transvariate modeling module includes:

[0133] The system consists of a self-attention mechanism submodule and a cross-variable dependency modeling submodule. The self-attention mechanism submodule takes the concatenated features as input, obtains the temporal dependency between the load features and global block features at each time scale based on the self-attention mechanism, updates the load features and global block features at multiple time scales based on the dependency, inputs the updated load features at multiple time scales into the cross-scale modeling module, and outputs the updated global block features at multiple time scales into the cross-variable dependency modeling submodule. The cross-variable dependency modeling submodule outputs updated global block features at multiple time scales based on the updated global block features at multiple time scales, the time features, and the weather features.

[0134] In some embodiments, the cross-variable dependency modeling submodule is used to take the updated global block features as a query for any time scale, concatenate the weather features and time features as a key value, and use the key value as the input of the cross-attention mechanism to output the updated global block features for multiple time scales.

[0135] In some embodiments, the cross-scale modeling module uses a top-down cross-attention mechanism to exchange load features and global block features at different time scales for updated load features and global block features at multiple time scales, and outputs the load features and global block features at the smallest time scale.

[0136] In some embodiments, the first acquisition module includes:

[0137] The acquisition unit is used to acquire load data and weather data corresponding to the timestamp data;

[0138] The preprocessing unit is used to preprocess the load data and weather data to obtain preprocessed load data and weather data;

[0139] The partitioning module is used to partition the preprocessed load data, weather data, and timestamp data based on a preset sliding step size to obtain sample load sequences, sample weather sequences, and sample timestamp sequences, so as to obtain the sample training set.

[0140] Based on the foregoing embodiments, this application further provides a forecasting device including: a second acquisition module, used to acquire load data and weather data corresponding to historical timestamps;

[0141] The prediction module is used to input the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load prediction model trained by the method described in any one of the first aspects, so as to obtain the predicted load.

[0142] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0143] In addition, the training device for the load prediction model shown above can be a software unit, a hardware unit, or a combination of software and hardware. It can also be integrated into electronic devices as an independent component, or exist as an independent terminal device.

[0144] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0145] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 3 in this embodiment may include: at least one processor 30 ( Figure 6Only one processor 30, memory 31, and computer program 32 stored in memory 31 and executable on at least one processor 30 are shown. When the processor 30 executes the computer program 32, it implements the steps in any of the above method embodiments, or the processor 30 executes the computer program 32 to implement the functions of each module / unit in the above device or system embodiments.

[0146] For example, computer program 32 may be divided into one or more modules / units, one or more of which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units may be a series of computer program 32 instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in electronic device 3.

[0147] This application also provides a computer-readable storage medium storing a computer program 32, which, when executed by a processor 30, implements the steps described in the above-described method embodiments.

[0148] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0149] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program 32 instructing related hardware. The computer program 32 can be stored in a computer-readable storage medium, and when executed by the processor 30, it can implement the steps of the various method embodiments described above. The computer program 32 includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to a terminal, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, a computer-readable medium cannot be an electrical carrier signal or a telecommunication signal.

[0150] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0151] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A training method for a load forecasting model, characterized in that, The method comprises: obtaining a sample training set, wherein each sample data in the sample training set comprises a sample timestamp sequence, a sample load sequence corresponding to the sample timestamp sequence, and a sample weather sequence; training an initial load prediction model based on the sample training set to obtain a load prediction model, wherein the initial load prediction model comprises a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module, and a prediction module, the multi-time scale feature extraction layer is used for inputting the sample timestamp sequence, the sample load sequence, and the sample weather sequence, and outputting load sequences, weather sequences, and timestamp sequences of multiple time scales, the embedding layer is used for inputting the load sequences, the weather sequences, and the timestamp sequences of multiple time scales, and outputting load features, weather features, and time features of multiple time scales to the multi-scale cross-variable modeling module, the multi-scale cross-variable modeling module comprises a self-attention mechanism submodule and a cross-variable dependency modeling submodule, the self-attention mechanism submodule is used for inputting concatenated features, obtaining a time dependency relationship between the load features and global block features at each time scale based on a self-attention mechanism, updating the load features and the global block features of multiple time scales based on the dependency relationship, inputting the updated load features of multiple time scales to the cross-scale modeling module, and outputting the updated global block features of multiple time scales to the cross-variable dependency modeling submodule, the cross-variable dependency modeling submodule is used for, for any time scale, taking the updated global block features as queries, concatenating weather features and time features as keys; taking the keys as inputs of a cross-attention mechanism, and outputting updated global block features of multiple time scales, the cross-scale modeling module adopts a top-down cross-attention mechanism to exchange load features and global block features of different time scales, and outputs load features and global block features of the smallest time scale, and the prediction module is used for inputting the updated load features and global block features of the smallest time scale, and outputting predicted load, wherein the global block features refer to global features extracted in the entire timestamp sequence, and the global features include statistical characteristics and trend information of data.

2. The method of claim 1, wherein, The embedding layer is used for dividing the sample load sequence into time-scale load sequences, encoding the time-scale load sequences into load features of multiple time scales through a linear layer, encoding all time-scale sample weather variables into a feature through a linear layer to obtain weather features, mapping target time attributes in all timestamps into a feature through a linear layer to obtain time features, and concatenating the load features of multiple time scales and the global block features through a linear layer, and outputting the concatenated features to the multi-scale cross-variable modeling module.

3. The method of claim 1, wherein, The method comprises: obtaining load data and weather data corresponding to the timestamp data; Preprocessing the load data and the weather data to obtain preprocessed load data and weather data; The preprocessed load data, weather data and timestamp data are divided based on a preset sliding step to obtain a sample load sequence, a sample weather sequence and a sample timestamp sequence, so as to obtain the sample training set.

4. A method of prediction, characterized by, Comprise: Obtaining load data and weather data corresponding to the historical timestamp; The historical timestamp, the load data and the weather data corresponding to the historical timestamp are input into the load prediction model trained by the method of any one of claims 1 to 3 to obtain a predicted load.

5. A load prediction model training device characterized by comprising: Comprise: An acquisition module is configured to acquire a sample training set, wherein each sample data in the sample training set comprises a sample timestamp sequence and sample load sequence and sample weather sequence corresponding to the sample timestamp sequence; A training module is configured to train an initial load prediction model based on the sample training set to obtain a load prediction model, wherein the initial load prediction model comprises a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module, the multi-time scale feature extraction layer is configured to input the sample timestamp sequence, the sample load sequence and the sample weather sequence, and output load sequences, weather sequences and timestamp sequences of multiple time scales, the embedding layer is configured to input the load sequences, the weather sequences and the timestamp sequences of multiple time scales, and output load features, weather features and time features of multiple time scales to the multi-scale cross-variable modeling module, the multi-scale cross-variable modeling module comprises a self-attention mechanism submodule and a cross-variable dependency modeling submodule, the self-attention mechanism submodule is configured to input the concatenated features, obtain the time dependency relationship between the load features and the global block features at each time scale based on the self-attention mechanism, update the load features and the global block features of multiple time scales based on the dependency relationship, input the updated load features of multiple time scales to the cross-scale modeling module, and output the updated global block features of multiple time scales to the cross-variable dependency modeling submodule, the cross-variable dependency modeling submodule is configured to, for any time scale, take the updated global block features as a query, concatenate the weather features and the time features as a key value; and take the key value as the input of the cross-attention mechanism, and output the updated global block features of multiple time scales, the cross-scale modeling module adopts a top-down cross-attention mechanism to exchange the load features and the global block features of different time scales, and outputs the load features and the global block features of the smallest time scale, and the prediction module is configured to input the updated load features and the global block features of the smallest time scale, and output a predicted load, wherein the global block features refer to global features extracted in the entire timestamp sequence, and the global features comprise statistical characteristics and trend information of the data.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 3 or the method of claim 4.

7. A computer-readable storage medium storing a computer program, wherein the computer program comprises the following steps of: The computer program is executed by the processor to implement the method of any one of claims 1 to 3 or the method of claim 4.

Images