Training method and device of load prediction model, prediction method and device and electronic equipment

Abstract

According to the load prediction model training method and device, the prediction method and device and the electronic equipment provided by the invention, the sample training set is acquired, the initial load prediction model is trained based on the sample training set, and the load prediction model is obtained; the initial load prediction model comprises a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module, and the load prediction model can improve the precision of load prediction through a set model structure.

Description

Technical Field

[0001] The present application belongs to the technical field of load forecasting, and in particular relates to a training method, a forecasting method, a device and an electronic device for a load forecasting model. Background Art

[0002] With the continuous development of smart grids and energy management, accurate load forecasting is crucial for ensuring safe and stable grid operation, optimizing resource allocation, and reducing operating costs. However, electricity load data is influenced by numerous factors and exhibits complex multi-timescale characteristics. Load patterns vary significantly across different timescales. For example, hourly load data exhibits morning and evening peaks and valleys, daily load data exhibits weekday-weekend variations, and weekly and monthly load data exhibit cyclical fluctuations influenced by factors such as seasons and holidays. Furthermore, load data is closely correlated with multiple source variables, such as weather and time of day, and these variables exhibit complex dependencies within and across multiple timescales. However, traditional statistical models, based on linear assumptions, struggle to capture nonlinear variations, resulting in weak generalization and low prediction accuracy. Existing deep learning models, on the other hand, often focus on single-dimensional dependency modeling and are unable to fully capture the complex relationships between multi-timescale load series and multiple source variables. Consequently, load forecasting accuracy falls short of meeting practical requirements. Summary of the Invention

[0003] The embodiments of the present application provide a training method, a prediction method, an apparatus, and an electronic device for a load prediction model, which can improve the accuracy of load prediction through the load prediction model.

[0004] In a first aspect, an embodiment of the present application provides a method for training a load forecasting model, comprising: Acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence; The initial load forecasting model is trained based on the sample training set to obtain a load forecasting model. The initial load forecasting model includes: a multi-time-scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. The multi-time-scale feature extraction layer is used to input a sample timestamp sequence, a sample load sequence and a sample weather sequence, and output a load sequence, a weather sequence and a timestamp sequence of multiple time scales. The embedding layer is used to input a load sequence, a weather sequence and a timestamp sequence of multiple time scales, and output load characteristics, weather characteristics and time characteristics of multiple time scales. The multi-scale cross-variable modeling module is used to input load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input updated load characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input updated load characteristics and global block characteristics of the minimum time scale, and output predicted load.

[0005] In some embodiments, the embedding layer is used to divide the sample load sequence into a load sequence of time scales, and encode the load sequence of time scales into load features of multiple time scales through a linear layer. The embedding layer is also used to encode the sample weather variables of all time scales into one feature using a linear layer to obtain weather features. The embedding layer is also used to map the target time attributes in all timestamps into features, and encode the mapped features into one feature using a linear layer to obtain time features. The embedding layer is also used to concatenate the load features of multiple time scales and the global block features, and output the concatenated features to the multi-scale cross-variable modeling module.

[0006] In some embodiments, the multi-scale cross-variable modeling module includes: A self-attention mechanism submodule and a cross-variable dependency modeling submodule, wherein the self-attention mechanism submodule is used to input the concatenated features, and obtain the time dependency between the load features and the global block features at each time scale based on the self-attention mechanism, and update the load features and global block features of multiple time scales based on the dependency, and input the updated load features of multiple time scales into the cross-scale modeling module, and output the updated global block features of multiple time scales to the cross-variable dependency modeling submodule, and the cross-variable dependency modeling submodule is used to output the updated global block features of multiple time scales based on the updated global block features of multiple time scales, the time features and the weather features.

[0007] In some embodiments, the cross-variable dependency modeling submodule is used to use the updated global block features as queries for any time scale, concatenate weather features and time features as key values; and use the key values ​​as input of the cross-attention mechanism, and output the updated global block features of multiple time scales.

[0008] In some embodiments, the cross-scale modeling module uses a top-down cross-attention mechanism to exchange the load features and global block features of different time scales for the updated load features and global block feature values ​​of multiple time scales, and outputs the load features and global block features of the minimum time scale.

[0009] In some embodiments, obtaining a sample training set includes: Get the load data and weather data corresponding to the timestamp data; Preprocessing the load data and weather data to obtain preprocessed load data and weather data; The pre-processed load data, weather data and timestamp data are divided based on a preset sliding step size to obtain a sample load sequence, a sample weather sequence and a sample timestamp sequence, so as to obtain the sample training set.

[0010] In a second aspect, an embodiment of the present application provides a prediction method, comprising: Get the load data and weather data corresponding to the historical timestamp; The historical timestamp, the load data corresponding to the historical timestamp, and the weather data are input into the load forecasting model trained by the method described in any one of the first aspects to obtain the predicted load.

[0011] In a third aspect, an embodiment of the present application provides a training device for a load forecasting model, comprising: A first acquisition module is configured to acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence; A training module is used to train the initial load forecasting model based on the sample training set to obtain a load forecasting model. The initial load forecasting model includes: a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. The multi-time scale feature extraction layer is used to input a sample timestamp sequence, a sample load sequence and a sample weather sequence, and output a load sequence, a weather sequence and a timestamp sequence of multiple time scales. The embedding layer is used to input a load sequence, a weather sequence and a timestamp sequence of multiple time scales, and output load characteristics, weather characteristics and time characteristics of multiple time scales. The multi-scale cross-variable modeling module is used to input load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input updated load characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input updated load characteristics and global block characteristics of the minimum time scale, and output predicted load.

[0012] In a fourth aspect, an embodiment of the present application provides a prediction device, comprising: The second acquisition module is used to obtain load data and weather data corresponding to historical timestamps; The prediction module is used to input the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load prediction model trained by the method described in any one of the first aspects to obtain the predicted load.

[0013] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-described methods when executing the computer program.

[0014] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements any of the methods described above.

[0015] In a seventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the electronic device to execute any of the methods described above.

[0016] Compared with the prior art, the embodiments of the present application have the following beneficial effects: The embodiment of the present application provides a method for training a load forecasting model, which obtains a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence; an initial load forecasting model is trained based on the sample training set to obtain a load forecasting model, wherein the initial load forecasting model includes: a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module, wherein the multi-time scale feature extraction layer is used to input a sample timestamp sequence, a sample load sequence and a sample weather sequence, and output a load sequence, a weather sequence and a timestamp sequence at multiple time scales, and the embedding layer is used to input a plurality of time scales. The multi-scale cross-variable modeling module is used to input the load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output the updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input the updated load characteristics and global block characteristics of multiple time scales, and output the updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input the updated load characteristics and global block characteristics of the minimum time scale, and output the predicted load. Through the set model structure, the load prediction model can improve the accuracy of load prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 A schematic diagram of the implementation flow of a load forecasting model training method provided for the implementation of this application; Figure 2 A schematic diagram of a framework of an initial load forecasting model provided in an embodiment of the present application; Figure 3 A schematic diagram of an implementation flow of a training method provided in an embodiment of the present application; Figure 4 A schematic diagram of a prediction method for the implementation of this application; Figure 5 A schematic diagram of the structure of a load forecasting model training device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0020] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0021] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0022] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if it is detected" can be interpreted as meaning "upon determining" or "in response to determining" or "upon detecting" or "in response to detecting," depending on the context.

[0023] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0025] Based on the problems in the related art, an embodiment of the present application provides a training method for a load forecasting model that can be applied to electronic devices. The electronic devices may include: mobile phones, tablet computers, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiments of the present application do not impose any restrictions on the specific types of electronic devices. Figure 1 A flow chart of a load forecasting model training method provided for the implementation of this application is shown in FIG. Figure 1 As shown in Figure 2, the training method of the load forecasting model includes: Step S101 : obtaining a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence, a sample load sequence, and a sample weather sequence corresponding to the sample timestamp sequence.

[0026] In an embodiment of the present application, the sample training set is a set including historical load data, weather data and corresponding timestamp data, which is used to train the load prediction model.

[0027] In an embodiment of the present application, historical load data, weather data, and corresponding timestamp data may be collected, preprocessed (such as eliminating outliers and missing values, and standardizing), and then the data may be divided using a sliding window to form a sample training set.

[0028] Step S102: train the initial load forecasting model based on the sample training set to obtain the load forecasting model. The initial load forecasting model includes: a multi-time-scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. The multi-time-scale feature extraction layer is used to input a sample timestamp sequence, a sample load sequence and a sample weather sequence, and output a load sequence, a weather sequence and a timestamp sequence of multiple time scales. The embedding layer is used to input a load sequence, a weather sequence and a timestamp sequence of multiple time scales, and output load characteristics, weather characteristics and time characteristics of multiple time scales. The multi-scale cross-variable modeling module is used to input load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input updated load characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input updated load characteristics and global block characteristics of the minimum time scale, and output predicted load.

[0029] In an embodiment of the present application, the initial load forecasting model is an untrained load forecasting model framework, which includes multiple modules for learning the complex relationship between load data and multi-source variables (such as weather). Figure 2 A schematic diagram of the framework of an initial load forecasting model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the initial load forecasting model consists of a multi-timescale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module, and a prediction module. The multi-timescale feature extraction layer extracts features at multiple timescales from time series data (load series, weather series, and timestamp series). It processes the series data at different timescales through one-dimensional convolutional layers and average pooling layers. The embedding layer converts discrete time series data (such as load values, weather variable values, and timestamps) into continuous vector representations, or features. Block embedding is used for load series, while variable embedding and time feature embedding are used for weather series and timestamp series. The multi-scale cross-variable modeling module models the complex dependencies between load features and multi-source variable features (weather features and time features) at each timescale. The multi-scale cross-variable modeling module includes a self-attention mechanism submodule and a cross-variable dependency modeling submodule. The cross-scale modeling module models the complex dependencies between load features at different timescales. It uses a top-down cross-scale attention mechanism to enable large-scale information to influence small-scale feature representations. The cross-scale modeling module includes a cross-scale dependency modeling layer and a fully connected layer. The prediction module is used to output the future load forecast value based on the feature representation learned by the model. The prediction module usually contains a linear layer and a random dropout layer.

[0030] In an embodiment of the present application, a sample training set is used to iteratively train each module of the initial load forecasting model, and the model parameters are updated by the back propagation algorithm until the model converges or reaches a specified number of iterations, thereby obtaining a load forecasting model. The multi-time scale feature extraction layer uses a one-dimensional convolution layer and an average pooling layer to extract features of different time scales from the original sequence. The multi-time scale feature extraction module can capture the change pattern of load data at different time scales and improve the model's adaptability to complex load changes. The embedding layer converts the load sequence, weather sequence, and timestamp sequence into load features, weather features, and time features, respectively. The embedding layer can convert discrete data into continuous vector representations, which facilitates model processing and learning of complex relationships between data. The multi-scale cross-variable modeling module models the dependency relationship between load features and multi-source variable features at each time scale through a self-attention mechanism submodule and a cross-variable dependency modeling submodule. The multi-scale cross-variable modeling module can capture the complex interactions between load data and multi-source variables and improve prediction accuracy. The cross-scale modeling module interacts load features of different time scales through a top-down cross-scale attention mechanism. The cross-scale modeling module enables the model to consider the information transmission and influence between different time scales, and enhances the model's ability to predict long-term and short-term load changes. The prediction module inputs the minimum time scale load features and global block features learned by the model into the prediction module and outputs the future load forecast value.

[0031] In this embodiment of the present application, the sample training set can be divided into batches according to the set size. Operation is done in batches, the total number of batches is .

[0032] The sample training set batch operation is based on the total number of samples in the training set and the batch size. The total number of batches is calculated as follows:

[0033] in, is the total number of training set samples.

[0034] You can set the hyperparameters required for the initial load forecasting model, including: time series pooling ratio , number of time scales , patch length , the number of multi-scale cross-variable modeling modules , number of cross-scale modeling modules A batch of training samples are randomly selected from the sample training set, where each sample data contains power stations Historical load observations at time steps power stations time steps Historical observations of weather variables (temperature, irradiance, humidity, wind speed, etc.) power stations Historical timestamps of time steps , through sample data for training, through the multi-time scale feature extraction layer, embedding layer, multi-scale cross-variable modeling module, cross-scale modeling module to obtain the updated minimum time scale load characteristics and global block characteristics, the updated minimum time scale load characteristics The features are concatenated and input into the prediction module, and the prediction module outputs the predicted value The prediction module consists of a linear layer and a random dropout layer, and the formula is as follows:

[0035] in, , Represents the forecast time step.

[0036] In this embodiment of the present application, the prediction loss of all samples in a batch can be calculated , that is, the true load observation value of all samples in a batch and the predicted value of the initial load forecasting model The mean absolute error between . Use the mean absolute error as the prediction loss , that is, the true load observation value of all samples in a batch and the predicted value of the initial load forecasting model The mean absolute error between , the prediction loss is calculated as follows:

[0037] After getting the prediction loss, according to the training loss , update the load forecasting model parameters through the back propagation algorithm. The network parameters that need to be updated in the load forecasting model To update, the formula is as follows:

[0038] in, A manually set learning rate.

[0039] The training is then repeated until all batches of the sample training set participate in the model training, until the early stopping condition or the specified number of iterations is reached, and the load forecasting model is obtained.

[0040] The method provided in the embodiment of the present application can improve the accuracy of load forecasting by setting a model structure and training a load forecasting model based on the model structure.

[0041] In some embodiments, see Figure 2 The embedding layer is used to divide the sample load sequence into a load sequence of time scales, and encode the load sequence of time scales into load features of multiple time scales through a linear layer. The embedding layer is also used to encode the sample weather variables of all time scales into one feature using a linear layer to obtain weather features. The embedding layer is also used to map the target time attributes in all timestamps into features, and encode the mapped features into one feature using a linear layer to obtain time features. The embedding layer is also used to concatenate the load features of multiple time scales and the global block features, and output the concatenated features to the multi-scale cross-variable modeling module.

[0042] In this embodiment of the present application, global block features refer to global features extracted from the entire time series or dataset, which may include statistical properties of the data (such as mean and variance) and trend information. Sample weather variables refer to weather data related to load forecasting, such as temperature, humidity, and wind speed. Feature concatenation refers to the process of concatenating multiple feature vectors along the dimensional direction to form a new, higher-dimensional feature vector.

[0043] In the embodiment of the present application, random parameters can be initialized Global block features at different time scales , is the feature dimension, Defined as time scale The load sequence, Defined as time scale The weather variable sequence, Defined as time scale Therefore, the historical load observations Can be expressed as a time scale Load sequence , historical weather variable observations Can be expressed as a time scale Weather sequence , historical timestamp Can be represented as a timestamp sequence of time scale 1 .Will 、 Input the multi-time scale feature extraction layer and get Load characteristics of each time scale , weather characteristics , time characteristics . In getting Load characteristics of each time scale , weather characteristics , time characteristics After that, it can be input into the embedding layer.

[0044] In the embodiment of the present application, the multi-time scale feature extraction layer is composed of a kernel size of A one-dimensional convolutional layer and a The average pooling layer is used to process 、 and , the parameters of the convolutional layers and average pooling layers of different time scales are not shared, and the processing formula of the multi-time scale feature extraction layer is as follows:

[0045] in, Representing time scale The load sequence on Representing time scale The weather sequence on Representing time scale The timestamp sequence on Time scale The number of time steps, is the set time series pooling ratio.

[0046] In the embodiment of this application, Load series on a time scale , Weather Sequence and timestamp sequence Input the embedding layer in the multi-scale cross-variable modeling module to obtain Load characteristics of each time scale , weather characteristics , time characteristics .

[0047] In the embodiment of the present application, the load sequence is embedded in blocks, which is based on the set patch length. Divide the load sequence into pieces of length The weather sequence is encoded into features using a linear layer. The weather sequence uses variable embedding, which converts the weather variables into All time steps of the time stamp sequence are encoded into a single feature using a linear layer. First, the time stamp sequence uses time feature embedding to linearly map the four time attributes of each time step's time stamp, namely, hour of day (HourOfDay), day of week (DayOfWeek), day of month (DayOfMonth), and day of year (DayOfYear), into features. Then, variable embedding is used to linearly encode all time steps of each time attribute into a single feature using a linear layer. The processing formula of the embedding layer can be expressed as:

[0048] in, is the load characteristic after block embedding, is the weather feature after variable embedding, is the time feature after time feature embedding and variable embedding, is the feature dimension.

[0049] The method provided in the embodiment of the present application converts the original data into a continuous feature vector through an embedding layer, which can better capture the potential patterns and relationships in the data. The encoding process of the linear layer further extracts useful information from the data, enhances the expressiveness of the features, divides the load series into different time scales, and extracts features separately, so that the model can simultaneously consider short-term and long-term load change patterns. This multi-scale feature extraction method improves the model's adaptability to complex load changes.

[0050] In some embodiments, see Figure 2 The multi-scale cross-variable modeling module includes: a self-attention mechanism sub-module and a cross-variable dependency modeling sub-module. The self-attention mechanism sub-module is used to input the concatenated features and obtain the time dependency between the load features and the global block features at each time scale based on the self-attention mechanism, and update the load features and global block features of multiple time scales based on the dependency, and input the updated load features of multiple time scales into the cross-scale modeling module, and output the updated global block features of multiple time scales to the cross-variable dependency modeling sub-module. The cross-variable dependency modeling sub-module is used to output the updated global block features of multiple time scales based on the updated global block features, time features and weather features of multiple time scales.

[0051] In this embodiment, the multi-scale cross-variable modeling module is a core component of the load forecasting model. It aims to capture the complex dependencies between load characteristics and multiple source variables (such as weather and time) at different time scales, while integrating global information to improve forecast accuracy. The multi-scale cross-variable modeling module includes a self-attention mechanism submodule and a cross-variable dependency modeling submodule. The self-attention mechanism submodule is responsible for modeling the temporal dependencies between load characteristics and global block features within a time scale; the cross-variable dependency modeling submodule is responsible for modeling the cross-variable dependencies between global block features and weather and time features. The self-attention mechanism submodule, based on the self-attention mechanism, dynamically captures the temporal dependencies between load characteristics and global block features at different time scales by calculating weights between features. The input to the self-attention mechanism submodule is the concatenated features (load characteristics at multiple time scales + global block features). The self-attention mechanism submodule calculates the query, key, and value matrices and uses the dot product attention mechanism to obtain the correlation weights between features. The features are weighted summed according to the weights to update the load characteristics and global block features. The output of the self-attention mechanism submodule is the updated load features of multiple time scales (passed to the cross-scale modeling module) and the updated global block features (passed to the cross-variable dependency modeling submodule).

[0052] Continuing with the above example, for the time scale , given load characteristics and global block features , the load feature and the global block feature are concatenated as the input of the self-attention mechanism submodule. The self-attention mechanism submodule models the temporal dependency between all patches at each time scale, and the global block receives feature information from all patches. The processing process of the self-attention mechanism submodule can be expressed by the following formula:

[0053] in, Indicates the A multi-scale cross-variable modeling module, , are the input load features and global block features of the self-attention submodule at layer 0, Time scale After the updated load characteristics, Time scale Updated global block features, is the feature dimension, Normalize the layer.

[0054] In this embodiment of the present application, the cross-variable dependency modeling submodule is responsible for modeling the cross-variable dependency relationships between global block features and weather and time features, enhancing the expressive power of global features by fusing multi-source information. The inputs to the cross-variable dependency modeling submodule include updated global block features (from the self-attention mechanism submodule), weather features (weather variables encoded via a linear layer), and time features (timestamp attributes encoded via a linear layer). The cross-variable dependency modeling submodule processes the global block features by concatenating or interacting them with the weather and time features (e.g., via a gating mechanism or attention mechanism), fusing the information via a nonlinear transformation (e.g., a fully connected layer), and generating updated global block features. The output of the cross-variable dependency modeling submodule is the updated global block features, which are fed back to other parts of the model or used for final prediction. For any time scale, the cross-variable dependency modeling submodule uses the updated global block features as a query, concatenating the weather and time features as a key, and using the key as input to the cross-attention mechanism. The module then outputs updated global block features for multiple time scales.

[0055] Following the above example, update the self-attention mechanism submodule Global partitioning of time scales , weather features output by the embedding layer , time characteristics Input the cross-variable dependency modeling submodule in the multi-scale cross-variable modeling module to obtain The global block features after time scale update .

[0056] For time scale ,Will As a query, The concatenated features are used as keys and as inputs to the cross-attention mechanism to model the dependencies among load, weather variables, and time feature variables at each time scale. The formula is as follows:

[0057] in, Time scale Updated global block features Indicates the A multi-scale cross-variable modeling module.

[0058] In the method provided in the embodiment of the present application, the self-attention mechanism submodule adaptively learns the correlation between load features and global features at different time scales through weight distribution based on the self-attention mechanism, avoiding information loss caused by fixed windows, and the cross-variable dependency modeling submodule deeply integrates weather, time features and global features to quantify the indirect impact of external factors on the load. Global block features act as an intermediary to unify the modeling of time scale features (local) and weather / time features (external) to form a more complete feature representation. Multi-scale and cross-variable modeling enable the model to process data of different time granularities and sources, and adapt to complex scenarios (such as sudden weather and holiday load fluctuations). Through the self-attention mechanism and cross-variable modeling, the model can automatically learn the complex relationships between features and reduce dependence on manually designed features.

[0059] In some embodiments, see Figure 2 ,The cross-scale modeling module uses a top-down cross-attention mechanism to exchange the load features and global block ,feature values ​​of the updated multiple time scales, and outputs the load ,features and global block features of the minimum time scale.

[0060] In the embodiment of the present application, the updated Load characteristics of each time scale and global block features Input to the cross-scale modeling module to obtain the updated minimum time scale load characteristics and global block features , is the number of cross-scale modeling modules set. Load characteristics of each time scale and global block features The cross-scale modeling module interacts with load features of different time scales through a top-down cross-scale attention mechanism, so that the large-scale load feature information is transmitted layer by layer to the small-scale load feature. Finally, the load feature of the smallest scale is and global block features Able to receive characteristic information from a larger time scale, Indicates the A cross-scale modeling module. The processing of the cross-scale modeling module can be expressed by the following formula:

[0061] in, Time scale go through The load characteristics updated by the cross-scale modeling module, Time scale go through The global block features are updated by a cross-scale modeling module.

[0062] In some embodiments, step S101 may be implemented by the following steps: Step S1011, obtaining load data and weather data corresponding to the timestamp data; Step S1012: pre-process the load data and weather data to obtain pre-processed load data and weather data.

[0063] In the embodiment of the present application, preprocessing the load data and weather data may include: performing outlier and missing value elimination and standardization processing on the load data and weather data.

[0064] In the embodiment of the present application, after eliminating the outliers and missing values ​​in the load data and weather data, the load and weather data can be standardized. The formula for the standardization is as follows:

[0065] in, is the load data of all power stations, is the average value of all loads within the time span of the training set, is the standard deviation of all loads within the training set time span, is the load data after standardization. Weather variables for all power stations data, is the weather variable within the time span of the training set The average value of is the weather variable within the time span of the training set The standard deviation of is the standardized weather variable data.

[0066] Step S1013 , dividing the pre-processed load data, weather data and timestamp data based on a preset sliding step size to obtain a sample load sequence, a sample weather sequence and a sample timestamp sequence, so as to obtain a sample training set.

[0067] In the embodiment of the present application, the time window size generally set in the load forecasting field is ,Based on the time sequence, the standardized load data, weather data and original timestamp data are divided using a fixed-length sliding step to obtain the sample training set.

[0068] Based on the above embodiments, the present invention further provides a training method. Figure 3 A schematic diagram of the implementation flow of a training method provided in an embodiment of the present application is shown as follows: Figure 3 Shown, including: Step 301: Obtain original load data, weather data and timestamp.

[0069] Step S302: outlier processing.

[0070] Step S303: standardization processing.

[0071] In an embodiment of the present application, outlier and missing value elimination and standardization are performed on the load and weather data, and the time span of the training set is determined, and then the processed load and weather data and the given original timestamp are divided using a sliding window to obtain a training set.

[0072] Step S304: Process the training set in batches.

[0073] In this embodiment of the present application, the training set is set according to the batch size Operation is done in batches, the total number of batches is .

[0074] Step S305: randomly select a batch of training samples from the training set.

[0075] In the embodiment of the present application, a batch of training samples are randomly selected from the training set, wherein the data of each sample contains power stations Historical load observations at time steps power stations time steps Historical observations of weather variables (temperature, irradiance, humidity, wind speed, etc.) power stations Historical timestamps of time steps .

[0076] Step S306: input to the initial load forecasting model.

[0077] In the embodiment of the present application, the hyper parameters required for the initial load forecasting model can be set, including: time series pooling ratio , number of time scales , patch length , the number of multi-scale cross-variable modeling modules , the number of cross-scale modeling modules . Random parameter initialization Global block features at different time scales , is the feature dimension. Defined as time scale The load sequence, Defined as time scale The weather variable sequence, Defined as time scale Therefore, the historical load observations Can be expressed as a time scale Load sequence , historical weather variable observations Can be expressed as a time scale Weather sequence , historical timestamp Can be represented as a timestamp sequence of time scale 1 .Will 、 Input the multi-time scale feature extraction layer in the multi-scale cross-variable modeling module to obtain Load characteristics of each time scale , weather characteristics , time characteristics .Will Load series on a time scale , Weather Sequence and timestamp sequence Input the embedding layer in the multi-scale cross-variable modeling module to obtain Load characteristics of each time scale , weather characteristics , time characteristics For the time scale , the load characteristics and global block features The features are concatenated and input into the self-attention mechanism submodule in the multi-scale cross-variable modeling module to obtain the updated Load characteristics of each time scale and global block features . Update the self-attention mechanism submodule Global partitioning of time scales , weather features output by the embedding layer , time characteristics Input the cross-variable dependency modeling layer in the multi-scale cross-variable modeling module to obtain The global block features after time scale update . The updated Load characteristics of each time scale and global block features Input to the cross-scale modeling module to obtain the updated minimum time scale load characteristics and global block features , The number of cross-scale modeling modules set for step 3. The updated minimum time scale load characteristics Concatenate the features and input them into the prediction module to get the predicted value .

[0078] Step S307: Obtain prediction results.

[0079] In the embodiment of the present application, the prediction module outputs a prediction value.

[0080] Step S308: Calculate the training loss.

[0081] In this embodiment of the present application, the prediction loss of all samples in a batch is calculated , that is, the true load observation value of all samples in a batch and the model prediction value The mean absolute error between .

[0082] Step S309: Update model parameters.

[0083] In the embodiment of the present application, according to the training loss , the model network parameters are updated through the back propagation algorithm.

[0084] Step S310: Check whether the training is completed.

[0085] In the embodiment of the present application, if the training is completed, step S311 is executed; if the training is not completed, step S305 is executed.

[0086] In the embodiment of the present application, training completion includes: all batches of the training set participate in model training until the early stopping condition or the specified number of iterations is reached.

[0087] Step S311, output the model.

[0088] The method provided in the embodiment of the present application first preprocesses the load and weather variable data, and constructs a training set based on the time-sequential sliding window partitioning; secondly, a multi-scale cross-variable modeling module is introduced to extract multi-time-scale features from the load and multi-source variable data, and a cross-variable attention mechanism is used to interact the load and multi-source variable features at each time scale, thereby modeling cross-variable dependencies at multiple time scales; then, a cross-multi-scale modeling module is introduced, and a top-down cross-scale attention mechanism is used to interact the load features of different time scales, thereby modeling the complex dependencies of load features across different time scales; finally, a prediction module is introduced to output the prediction results.

[0089] The method provided in the embodiments of this application introduces a multi-scale cross-variable modeling module based on the extracted load characteristics and multi-source variable characteristics at multiple time scales. This module utilizes a cross-variable attention mechanism to interact with the load characteristics and multi-source variable characteristics at each time scale, enabling the modeling of complex cross-variable dependencies at multiple time scales. The cross-scale modeling module utilizes a top-down cross-scale attention mechanism to interact with load characteristics at different time scales, enabling the modeling of complex dependencies between load characteristics at different time scales.

[0090] Based on the aforementioned embodiments, an embodiment of the present application further provides a prediction method. The embodiment of the present application provides a prediction method that can be applied to electronic devices. The electronic devices may include: mobile phones, tablet computers, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). The embodiment of the present application does not impose any restrictions on the specific types of electronic devices. Figure 4 A schematic diagram of the implementation flow of a prediction method provided for the implementation of this application is as follows: Figure 4 As shown, the prediction methods include: Step S401, obtaining load data and weather data corresponding to historical timestamps; Step S402: input the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load forecasting model trained by the method provided in the above embodiment to obtain the forecasted load.

[0091] The method provided in the embodiment of the present application can accurately predict the load through a load prediction model.

[0092] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0093] According to the aforementioned embodiments, the embodiments of the present application provide a training device for a load forecasting model. The modules included in the device, and the units included in each module, can be implemented by a processor in a computer device; of course, they can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0094] The present invention provides a load forecasting model training device. Figure 5 A structural diagram of a load forecasting model training device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the load forecasting model training device 500 includes: The first acquisition module 501 is configured to acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence; The training module 502 is used to train the initial load forecasting model based on the sample training set to obtain a load forecasting model. The initial load forecasting model includes: a multi-time-scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. The multi-time-scale feature extraction layer is used for sample timestamp sequences, sample load sequences and sample weather sequences, and outputs load sequences, weather sequences and timestamp sequences of multiple time scales. The embedding layer is used to input load sequences, weather sequences and timestamp sequences of multiple time scales, and output load characteristics, weather characteristics and time characteristics of multiple time scales. The multi-scale cross-variable modeling module is used to input load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input updated load characteristics and global block feature values ​​of multiple time scales, and output updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input updated load characteristics and global block characteristics of the minimum time scale, and output predicted load.

[0095] In some embodiments, the embedding layer is used to divide the sample load sequence into a load sequence of time scales, and encode the load sequence of time scales into load features of multiple time scales through a linear layer. The embedding layer is also used to encode the sample weather variables of all time scales into one feature using a linear layer to obtain weather features. The embedding layer is also used to map the target time attributes in all timestamps into features, and encode the mapped features into one feature using a linear layer to obtain time features. The embedding layer is also used to concatenate the load features of multiple time scales and the global block features, and output the concatenated features to the multi-scale cross-variable modeling model.

[0096] In some embodiments, the multi-scale cross-variable modeling module includes: A self-attention mechanism submodule and a cross-variable dependency modeling submodule, wherein the self-attention mechanism submodule is used to input the concatenated features, and obtain the time dependency between the load features and the global block features at each time scale based on the self-attention mechanism, and update the load features and global block features of multiple time scales based on the dependency, and input the updated load features of multiple time scales into the cross-scale modeling module, and output the updated global block features of multiple time scales to the cross-variable dependency modeling submodule, and the cross-variable dependency modeling submodule is used to output the updated global block features of multiple time scales based on the updated global block features of multiple time scales, the time features and the weather features.

[0097] In some embodiments, the cross-variable dependency modeling submodule is used to use the updated global block features as queries for any time scale, concatenate weather features and time features as key values; and use the key values ​​as input of the cross-attention mechanism, and output the updated global block features of multiple time scales.

[0098] In some embodiments, the cross-scale modeling module uses a top-down cross-attention mechanism to exchange the load features and global block features of different time scales for the updated load features and global block feature values ​​of multiple time scales, and outputs the load features and global block features of the minimum time scale.

[0099] In some embodiments, the first acquisition module includes: An acquisition unit, used to acquire load data and weather data corresponding to the timestamp data; a preprocessing unit, configured to preprocess the load data and weather data to obtain preprocessed load data and weather data; The partitioning module is used to partition the preprocessed load data, weather data and timestamp data based on a preset sliding step size to obtain a sample load sequence, a sample weather sequence and a sample timestamp sequence to obtain the sample training set.

[0100] Based on the aforementioned embodiments, an embodiment of the present application further provides a prediction device comprising: a second acquisition module, configured to acquire load data and weather data corresponding to a historical timestamp; The prediction module is used to input the historical timestamp, the load data corresponding to the historical timestamp, and the weather data into the load prediction model trained by the method described in any one of the first aspects to obtain the predicted load.

[0101] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0102] In addition, the training device of the load forecasting model shown above can be a software unit, a hardware unit, or a combination of software and hardware. It can also be integrated into an electronic device as an independent pendant, or exist as an independent terminal device.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0104] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. Figure 6 As shown, the electronic device 3 of this embodiment may include: at least one processor 30 ( Figure 6Only one processor 30 is shown in the figure), a memory 31, and a computer program 32 stored in the memory 31 and executable on at least one processor 30. When the processor 30 executes the computer program 32, the steps in any of the above-mentioned method embodiments are implemented, or when the processor 30 executes the computer program 32, the functions of the modules / units in the above-mentioned apparatus or system embodiments are implemented.

[0105] Exemplarily, the computer program 32 may be divided into one or more modules / units, one or more of which are stored in the memory 31 and executed by the processor 30 to implement the present application. The one or more modules / units may be a series of computer program 32 instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3.

[0106] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program 32. When the computer program 32 is executed by the processor 30, the steps in the above-mentioned method embodiments can be implemented.

[0107] An embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0108] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the process steps in the above-mentioned method embodiments can be implemented by computer program 32 instructing the relevant hardware. Computer program 32 can be stored in a computer-readable storage medium. When executed by processor 30, computer program 32 can implement the steps of each of the above-mentioned method embodiments. Computer program 32 includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. Computer-readable media can include at least: any entity or device capable of carrying computer program code to a terminal, recording media, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0109] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0110] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0111] In the embodiments provided in this application, it should be understood that the disclosed devices / network equipment and methods can be implemented in other ways. For example, the device / network equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0112] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0113] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for training a load forecasting model, characterized in that: include: Acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence; The initial load forecasting model is trained based on the sample training set to obtain a load forecasting model. The initial load forecasting model includes: a multi-time-scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. The multi-time-scale feature extraction layer is used to input a sample timestamp sequence, a sample load sequence and a sample weather sequence, and output a load sequence, a weather sequence and a timestamp sequence of multiple time scales. The embedding layer is used to input a load sequence, a weather sequence and a timestamp sequence of multiple time scales, and output load characteristics, weather characteristics and time characteristics of multiple time scales. The multi-scale cross-variable modeling module is used to input load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input updated load characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input updated load characteristics and global block characteristics of the minimum time scale, and output predicted load.

2. The method according to claim 1, characterized in that The embedding layer is used to divide the sample load sequence into a load sequence of time scales, and encode the load sequence of time scales into load features of multiple time scales through a linear layer. The embedding layer is also used to encode the sample weather variables of all time scales into one feature using a linear layer to obtain weather features. The embedding layer is also used to map the target time attributes in all timestamps into features, and encode the mapped features into one feature using a linear layer to obtain time features. The embedding layer is also used to concatenate the load features of multiple time scales and the global block features, and output the concatenated features to the multi-scale cross-variable modeling module.

3. The method according to claim 1, characterized in that The multi-scale cross-variable modeling module includes: A self-attention mechanism submodule and a cross-variable dependency modeling submodule, wherein the self-attention mechanism submodule is used to input the concatenated features, and obtain the time dependency between the load features and the global block features at each time scale based on the self-attention mechanism, and update the load features and global block features of multiple time scales based on the dependency, and input the updated load features of multiple time scales into the cross-scale modeling module, and output the updated global block features of multiple time scales to the cross-variable dependency modeling submodule, and the cross-variable dependency modeling submodule is used to output the updated global block features of multiple time scales based on the updated global block features of multiple time scales, the time features and the weather features.

4. The method according to claim 3, characterized in that The cross-variable dependency modeling submodule is used to use the updated global block features as queries for any time scale, concatenate weather features and time features as key values; and use the key values ​​as input to the cross-attention mechanism, and output the updated global block features of multiple time scales.

5. The method according to claim 1, wherein The cross-scale modeling module uses a top-down cross-attention mechanism to exchange the load features and global block features of different time scales for the updated load features and global block feature values ​​of multiple time scales, and outputs the load features and global block features of the minimum time scale.

6. The method according to claim 1, characterized in that The obtaining of the sample training set includes: Get the load data and weather data corresponding to the timestamp data; Preprocessing the load data and weather data to obtain preprocessed load data and weather data; The pre-processed load data, weather data and timestamp data are divided based on a preset sliding step size to obtain a sample load sequence, a sample weather sequence and a sample timestamp sequence, so as to obtain the sample training set.

7. A prediction method, characterized in that: include: Get the load data and weather data corresponding to the historical timestamp; The historical timestamp, the load data corresponding to the historical timestamp, and the weather data are input into the load forecasting model trained by the method according to any one of claims 1 to 6 to obtain the predicted load.

8. A training device for a load forecasting model, characterized in that: include: An acquisition module is used to acquire a sample training set, wherein each sample data in the sample training set includes: a sample timestamp sequence and a sample load sequence and a sample weather sequence corresponding to the sample timestamp sequence; A training module is used to train the initial load forecasting model based on the sample training set to obtain a load forecasting model. The initial load forecasting model includes: a multi-time scale feature extraction layer, an embedding layer, a multi-scale cross-variable modeling module, a cross-scale modeling module and a prediction module. The multi-time scale feature extraction layer is used to input a sample timestamp sequence, a sample load sequence and a sample weather sequence, and output a load sequence, a weather sequence and a timestamp sequence of multiple time scales. The embedding layer is used to input a load sequence, a weather sequence and a timestamp sequence of multiple time scales, and output load characteristics, weather characteristics and time characteristics of multiple time scales. The multi-scale cross-variable modeling module is used to input load characteristics, weather characteristics, time characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of multiple time scales. The cross-scale modeling module is used to input updated load characteristics and global block characteristics of multiple time scales, and output updated load characteristics and global block characteristics of the minimum time scale. The prediction module is used to input updated load characteristics and global block characteristics of the minimum time scale, and output predicted load.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 or the method according to claim 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 or the method according to claim 7 is implemented.

Images