Cross-regional load prediction method, system and device based on shared encoder and storage medium

Abstract

The invention discloses a cross-regional load prediction method, system and device based on a shared encoder, and a storage medium. The method comprises the following steps: constructing a common covariable pre-training data set and a specific region data set; constructing a time sequence prediction backbone network, wherein the time sequence prediction backbone network comprises an input coding layer and a shared encoder based on a self-attention mechanism; designing a multi-task learning framework which comprises a multi-task learning network architecture and a dynamic weighting loss function; pre-training the encoder by using the common covariable pre-training data set; performing fine adjustment on the encoder in a specific region by utilizing the specific region data set of each region and combining with a multi-task learning framework; performing time sequence prediction on large-scale and cross-regional load data by using an encoder; and evaluating the performance of the encoder according to the time sequence prediction result. According to the method, the problems of data sparsity, regional difference, long-term dependency relationship capture, data imbalance and the like in cross-regional load prediction are effectively solved, and the prediction precision and the model generalization ability are remarkably improved.

Description

Technical Field

[0001] The present invention relates to a power system load forecasting technology, and in particular to a cross-regional load forecasting method, system, device and storage medium based on a shared encoder. Background Art

[0002] Power system load forecasting is an important basis for power system planning and operation. With the continuous expansion of power system scale and the increasing interconnection across regions, the importance of cross-regional load forecasting has become increasingly prominent. However, existing load forecasting methods face the following challenges when processing cross-regional load data:

[0003] (1) Data sparsity: Historical data in some regions may not be sufficient, resulting in reduced prediction accuracy.

[0004] (2) Regional differences: The load characteristics and influencing factors in different regions may vary significantly, making it difficult to make accurate predictions using a unified model.

[0005] (3) Long-term dependencies: Load data usually have complex long-term dependencies, which are difficult to capture effectively using traditional methods.

[0006] (4) Data imbalance: The distribution of different types of samples in the dataset may be uneven, affecting the generalization ability of the model.

[0007] Therefore, there is an urgent need for a cross-regional load forecasting method that can effectively deal with the above problems. Summary of the invention

[0008] Purpose of the invention: The cross-regional load forecasting method, system, device and storage medium based on a shared encoder of the present invention can solve the problems of data sparsity, regional differences, long-term dependency capture and data imbalance existing in the existing load forecasting methods when processing cross-regional load data.

[0009] Technical solution: A cross-region load prediction method based on a shared encoder of the present invention comprises:

[0010] Collect the historical load data and covariate characteristics of different regions, and perform data preprocessing. Use the preprocessed historical load data and covariate characteristics of different regions to build a specific regional data set for each region; Considering the different data bases of different regions, select the intersection of covariates in each region to build a common covariate pre-training data set;

[0011] Construct a time series prediction backbone network, which includes an input encoding layer and a shared encoder based on a self-attention mechanism. The input encoding layer implements embedded encoding of time series data of different frequencies, and the shared encoder based on a self-attention mechanism is used to capture the long-term dependencies of time series data from different regions.

[0012] Design a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample rarity;

[0013] The shared encoder based on the self-attention mechanism is pre-trained using the common covariate pre-training dataset. After pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific region using the specific region dataset of each region in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism.

[0014] Use a trained shared encoder based on self-attention mechanism to perform time series forecasting on large-scale, cross-regional load data;

[0015] The performance of the shared encoder based on the self-attention mechanism is evaluated according to the time series prediction results. If the performance of the shared encoder based on the self-attention mechanism does not meet the standard, the shared encoder based on the self-attention mechanism is fine-tuned again.

[0016] Furthermore, the expression of the self-attention mechanism is as follows:

[0017] Q=XW Q , K = XW K , V = XW V

[0018]

[0019] Where X represents the input sequence; W Q , W K , W V represents the learnable weight matrix; d k represents the dimension of the attention head; Q represents the query vector; K represents the key vector; V represents the value vector.

[0020] Furthermore, the improved position encoding method comprises:

[0021]

[0022]

[0023] Among them, pos represents the position index; i represents the dimension index, d model Represents the dimension of the model; in addition, an additional periodic encoding is introduced:

[0024]

[0025] Among them, T u Indicates different cycles.

[0026] Furthermore, the multi-task learning network architecture includes:

[0027] Input encoding layer:

[0028] According to the frequency of the data, different lengths are selected to divide the data into time series blocks, and then the divided time series blocks are calculated by the fully connected layer, and then the time series code of the specified dimension is output.

[0029] Shared encoder:

[0030] Based on the time series prediction backbone network, the features shared by all tasks are extracted, and its mathematical expression is:

[0031] H=Encoder(X)

[0032] Among them, X represents the input sequence; H represents the shared feature;

[0033] Task specific layers:

[0034] For each task, a task-specific layer is added to capture the unique features of the task; for the i-th task, its specific layer can be expressed as:

[0035] H_i=f_i(H)

[0036] Among them, fi represents the task-specific transformation function, which adopts the cascade of multiple convolutional layers and fully connected layers;

[0037] Output layer:

[0038] Each task has an output layer to generate the final prediction; for the i-th task, its output can be expressed as:

[0039] Y_i=g_i(H_i)

[0040] Among them, g_i represents the output function, a fully connected layer is used, and linear activation is used.

[0041] Furthermore, the dynamic weighted loss function that automatically adjusts the weight based on the sample rarity is expressed as follows:

[0042]

[0043] In the formula, l i represents the loss of the i-th sample; w i represents the sample weight, and N represents the total number of samples;

[0044] Among them, the sample weight w i The calculation formula is as follows:

[0045]

[0046] In the formula, f i Represents the frequency of the i-th sample in the data set; represents the sum of all sample frequencies;

[0047] In order to further balance the weights, the softmax function is used to weight the sample w i Normalize:

[0048]

[0049] Furthermore, the shared encoder based on the self-attention mechanism is pre-trained using the common covariate pre-training dataset, including:

[0050] The shared encoder based on the self-attention mechanism is pre-trained on the common covariate pre-training dataset, and the optimization goal is to minimize the prediction error:

[0051]

[0052] In the formula, y represents the real load value; represents the predicted value; L reg represents the regularization term; L pre represents the loss function; λ represents the regularization coefficient.

[0053] Furthermore, after pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific region using the specific region dataset of each region in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism, including:

[0054] Using the region-specific datasets of each region, the shared encoder based on the self-attention mechanism is fine-tuned in specific regions in combination with the multi-task learning framework:

[0055]

[0056] Where, L r represents the task loss of the rth region; L share represents the shared task loss; α r and β represent the balance coefficient.

[0057] Based on the same inventive concept, a cross-regional load forecasting system based on a shared encoder of the present invention includes:

[0058] The data acquisition and preprocessing module is used to collect the historical load data and covariate characteristics of different regions, and perform data preprocessing. The preprocessed historical load data and covariate characteristics of different regions are used to build a specific regional data set for each region. Considering the different data bases of different regions, the intersection of covariates in each region is selected to build a common covariate pre-training data set.

[0059] The model building module is used to build a time series prediction backbone network. The time series prediction backbone network includes an input encoding layer and a shared encoder based on a self-attention mechanism. The input encoding layer implements embedded encoding of time series data of different frequencies, and the shared encoder based on a self-attention mechanism is used to capture the long-term dependencies of time series data from different regions.

[0060] A multi-task learning framework building module is used to design a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample rarity;

[0061] The model training module is used to pre-train the shared encoder based on the self-attention mechanism using the common covariate pre-training dataset; after pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific region using the specific region dataset of each region in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism;

[0062] The time series prediction module is used to perform time series prediction on large-scale, cross-regional load data using a trained shared encoder based on the self-attention mechanism;

[0063] The performance evaluation module is used to evaluate the performance of the shared encoder based on the self-attention mechanism according to the time series prediction results. If the performance of the shared encoder based on the self-attention mechanism does not meet the standard, the shared encoder based on the self-attention mechanism is fine-tuned.

[0064] Based on the same inventive concept, a cross-regional load forecasting device based on a shared encoder of the present invention is characterized in that it includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the electronic device implements the steps of the above-mentioned cross-regional load forecasting method based on a shared encoder.

[0065] Based on the same inventive concept, a computer-readable storage medium of the present invention stores a computer program thereon, and when the program is executed by a processor, the steps of the above-mentioned cross-regional load prediction method based on a shared encoder are implemented.

[0066] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are:

[0067] By sharing encoders and pre-training strategies, the data sparsity problem in cross-regional load forecasting is effectively solved, and the prediction performance of the model in data-scarce areas is improved.

[0068] By adopting a multi-task learning framework, adaptive learning of data features from different regions is achieved, overcoming the challenges brought by regional differences.

[0069] The introduction of a shared encoder based on the self-attention mechanism effectively captures the long-term dependencies of load data and improves the accuracy of prediction.

[0070] The dynamic weighted loss function is used to solve the data imbalance problem and enhance the generalization ability of the model.

[0071] The improved position encoding method is used to better capture the periodic pattern of load data and further improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a flow chart of a cross-region load prediction method based on a shared encoder disclosed in an embodiment of the present invention;

[0073] Figure 2 It is a verification effect diagram of a cross-regional load prediction method based on a shared encoder in different regions disclosed in an embodiment of the present invention;

[0074] Figure 3 It is a structural schematic diagram of a cross-regional load forecasting system based on a shared encoder disclosed in an embodiment of the present invention;

[0075] Figure 4 It is a structural schematic diagram of a cross-regional load prediction device based on a shared encoder disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It will be understood by those skilled in the art that the purposes and advantages that can be achieved with the present invention are not limited to the specific description of the above beneficial effects, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description.

[0077] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed in the present invention can be implemented in hardware, software or a combination of the two. Whether it is implemented in hardware or software depends on the specific application and design of the technical solution. Professional and technical personnel can use different methods to implement the functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0078] Reference to "embodiments" in the present invention means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0079] Example 1

[0080] See also Figure 1 , Figure 1 is a flow chart of a cross-region load prediction method based on a shared encoder disclosed in an embodiment of the present invention. Figure 1 The cross-regional load forecasting method described is applied to the power system, such as for power system load forecasting, etc., which is not limited in the embodiments of the present invention. Figure 1 As shown, the cross-region load prediction method based on the shared encoder may include the following operations:

[0081] S1. Collect historical load data and covariate characteristics of different regions, and perform data preprocessing. Use the preprocessed historical load data and covariate characteristics of different regions to build a specific regional data set for each region; considering the different data bases of different regions, select the intersection of covariates in each region to build a common covariate pre-training data set;.

[0082] Among them, the covariate characteristics include:

[0083] Weather characteristics: temperature (T), humidity (H), wind speed (W)

[0084] Date type characteristics: working day (D worl ), weekend (D weekend )、Holidays(D holid )

[0085] Seasonal characteristics: month (M), seasonal index (S)

[0086] Considering the different data bases in different regions, the intersection of covariates in multiple regions is selected as the common covariate pre-training dataset. For example, region A has temperature, humidity, and irradiance. Region B only has temperature and humidity. Then, in the pre-training dataset, the data only contains temperature and humidity. The dataset for a specific region retains all the data of each region.

[0087] The eigenvector for a certain time t can be expressed as:

[0088]

[0089] Among them, Lt Indicates the load value at that moment.

[0090] S2. Construct a time series prediction backbone network. The time series prediction backbone network includes an input coding layer and a shared encoder based on a self-attention mechanism. The input coding layer implements embedded coding of time series data of different frequencies, and the shared encoder based on the self-attention mechanism is used to capture the long-term dependencies of time series data in different regions.

[0091] The backbone network of the present invention adopts a pure encoder architecture. The core component is a shared encoder based on the self-attention mechanism, which is used to capture the long-term dependencies of time series data from different regions.

[0092] Step S2 specifically includes the following steps:

[0093] S2.1. Design input coding layer:

[0094] In this embodiment, different lengths are selected according to the frequency of the data to divide the data into time series blocks, and then the divided time series blocks are calculated by the fully connected layer, and then the time series encoding of the specified dimension is output. For hourly data, the data segmentation length is usually [32, 64], and the minute-level data segmentation length is usually [32, 64, 128].

[0095] S2.2. Design a shared encoder based on self-attention mechanism.

[0096] In this embodiment, the expression of the self-attention mechanism is as follows:

[0097] Q=XW Q , K = XW K , V = XW V

[0098]

[0099] Where X represents the input sequence; W Q , W K , W V represents the learnable weight matrix; d k represents the dimension of the attention head; Q represents the query vector, which is used to match the key vector to find the most relevant value vector; K represents the key vector, which performs a dot product operation with the query vector to calculate the relevance of the query to each key; V represents the value vector, which contains the actual information. When the query matches a key, the corresponding value vector will be selected as part of the output.

[0100] S2.3. Design of improved position encoding method

[0101] In order to better capture the periodic pattern of load data, the present invention adopts an improved position encoding method.

[0102] Improved position encoding methods, including:

[0103]

[0104]

[0105] Among them, pos represents the position index; i represents the dimension index, d model represents the dimensionality of the model; in addition, an additional periodic encoding is introduced:

[0106]

[0107] Among them, T i Indicates different periods (such as daily, weekly, monthly, yearly).

[0108] The shared encoder is used to extract the common features of load data in different regions, which includes a multi-layer self-attention mechanism and a feedforward neural network.

[0109] The time series prediction backbone network can adaptively learn the common features and unique patterns of time series data in different regions; wherein, the backbone network simultaneously optimizes shared feature extraction and region-specific prediction tasks through a multi-task learning framework to achieve adaptive learning of data features from different regions.

[0110] S3, design a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample rarity. Step S3 specifically includes the following:

[0111] S3.1. Establish a multi-task learning network architecture, including the following main parts:

[0112] (1) Input coding layer:

[0113] According to the frequency of the data, different lengths are selected to divide the data into time series blocks, and then the divided time series blocks are calculated by the fully connected layer, and then the time series encoding of the specified dimension is output. For hourly data, the data segmentation length is usually [32, 64], and the minute-level data segmentation length is usually [32, 64, 128].

[0114] (2) Shared encoder:

[0115] Based on the time series prediction backbone network constructed in step S2, the features shared by all tasks are extracted, and its mathematical expression is:

[0116] H=Encoder(X)

[0117] Among them, X represents the input sequence; H represents the shared feature.

[0118] (3) Task-specific layer:

[0119] For each task, a task-specific layer is added to capture the unique features of the task. For the i-th task, its specific layer can be expressed as:

[0120] H_i=f_i(H)

[0121] Among them, fi represents the task-specific transformation function, which adopts the cascade of multiple convolutional layers and fully connected layers.

[0122] (4) Output layer:

[0123] Each task has its own output layer to generate the final prediction result. For the i-th task, its output can be expressed as:

[0124] Y_i=g_i(H_i)

[0125] Among them, g_i represents the output function, a fully connected layer is used, and linear activation is used.

[0126] S3.1. Establish a dynamic weighted loss function that automatically adjusts weights based on sample rarity.

[0127] In order to solve the problem of data imbalance, the present invention designs a dynamic weighted loss function that automatically adjusts the weight based on the rarity of the sample, and its expression is as follows:

[0128]

[0129] In the formula, l i represents the loss of the i-th sample; w i represents the sample weight, and N represents the total number of samples.

[0130] Among them, the sample weight w i The calculation formula is as follows:

[0131]

[0132] In the formula, f i Represents the frequency of the i-th sample in the data set; Represents the sum of all sample frequencies.

[0133] In order to further balance the weights, the softmax function is used to weight the sample w i Normalize:

[0134]

[0135] The multi-task learning network architecture allows different output layers to learn shared features and region-specific features simultaneously. Different output layers obtain the output of the shared encoder. The dynamic weighted loss function adaptively adjusts the weights of different samples according to the distribution of samples in the dataset, achieving unified training of cross-regional load samples.

[0136] S4. Model training: Use the common covariate pre-training dataset to pre-train the shared encoder based on the self-attention mechanism. After pre-training, use the specific area datasets of each region and combine the multi-task learning framework to fine-tune the shared encoder based on the self-attention mechanism in the specific area to complete the training of the shared encoder based on the self-attention mechanism.

[0137] The present invention adopts a phased training strategy, and step S4 specifically includes the following:

[0138] S4.1. Pre-training stage: Use the common covariate pre-training dataset to pre-train the shared encoder based on the self-attention mechanism, including:

[0139] The shared encoder based on the self-attention mechanism is pre-trained on the common covariate pre-training dataset, and the optimization goal is to minimize the prediction error:

[0140]

[0141] Where y represents the true load value; represents the predicted value; L reg represents the regularization term; L pre represents the loss function; λ represents the regularization coefficient; λ represents the regularization coefficient.

[0142] S4.2, fine-tuning stage: After pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific area using the specific area dataset of each area and combined with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism, including:

[0143] Using the region-specific datasets of each region, the shared encoder based on the self-attention mechanism is fine-tuned in specific regions in combination with the multi-task learning framework:

[0144]

[0145] Where, L r represents the task loss of the rth region; L sha represents the shared task loss; α r and β represent the balance coefficient.

[0146] S5. Load forecast: Figure 2As shown, a trained shared encoder based on self-attention mechanism is used to perform time series forecasting on large-scale, cross-regional load data.

[0147] Load forecasting is performed using a trained shared encoder based on self-attention mechanism. For the input sequence X = [x1, x2, ..., x T ], predict the load value of the next H time steps:

[0148]

[0149] Among them, f(·) represents the trained model and θ is the model parameter.

[0150] S6. Model evaluation: The performance of the shared encoder based on the self-attention mechanism is evaluated according to the time series prediction results. If the performance of the shared encoder based on the self-attention mechanism does not meet the standard, the shared encoder based on the self-attention mechanism is fine-tuned.

[0151] Model performance was evaluated using multiple metrics, including mean absolute percentage error (MAPE) and root mean square error (RMSE):

[0152]

[0153]

[0154] Among them, y i represents the true value; represents the predicted value; n represents the number of samples.

[0155] Through the above steps, the present invention realizes efficient and accurate cross-regional load forecasting. The method effectively solves the problems of data sparsity, regional differences, long-term dependency capture and data imbalance, and significantly improves the prediction accuracy and model generalization ability.

[0156] The cross-region load forecasting method based on a shared encoder of the present invention has the main task of forecasting the load time series data of different regions, and regards the load forecasting of each region as a separate task.

[0157] Example 2

[0158] See also Figure 3 , Figure 3 1 is a schematic diagram of a cross-regional load forecasting system based on a shared encoder disclosed in an embodiment of the present invention. The system can realize load forecasting of a power system, and specifically includes:

[0159] The data acquisition and preprocessing module is used to collect the historical load data and covariate characteristics of different regions, and perform data preprocessing. The preprocessed historical load data and covariate characteristics of different regions are used to build a specific regional data set for each region; considering the different data bases of different regions, the intersection of covariates in each region is selected to build a common covariate pre-training data set;

[0160] The model building module is used to build a time series prediction backbone network. The time series prediction backbone network includes an input encoding layer and a shared encoder based on a self-attention mechanism. The input encoding layer implements embedded encoding of time series data of different frequencies, and the shared encoder based on a self-attention mechanism is used to capture the long-term dependencies of time series data from different regions.

[0161] A multi-task learning framework building module is used to design a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample rarity;

[0162] The model training module is used to pre-train the shared encoder based on the self-attention mechanism using the common covariate pre-training dataset; after pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific region using the specific region dataset of each region in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism;

[0163] The time series prediction module is used to perform time series prediction on large-scale, cross-regional load data using a trained shared encoder based on the self-attention mechanism;

[0164] The performance evaluation module is used to evaluate the performance of the shared encoder based on the self-attention mechanism according to the time series prediction results. If the performance of the shared encoder based on the self-attention mechanism does not meet the standard, the shared encoder based on the self-attention mechanism is fine-tuned.

[0165] In an optional embodiment, a cross-regional load forecasting method based on a shared encoder includes: a) constructing a common covariate pre-training dataset and a specific regional dataset; b) constructing a time series prediction backbone network; c) designing a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample scarcity; d) using the common covariate pre-training dataset to pre-train a shared encoder based on a self-attention mechanism; using the specific regional datasets of each region, combined with the multi-task learning framework, fine-tuning the pre-trained shared encoder based on the self-attention mechanism in a specific region; e) using the trained shared encoder based on the self-attention mechanism to perform time series prediction on large-scale, cross-regional load data; f) evaluating the performance of the shared encoder based on the self-attention mechanism based on the time series prediction results.

[0166] Example 3

[0167] See also Figure 4 , Figure 4 : is a schematic diagram of a cross-region load forecasting device based on a shared encoder disclosed in an embodiment of the present invention. Figure 4 The described device can be applied in an electric power system, such as for load forecasting of the electric power system, and the embodiments of the present invention are not limited thereto.

[0168] like Figure 4 As shown, the device may include a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the method described in the above embodiment and can achieve the technical effect consistent with the above method.

[0169] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, the memory, such program modules including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include the implementation of a network environment. The program modules typically perform the functions and / or methods in the embodiments described herein.

[0170] The processor executes various functional applications and data processing by running the programs stored in the memory, such as implementing the method provided in Embodiment 1 of the present invention.

[0171] Example 4

[0172] Embodiment 4 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in the above embodiment are implemented and the technical effect consistent with the above method can be achieved.

[0173] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0174] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0175] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0176] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0177] Of course, the storage medium containing computer executable instructions provided by an embodiment of the present invention, whose computer executable instructions are not limited to the above method operations, can also execute related operations in the method provided by any embodiment of the present invention.

[0178] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cross-region load forecasting method based on a shared encoder, characterized in that: include: Collect historical load data and covariate characteristics of different regions, perform data preprocessing, and use the preprocessed historical load data and covariate characteristics of different regions to build a specific regional data set for each region; Considering the different data bases of different regions, select the intersection of covariates in each region to build a common covariate pre-training data set; Construct a time series prediction backbone network, which includes an input encoding layer and a shared encoder based on a self-attention mechanism. The input encoding layer implements embedded encoding of time series data of different frequencies, and the shared encoder based on a self-attention mechanism is used to capture the long-term dependencies of time series data from different regions. Design a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample rarity; The shared encoder based on the self-attention mechanism is pre-trained using the common covariate pre-training dataset. After pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific region using the specific region dataset of each region in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism. Use a trained shared encoder based on self-attention mechanism to perform time series forecasting on large-scale, cross-regional load data; The performance of the shared encoder based on the self-attention mechanism is evaluated according to the time series prediction results. If the performance of the shared encoder based on the self-attention mechanism does not meet the standard, the shared encoder based on the self-attention mechanism is fine-tuned.

2. The cross-region load prediction method based on shared encoder according to claim 1 is characterized in that: The expression of the self-attention mechanism is as follows: Q=XW Q ,K=XWK,V=XW V Where X represents the input sequence; W Q , W K , W V represents the learnable weight matrix; dk represents the dimension of the attention head; Q represents the query vector; K represents the key vector; V represents the value vector.

3. The cross-region load prediction method based on shared encoder according to claim 1, characterized in that: The improved position encoding method comprises: Among them, pos represents the position index; i represents the dimension index, d model Represents the dimension of the model; in addition, an additional periodic encoding is introduced: Among them, T i Indicates different cycles.

4. The cross-region load prediction method based on shared encoder according to claim 1, characterized in that: The multi-task learning network architecture includes: Input encoding layer: According to the frequency of the data, different lengths are selected to divide the data into time series blocks, and then the divided time series blocks are calculated by the fully connected layer, and then the time series code of the specified dimension is output; Shared encoder: Based on the time series prediction backbone network, the features shared by all tasks are extracted, and its mathematical expression is: H=Encoder(X) Among them, X represents the input sequence; H represents the shared feature; Task specific layers: For each task, a task-specific layer is added to capture the unique features of the task; for the i-th task, its specific layer can be expressed as: H_i=f_i(H) Among them, fi represents the task-specific transformation function, which adopts the cascade of multiple convolutional layers and fully connected layers; Output layer: Each task has an output layer to generate the final prediction result; for the i-th task, its output can be expressed as: Y_i=g_i(H_i) Among them, g_i represents the output function, a fully connected layer is used, and linear activation is used.

5. The cross-region load prediction method based on shared encoder according to claim 1, characterized in that: The dynamic weighted loss function that automatically adjusts weights based on sample rarity is expressed as follows: In the formula, l i represents the loss of the i-th sample; w i represents the sample weight, and N represents the total number of samples; Among them, the sample weight w i The calculation formula is as follows: In the formula, f i Represents the frequency of the i-th sample in the data set; represents the sum of all sample frequencies; In order to further balance the weights, the softmax function is used to weight the sample w i Normalize:

6. The cross-region load forecasting method based on shared encoder according to claim 1 is characterized in that: The shared encoder based on the self-attention mechanism is pre-trained using the common covariate pre-training dataset, including: The shared encoder based on the self-attention mechanism is pre-trained on the common covariate pre-training dataset, and the optimization goal is to minimize the prediction error: Where y represents the true load value; represents the predicted value; L reg represents the regularization term; L pre represents the loss function; λ represents the regularization coefficient.

7. The cross-region load prediction method based on shared encoder according to claim 1, characterized in that: After pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific area using the specific area dataset of each area in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism, including: Using the region-specific datasets of each region, the shared encoder based on the self-attention mechanism is fine-tuned in specific regions in combination with the multi-task learning framework: Where, L r represents the task loss of the rth region; L shdred represents the shared task loss; α r and β represent the balance coefficient.

8. A cross-regional load forecasting system based on a shared encoder, characterized in that: include: The data acquisition and preprocessing module is used to collect the historical load data and covariate characteristics of different regions, and perform data preprocessing. The preprocessed historical load data and covariate characteristics of different regions are used to build a specific regional data set for each region; considering the different data bases of different regions, the intersection of covariates in each region is selected to build a common covariate pre-training data set; The model building module is used to build a time series prediction backbone network. The time series prediction backbone network includes an input encoding layer and a shared encoder based on a self-attention mechanism. The input encoding layer implements embedded encoding of time series data of different frequencies, and the shared encoder based on a self-attention mechanism is used to capture the long-term dependencies of time series data from different regions. A multi-task learning framework building module is used to design a multi-task learning framework, which includes a multi-task learning network architecture and a dynamic weighted loss function that automatically adjusts weights based on sample rarity; The model training module is used to pre-train the shared encoder based on the self-attention mechanism using the common covariate pre-training dataset; after pre-training, the shared encoder based on the self-attention mechanism is fine-tuned in the specific region using the specific region dataset of each region in combination with the multi-task learning framework to complete the training of the shared encoder based on the self-attention mechanism; The time series prediction module is used to perform time series prediction on large-scale, cross-regional load data using a trained shared encoder based on the self-attention mechanism; The performance evaluation module is used to evaluate the performance of the shared encoder based on the self-attention mechanism according to the time series prediction results. If the performance of the shared encoder based on the self-attention mechanism does not meet the standard, the shared encoder based on the self-attention mechanism is fine-tuned.

9. A cross-region load forecasting device based on a shared encoder, characterized in that: The electronic device comprises a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the electronic device implements the steps of the cross-regional load forecasting method based on a shared encoder as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the cross-regional load prediction method based on a shared encoder as described in any one of claims 1 to 7.

Images