Energy load increment prediction method, device and storage medium

Abstract

The present invention proposes a method, device and storage medium for predicting energy load increments. The method includes: using pre-collected medium- and long-term energy load data to train a main network; constructing an incremental network based on the main network combined with a weight network and a deviation correction network; wherein the main network and the deviation correction network are two parallel network branches, and the sum of their outputs constitutes the output of the incremental network; the weight network assigns weights to the input of the incremental network and serves as the input of the main network and the deviation correction network respectively; freezing the network parameters of the main network, and using the short-term energy load data to train the weight network and the deviation correction network based on a preset loss function; and using the trained incremental network to predict future energy loads based on current energy load data. The technical solution in the embodiment of the present invention can use continuously collected data to quickly update the pre-trained model and automatically process distribution offsets to achieve accurate prediction of energy loads.

Description

Technical Field

[0001] The present invention relates to the field of energy load forecasting, and in particular to an energy load increment forecasting method, device and storage medium. Background Art

[0002] Load forecasting is a crucial component of energy management systems. Data sets are often collected sequentially, and deep learning methods can be used to generate forecasts on this data. As we know, training deep learning methods is time-consuming and therefore difficult for end users to manage. A common approach is to train a model based on long-term or median series and simultaneously perform forecasts on short-term series. This approach works well when the data to be forecasted follows the same pattern as the training data. However, in the real world, time series data can be non-stationary, and distribution shifts can occur at any point in time that cannot be predicted in advance. Summary of the Invention

[0003] To achieve the above objectives, the present invention proposes a method, device and storage medium for predicting energy load increments, which can use continuously collected data to quickly update the pre-trained model and automatically process distribution offsets to achieve accurate prediction of energy load.

[0004] A method for predicting incremental energy loads proposed in an embodiment of the present invention includes: training a main network using pre-collected medium- and long-term energy load data; constructing an incremental network based on the main network in combination with a weight network and a deviation correction network; wherein the main network and the deviation correction network are two parallel network branches, and the sum of their outputs constitutes the output of the incremental network; the weight network assigns weights to the inputs of the incremental network and uses them as inputs to the main network and the deviation correction network respectively; training the incremental network includes: freezing the network parameters of the main network, training the weight network and the deviation correction network using short-term energy load data based on a preset loss function; and using the trained incremental network to predict future energy loads based on current energy load data. Through the combination of the main network and the incremental network, continuous incremental learning of long- and short-term data is achieved, and the weight network is used to automatically distribute and process data offsets, achieving accurate prediction based on sample drive.

[0005] In one embodiment, the loss function is used to penalize the deviation between the incremental network output and the actual energy load, as well as the deviation between the output of the independent main network and the output of the main network portion of the constructed incremental network. By penalizing the deviation between the incremental network and the actual energy load and the deviation between the main network and the previous and next steps of the main network, catastrophic forgetting is avoided and accurate learning is achieved for data offsets.

[0006] An energy load increment prediction device proposed in an embodiment of the present invention includes: a main network training module configured to train a main network using pre-collected medium- and long-term energy load data; an incremental network construction module configured to construct an incremental network based on the main network in combination with a weight network and a deviation correction network; wherein the main network and the deviation correction network are two parallel network branches, and the sum of their outputs constitutes the output of the incremental network; the weight network assigns weights to the inputs of the incremental network and serves as the inputs of the main network and the deviation correction network respectively; the incremental network training module configured to freeze the network parameters of the main network and train the weight network and the deviation correction network using short-term energy load data based on a preset loss function; and a predicted energy load prediction module configured to use the trained incremental network to predict future energy load based on current energy load data. Through the combination of the main network and the incremental network, continuous incremental learning of long- and short-term data is achieved, and the weight network is used to automatically distribute and process data offsets, achieving accurate sample-driven prediction.

[0007] In one embodiment, the loss function is used to penalize the deviation between the incremental network output and the actual energy load, as well as the deviation between the output of the independent main network and the output of the main network portion of the constructed incremental network. By penalizing the deviation between the incremental network and the actual energy load and the deviation between the main network and the previous and next steps of the main network, catastrophic forgetting is avoided and accurate learning is achieved for data offsets.

[0008] An electronic device proposed in an embodiment of the present invention includes: at least one processor; and a memory coupled to the at least one processor, wherein the memory is used to store instructions, and when the instructions are executed by the at least one processor, the processor executes the method described above.

[0009] An embodiment of the present invention provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed, the method described in any of the above embodiments is executed.

[0010] A computer program product provided in an embodiment of the present invention includes a computer program. When the computer program is executed by a processor, the method described in any one of the above embodiments is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The following drawings are only intended to illustrate and explain the present invention, and do not limit the scope of the present invention.

[0012] Figure 1 FIG. 1 is a flow chart of an energy load increment prediction method 100 according to an embodiment of the present invention.

[0013] Figure 2 2 is a schematic structural diagram of an energy load increment prediction device 200 according to an embodiment of the present invention.

[0014] Figure 3 is a schematic diagram of an electronic device 300 according to an embodiment of the present invention.

[0015] Figure 4 FIG. 1 is a schematic diagram of an incremental network according to an embodiment of the present invention.

[0016] The following are the descriptions of the reference numerals:

[0017]

[0018] DETAILED DESCRIPTION

[0019] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described with reference to the accompanying drawings.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0021] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0022] Incremental learning, sometimes called lifelong learning, continuous learning, or progressive learning, is a machine learning framework that enables a model to learn and improve on continuously collected data without using old data. Methods for performing incremental learning using neural networks as base learners can be roughly divided into three categories: fine-tuning, feature extraction, and learning without forgetting.

[0023] To fine-tune, you build a model with the same structure as the one trained on the old data, and then use the weights obtained previously to initialize the new model. This approach can run very fast if the new data follows similar patterns to the old data, but it can lead to catastrophic forgetting, where patterns trained on the old data are lost, especially when the new data has a distribution shift.

[0024] To remember the information learned from old data, feature extraction freezes parts of the network that will be reused. The outputs of these frozen layers are then used as inputs to downstream layers. It works well when those frozen layers have learned a good representation that also appears on new data and can run quickly, often with a much smaller number of parameters than the entire network. However, these representation layers can be counterproductive because if there is a distribution shift in them, new patterns may be lost as data passes through them.

[0025] Forget-free learning updates the entire network with new data while using regularization techniques to mitigate catastrophic forgetting. Compared to the previous two methods, this method requires updating all parameters in the network and incorporating regularization into the loss, so it takes longer.

[0026] In an embodiment of the present invention, a more effective energy load increment prediction method, device, and storage medium are provided, taking into account problems such as data distribution deviation, excessive training time, and catastrophic forgetting.

[0027] Figure 4 FIG is a schematic diagram of an incremental network according to an embodiment of the present invention. Figure 4 As shown, the incremental network may include:

[0028] Main network 401, bias correction network 402 and weight network 403.

[0029] Figure 1 FIG. 1 is a flow chart of an energy load increment prediction method 100 according to an embodiment of the present invention. Figure 1 As shown, the method 100 may include:

[0030] Step S101: training the main network using pre-collected medium- and long-term energy load data.

[0031] Step S102: constructing an incremental network based on the main network in combination with a weight network and a deviation correction network; wherein the main network and the deviation correction network are two parallel network branches, and the sum of their outputs constitutes the output of the incremental network; the weight network assigns weights to the inputs of the incremental network and uses them as the inputs of the main network and the deviation correction network respectively.

[0032] The incremental architecture consists of two subnetworks, or branches, for prediction. The first is the main network, which is constructed as usual. The second, which we call the bias correction network, is much smaller than the main network. The outputs of the two branches are summed to produce the final output. Furthermore, the incremental network includes a weight network to predict the attention weight for each sample, separating the input values ​​to the two prediction branches.

[0033] Step S103 , training the incremental network, freezing the network parameters of the main network, and training the weight network and the deviation correction network using short-term energy load data based on a preset loss function.

[0034] In one embodiment, before training the incremental network, the network parameters of the weight network are randomly generated; and / or are obtained by training the weight network using the medium- and long-term data of the energy load.

[0035] In one embodiment, the loss function is used to penalize the deviation between the incremental network output and the actual energy load, and the deviation between the output of the main network alone and the output of the main network part in the constructed incremental network.

[0036] In the initial phase, the main network is trained using previously collected long-term or median data, without the weights and bias correction network. Then, the weight network and bias correction network are added. The main network is initialized using the previously acquired weights. Furthermore, a regularization term and loss function are added to penalize deviations in the target predictions of the entire network from those previously obtained using only the main network. This allows the main network to learn a universal representation from long-term or medium-term data. The bias correction network focuses on modeling heterogeneous patterns. The weight network determines the extent to which each sample deviates from the common structure shared by the data.

[0037] Among them, the loss function is specifically:

[0038]

[0039] in, y is the actual energy load, is the incremental network output, is the output of the main network part in the constructed incremental network, For the output of the main network alone, Correct the output of the network for bias.

[0040] Among them, the input of the weight network is x, its output to the main network is wx, and the output to the deviation correction network is (1-w)x.

[0041] During the incremental learning phase, the layers in the main network are frozen, while the parameters in the weight and bias correction network are allowed to be updated. To reduce computation time, both networks can be initialized with previously learned weights.

[0042] Step S104: using the trained incremental network to predict future energy load based on current energy load data.

[0043] In one embodiment, the method further includes step S105 of updating the model, continuously training the incremental network based on the newly collected energy load data, and updating the network parameters of the weight network and the deviation correction network.

[0044] To test the performance of the incremental energy load forecasting method, 33 days of data were collected, with one point every 15 minutes. The dataset was divided into Dataset-1 (records from the first 23 days) and Dataset-2 (records from the last 10 days), which were used for the initial incremental learning phase. Predictions for Dataset-2 were compared using different strategies: S1: training a neural network on Dataset-1; S2: training a neural network on Dataset-2; S3: running fine-tuning incremental learning, initializing the second network with the network trained on Dataset-1; S4: running incremental learning and extracting features, freezing some of the first network and continuing training on Dataset-2; and S5: the method of this application. The prediction results were compared using R2 scores, root mean squared error, and mean absolute error, as shown in the table below.

[0045]

[0046] Figure 2 FIG. 2 is a schematic diagram of the structure of an energy load increment prediction device 200 according to an embodiment of the present invention. Figure 2 As shown, the apparatus 200 may include:

[0047] The main network training module 201 is configured to train the main network using pre-collected medium- and long-term energy load data.

[0048] The incremental network construction module 202 is configured to construct an incremental network based on the main network in combination with a weight network and a deviation correction network; wherein the main network and the deviation correction network are two parallel network branches, and the sum of their outputs constitutes the output of the incremental network; the weight network assigns weights to the inputs of the incremental network and serves as the inputs of the main network and the deviation correction network respectively.

[0049] The incremental architecture consists of two subnetworks, or branches, for prediction. The first is the main network, which is constructed as usual. The second, which we call the bias correction network, is much smaller than the main network. The outputs of the two branches are summed to produce the final output. Furthermore, the incremental network includes a weight network to predict the attention weight for each sample, separating the input values ​​to the two prediction branches.

[0050] The incremental network training module 203 is configured to freeze the network parameters of the main network and train the weight network and the deviation correction network using short-term energy load data based on a preset loss function.

[0051] In one embodiment, before training the incremental network, the network parameters of the weight network are randomly generated; and / or are obtained by training the weight network using the medium- and long-term data of the energy load.

[0052] In one embodiment, the loss function is used to penalize the deviation between the incremental network output and the actual energy load, and the deviation between the output of the main network alone and the output of the main network part in the constructed incremental network.

[0053] In the initial phase, the main network is trained using previously collected long-term or median data, without the weights and bias correction network. Then, the weight network and bias correction network are added. The main network is initialized using the previously acquired weights. Furthermore, a regularization term and loss function are added to penalize deviations in the target predictions of the entire network from those previously obtained using only the main network. This allows the main network to learn a universal representation from long-term or medium-term data. The bias correction network focuses on modeling heterogeneous patterns. The weight network determines the extent to which each sample deviates from the common structure shared by the data.

[0054] Among them, the loss function is specifically:

[0055]

[0056] in, y is the actual energy load, is the incremental network output, is the output of the main network part in the constructed incremental network, For the output of the main network alone, Correct the output of the network for bias.

[0057] Among them, the input of the weight network is x, its output to the main network is wx, and the output to the deviation correction network is (1-w)x.

[0058] During the incremental learning phase, the layers in the main network are frozen, while the parameters in the weight and bias correction network are allowed to be updated. To reduce computation time, both networks can be initialized with previously learned weights.

[0059] The energy load prediction module 204 is configured to use the trained incremental network to predict future energy load based on current energy load data.

[0060] In one embodiment, the apparatus further includes a model updating module 205 configured to continuously train the incremental network based on newly collected energy load data, and update network parameters of the weight network and the deviation correction network.

[0061] The present invention also provides an electronic device 300 . Figure 3 FIG. 3 is a schematic diagram of an electronic device 300 according to an embodiment of the present invention. Figure 3 As shown, the electronic device 300 includes a processor 310 and a memory 320 , wherein the memory 320 stores instructions, wherein when the instructions are executed by the processor 310 , the method 100 described above is implemented.

[0062] The present invention further provides a computer-readable storage medium having computer instructions stored thereon. When the computer instructions are executed, the method described above is executed.

[0063] The present invention also provides a computer program product, comprising a computer program, which implements the above-mentioned method when executed by a processor.

[0064] Some aspects of the methods and apparatus of the present invention may be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may be referred to as "data blocks," "modules," "engines," "units," "components," or "systems." The processor may be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLCs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. In addition, various aspects of the present invention may be embodied as computer products in one or more computer-readable media, the product including computer-readable program code. For example, computer-readable media may include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, tapes, etc.), optical disks (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., cards, sticks, key drives, etc.).

[0065] Flowcharts are used herein to illustrate the operations performed by the methods according to the embodiments of the present application. It should be understood that the preceding operations are not necessarily performed in exact order. Instead, the various steps may be performed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0066] It should be understood that although this specification is described according to various embodiments, not every embodiment contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

[0067] The above description is only an illustrative embodiment of the present invention and is not intended to limit the scope of the present invention. Any equivalent changes, modifications and combinations made by those skilled in the art without departing from the concept and principle of the present invention shall fall within the scope of protection of the present invention.

Claims

1. A method (100) for predicting energy load increments, characterized in that: The method comprises: The main network is trained using pre-collected medium- and long-term energy load data (S101): Constructing an incremental network based on the main network combined with a weight network and a deviation correction network (S102); The main network and the deviation correction network are two parallel network branches, and the sum of the outputs of the main network and the deviation correction network constitutes the output of the incremental network; the weight network assigns weights to the inputs of the incremental network and serves as the inputs of the main network and the deviation correction network respectively; Training the incremental network (S103), including: freezing the network parameters of the main network, and training the weight network and the deviation correction network based on a preset loss function using short-term energy load data; The trained incremental network is used to predict future energy load based on current energy load data (S104).

2. The method according to claim 1, wherein The method further includes: before training the incremental network, the network parameters of the weight network are randomly generated; and / or, the parameters are obtained by training the weight network using the medium- and long-term data of the energy load.

3. The method according to claim 1, wherein The method further includes: the loss function is used to penalize the deviation between the incremental network output and the actual energy load, and the deviation between the output of the separate main network and the output of the main network part in the constructed incremental network.

4. The method according to claim 1, wherein The method further comprises: The model is updated (S105), and the incremental network is continuously trained based on the newly collected energy load data, and the network parameters of the weight network and the deviation correction network are updated.

5. An energy load increment prediction device (200), characterized in that: The device comprises: The main network training module (201) is configured to train the main network using pre-collected medium- and long-term energy load data: An incremental network construction module (202) is configured to construct an incremental network based on the main network in combination with a weight network and a deviation correction network; The main network and the deviation correction network are two parallel network branches, and the sum of their outputs constitutes the output of the incremental network; the weight network assigns weights to the inputs of the incremental network and serves as the inputs of the main network and the deviation correction network respectively; The incremental network training module (203) is configured to: freeze the network parameters of the main network, and train the weight network and the deviation correction network using short-term energy load data based on a preset loss function; The energy load prediction module (204) is configured to use the trained incremental network to predict future energy load based on current energy load data.

6. The device according to claim 5, wherein The incremental network training module (203) is further configured to: before training the incremental network, randomly generate network parameters of the weight network; and / or obtain the parameters by training the weight network using the medium- and long-term data of the energy load.

7. The device according to claim 5, wherein The loss function is used to penalize the deviation between the output of the incremental network and the actual energy load, and the deviation between the output of the main network alone and the output of the main network part in the constructed incremental network.

8. The device according to claim 5, wherein The device further comprises: The model updating module (205) is configured to continuously train the incremental network based on the newly collected energy load data, and update the network parameters of the weight network and the deviation correction network.

9. An electronic device (300), comprising: at least one processor (310); as well as A memory (320) coupled to the at least one processor (310), the memory (320) being configured to store instructions, which, when executed by the at least one processor (310), causes the processor (310) to perform the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having computer instructions stored thereon, wherein the computer instructions, when executed, perform the method according to any one of claims 1 to 4.

11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 4 when executed by a processor.

Images