Thermal load prediction method and system based on self-attention mechanism
Through the thermal load prediction method of self-attention mechanism, combined with periodic feature extraction and adversarial sample training, the problems of inaccurate and insufficient safety of thermal load prediction in the prior art are solved, and high accuracy over long-term spans are achieved Security prediction.
Patent Information
- Application Number
- CN202411407578.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-10
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art lacks feature extraction in thermal load prediction, making it difficult to accurately predict over a long span, and lacks defense measures for foreign data attacks, resulting in poor prediction results.
The thermal load prediction method based on the self-attention mechanism is adopted, and the data is preprocessed and predicted by constructing a real thermal load data set using the periodic feature extraction module and attention mechanism model, and adversarial sample training is carried out to resist attacks.
It improves the accuracy and safety of thermal load prediction, can accurately predict thermal load over a long span, and effectively resist attacks against samples.
Smart Images

Figure CN118917497A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of building heat load prediction, and in particular to a heat load prediction method and system based on a self-attention mechanism. Background Art
[0002] With the advancement of the fourth generation of heating methods, the heating system uses more renewable energy, making it cleaner and more environmentally friendly, enabling it to better cope with challenges such as energy shortages and environmental pollution. At the same time, with the emergence of the fifth generation of heating methods, the refinement and intelligent control of the heating network represents the direction of change in the district heating system. For district heating systems, optimal control should focus on achieving efficient and accurate operation management, which requires the heating system to closely track and accurately match the actual thermal energy needs of end users. In short, the optimized control strategy should ensure a real-time and dynamic balance between heat production and user needs.
[0003] Therefore, how to accurately predict the heat load is a problem we need to solve. Summary of the invention
[0004] The present invention aims to provide a heat load prediction method and system based on a self-attention mechanism, which can improve the accuracy of heat load prediction.
[0005] In order to achieve the above object, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides a heat load prediction method based on a self-attention mechanism, the method comprising the following steps: Constructing a real heat load data set, wherein the real heat load data set includes a plurality of target data, wherein the target data is data obtained by coupling weather data and corresponding heat load data according to a time series; Preprocessing the target data in the real heat load data set by embedding technology to obtain a processed real heat load data set; An initial heat load prediction model is constructed, and the initial heat load prediction model is trained according to the processed real heat load data set to obtain a trained target heat load prediction model for heat load prediction of buildings in a target area.
[0006] Optionally, the target data in the real heat load data set is preprocessed by embedding technology, and the processed real heat load data set includes: The discrete target data in the real heat load data set is mapped into a low-dimensional continuous vector space through embedding technology, and the position vector is added to obtain a processed real heat load data set.
[0007] Optionally, the constructing of the initial heat load prediction model includes: The initial heat load prediction model includes a period feature extraction module and an attention mechanism model; The periodic feature extraction module is used to perform sequence conversion processing on the target data of the processed real heat load data set to obtain a deep-level feature embedding vector; the deep-level feature embedding vector is input into the attention mechanism model, and the predicted heat load is output.
[0008] Optionally, the sequence conversion process includes: The target data of the real heat load data set is converted from a 1D sequence to a 2D sequence, and after feature extraction, it is converted back to a 1D sequence through adaptive aggregation.
[0009] Optionally, the attention mechanism model includes an Encorder encoder and a Decorder decoder.
[0010] Optionally, the method further includes: After the target Loss is calculated and the target gradient is obtained in the forward propagation, the adversarial sample data is determined; According to the adversarial sample, the parameters of the attention mechanism model are updated.
[0011] Optionally, the heat load data includes historical heat load, and supply water temperature, return water temperature and building information corresponding to the historical heat load.
[0012] In a second aspect, an embodiment of the present invention provides a heat load prediction system based on a self-attention mechanism, the system comprising: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the heat load prediction method based on the self-attention mechanism as described in any one of the above.
[0013] The beneficial effects of the present invention are as follows: the present invention discloses a heat load prediction method and system based on a self-attention mechanism, the present invention constructs a real heat load data set, the real heat load data set includes multiple target data, the target data is the data obtained by coupling weather data and corresponding heat load data in a time series; through embedding technology, the target data in the real heat load data set is preprocessed to obtain a processed real heat load data set; an initial heat load prediction model is constructed, and according to the processed real heat load data set, the initial heat load prediction model is trained to obtain a trained target heat load prediction model, which is used to predict the heat load of buildings in the target area. In this way, the periodic feature extraction mechanism can be used to fully explore the time series characteristics of the heat load characteristics. And it can perform heat load prediction over a long time span, improving the accuracy of heat load prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0015] Figure 1 A flow chart of a heat load prediction method based on a self-attention mechanism in an embodiment of the present invention is shown.
[0016] Figure 2 A schematic diagram of constructing an initial heat load prediction model in an embodiment of the present invention is shown.
[0017] Figure 3 A block diagram of a heat load prediction device based on a self-attention mechanism in an embodiment of the present invention is shown. DETAILED DESCRIPTION
[0018] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.
[0019] See also Figure 1 , Figure 1 The present invention provides a heat load prediction method based on a self-attention mechanism, and the method comprises the following steps: With the increasing emphasis on building heat loads in recent years (due to their majority in number), improving the energy efficiency of buildings is a top priority. Accurate prediction of heat loads is essential in order to optimize the operation and planning of district heating systems.
[0020] Traditional district heating distributes heat load based on past experience, but this empirical distribution method is often time-consuming and labor-intensive, and also causes waste. Therefore, it is necessary to accurately predict the heat load. The prediction algorithms based on the past are mainly divided into three categories: black box model, gray box model and white box model.
[0021] The prior art generally has the following disadvantages: 1. The feature extraction for heat load is insufficient, and the features of time series are not specifically mined; 2. Other models have poor prediction effects when dealing with long time spans; 3. There are no defense measures against external data attacks.
[0022] Based on the above situation, the inventors of the present application adopted the Robustransblock model to predict the heat load data based on the attention mechanism, periodic feature extractor and adversarial attack training.
[0023] Compared with the existing technology, it has the following advantages: 1. Use periodic feature extraction mechanism to fully explore the time series characteristics of heat load characteristics; 2. Introducing the attention mechanism to enable heat load prediction over a long time span; 3. The introduction of adversarial attack training effectively resists the attack of adversarial samples.
[0024] Specifically, combined with Figure 1 , Figure 2 As shown, an embodiment of the present invention provides a heat load prediction method based on a self-attention mechanism, the method comprising steps 101 to 103: Step 101: constructing a real heat load data set, wherein the real heat load data set includes a plurality of target data, wherein the target data is data obtained by coupling weather data and corresponding heat load data according to a time series; In this embodiment, the real heat load data set includes a plurality of collected target data. The target data is the data obtained by coupling the weather data and the corresponding heat load data in a time series. For example, the heat load data can use the filtered, interpolated and filled data from 3021 commercial intelligent heat meters installed in residential buildings in Denmark from January 3, 2018 to December 31, 2020, with a resolution of days. It includes calendar data, historical heat load, water supply temperature, return water temperature, etc. In addition, the heat load data also provides building information, such as residential types such as apartments, townhouses, single-family houses and non-residential buildings. This embodiment takes the heat load prediction of a single-family house as an example for explanation.
[0025] In this embodiment, weather data refers to relevant weather information corresponding to heat load data. Weather data includes relative humidity and outdoor temperature, with a resolution of days. Local weather data and heat load data are coupled in time series to obtain sufficient and high-quality real heat load data sets.
[0026] Step 102: preprocessing the target data in the real heat load data set by embedding technology to obtain a processed real heat load data set; In this embodiment, the discrete target data (i.e., time series data) is mapped into a low-dimensional continuous vector space through embedding technology, and positional encoding is added to extract rich features to obtain a processed real heat load data set, that is, the original time series data is converted into the dimension of input data that can be recognized by the model, thereby improving the processing and analysis capabilities of heat load data.
[0027] Step 103: construct an initial heat load prediction model, and train the initial heat load prediction model according to the processed real heat load data set to obtain a trained target heat load prediction model for heat load prediction of buildings in the target area.
[0028] In this embodiment, the initial heat load prediction model includes a period feature extraction module and an attention mechanism model. The attention mechanism model includes an Encorder encoder and a Decorder decoder.
[0029] Among them, the periodic feature extraction module is used to perform sequence conversion processing on the target data of the processed real heat load data set to obtain a deep-level feature embedding vector; the deep-level feature embedding vector is input into the attention mechanism model and outputs the predicted heat load.
[0030] In one embodiment, the sequence conversion process includes: The target data of the real heat load data set is converted from a 1D sequence to a 2D sequence to facilitate the extraction of periodic features through a convolution operation. After the features are extracted, they are converted to a 1D sequence through adaptive aggregation.
[0031] It is easy to understand that due to the obvious periodicity of heat load, such as the obvious regularity of indoor temperature and building heat load during the day and night, and the obvious regularity of heat load in some office buildings during the week, which is reflected in the difference in heat load between working days and weekends. Therefore, we connect the feature extractor Timesblock after processing the target data, including the conversion from 1D sequence to 2D sequence, and then convert it to 1D sequence through adaptive aggregation after feature extraction. This feature extraction method of converting time series from 1D to 2D tensors can extract deeper features, capture multi-scale time features, and greatly improve the accuracy of prediction tasks.
[0032] The embedded vector containing a large number of features after data processing is input into the periodic feature extraction module, which includes converting from 1D sequence to 2D sequence, and then converting to 1D sequence through adaptive aggregation after feature extraction.
[0033] First, technically we use fast Fourier transform to get the time series in the frequency domain: A = avg(Amp(FFT(X1D))); f1,......,fk=argTopk(A); ; ; The above formula can be summarized as follows: A,{f1,...,fk},{p1,...,pk}=Period(X1D); We obtained the first k most significant frequencies, and based on the obtained frequencies, we calculated the period length, divided the heat load data into different segments according to the period length, and converted the 1D sequence data into 2D sequence data through the Reshape module: ; Next, we perform convolution on the extracted 2D sequence data to extract features. Since the heat load data has been converted into 2D sequence data according to the period dimension, it is easier to extract the feature relationships within and between periods. We use the same shared Inception module to extract different two-dimensional time series.
[0034] After extracting the features between cycles, the 2D heat load data after feature extraction is converted into a 1D original time series, and the time series is truncated by Trunc (·) to ensure that the original sequence structure does not change, but at this time, multi-scale features have been added to the time series. The formula is as follows: ; Finally, we need to adaptively aggregate the k different 1D tensors generated. This module uses the auto-correlation method. The corresponding amplitude A can reflect the relative importance of different period lengths and frequency selections. We use the amplitude obtained by the FFT for Period module to pass softmax (·) and then aggregate the 1D sequence: ; ; In addition, after optimizing the feature extraction of heat load data, we input the generated deep feature embedding vector into the attention mechanism model. The attention mechanism model can adopt the most classic attention mechanism Transformer, which can model the heat load data to extract key information from the heat load data. Compared with traditional recurrent neural networks, the attention mechanism model can process time series data in parallel, so it has an advantage in processing long sequence data. In addition, the attention mechanism model can capture the dependencies between different time steps, especially long-range dependencies, so as to more comprehensively understand the information interaction in time series data.
[0035] Specifically, the attention mechanism model implementation process is divided into two parts, the Encorder encoder module and the Decorder decoder module. Each encoder consists of two sub-layers: the attention layer (Multi-Head Attention layer) and the position-wise feed forward network (FFN). The output of the attention layer will flow into the feed forward network. The input of the encoder will first flow into the attention layer, and then the output of the attention layer will flow into the feed forward network.
[0036] In the Decorator stage, we use the output of the encoder and partial information of the target sequence to generate the output sequence.
[0037] The decoder also uses the self-attention mechanism to calculate attention weights, but unlike the encoder, the decoder also calculates encoder-decoder attention weights to guide the decoder to generate outputs related to the input sequence.
[0038] In the output generation stage, we use linear transformation and softmax function to convert the output of the decoder into a probability distribution. This probability distribution represents the generation probability of each vector in the output sequence. The decoder also consists of two sublayers: the self-attention layer and the feedforward network. But unlike the encoder, the decoder has an additional attention layer between the self-attention layer and the feedforward network, namely the encoder-decoder attention layer. The attention formula reference is as follows: ; In this embodiment, in the energy management system, the security of the target heat load prediction model is of vital importance. If the target heat load prediction model is used by an attacker, it may lead to inaccurate prediction results, thereby affecting the normal operation of the heating system. Therefore, it is necessary to conduct anti-attack training on the target heat load prediction model so that the target heat load prediction model can resist the attacker's attack and protect the security of the target heat load prediction model.
[0039] During the training process, after the model calculates the normal target Loss and obtains the correct target gradient in the forward propagation, the perturbation x+r is obtained according to the FGM formula adv , and then attack the Embeding sample, that is, x+r adv , get the adversarial sample, input the adversarial sample into the attention mechanism model to calculate the new Loss, calculate the gradient of the adversarial sample through back propagation, and then restore the Embedding value to the original state, and finally use the gradient of the adversarial sample to update the parameters of the attention mechanism model to complete an adversarial training. Among them: ; ; In summary, the embodiment of this specification provides a heat load prediction method based on the self-attention mechanism, by constructing a real heat load data set, the real heat load data set includes multiple target data, the target data is the data obtained by coupling weather data and corresponding heat load data in time series; through embedding technology, the target data in the real heat load data set is preprocessed to obtain a processed real heat load data set; an initial heat load prediction model is constructed, and the initial heat load prediction model is trained according to the processed real heat load data set to obtain a trained target heat load prediction model, which is used to predict the heat load of buildings in the target area. In this way, the periodic feature extraction mechanism can be used to fully explore the time series characteristics of the heat load characteristics. And it can perform heat load prediction over a long time span, improving the accuracy of heat load prediction. In addition, the introduction of adversarial attack training effectively resists the attack of adversarial samples.
[0040] and Figure 1 Corresponding to the method, refer to Figure 3 , an embodiment of the present invention provides a heat load prediction system based on a self-attention mechanism, comprising: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above method.
[0041] It can be seen that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0042] In addition, an embodiment of the present invention further discloses a computer program product or a computer program, which is stored in a computer-readable storage medium. A processor of a computer device can read the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above method. Similarly, the contents of the above method embodiment are all applicable to the storage medium embodiment, and the functions specifically implemented by the storage medium embodiment are the same as those of the above method embodiment, and the beneficial effects achieved are also the same as those achieved by the above method embodiment.
[0043] It will be appreciated by those skilled in the art that all or some of the methods disclosed above and the system may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0044] The above is a specific description of the preferred implementation of the present disclosure, but the present disclosure is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present disclosure. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present disclosure.
Claims
1. A heat load prediction method based on self-attention mechanism, characterized in that: include: Constructing a real heat load data set, wherein the real heat load data set includes a plurality of target data, wherein the target data is data obtained by coupling weather data and corresponding heat load data according to a time series; Preprocessing the target data in the real heat load data set by embedding technology to obtain a processed real heat load data set; An initial heat load prediction model is constructed, and the initial heat load prediction model is trained according to the processed real heat load data set to obtain a trained target heat load prediction model for heat load prediction of buildings in a target area.
2. The method according to claim 1, characterized in that The target data in the real heat load data set is preprocessed by the embedding technology, and the processed real heat load data set includes: The discrete target data in the real heat load data set is mapped into a low-dimensional continuous vector space through embedding technology, and the position vector is added to obtain a processed real heat load data set.
3. The method according to claim 1, characterized in that The constructing of the initial heat load prediction model comprises: The initial heat load prediction model includes a period feature extraction module and an attention mechanism model; The periodic feature extraction module is used to perform sequence conversion processing on the target data of the processed real heat load data set to obtain a deep-level feature embedding vector; the deep-level feature embedding vector is input into the attention mechanism model, and the predicted heat load is output.
4. The method according to claim 3, characterized in that The sequence conversion process includes: The target data of the real heat load data set is converted from a 1D sequence to a 2D sequence, and after feature extraction, it is converted back to a 1D sequence through adaptive aggregation.
5. The method according to claim 3, characterized in that: The attention mechanism model includes an Encorder encoder and a Decorder decoder.
6. The method according to claim 3, characterized in that The method further comprises: After the target Loss is calculated and the target gradient is obtained in the forward propagation, the adversarial sample data is determined; According to the adversarial sample, the parameters of the attention mechanism model are updated.
7. The method according to claim 1, characterized in that The heat load data includes historical heat load, and supply water temperature, return water temperature and building information corresponding to the historical heat load.
8. A heat load prediction system based on self-attention mechanism, characterized in that: The system comprises: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the heat load prediction method based on the self-attention mechanism as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Training method, prediction method, device and equipment of time sequence signal prediction model
CN117852624A
Generative adversarial model training method, adversarial sample generation method and related devices
CN117993476A
Data processing method and device, electronic equipment and storage medium
CN118260549A
Cited By
Transformer substation locating and sizing method and device, electronic equipment and storage medium
CN120952472A
Method and system for adjusting and relieving DLAA by using network parameters
CN121076855A
